@extrafemi-ai/lodestone 0.3.1 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -87,6 +87,36 @@ const { ok, withdrawn, shortTiers } = await ai.checkModels();
87
87
 
88
88
  `withdrawn` lists preferred models that have gone. That is harmless, they are already replaced. `ok` is false only when the catalogue can't be read or a tier is left with fewer than two real models. Without a daily run, a request that finds the saved catalogue over a day old refreshes it in the background.
89
89
 
90
+ ### Free models that are not worth waiting for (`paidFirst`)
91
+
92
+ Free models can be down for hours (rate-limited, or holding a connection open for 45 seconds and never answering). On the platform's own key, `openrouter: { paidFirst: true }` tries the low-cost paid model first and the free ones only if it fails or the spend limit is reached, so a request is not slowed by free models that are not going to answer. A caller's own key is not affected: the paid models are their money, so their free models still come first. Reasoning is set per model with `openrouter({ reasoning: (model) => ... })`, because models differ: one thinks for a thousand tokens unless told not to (measured: turning it off halved scoring time, 10 s to 5 s, with the same answers), another refuses to run with reasoning off.
93
+
94
+ ### Not waiting on a model that is down (circuit breaker)
95
+
96
+ Free models fail in bursts: rate-limited, stuck, or answering nothing. Without memory, every request pays for the same dead models again. `createHealth()` remembers:
97
+
98
+ ```ts
99
+ import { createHealth } from '@extrafemi-ai/lodestone';
100
+ const health = createHealth({ store }); // the same store as the catalogue: every serverless function shares it
101
+ const ai = createLodestone({ /* ... */, health });
102
+ ```
103
+
104
+ A model whose last three attempts (within ten minutes) all failed is skipped, so the next request goes straight to one that works. Every two minutes one attempt is let through to see whether it has recovered, and one success closes it again. A timeout, a rate limit, a server error and an unusable answer count as failures; a rejected key or missing credit (401, 402, 403) say nothing about the model and never do. If every free model looks down and a paid one can follow, the free ones are skipped entirely; if nothing can follow, they are still tried, because trying is better than not answering. `health.snapshot()` lists what it has seen, for a health page.
105
+
106
+ ### Limiting what the platform spends
107
+
108
+ ```ts
109
+ const ai = createLodestone({
110
+ /* ... */
111
+ spend: {
112
+ allow: async ({ model, phase }) => (await spentToday()) < DAILY_LIMIT_USD,
113
+ record: async ({ costUsd }) => addToday(costUsd ?? 0),
114
+ },
115
+ });
116
+ ```
117
+
118
+ `allow` is asked before each attempt that spends the platform's money (the paid models and fallback providers on the platform's keys), and a `false` means that model is not used. `record` is told what each such call cost: the provider's own figure when it gives one (OpenRouter does), otherwise an estimate from the model's catalogue price. Free models and a caller's own key (their money, not yours) never go through either. When the limit stops every paid option the error is `kind: 'budget'`. A `record` that throws never fails the answer.
119
+
90
120
  ### Letting measurements decide the order
91
121
 
92
122
  A scorecard (a small JSON file of measured results) can put the models that did best on your kind of task first, and Lodestone can publish its own live success rates and speeds in the same format. [Naija Eval](https://naija-eval.extrafemi.com) publishes translation and speech quality for Hausa, Igbo and Yoruba this way. It is opt-in, per task, and only changes the order models are tried in. See [docs/scorecards.md](docs/scorecards.md).
package/dist/errors.d.ts CHANGED
@@ -22,6 +22,8 @@ export type FailureKind =
22
22
  | 'unavailable'
23
23
  /** A model answered but the answer was not usable (empty, not valid JSON, failed validation). */
24
24
  | 'bad_output'
25
+ /** The platform's spend limit has been reached, so no paid model was used. */
26
+ | 'budget'
25
27
  /** No provider has a key. */
26
28
  | 'not_configured' | 'other';
27
29
  /** Thrown when no model could answer. `kind` says why, so the caller can decide what
package/dist/explain.js CHANGED
@@ -42,6 +42,8 @@ export function explain(err, opts = {}) {
42
42
  return "The AI gave back something that couldn't be read. Try again; it usually works on a second attempt.";
43
43
  case 'credit':
44
44
  return 'The AI account has no credit for the backup model. Add a little credit and try again.';
45
+ case 'budget':
46
+ return `${product}'s AI budget for now is used up, and the free models are busy. Try again later${opts.settingsHint ? `, or add your own OpenRouter key ${opts.settingsHint.replace(/^in /, 'in ')} to keep going` : ''}.`;
45
47
  case 'not_configured':
46
48
  return `${product}'s AI isn't set up yet.`;
47
49
  default:
@@ -0,0 +1,39 @@
1
+ import type { KeyValueStore } from './types.js';
2
+ /** A circuit breaker: remembers which models have been failing, so a request does not spend its
3
+ * time on a model that is down. A model whose last few attempts all failed is skipped; every
4
+ * couple of minutes one attempt is let through to see whether it has recovered, and one success
5
+ * closes the breaker again. With a store, every instance (every serverless function) shares what
6
+ * it has learned, so one request's failure saves the next request's time.
7
+ *
8
+ * Counts a timeout, a rate limit, a server error and an unusable answer (empty, not JSON) as
9
+ * failures. A rejected key or missing credit (401, 402, 403) says nothing about the model and is
10
+ * never recorded. */
11
+ export interface HealthOptions {
12
+ /** Where to keep what it has learned, shared across instances. Default: memory only. */
13
+ store?: KeyValueStore;
14
+ key?: string;
15
+ /** A model is skipped when its last `failures` attempts all failed within `windowMs`. Default 3 and 10 minutes. */
16
+ failures?: number;
17
+ windowMs?: number;
18
+ /** While a model is skipped, one attempt is let through this often. Default 2 minutes. */
19
+ probeEveryMs?: number;
20
+ now?: () => number;
21
+ }
22
+ export interface ModelHealth {
23
+ provider: string;
24
+ model: string;
25
+ /** Skipped right now. */
26
+ open: boolean;
27
+ /** Failures among its last few attempts. */
28
+ recentFailures: number;
29
+ recentAttempts: number;
30
+ lastFailureAt: string | null;
31
+ }
32
+ export interface Health {
33
+ /** Should this model be skipped right now? */
34
+ shouldSkip(provider: string, model: string): Promise<boolean>;
35
+ record(provider: string, model: string, ok: boolean): Promise<void>;
36
+ /** Every model it has seen, for a health page. Models not seen in the window are left out. */
37
+ snapshot(): Promise<ModelHealth[]>;
38
+ }
39
+ export declare function createHealth(opts?: HealthOptions): Health;
package/dist/health.js ADDED
@@ -0,0 +1,73 @@
1
+ const KEEP = 6;
2
+ const SHARE_MS = 15_000;
3
+ const id = (provider, model) => `${provider}\u0000${model}`;
4
+ export function createHealth(opts = {}) {
5
+ const need = opts.failures ?? 3;
6
+ const windowMs = opts.windowMs ?? 10 * 60_000;
7
+ const probeMs = opts.probeEveryMs ?? 2 * 60_000;
8
+ const now = opts.now ?? Date.now;
9
+ const key = opts.key ?? 'lodestone:health';
10
+ let state = {};
11
+ let loadedAt = -Infinity;
12
+ /** Brings in what other instances have saved, at most every 15 seconds. */
13
+ async function refresh() {
14
+ if (!opts.store || now() - loadedAt < SHARE_MS)
15
+ return;
16
+ loadedAt = now();
17
+ try {
18
+ state = merge(state, (await opts.store.get(key)) ?? {});
19
+ }
20
+ catch { /* an unreadable store is the same as an empty one */ }
21
+ }
22
+ const merge = (a, b) => {
23
+ const out = {};
24
+ for (const k of new Set([...Object.keys(a), ...Object.keys(b)])) {
25
+ const seen = new Set();
26
+ out[k] = [...(a[k] ?? []), ...(b[k] ?? [])].filter(([at, , sid]) => { const u = `${at}:${sid}`; return seen.has(u) ? false : (seen.add(u), true); }).sort((x, y) => x[0] - y[0]).slice(-KEEP);
27
+ }
28
+ return out;
29
+ };
30
+ const recent = (k) => (state[k] ?? []).filter(([at]) => now() - at <= windowMs);
31
+ const isOpen = (r) => {
32
+ const last = r.slice(-need);
33
+ if (last.length < need || last.some(([, ok]) => ok === 1))
34
+ return false;
35
+ return now() - last[last.length - 1][0] < probeMs; // after a while, let one attempt through to see if it has recovered
36
+ };
37
+ return {
38
+ async shouldSkip(provider, model) {
39
+ await refresh();
40
+ return isOpen(recent(id(provider, model)));
41
+ },
42
+ async record(provider, model, ok) {
43
+ const k = id(provider, model);
44
+ const before = recent(k);
45
+ const recovering = before.length > 0 && before[before.length - 1][1] === 0; // this success follows a failure
46
+ state[k] = [...(state[k] ?? []), [now(), ok ? 1 : 0, Math.random().toString(36).slice(2, 8)]].slice(-KEEP);
47
+ // Only failures, and a success right after a failure, change what anyone else needs to know;
48
+ // a model that is simply working costs nothing extra.
49
+ if (!opts.store || (ok && !recovering))
50
+ return;
51
+ try {
52
+ const theirs = (await opts.store.get(key)) ?? {};
53
+ state = merge(state, theirs);
54
+ await opts.store.set(key, state);
55
+ loadedAt = now();
56
+ }
57
+ catch { /* not saved: the next instance just has to learn it again */ }
58
+ },
59
+ async snapshot() {
60
+ await refresh();
61
+ const out = [];
62
+ for (const k of Object.keys(state)) {
63
+ const r = recent(k);
64
+ if (!r.length)
65
+ continue;
66
+ const [provider, model] = k.split('\u0000');
67
+ const fails = r.filter(([, ok]) => ok === 0);
68
+ out.push({ provider, model, open: isOpen(r), recentFailures: fails.length, recentAttempts: r.length, lastFailureAt: fails.length ? new Date(fails[fails.length - 1][0]).toISOString() : null });
69
+ }
70
+ return out;
71
+ },
72
+ };
73
+ }
package/dist/index.d.ts CHANGED
@@ -1,5 +1,5 @@
1
1
  export { createLodestone } from './lodestone.js';
2
- export type { Lodestone, LodestoneConfig, CompleteOptions, CompleteResult, Profile, FallbackConfig, AttemptEvent, RouteStep } from './lodestone.js';
2
+ export type { Lodestone, LodestoneConfig, CompleteOptions, CompleteResult, Profile, FallbackConfig, AttemptEvent, RouteStep, SpendControl } from './lodestone.js';
3
3
  export { LodestoneError, ProviderError, messageFrom } from './errors.js';
4
4
  export type { FailureKind } from './errors.js';
5
5
  export { explain } from './explain.js';
@@ -22,3 +22,5 @@ export { createStats } from './stats.js';
22
22
  export type { Stats, StatsOptions, StatsState } from './stats.js';
23
23
  export { CANDIDATES_SCHEMA } from './candidates.js';
24
24
  export type { Candidates, Candidate } from './candidates.js';
25
+ export { createHealth } from './health.js';
26
+ export type { Health, HealthOptions, ModelHealth } from './health.js';
package/dist/index.js CHANGED
@@ -13,3 +13,4 @@ export { mock } from './providers/mock.js';
13
13
  export { SCORECARD_SCHEMA, ScorecardError, parseScorecard, fetchScorecard, rankModels, scoreFor, baseModelId } from './scorecard.js';
14
14
  export { createStats } from './stats.js';
15
15
  export { CANDIDATES_SCHEMA } from './candidates.js';
16
+ export { createHealth } from './health.js';
@@ -1,5 +1,6 @@
1
1
  import { type CatalogModel } from './catalog.js';
2
2
  import { type Candidates } from './candidates.js';
3
+ import type { Health } from './health.js';
3
4
  import { type QualityQuery, type Scorecard } from './scorecard.js';
4
5
  import type { Stats } from './stats.js';
5
6
  import type { KeyValueStore, Message, Provider, Tier, Usage } from './types.js';
@@ -22,16 +23,37 @@ export interface AttemptEvent {
22
23
  provider: string;
23
24
  model: string;
24
25
  phase: 'free' | 'paid' | 'fallback' | 'route';
25
- outcome: 'ok' | 'timeout' | 'failed';
26
+ /** 'skipped': not tried, because the breaker had it down or the spend limit was reached. */
27
+ outcome: 'ok' | 'timeout' | 'failed' | 'skipped';
26
28
  status?: number;
27
29
  ms: number;
28
30
  }
31
+ /** Where the platform's money is counted. Only attempts that spend it go through: the paid models and the
32
+ * fallback providers on the platform's keys. A call on the caller's own key never does. */
33
+ export interface SpendControl {
34
+ /** Asked before each such attempt. Return false once the limit is reached and that model is not used. */
35
+ allow(info: {
36
+ provider: string;
37
+ model: string;
38
+ phase: 'paid' | 'fallback';
39
+ }): boolean | Promise<boolean>;
40
+ /** Told after each such attempt that answered, with what it cost: the provider's own figure when it gives one (OpenRouter does), otherwise an estimate from the model's catalogue price, otherwise nothing. */
41
+ record(info: {
42
+ provider: string;
43
+ model: string;
44
+ phase: 'paid' | 'fallback';
45
+ usage?: Usage;
46
+ costUsd?: number;
47
+ }): void | Promise<void>;
48
+ }
29
49
  export interface LodestoneConfig {
30
50
  openrouter?: {
31
51
  /** The platform's key. A caller's own key (`apiKey` on a call) takes priority. */
32
52
  apiKey?: string;
33
53
  /** Use the paid models after the free ones fail, on the platform's key. A caller's own key always allows it. */
34
54
  allowPaid?: boolean;
55
+ /** On the platform's key, try the paid models first and the free ones only if they fail or the spend limit is reached. For when the free models are too unreliable to be worth waiting for. Not applied to a caller's own key: it is their money, so their free models are still tried first. */
56
+ paidFirst?: boolean;
35
57
  models?: Partial<Record<Tier, string[]>>;
36
58
  paidModels?: string[];
37
59
  /** Replace the OpenRouter adapter (tests, proxies). */
@@ -46,6 +68,10 @@ export interface LodestoneConfig {
46
68
  * nobody has measured keep their place. A function is called per request, so it can fetch
47
69
  * and cache; if it fails or takes over 1.5 s the order is left alone. */
48
70
  scorecards?: Scorecard[] | (() => Promise<Scorecard[]>);
71
+ /** Skips models that have been failing, so a request does not spend its time on them (see health.ts). */
72
+ health?: Health;
73
+ /** Counts and limits the platform's spend. Without it nothing is counted or limited. */
74
+ spend?: SpendControl;
49
75
  /** Records every attempt so you can publish how each model does (stats.scorecard()). */
50
76
  stats?: Stats;
51
77
  /** Named tasks, so calls say what they are doing and not which model to use. */
@@ -60,6 +86,7 @@ export interface LodestoneConfig {
60
86
  freeSliceWithLaterMs?: number;
61
87
  freePhaseWithLaterMs?: number;
62
88
  minSliceMs?: number;
89
+ paidSliceMs?: number;
63
90
  };
64
91
  fetch?: typeof fetch;
65
92
  now?: () => number;
package/dist/lodestone.js CHANGED
@@ -17,6 +17,7 @@ export function createLodestone(config = {}) {
17
17
  freeSliceLater: config.timing?.freeSliceWithLaterMs ?? 5_000,
18
18
  freePhaseLater: config.timing?.freePhaseWithLaterMs ?? 9_000,
19
19
  minSlice: config.timing?.minSliceMs ?? 3_000,
20
+ paidSlice: config.timing?.paidSliceMs ?? 12_000,
20
21
  };
21
22
  const now = config.now ?? Date.now;
22
23
  const store = config.store ?? memoryStore();
@@ -50,6 +51,15 @@ export function createLodestone(config = {}) {
50
51
  return [];
51
52
  }
52
53
  }
54
+ /** What a call cost, estimated from the model's catalogue price, for providers that do not report it. */
55
+ async function estimateCost(model, usage) {
56
+ if (!usage)
57
+ return undefined;
58
+ const m = (await catalog.load().catch(() => null))?.find((x) => x.id === model);
59
+ const inP = Number(m?.pricing?.prompt);
60
+ const outP = Number(m?.pricing?.completion);
61
+ return Number.isFinite(inP) && Number.isFinite(outP) ? usage.inputTokens * inP + usage.outputTokens * outP : undefined;
62
+ }
53
63
  async function complete(opts) {
54
64
  const messages = opts.messages ?? (opts.prompt !== undefined ? [{ role: 'user', content: opts.prompt }] : []);
55
65
  if (messages.length === 0)
@@ -86,6 +96,31 @@ export function createLodestone(config = {}) {
86
96
  let rateLimited = null;
87
97
  let authFailed = false;
88
98
  let needsCredit = false;
99
+ let spendBlocked = false;
100
+ const callerKey = Boolean(opts.apiKey);
101
+ /** Drops the models the breaker has down, saying so. */
102
+ const healthy = async (provider, list, phase) => {
103
+ if (!config.health)
104
+ return list;
105
+ const keep = [];
106
+ for (const m of list) {
107
+ if (await config.health.shouldSkip(provider.name, m))
108
+ config.onAttempt?.({ provider: provider.name, model: m, phase, outcome: 'skipped', ms: 0 });
109
+ else
110
+ keep.push(m);
111
+ }
112
+ return keep;
113
+ };
114
+ /** May the platform's money be spent on this attempt? A caller's own key is their money, not ours. */
115
+ const mayUse = async (provider, model, phase) => {
116
+ if (callerKey || !config.spend)
117
+ return true;
118
+ if (await config.spend.allow({ provider: provider.name, model, phase }))
119
+ return true;
120
+ spendBlocked = true;
121
+ config.onAttempt?.({ provider: provider.name, model, phase, outcome: 'skipped', ms: 0 });
122
+ return false;
123
+ };
89
124
  const attempt = async (provider, apiKey, model, phase, ms) => {
90
125
  attempted = true;
91
126
  const at = now();
@@ -110,14 +145,25 @@ export function createLodestone(config = {}) {
110
145
  }
111
146
  const event = { provider: provider.name, model, phase, outcome: out.kind === 'ok' ? 'ok' : out.kind === 'timeout' ? 'timeout' : 'failed', status: out.kind === 'fail' ? out.status : undefined, ms: now() - at };
112
147
  // A key or credit problem (401, 402, 403) says nothing about the model, so it is not counted against it.
113
- if (!(out.kind === 'fail' && [401, 402, 403].includes(out.status)))
148
+ const credential = out.kind === 'fail' && [401, 402, 403].includes(out.status);
149
+ if (!credential) {
114
150
  config.stats?.record(event);
151
+ await config.health?.record(provider.name, model, out.kind === 'ok');
152
+ }
115
153
  config.onAttempt?.(event);
116
154
  last = out;
117
155
  return out;
118
156
  };
119
- const finish = async (provider, model, reply) => {
157
+ const finish = async (provider, model, reply, phase) => {
120
158
  const result = { text: reply.text, provider: provider.name, model, usage: reply.usage };
159
+ if (phase && !callerKey && config.spend) {
160
+ const costUsd = reply.usage?.costUsd ?? (await estimateCost(model, reply.usage));
161
+ // A ledger that fails must never fail an answer the person is waiting for.
162
+ try {
163
+ await config.spend.record({ provider: provider.name, model, phase, usage: reply.usage, costUsd });
164
+ }
165
+ catch { /* not counted */ }
166
+ }
121
167
  if (cacheSlot)
122
168
  await store.set(cacheSlot, { ...result, at: now() }).catch(() => { });
123
169
  return result;
@@ -150,11 +196,19 @@ export function createLodestone(config = {}) {
150
196
  throw new LodestoneError('slow');
151
197
  throw classify(firstFail ?? (lastRoute?.kind === 'fail' ? lastRoute : null), Boolean(opts.apiKey), false, false);
152
198
  }
153
- // Phase 1: OpenRouter's free models. A withdrawn one has already been replaced by resolveModels.
154
- if (orKey) {
155
- const models = await order(await resolveModels(tier));
156
- for (let i = 0; i < models.length && freeLeft() >= t.minSlice; i++) {
157
- const slice = i >= models.length - 1 ? freeLeft() : Math.min(later ? t.freeSliceLater : t.freeSlice, freeLeft());
199
+ // OpenRouter: the free models and the low-cost paid ones. Free first by default; paid first when the
200
+ // platform's key is used and the free models are not worth waiting for (see paidFirst).
201
+ const paidFirst = Boolean(config.openrouter?.paidFirst) && !callerKey && paidAvailable;
202
+ /** The free models. A withdrawn one has already been replaced by resolveModels. */
203
+ const runFree = async () => {
204
+ const all = await order(await resolveModels(tier));
205
+ let models = await healthy(router, all, 'free');
206
+ // With nothing after the free models to fall back on, a model that may be down is still better than no attempt.
207
+ if (models.length === 0 && !later && !paidFirst)
208
+ models = all;
209
+ const leftHere = () => (paidFirst ? left() : freeLeft());
210
+ for (let i = 0; i < models.length && leftHere() >= t.minSlice; i++) {
211
+ const slice = i >= models.length - 1 ? leftHere() : Math.min(later && !paidFirst ? t.freeSliceLater : t.freeSlice, leftHere());
158
212
  const out = await attempt(router, orKey, models[i], 'free', slice);
159
213
  if (out.kind === 'ok')
160
214
  return finish(router, models[i], out.reply);
@@ -167,39 +221,78 @@ export function createLodestone(config = {}) {
167
221
  rateLimited = out;
168
222
  }
169
223
  }
170
- // Phase 2: a low-cost paid model, which is not under the free-model limits.
171
- if (paidAvailable && !authFailed) {
172
- for (const m of await order(paidModels)) {
173
- if (left() < t.minSlice)
224
+ return null;
225
+ };
226
+ /** A low-cost paid model, which is not under the free-model limits. */
227
+ const runPaid = async () => {
228
+ const paidAll = await order(paidModels);
229
+ let paidList = await healthy(router, paidAll, 'paid');
230
+ if (paidList.length === 0)
231
+ paidList = paidAll; // never leave no paid attempt because of the breaker
232
+ for (let i = 0; i < paidList.length; i++) {
233
+ const m = paidList[i];
234
+ if (left() < t.minSlice)
235
+ break;
236
+ if (!(await mayUse(router, m, 'paid')))
237
+ continue;
238
+ // Going first, a paid model may not use all the time: the free models and the rest need some.
239
+ const ms = paidFirst && i < paidList.length - 1 ? Math.min(left(), t.paidSlice) : left();
240
+ const out = await attempt(router, orKey, m, 'paid', ms);
241
+ if (out.kind === 'ok')
242
+ return finish(router, m, out.reply, 'paid');
243
+ if (out.kind === 'fail') {
244
+ if (out.status === 402) {
245
+ needsCredit = true;
174
246
  break;
175
- const out = await attempt(router, orKey, m, 'paid', left());
176
- if (out.kind === 'ok')
177
- return finish(router, m, out.reply);
178
- if (out.kind === 'fail') {
179
- if (out.status === 402) {
180
- needsCredit = true;
181
- break;
182
- }
183
- if (out.status === 401)
184
- break;
185
247
  }
248
+ if (out.status === 401)
249
+ break;
250
+ }
251
+ }
252
+ return null;
253
+ };
254
+ if (orKey) {
255
+ if (paidFirst) {
256
+ const a = await runPaid();
257
+ if (a)
258
+ return a;
259
+ const b = authFailed ? null : await runFree();
260
+ if (b)
261
+ return b;
262
+ }
263
+ else {
264
+ const b = await runFree();
265
+ if (b)
266
+ return b;
267
+ if (paidAvailable && !authFailed) {
268
+ const a = await runPaid();
269
+ if (a)
270
+ return a;
186
271
  }
187
272
  }
188
273
  }
189
274
  // Phase 3: other providers, with their own keys.
190
275
  for (const fb of fbs) {
191
- for (const m of await order(fb.models)) {
276
+ const fbAll = await order(fb.models);
277
+ let fbList = await healthy(fb.provider, fbAll, 'fallback');
278
+ if (fbList.length === 0)
279
+ fbList = fbAll;
280
+ for (const m of fbList) {
192
281
  if (left() < t.minSlice)
193
282
  break;
283
+ if (!(await mayUse(fb.provider, m, 'fallback')))
284
+ continue;
194
285
  const out = await attempt(fb.provider, fb.apiKey, m, 'fallback', left());
195
286
  if (out.kind === 'ok')
196
- return finish(fb.provider, m, out.reply);
287
+ return finish(fb.provider, m, out.reply, 'fallback');
197
288
  if (out.kind === 'fail' && out.status === 401)
198
289
  break; // this provider's key is wrong: skip its other models
199
290
  }
200
291
  }
201
292
  // Nothing answered. Say why, using what the first useful failure told us.
202
293
  const ownKey = Boolean(opts.apiKey);
294
+ if (spendBlocked)
295
+ throw new LodestoneError('budget');
203
296
  if (!attempted)
204
297
  throw new LodestoneError('not_configured');
205
298
  const lastFail = last;
@@ -3,6 +3,8 @@ export interface OpenRouterOptions {
3
3
  fetch?: typeof fetch;
4
4
  /** Ask the model not to think at length. Default true: most tasks need an answer, not a long chain of thought, and thinking is what made some free models slower than the time allowed. */
5
5
  lowReasoning?: boolean;
6
+ /** How much each model should think, by model id: an OpenRouter `reasoning` setting, or null to send none. Overrides `lowReasoning`. Models differ: some will not run with reasoning off ("mandatory for this endpoint"), some think for thousands of tokens unless told not to, so one setting for every model is wrong. For example `(m) => m.startsWith('anthropic/claude-haiku') ? { enabled: false } : { effort: 'low', exclude: true }`. */
7
+ reasoning?: (model: string) => Record<string, unknown> | null;
6
8
  /** Sent as HTTP-Referer and X-Title, which OpenRouter uses to credit apps. */
7
9
  referer?: string;
8
10
  title?: string;
@@ -3,6 +3,13 @@ const part = (p) => (p.type === 'text' ? { type: 'text', text: p.text } : { type
3
3
  const wire = (m) => ({ role: m.role, content: typeof m.content === 'string' ? m.content : m.content.map(part) });
4
4
  export function openrouter(opts = {}) {
5
5
  const f = opts.fetch ?? fetch;
6
+ const reasoningFor = (model) => {
7
+ if (opts.reasoning) {
8
+ const r = opts.reasoning(model);
9
+ return r ? { reasoning: r } : {};
10
+ }
11
+ return opts.lowReasoning === false ? {} : { reasoning: { effort: 'low', exclude: true } };
12
+ };
6
13
  return {
7
14
  name: 'openrouter',
8
15
  vision: true,
@@ -17,9 +24,10 @@ export function openrouter(opts = {}) {
17
24
  },
18
25
  body: JSON.stringify({
19
26
  model: req.model,
27
+ usage: { include: true }, // asks OpenRouter to report what the call cost
20
28
  ...(req.temperature === undefined ? {} : { temperature: req.temperature }),
21
29
  max_tokens: req.maxTokens,
22
- ...(opts.lowReasoning === false ? {} : { reasoning: { effort: 'low', exclude: true } }),
30
+ ...reasoningFor(req.model),
23
31
  messages: [...(req.system ? [{ role: 'system', content: req.system }] : []), ...req.messages.map(wire)],
24
32
  }),
25
33
  signal: req.signal,
@@ -29,7 +37,7 @@ export function openrouter(opts = {}) {
29
37
  const data = (await res.json());
30
38
  return {
31
39
  text: data.choices?.[0]?.message?.content ?? '',
32
- usage: data.usage ? { inputTokens: data.usage.prompt_tokens ?? 0, outputTokens: data.usage.completion_tokens ?? 0 } : undefined,
40
+ usage: data.usage ? { inputTokens: data.usage.prompt_tokens ?? 0, outputTokens: data.usage.completion_tokens ?? 0, ...(typeof data.usage.cost === 'number' ? { costUsd: data.usage.cost } : {}) } : undefined,
33
41
  };
34
42
  },
35
43
  };
package/dist/stats.js CHANGED
@@ -8,6 +8,8 @@ export function createStats(opts = {}) {
8
8
  let state = {};
9
9
  return {
10
10
  record(e) {
11
+ if (e.outcome === 'skipped')
12
+ return; // not tried, so nothing to count
11
13
  const k = key(e.provider, e.model);
12
14
  const list = (state[k] ??= []);
13
15
  list.push({ ok: e.outcome === 'ok', ms: e.ms, at: now() });
package/dist/types.d.ts CHANGED
@@ -13,6 +13,8 @@ export interface Message {
13
13
  export interface Usage {
14
14
  inputTokens: number;
15
15
  outputTokens: number;
16
+ /** What the provider charged for this call, in US dollars, when it says (OpenRouter does). */
17
+ costUsd?: number;
16
18
  }
17
19
  /** What a provider adapter is asked to do: one model, one request. */
18
20
  export interface ProviderRequest {
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@extrafemi-ai/lodestone",
3
- "version": "0.3.1",
3
+ "version": "0.4.0",
4
4
  "description": "Calls AI models so your tool keeps working when a model is withdrawn, slow or rate-limited.",
5
5
  "license": "MIT",
6
6
  "repository": {