@blockrun/llm 3.20.1 → 3.22.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +90 -16
- package/dist/index.cjs +821 -191
- package/dist/index.d.cts +276 -38
- package/dist/index.d.ts +276 -38
- package/dist/index.js +814 -188
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
|
|
5
5
|
### Cut your LLM bill by <!-- br:savings.autoVsBaselinePct -->84<!-- /br:savings.autoVsBaselinePct -->%. One line of TypeScript.
|
|
6
6
|
|
|
7
|
-
The smart-routing SDK for <!-- br:models.chatVisible -->
|
|
7
|
+
The smart-routing SDK for <!-- br:models.chatVisible -->88<!-- /br:models.chatVisible --> models — every request goes to the cheapest model that can handle it,
|
|
8
8
|
paid with an API key or per-request USDC on Solana or Base. No vendor lock-in.
|
|
9
9
|
|
|
10
10
|
[](https://www.npmjs.com/package/@blockrun/llm)
|
|
@@ -44,7 +44,7 @@ console.log(r.response); // the proof
|
|
|
44
44
|
## Why This SDK
|
|
45
45
|
|
|
46
46
|
- 🧠 **Smart routing that pays for itself** — the bundled [Router Core V3](https://github.com/BlockRunAI/router-core) engine (shared with [ClawRouter](https://github.com/BlockRunAI/ClawRouter)) classifies every request locally in <1ms across <!-- br:clawrouter.dimensions -->15<!-- /br:clawrouter.dimensions --> dimensions and routes to the cheapest capable model. The main event.
|
|
47
|
-
- 🆓 **<!-- br:models.free -->
|
|
47
|
+
- 🆓 **<!-- br:models.free -->7<!-- /br:models.free --> genuinely free models** — $0 in and out, incl. two 1M-context Nemotrons, a multimodal one, and free coding models from Cohere and Poolside. No rate-limit gimmicks.
|
|
48
48
|
- 🔐 **Two ways to connect** — a **BlockRun API key** billed against account credit ([sign up at user.blockrun.ai](https://user.blockrun.ai), [create a key](https://user.blockrun.ai/dashboard/keys), [add credit](https://user.blockrun.ai/dashboard/credits)), or a wallet signature with x402 micropayments and no account at all. Same code either way.
|
|
49
49
|
- 💸 **Pay per request in USDC** — x402 micropayments on Solana or Base. $5 covers thousands of requests; agents can pay their own way.
|
|
50
50
|
- 🛡️ **Automatic failover** — transient errors (timeouts, 429, 5xx) before payment walk the router's ranked fallback chain instead of failing your request; a failure after a payment was sent never buys a second model.
|
|
@@ -56,8 +56,8 @@ console.log(r.response); // the proof
|
|
|
56
56
|
| | OpenAI SDK | OpenRouter | LiteLLM | **@blockrun/llm** |
|
|
57
57
|
| ------------------ | -------------- | ----------------- | ---------------- | ----------------------------------------------------------------------- |
|
|
58
58
|
| **Cost routing** | ✗ one vendor | Manual selection | Manual selection | **Automatic — <!-- br:savings.autoVsBaselinePct -->84<!-- /br:savings.autoVsBaselinePct -->% cheaper** |
|
|
59
|
-
| **Models** | GPT only | 200+ | 100+ (BYO keys) | **<!-- br:models.chatVisible -->
|
|
60
|
-
| **Free tier** | ✗ | Rate-limited | ✗ | **<!-- br:models.free -->
|
|
59
|
+
| **Models** | GPT only | 200+ | 100+ (BYO keys) | **<!-- br:models.chatVisible -->88<!-- /br:models.chatVisible -->, one credential** |
|
|
60
|
+
| **Free tier** | ✗ | Rate-limited | ✗ | **<!-- br:models.free -->7<!-- /br:models.free --> models, no signup** |
|
|
61
61
|
| **Auth** | API key | Account + API key | Your API keys | **API key *or* wallet signature** |
|
|
62
62
|
| **Payment** | Card + invoice | Credit card | BYO keys | **Account credit or USDC per-request** |
|
|
63
63
|
| **Agent-ready** | ✗ | ✗ | ✗ | **✓ — one key, or agents fund their own wallet** |
|
|
@@ -359,7 +359,7 @@ const reply = await client.chat('nvidia/nemotron-3.5-lightning', 'hello', {
|
|
|
359
359
|
|
|
360
360
|
| Profile | Strategy | Savings vs Opus 5 | Best For |
|
|
361
361
|
|---------|----------|-------------------|----------|
|
|
362
|
-
| `eco` | Cheapest capable model — ranks the <!-- br:models.free -->
|
|
362
|
+
| `eco` | Cheapest capable model — ranks the <!-- br:models.free -->7<!-- /br:models.free -->-model free tier first | **<!-- br:savings.ecoVsBaselinePct -->98<!-- /br:savings.ecoVsBaselinePct -->%** | Cost-sensitive production, zero-cost testing |
|
|
363
363
|
| `auto` | Best balance of cost/quality (default) | **<!-- br:savings.autoVsBaselinePct -->84<!-- /br:savings.autoVsBaselinePct -->%** | General use |
|
|
364
364
|
| `premium` | Top-tier models (OpenAI, Anthropic) | 0% | Quality-critical tasks |
|
|
365
365
|
|
|
@@ -441,6 +441,14 @@ picked:
|
|
|
441
441
|
| `reasoning` | Human-readable explanation of the decision |
|
|
442
442
|
| `tierConfigs` | The tier → primary/fallback map the decision was made against |
|
|
443
443
|
|
|
444
|
+
**Search and routing.** xAI search (`x_search` / `web_search` tools, or
|
|
445
|
+
`search` / `searchParameters` not set to `mode: 'off'`) runs on `xai/*` models
|
|
446
|
+
only. When a routed call asks for search, the candidate chain is narrowed to
|
|
447
|
+
its `xai/*` models in the router's order; when the route has none, `route()` /
|
|
448
|
+
`smartChat()` / `smartChatCompletion()` throw before anything is sent or paid.
|
|
449
|
+
For search, calling an `xai/*` model directly (e.g. `xai/grok-4.3`) is the
|
|
450
|
+
dependable path.
|
|
451
|
+
|
|
444
452
|
### TypeScript Types
|
|
445
453
|
|
|
446
454
|
`RoutingDecision`, `RoutingProfile`, `RoutingTier`, `RoutingTaskType`,
|
|
@@ -480,8 +488,26 @@ console.log(response);
|
|
|
480
488
|
|
|
481
489
|
// Live Search with Grok (Solana payment)
|
|
482
490
|
const tweet = await client.chat('xai/grok-4.5', 'What is trending on X?', { search: true });
|
|
491
|
+
|
|
492
|
+
// xAI x_search on chosen accounts, structured output, real cost per call
|
|
493
|
+
const res = await client.chatCompletion('xai/grok-4.3', [
|
|
494
|
+
{ role: 'user', content: 'Which new models did these labs announce today? Return JSON.' },
|
|
495
|
+
], {
|
|
496
|
+
tools: [{ type: 'x_search', allowedXHandles: ['OpenAI', 'AnthropicAI', 'xai'], fromDate: '2026-10-04' }],
|
|
497
|
+
searchParameters: { maxTurns: 2 }, // bounds xAI's per-post search bill
|
|
498
|
+
responseFormat: { type: 'json_object' },
|
|
499
|
+
timeout: 180_000,
|
|
500
|
+
});
|
|
501
|
+
console.log(res.choices[0].message.annotations); // url_citation with positions
|
|
502
|
+
console.log(res.usage?.server_side_tool_usage); // { x_posts_fetched, ... }
|
|
503
|
+
console.log(res.costUsd, res.settlement); // what you paid, and the tx
|
|
483
504
|
```
|
|
484
505
|
|
|
506
|
+
`costUsd`, `settlement` and the per-call `timeout` work the same on `LLMClient`
|
|
507
|
+
(Base / Arc). `costUsd` is what the call added to `getSpending()`: the signed
|
|
508
|
+
price under `exact`, the metered `extra.chargedAmount` under batch settlement,
|
|
509
|
+
`0` for a free model; it is undefined in API-key mode (billed to the account).
|
|
510
|
+
|
|
485
511
|
**Setup:**
|
|
486
512
|
1. Export your Solana wallet key: `export SOLANA_WALLET_KEY="your-bs58-key"`
|
|
487
513
|
2. Fund with USDC on Solana mainnet
|
|
@@ -490,6 +516,25 @@ const tweet = await client.chat('xai/grok-4.5', 'What is trending on X?', { sear
|
|
|
490
516
|
**Supported endpoint:** `https://sol.blockrun.ai/api`
|
|
491
517
|
**Payment:** Solana USDC (SPL, mainnet)
|
|
492
518
|
|
|
519
|
+
**Images, music and video on Solana:**
|
|
520
|
+
|
|
521
|
+
```typescript
|
|
522
|
+
const image = await client.image('a lighthouse at dusk', { model: 'openai/gpt-image-2' });
|
|
523
|
+
const track = await client.music('chill lo-fi beats with piano');
|
|
524
|
+
const video = await client.video('a red apple slowly spinning', { duration_seconds: 5 });
|
|
525
|
+
```
|
|
526
|
+
|
|
527
|
+
Slow jobs answer `202 { id, poll_url }` and the client polls for you. Each poll
|
|
528
|
+
proves the job is yours with a wallet signature (`x-poll-wallet`,
|
|
529
|
+
`x-poll-timestamp`, `x-poll-signature`: ed25519 over
|
|
530
|
+
`blockrun-poll:v1:<kind>:<job id>:<unix seconds>`), not a payment, so a
|
|
531
|
+
multi-minute job no longer signs a Solana transaction every few seconds. Image
|
|
532
|
+
and music are paid when submitted. A video is charged only once it is finished:
|
|
533
|
+
the client signs one payment for the completed job, and a failed job costs
|
|
534
|
+
nothing. If the gateway does not accept the poll signature for a job, the client
|
|
535
|
+
falls back to paying each poll the old way (logged to stderr). Base polling is
|
|
536
|
+
unchanged.
|
|
537
|
+
|
|
493
538
|
### Batch settlement (optional metered billing)
|
|
494
539
|
|
|
495
540
|
By default, every Solana call uses x402 `exact`: one SPL transfer per call,
|
|
@@ -591,12 +636,37 @@ gateway's answer proves, in one place:
|
|
|
591
636
|
| The gateway's answer to the payment | What the SDK does |
|
|
592
637
|
| --- | --- |
|
|
593
638
|
| a 2xx | the call is served and booked (a missing or unreconciled receipt only makes the SDK re-read the channel before the next payment) |
|
|
594
|
-
| first send: a 402, or a recognised refusal (`batch_payer_not_allowed`, `batch_payer_not_admitted`, `batch_admission_paused`, `batch_server_signed_only`, `batch_unavailable`, `batch_channel_limit`, `PAYMENT_VERIFICATION_UNAVAILABLE`), with no receipt or a failed one with no transaction | nothing was charged: pays `exact`. A deposit refused with a 402 or `PAYMENT_VERIFICATION_UNAVAILABLE` may still have been broadcast, so its channel is re-read from the chain before anything pays into it again |
|
|
639
|
+
| first send: a 402, or a recognised refusal (`batch_payer_not_allowed`, `batch_payer_not_admitted`, `batch_admission_paused`, `batch_server_signed_only`, `batch_unavailable`, `batch_channel_limit`, `PAYMENT_VERIFICATION_UNAVAILABLE`), with no receipt or a failed one with no transaction | nothing was charged: pays `exact`. A deposit refused with a 402 or `PAYMENT_VERIFICATION_UNAVAILABLE` may still have been broadcast, so its channel is re-read from the chain before anything pays into it again, unless the 402's failed receipt names one of PayAI's `batch_*` admission refusals (capacity, rate, new channels disabled, account paused, fee-payer lane, deposit terms): that proves nothing was broadcast, so the deposit is forgotten, and the wallet's calls pay `exact` at once, without trying batch (`fallback` / `admission_refused`), for `Retry-After` (at most 15 min), else 60s, or 10 min when the deposit's own amount, asset or withdraw delay was refused |
|
|
595
640
|
| first send: a 429 whose failed receipt proves nothing was broadcast (`batch_account_channel_capacity_exhausted`, `batch_channel_capacity_exhausted`, `batch_deposit_rate_limited`) | nothing was charged: backs off, then pays again with a new batch payment from a fresh 402; `exact` once `rateLimit` runs out |
|
|
596
|
-
| first send: a 429 with no receipt | **in doubt**: after the backoff, replays the identical payment **once**; a 2xx with a success receipt ends it, anything else raises |
|
|
641
|
+
| first send: a 429 with no receipt | **in doubt**: after the backoff, replays the identical payment **once**; a 2xx with a success receipt ends it, anything else raises. A payment carrying a payment identifier is replayed as in the last row instead |
|
|
597
642
|
| first send of an authorization: an error status with a cancelled receipt (`success: false`, `errorReason: "batch_cancelled"`, no transaction) | nothing was charged: the error is raised with retry disposition `'unpaid'`, so `fallbackModels` / `smartChat()` move on to the next model |
|
|
598
|
-
| anything else: a timeout, an abort or a network error after sending; a 5xx or other error status without a recognised refusal; a receipt naming a transaction, saying `settlement_pending` or saying it succeeded on an error status; a 429 with any other receipt | **in doubt**: raises `BatchPaymentUnresolvedError
|
|
599
|
-
|
|
643
|
+
| anything else: a timeout, an abort or a network error after sending; a 5xx or other error status without a recognised refusal; a receipt naming a transaction, saying `settlement_pending` or saying it succeeded on an error status; a 429 with any other receipt | **in doubt**: raises `BatchPaymentUnresolvedError`, unless the payment carries a payment identifier (below) |
|
|
644
|
+
| a payment carrying a payment identifier, in doubt: no answer, a 5xx, a 429, or a 409 with `Retry-After` | **in doubt**: waits `Retry-After` (at least 1 s, at most 30 s; without one about 1s, 2s, 4s... with jitter), then replays the identical payment, and again for as long as the answer is one of these, within `inDoubt.maxWaitMs` (default 5 minutes). A 2xx with a success receipt ends it. Any other answer (a 409 without `Retry-After`, a 402, a 2xx without a receipt the SDK can verify), or running out of time, raises |
|
|
645
|
+
|
|
646
|
+
- **Payment identifier.** When the gateway's 402 declares the x402
|
|
647
|
+
`payment-identifier` extension, every batch payment carries a new id in
|
|
648
|
+
its `extensions` (`pay_` and 32 hex digits). A gateway that stores each
|
|
649
|
+
paid response under its id answers a replay of a completed request with
|
|
650
|
+
the original response and receipt, and the call is served and booked
|
|
651
|
+
once; while the original is still running, it answers 409 with a
|
|
652
|
+
`Retry-After`. Every replay repeats the payment byte for byte: the same
|
|
653
|
+
`PAYMENT-SIGNATURE` header (same id, same request id, same signed
|
|
654
|
+
authorization and deposit transaction) and the same request body, which
|
|
655
|
+
the SDK serializes once per call. This also recovers a generation that
|
|
656
|
+
outlasts the client's `timeout` (60 s by default): the request is aborted,
|
|
657
|
+
and the replays pick up its stored response once the gateway has it. The
|
|
658
|
+
replay resolves the call only on a 2xx whose receipt the SDK can verify
|
|
659
|
+
(`success: true`, and an `extra` with the operator's `voucher` for this
|
|
660
|
+
request's charge and a `commitmentId`; a `channelState`, when present,
|
|
661
|
+
must match). Every error answer, and a replay's 2xx without a success
|
|
662
|
+
receipt, is held back from the channel's bookkeeping until no replay
|
|
663
|
+
follows it; the bookkeeping then takes the last answer, unless an earlier
|
|
664
|
+
one carried a success receipt and the later ones do not, so a charge the
|
|
665
|
+
gateway reported on an error status is never rolled back by a replay's
|
|
666
|
+
refusal. These replays do not count against `rateLimit`. An `exact`
|
|
667
|
+
payment carries its own new id when the extension is declared, so a
|
|
668
|
+
gateway that declares it `required` takes the exact fallback too. Without
|
|
669
|
+
the extension nothing changes.
|
|
600
670
|
- **Trust model.** The gateway is already trusted with the escrow up to
|
|
601
671
|
`maxDeposit`, so its explicit answer to a payment's first send is taken as
|
|
602
672
|
proof that nothing was charged. Nothing else is: not an error's type or
|
|
@@ -613,8 +683,11 @@ gateway's answer proves, in one place:
|
|
|
613
683
|
`'top-up'` or `'authorization'`), `depositInDoubt`, `status` (the gateway's
|
|
614
684
|
last answer, when there was one), `cause` (that answer as an `APIError`, or
|
|
615
685
|
the transport error) and `reason`:
|
|
616
|
-
- `'replay_unresolved'`: a 429 without a receipt
|
|
617
|
-
success receipt (or `rateLimit` left no room
|
|
686
|
+
- `'replay_unresolved'`: a 429 without a receipt whose one replay got no
|
|
687
|
+
success receipt (or `rateLimit` left no room to replay it), or a payment
|
|
688
|
+
carrying a payment identifier whose replays got none: the last answer
|
|
689
|
+
was one no later replay can change, or `inDoubt.maxWaitMs` ran out (the
|
|
690
|
+
message says how many replays were sent);
|
|
618
691
|
- `'ambiguous_rate_limit'`: a 429 whose receipt does not prove nothing was
|
|
619
692
|
broadcast;
|
|
620
693
|
- `'no_response'`: sending it threw (timeout, abort, network error);
|
|
@@ -659,7 +732,7 @@ gateway's answer proves, in one place:
|
|
|
659
732
|
|
|
660
733
|
Parallel calls never queue behind the channel, except during a 429 cooldown (below).
|
|
661
734
|
- **Deposit size.** An open or top-up deposits the 402's `extra.minDeposit` when it is at least the call's ceiling (the `@x402/svm` server defaults it to 3x the ceiling), otherwise 5x the ceiling. It is never less than the call needs, and never more than the room `maxDeposit` leaves: `maxDeposit - (deposit - settled)`, which for an open is all of `maxDeposit`. A call whose shortfall does not fit pays `exact` (`deposit_over_cap`).
|
|
662
|
-
- **429 means wait, not give up.** After a 429 the call waits for `Retry-After` (seconds or an HTTP date; without one, about 1s, 2s, 4s with jitter). A `Retry-After` that is not a finite delay of at most a day is ignored, and the default backoff applies. While the wallet cools down, its other calls wait as well instead of sending new channel opens. `batch.rateLimit` bounds this per call: `maxAttempts` (default `3` batch payments sent, the first and the one replay included) and `maxWaitMs` (default `60000` ms spent waiting). A call that cannot wait out `Retry-After` within what is left of `maxWaitMs` pays `exact` at once, and the wallet's shared cooldown never lasts longer than `maxWaitMs`, so no single header keeps batch off for longer than that. A 429 that proves nothing was broadcast is paid again with a new payment, rebuilt from a fresh 402 challenge (one unpaid request) because the original's blockhash and slot hints may have gone stale. That unpaid request is the call itself: if the gateway serves it (a 2xx), that response is the call's result and nothing more is paid (a `recovered` event, `served_unpaid_on_rechallenge`); if it gets any other answer or none, the call throws that error with retry disposition `'unpaid'` (nothing was paid for it, so `fallbackModels` may take over). It is never paid `exact` against the stale challenge. A 429 without a receipt is never rebuilt: it gets its one replay, byte for byte (same request id, same signed authorization, same signed deposit transaction).
|
|
735
|
+
- **429 means wait, not give up.** After a 429 the call waits for `Retry-After` (seconds or an HTTP date; without one, about 1s, 2s, 4s with jitter). A `Retry-After` that is not a finite delay of at most a day is ignored, and the default backoff applies. While the wallet cools down, its other calls wait as well instead of sending new channel opens. `batch.rateLimit` bounds this per call: `maxAttempts` (default `3` batch payments sent, the first and the one replay included) and `maxWaitMs` (default `60000` ms spent waiting). The replays of a payment carrying a payment identifier are bound by `batch.inDoubt.maxWaitMs` instead (default `300000` ms, counted from its first answer in doubt, waits and replays included; no replay starts once its wait would end past it, and one already sent runs until its answer or the client's `timeout`; `0` never replays). A call that cannot wait out `Retry-After` within what is left of `maxWaitMs` pays `exact` at once, and the wallet's shared cooldown never lasts longer than `maxWaitMs`, so no single header keeps batch off for longer than that. A 429 that proves nothing was broadcast is paid again with a new payment, rebuilt from a fresh 402 challenge (one unpaid request) because the original's blockhash and slot hints may have gone stale. That unpaid request is the call itself: if the gateway serves it (a 2xx), that response is the call's result and nothing more is paid (a `recovered` event, `served_unpaid_on_rechallenge`); if it gets any other answer or none, the call throws that error with retry disposition `'unpaid'` (nothing was paid for it, so `fallbackModels` may take over). It is never paid `exact` against the stale challenge. A 429 without a receipt is never rebuilt: it gets its one replay, byte for byte (same request id, same signed authorization, same signed deposit transaction).
|
|
663
736
|
- **A deposit in doubt blocks new deposits.** When an open or top-up is in doubt, or a deposit-bearing call was served without a receipt the SDK could reconcile, the wallet signs no batch payment at all (so no second deposit can land next to it) and its calls pay `exact` (`channel_resync_pending`) until the chain settles the deposit. The SDK reads channels at `finalized` commitment only. A deposit has landed once the finalized channel holds it. It has provably never landed once the finalized chain is 300 blocks past a block height the SDK read itself after the payment was built (a transaction expires 150 blocks after its blockhash), and a read at least that recent still does not show it: then an open's record is dropped and a top-up's channel is rewritten with the deposit it really has. Wall-clock time, and any `lastValidBlockHeight` a 402 names, are never used. This settles only the deposit, never whether the gateway charged the payment that carried it.
|
|
664
737
|
- **No silent fallbacks.** Every fallback, every 429 backoff, every recovery, every channel re-read and every payment raised in doubt writes one line to stderr, each time it happens:
|
|
665
738
|
|
|
@@ -668,13 +741,14 @@ gateway's answer proves, in one place:
|
|
|
668
741
|
[@blockrun/llm] batch-settlement event=unresolved reason=no_response attempt=1 wallet=<address> next=raise detail="the payment got no answer (AbortError: This operation was aborted); it may have been charged, so it is not paid again"
|
|
669
742
|
```
|
|
670
743
|
|
|
671
|
-
`batch.onEvent` receives each one as a `SolanaBatchEvent`: `{ type: 'fallback' | 'backoff' | 'recovered' | 'resync' | 'unresolved', reason, status?, errorReason?, retryAfterMs?, attempt?, detail?, wallet, at }`. `reason` is a stable code: `rate_limited` or `
|
|
744
|
+
`batch.onEvent` receives each one as a `SolanaBatchEvent`: `{ type: 'fallback' | 'backoff' | 'recovered' | 'resync' | 'unresolved', reason, status?, errorReason?, retryAfterMs?, attempt?, detail?, wallet, at }`. `reason` is a stable code: `rate_limited`, `cooldown` or `in_doubt` (backoffs, and the `recovered` event after one, or `served_unpaid_on_rechallenge`); for fallbacks `not_offered`, `client_signed_not_supported`, `channel_busy`, `untrusted_operator`, `deposit_over_cap`, `channel_pending`, `closed_during_call`, `peer_dependency_missing`, `payment_creation_failed`, `wallet_config_conflict`, `channel_store_locked`, `deposit_journal_unreadable`, `deposit_journal_failed`, `channel_resync_pending`, `channel_resync_failed`, `channel_unreadable`, `payment_required`, `rate_limited` or one of the recognised refusal codes above; for `unresolved`, the `BatchPaymentUnresolvedError` reasons; for `resync`, the re-read reasons below. The callback may be async; one that throws or rejects is logged and never affects the payment. `client.getBatchStats()` returns this client's counters:
|
|
672
745
|
|
|
673
746
|
```typescript
|
|
674
747
|
const client = new SolanaLLMClient({
|
|
675
748
|
batch: {
|
|
676
749
|
operators: [BLOCKRUN_SOL_OPERATOR],
|
|
677
750
|
rateLimit: { maxAttempts: 3, maxWaitMs: 60_000 }, // the defaults
|
|
751
|
+
inDoubt: { maxWaitMs: 300_000 }, // the default
|
|
678
752
|
onEvent: (event) => metrics.increment(`batch.${event.type}`, { reason: event.reason }),
|
|
679
753
|
},
|
|
680
754
|
});
|
|
@@ -686,7 +760,7 @@ gateway's answer proves, in one place:
|
|
|
686
760
|
- **Scope.** Batch covers non-streaming chat: `chat`, `chatCompletion`, `smartChat`, and `smartChatCompletion`. `stream()` and the image and media jobs still pay with `exact`.
|
|
687
761
|
- **One channel per wallet.** Every `SolanaLLMClient` for a wallet in one process shares a single channel, including clients built from different loaded copies of this package (its CJS and ESM builds, or two installs). A second client with different `batch` options pays `exact`. On disk, one live process owns a wallet's channel file through a lock file holding its pid, in the same format earlier releases write and read, so a process still on an older version that shares the channel store (say, during a rolling upgrade) sees the lock as live too; a random ownership token sits beside it in `<lock>.owner`. Other processes pay `exact` until that process exits. A lock that names the current process but was not taken by this SDK version is never treated as stale; if no process uses it (say, a crashed process whose pid was reused), delete the `.lock` and `.lock.owner` files.
|
|
688
762
|
- **State and crash safety.** The open channel is saved to `~/.blockrun/solana-batch/<wallet>.json` (mode `0600`), so a restart reuses it; a custom `channelStore` path must also be per wallet. Before an open or top-up is sent, its intent is written to `<wallet>.json.deposit-intents` beside it (the store's full path plus `.deposit-intents`, so every store has its own; mode `0600`; the file and its directory are fsynced where the platform allows), and removed once the deposit is reconciled. After a crash, every intent left there makes the new process treat that deposit as in doubt (above): it signs no deposit until the chain settles it, and it never re-sends the old payment or pays for its request again. A deposit that cannot be journaled is not sent (`deposit_journal_failed`, paid `exact`). Intents an earlier version journaled under its old name (`<wallet>.deposit-intents.json`, which a store named `<wallet>` without `.json` shared) are moved over on start, only the ones for this wallet; an old journal that cannot be read keeps batch off as below. Only a missing journal counts as empty: one that cannot be read, or is not a well-formed journal this version knows (a missing `intents`, an unknown `version`, a malformed entry), keeps batch off for the wallet and its calls pay `exact` (`deposit_journal_unreadable`); the file is left as it is. **With `channelStore: false` this crash recovery is not available:** the channel is kept in memory and found again on-chain, and a deposit that was in flight when the process died is not known to be in doubt. Nothing coordinates processes in that mode either, so use it for a single process per wallet.
|
|
689
|
-
- **Closing.** `closeBatchChannel()` closes the channel the gateway's current 402 names: the one for its trusted operator, receiver authorizer, fee payer, asset and receiver. It first re-reads that record if it is in doubt (for example after a process died mid-deposit) and loads that channel alone, so the escrow can be recovered without another paid call; after the close the SDK forgets that channel only. During a key rotation, a channel opened with the other operator key keeps its record and can be closed once the gateway names that operator. A client-signed channel left by an earlier SDK version is closed the same way once no trusted server-signed record is left for that challenge, through the challenge's client-signed accept. It closes nothing and throws `BatchCloseDeferredError` while a chat call for the wallet is in flight (`reason: 'call_in_flight'`, including one waiting out a 429, so that retry can never open a channel you believe is closed) or while any of its deposits is in doubt (`reason: 'deposit_in_doubt'`, with `channelIds`); call it again once the call returns or the deposit is settled (usually a few minutes: it waits for finality). A chat that was still waiting for its first answer when a close completed pays `exact` instead of opening a new channel (`closed_during_call`).
|
|
763
|
+
- **Closing.** `closeBatchChannel()` closes the channel the gateway's current 402 names: the one for its trusted operator, receiver authorizer, fee payer, asset and receiver. It first re-reads that record if it is in doubt (for example after a process died mid-deposit) and loads that channel alone, so the escrow can be recovered without another paid call; after the close the SDK forgets that channel only. During a key rotation, a channel opened with the other operator key keeps its record and can be closed once the gateway names that operator. A client-signed channel left by an earlier SDK version is closed the same way once no trusted server-signed record is left for that challenge, through the challenge's client-signed accept. It closes nothing and throws `BatchCloseDeferredError` while a chat call for the wallet is in flight (`reason: 'call_in_flight'`, including one waiting out a 429 or between the replays of a payment in doubt, so that retry can never open a channel you believe is closed) or while any of its deposits is in doubt (`reason: 'deposit_in_doubt'`, with `channelIds`); call it again once the call returns or the deposit is settled (usually a few minutes: it waits for finality). A chat that was still waiting for its first answer when a close completed pays `exact` instead of opening a new channel (`closed_during_call`).
|
|
690
764
|
- **RPC.** Batch uses the same Solana RPC as `exact`: `rpcUrl` / `SOLANA_RPC_URL`, with `rpcHeaders` / `SOLANA_RPC_HEADERS` / `SOLANA_RPC_API_KEY` for header-auth endpoints. `@x402/svm` builds its RPC client from a URL alone, so when headers are configured the SDK adds them through a thin wrapper around the global `fetch` that only touches requests to exactly that RPC URL made during a batch payment. The RPC must support `finalized` commitment and `minContextSlot` (standard on current Solana RPCs); one that does not keeps calls on `exact`.
|
|
691
765
|
- **A funded channel is never forgotten.** A refused or rate-limited open or top-up that charged nothing leaves the saved channel exactly as it was confirmed. When the SDK cannot trust its record — a deposit in doubt; a response whose receipt is missing or was rebuilt without a voucher; a pending deposit or journaled intent left by a process that died; a chain that shows more deposit than the record before a top-up — it re-reads that one channel by its address (`getAccountInfo` at `finalized`, no program scan) before the next payment, and rewrites the record with the on-chain deposit and the highest of its confirmed cumulative charge, the one the saved record already holds, and the on-chain settled amount. The saved cumulative never moves backwards, even when a journaled intent outlived the receipt that reconciled its deposit (vouchers are redeemed on-chain later, so the settled amount can lag). Each re-read emits a `resync` event (`reason`: `deposit_unanswered`, `deposit_failed`, `deposit_rate_limited`, `receipt_missing`, `receipt_unreconciled`, `orphaned_deposit`, `deposit_unrecorded`, `channel_unusable`; `detail`: what the chain showed). A top-up never goes into a channel the chain shows closing or closed, or that is not your wallet's for its operator and mint: that record is re-read at once (`channel_unusable`), a closed channel's record is dropped and the call carries on as with no channel (a fresh open when it fits in `maxDeposit`), and anything else keeps the record and pays `exact` (`channel_unreadable`). Until it succeeds, calls pay `exact` (`channel_resync_pending`, or `channel_resync_failed` when the chain cannot be read). Only an absent account counts as "no channel": an account at that address that cannot be read as your channel (another owner, layout, encoding, payer, operator or mint) keeps the record and pays `exact` (`channel_unreadable`), and is never a reason to sign a new deposit. Every channel read (these re-reads and the settled read before a top-up) first checks that the installed `@x402/svm` still uses the channel layout the SDK decodes; when it does not, nothing is decoded and those calls pay `exact` (`channel_unreadable`) with the record kept.
|
|
692
766
|
- **Rollout.** `sol.blockrun.ai` lists `exact` first and advertises
|
|
@@ -1488,7 +1562,7 @@ const free = await client.chat('nvidia/nemotron-3.5-lightning', 'Hello!');
|
|
|
1488
1562
|
|
|
1489
1563
|
| Profile | Description | Best For |
|
|
1490
1564
|
|---------|-------------|----------|
|
|
1491
|
-
| `eco` | Budget-optimized — ranks the <!-- br:models.free -->
|
|
1565
|
+
| `eco` | Budget-optimized — ranks the <!-- br:models.free -->7<!-- /br:models.free -->-model free tier first | Cost-sensitive workloads, zero-cost testing |
|
|
1492
1566
|
| `auto` | Intelligent routing (default) | General use |
|
|
1493
1567
|
| `premium` | Best quality models | Critical tasks |
|
|
1494
1568
|
|
|
@@ -2087,7 +2161,7 @@ The `AnthropicClient` wraps the official `@anthropic-ai/sdk` with a custom fetch
|
|
|
2087
2161
|
## Frequently Asked Questions
|
|
2088
2162
|
|
|
2089
2163
|
### What is @blockrun/llm?
|
|
2090
|
-
@blockrun/llm is a TypeScript SDK that cuts LLM costs by up to <!-- br:savings.autoVsBaselinePct -->84<!-- /br:savings.autoVsBaselinePct -->% with built-in smart routing: every request is routed to the cheapest of <!-- br:models.chatVisible -->
|
|
2164
|
+
@blockrun/llm is a TypeScript SDK that cuts LLM costs by up to <!-- br:savings.autoVsBaselinePct -->84<!-- /br:savings.autoVsBaselinePct -->% with built-in smart routing: every request is routed to the cheapest of <!-- br:models.chatVisible -->88<!-- /br:models.chatVisible --> models (OpenAI, Anthropic, Google, xAI, DeepSeek, Moonshot, and more) that can handle it, then paid per-request in USDC via the x402 protocol — with API key account billing or x402 wallet payments on Solana or Base.
|
|
2091
2165
|
|
|
2092
2166
|
### How does payment work?
|
|
2093
2167
|
When you make an API call, the SDK automatically handles x402 payment. It signs a USDC transaction locally using your wallet private key (which never leaves your machine), and includes the payment proof in the request header. Settlement is non-custodial and instant on Base or Solana.
|