bare-agent 0.33.1 → 0.35.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bareagent.context.md +5 -1
- package/package.json +1 -1
- package/src/provider-anthropic.d.ts +16 -2
- package/src/provider-anthropic.js +15 -3
- package/src/provider-gemini.d.ts +16 -2
- package/src/provider-gemini.js +15 -3
- package/src/provider-http.d.ts +80 -0
- package/src/provider-http.js +126 -0
- package/src/provider-ollama.d.ts +16 -2
- package/src/provider-ollama.js +15 -3
- package/src/provider-openai.d.ts +32 -2
- package/src/provider-openai.js +23 -3
package/bareagent.context.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# bareagent — Integration Guide
|
|
2
2
|
|
|
3
3
|
> For AI assistants and developers wiring bareagent into a project.
|
|
4
|
-
> v0.
|
|
4
|
+
> v0.35.0 | Node.js >= 18 | zero required deps (`bareguard >=0.9.0 <0.13.0` optional peer for governance) | Apache 2.0
|
|
5
5
|
>
|
|
6
6
|
> Full human guide with composition examples, design philosophy, and recipes: [Usage Guide](docs/02-features/usage-guide.md)
|
|
7
7
|
|
|
@@ -829,6 +829,10 @@ All return `{ text, toolCalls, usage: { inputTokens, outputTokens }, model?, cos
|
|
|
829
829
|
|
|
830
830
|
**Error body (v0.11.0):** on an HTTP error the OpenAI/Anthropic/Ollama providers throw a `ProviderError` whose `message` carries the upstream error string. The full parsed response is **not** attached to `err.body` by default (so an unexpected field can't leak through logs that dump the error object). Pass `{ exposeErrorBody: true }` to attach it for debugging.
|
|
831
831
|
|
|
832
|
+
**Request/idle timeout (BA-18, v0.34.0):** the four http(s) providers (Anthropic, OpenAI, Gemini, Ollama) accept a `timeoutMs` option — constructor default **600000 (10 min)**, overridable per call via `generate(..., { timeoutMs })`, and `0`/`Infinity` disables it. Before this they wired only `req.on('error')`, so a socket the server silently dropped — or a response that never starts — hung `generate()` until the OS TCP timeout (~2h): a hang, not an error, so retry/casualty policy above it never fired. `timeoutMs` bounds on socket **inactivity** (`req.setTimeout`), so a slow-but-streaming response is not killed — only a silent/never-answering socket trips it; the 10-min default clears any single non-streaming completion (TTFB ≈ generation time). On trip, `generate()` rejects with a retryable `TimeoutError` (`code: 'ETIMEDOUT'`, `context.bound: 'idle'`, `retryable: true`). **Retry is caller-side and already wired:** `new Loop({ provider, retry: new Retry() })` wraps `provider.generate`, and `DEFAULT_RETRY_ON` classifies `ETIMEDOUT` (and `ECONNRESET`/`ENOTFOUND`/429/5xx) as transient — so a wired `Retry` retries a timed-out request and rethrows under `retryOn: () => false`, with no extra wiring (`run-plan`'s `stepRetry` is a second consumer of the same seam). CLIPipe already bounded its child process (`timeout`, default 30000 for one-shot) and is unchanged.
|
|
833
|
+
|
|
834
|
+
**Total call-duration deadline (BA-19, v0.35.0):** `timeoutMs` bounds socket *inactivity*, and `req.setTimeout` resets on any activity by design — so a "zombie stream" that trickles a byte forever (bytes arriving, the response never completing) never trips it and hangs the caller for hours (an adopter saw one `generate()` run **274 min** and end in `ECONNRESET`, not a `TimeoutError`: the reset proves bytes *were* flowing, so the idle timer never fired). The four http(s) providers now also accept a `deadlineMs` option — an absolute, **non-resetting** wall-clock ceiling on the whole request. **Disabled by default** (a deliberately long single call — large `maxTokens`, slow model — is legitimate; a default here would kill it), overridable per call via `generate(..., { deadlineMs })`, `0`/`Infinity` disable. An *unset* deadline resolves to disabled, but an *explicitly-set* garbage value (`NaN`, a non-numeric string) throws a `ValidationError` at resolve time rather than silently disabling the bound and running unbounded — unlike `timeoutMs`, the deadline has no safe default to fall back to, so a config mistake must surface loudly. On trip, `generate()` rejects with a **terminal** `TimeoutError` distinguishable from the idle trip: `code: 'EDEADLINE'`, `context.bound: 'deadline'`, `retryable: false` — a deadline is a hard ceiling meant to STOP, so it is *not* auto-retried (retrying would re-spend up to another full `deadlineMs`); a consumer that wants retry opts in via `retryOn`. When both are armed and `timeoutMs < deadlineMs`, a silent socket trips the idle bound first; only a still-active-but-never-completing stream reaches the deadline. The idle bound (BA-18) and the deadline (BA-19) are two independent failure modes — a silent socket vs a zombie stream.
|
|
835
|
+
|
|
832
836
|
**Plaintext-key warning (Unreleased):** the OpenAI provider's `baseUrl` accepts `http://` (for local/OpenAI-compatible endpoints), but a `Bearer` key sent over plaintext http to a **non-loopback** host is exposed on the wire. The provider now warns once when that happens. Loopback hosts (`localhost`/`127.0.0.0/8`/`::1` — local proxies, Ollama-style endpoints) stay silent, since that's the legitimate keyless-local case. The header is **not** stripped (some local proxies want a key), so use `https` for any remote endpoint, or drop `apiKey` when the local endpoint needs none.
|
|
833
837
|
|
|
834
838
|
**Cost estimation:** Loop automatically estimates USD cost per run based on model and token usage. The `cost` field appears in every `loop.run()` result and in `loop:done` stream events. Pricing covers OpenAI and Anthropic models; unknown models use a default average. To adjust rates, edit `COST_PER_1K` at the top of `src/loop.js`. The model is resolved as `result.model || provider.model` (v0.16.1+) — providers now echo the model in their `generate()` result, so cost accounting holds even when `provider.model` is absent or varies per response, e.g. behind `FallbackProvider` or `CircuitBreaker.wrapProvider` (the wrapper also preserves `model`/`name` passthrough props). Wire `onLlmResult` (via `wireGate`) and a `budget.maxCostUsd` cap then halts on token-heavy workloads too.
|
package/package.json
CHANGED
|
@@ -32,6 +32,14 @@ export type AnthropicOptions = {
|
|
|
32
32
|
* - Attach the full upstream response to `err.body` on HTTP errors (off by default to avoid leaking unexpected fields through error logs; `err.message` still carries the API error).
|
|
33
33
|
*/
|
|
34
34
|
exposeErrorBody?: boolean | undefined;
|
|
35
|
+
/**
|
|
36
|
+
* - BA-18: request/idle timeout in ms. A silent or never-answering socket (dropped by the server, or a response that never starts) was otherwise bounded only by the OS TCP timeout (~2h) — a hang, not an error, so every retry policy above it was inert. Bounds on socket INACTIVITY (the timer resets on activity, so a slow-but-streaming response is not killed); on trip, `generate()` rejects with a retryable `TimeoutError` (`code: 'ETIMEDOUT'`, `context.bound: 'idle'`) that a wired `Retry` (`Loop({ retry })`) retries. Default 10 min sits above any single non-streaming completion; `0` or `Infinity` disables it (pre-BA-18 behaviour). Overridable per call via `generate(..., { timeoutMs })`.
|
|
37
|
+
*/
|
|
38
|
+
timeoutMs?: number | undefined;
|
|
39
|
+
/**
|
|
40
|
+
* - BA-19: TOTAL call-duration deadline in ms, beside `timeoutMs`. The idle bound resets on any socket activity, so a response that trickles a byte forever (a "zombie stream") never trips it and hangs the caller for hours. This is an absolute, non-resetting wall-clock ceiling; on trip, `generate()` rejects with a TERMINAL `TimeoutError` (`code: 'EDEADLINE'`, `context.bound: 'deadline'`, `retryable: false` — a hard ceiling meant to STOP, not re-spend). DISABLED by default (a deliberately long single call is legitimate); `0`/`Infinity` disable. When both are set with `timeoutMs < deadlineMs`, a silent socket trips the idle bound first. Overridable per call via `generate(..., { deadlineMs })`.
|
|
41
|
+
*/
|
|
42
|
+
deadlineMs?: number | undefined;
|
|
35
43
|
};
|
|
36
44
|
/** @typedef {import('../types').Message} Message */
|
|
37
45
|
/** @typedef {import('../types').ToolDef} ToolDef */
|
|
@@ -47,6 +55,8 @@ export type AnthropicOptions = {
|
|
|
47
55
|
*
|
|
48
56
|
* **MEASURED CAVEAT — this option does not "turn thinking on".** On `claude-sonnet-5` adaptive thinking is ALREADY the default: sending this changed the observed thinking rate not at all (2/10 rounds with it vs 3/10 without — `poc/ba7-adaptive-default.mjs`). Its real use is pinning the mode and reaching `display`/`effort`. The change that mattered is that thinking blocks are now PRESERVED and replayed (see `Message.providerBlocks`), which happens whether or not you ever set this.
|
|
49
57
|
* @property {boolean} [exposeErrorBody=false] - Attach the full upstream response to `err.body` on HTTP errors (off by default to avoid leaking unexpected fields through error logs; `err.message` still carries the API error).
|
|
58
|
+
* @property {number} [timeoutMs=600000] - BA-18: request/idle timeout in ms. A silent or never-answering socket (dropped by the server, or a response that never starts) was otherwise bounded only by the OS TCP timeout (~2h) — a hang, not an error, so every retry policy above it was inert. Bounds on socket INACTIVITY (the timer resets on activity, so a slow-but-streaming response is not killed); on trip, `generate()` rejects with a retryable `TimeoutError` (`code: 'ETIMEDOUT'`, `context.bound: 'idle'`) that a wired `Retry` (`Loop({ retry })`) retries. Default 10 min sits above any single non-streaming completion; `0` or `Infinity` disables it (pre-BA-18 behaviour). Overridable per call via `generate(..., { timeoutMs })`.
|
|
59
|
+
* @property {number} [deadlineMs=0] - BA-19: TOTAL call-duration deadline in ms, beside `timeoutMs`. The idle bound resets on any socket activity, so a response that trickles a byte forever (a "zombie stream") never trips it and hangs the caller for hours. This is an absolute, non-resetting wall-clock ceiling; on trip, `generate()` rejects with a TERMINAL `TimeoutError` (`code: 'EDEADLINE'`, `context.bound: 'deadline'`, `retryable: false` — a hard ceiling meant to STOP, not re-spend). DISABLED by default (a deliberately long single call is legitimate); `0`/`Infinity` disable. When both are set with `timeoutMs < deadlineMs`, a silent socket trips the idle bound first. Overridable per call via `generate(..., { deadlineMs })`.
|
|
50
60
|
*/
|
|
51
61
|
export class AnthropicProvider {
|
|
52
62
|
/**
|
|
@@ -61,11 +71,13 @@ export class AnthropicProvider {
|
|
|
61
71
|
cacheMessages: boolean;
|
|
62
72
|
thinking: any;
|
|
63
73
|
exposeErrorBody: boolean;
|
|
74
|
+
timeoutMs: number | undefined;
|
|
75
|
+
deadlineMs: number | undefined;
|
|
64
76
|
/**
|
|
65
77
|
* Generate a response from the Anthropic API.
|
|
66
78
|
* @param {Message[]} messages - Conversation messages (OpenAI format, auto-converted).
|
|
67
79
|
* @param {ToolDef[]} [tools=[]] - Tool definitions.
|
|
68
|
-
* @param {Record<string, any>} [options={}] - Options (temperature, maxTokens, system).
|
|
80
|
+
* @param {Record<string, any>} [options={}] - Options (temperature, maxTokens, system, timeoutMs — a per-call override of the constructor's `timeoutMs`, see BA-18; deadlineMs — a per-call override of the constructor's `deadlineMs`, see BA-19).
|
|
69
81
|
* @returns {Promise<GenerateResult>}
|
|
70
82
|
* @throws {Error} `[AnthropicProvider] ...` — on HTTP errors (4xx/5xx) or invalid JSON response.
|
|
71
83
|
*/
|
|
@@ -94,8 +106,10 @@ export class AnthropicProvider {
|
|
|
94
106
|
_toAnthropicMessage(msg: Message): any;
|
|
95
107
|
/**
|
|
96
108
|
* @param {Record<string, any>} body
|
|
109
|
+
* @param {number} [timeoutMs=0] - Idle-socket timeout (ms); 0 disables. See BA-18 / provider-http.
|
|
110
|
+
* @param {number} [deadlineMs=0] - Total call-duration deadline (ms); 0 disables. See BA-19 / provider-http.
|
|
97
111
|
* @returns {Promise<any>}
|
|
98
112
|
*/
|
|
99
|
-
_request(body: Record<string, any
|
|
113
|
+
_request(body: Record<string, any>, timeoutMs?: number, deadlineMs?: number): Promise<any>;
|
|
100
114
|
_warnedInsecure: boolean | undefined;
|
|
101
115
|
}
|
|
@@ -5,6 +5,7 @@ const http = require('http');
|
|
|
5
5
|
const { ProviderError } = require('./errors');
|
|
6
6
|
const { requestWithTemperatureFallback } = require('./provider-temperature');
|
|
7
7
|
const { normalizeStopReason } = require('./provider-stop-reason');
|
|
8
|
+
const { resolveTimeoutMs, applyRequestBounds } = require('./provider-http');
|
|
8
9
|
|
|
9
10
|
/** @param {string} hostname @returns {boolean} */
|
|
10
11
|
function isLoopbackHost(hostname) {
|
|
@@ -27,6 +28,8 @@ function isLoopbackHost(hostname) {
|
|
|
27
28
|
*
|
|
28
29
|
* **MEASURED CAVEAT — this option does not "turn thinking on".** On `claude-sonnet-5` adaptive thinking is ALREADY the default: sending this changed the observed thinking rate not at all (2/10 rounds with it vs 3/10 without — `poc/ba7-adaptive-default.mjs`). Its real use is pinning the mode and reaching `display`/`effort`. The change that mattered is that thinking blocks are now PRESERVED and replayed (see `Message.providerBlocks`), which happens whether or not you ever set this.
|
|
29
30
|
* @property {boolean} [exposeErrorBody=false] - Attach the full upstream response to `err.body` on HTTP errors (off by default to avoid leaking unexpected fields through error logs; `err.message` still carries the API error).
|
|
31
|
+
* @property {number} [timeoutMs=600000] - BA-18: request/idle timeout in ms. A silent or never-answering socket (dropped by the server, or a response that never starts) was otherwise bounded only by the OS TCP timeout (~2h) — a hang, not an error, so every retry policy above it was inert. Bounds on socket INACTIVITY (the timer resets on activity, so a slow-but-streaming response is not killed); on trip, `generate()` rejects with a retryable `TimeoutError` (`code: 'ETIMEDOUT'`, `context.bound: 'idle'`) that a wired `Retry` (`Loop({ retry })`) retries. Default 10 min sits above any single non-streaming completion; `0` or `Infinity` disables it (pre-BA-18 behaviour). Overridable per call via `generate(..., { timeoutMs })`.
|
|
32
|
+
* @property {number} [deadlineMs=0] - BA-19: TOTAL call-duration deadline in ms, beside `timeoutMs`. The idle bound resets on any socket activity, so a response that trickles a byte forever (a "zombie stream") never trips it and hangs the caller for hours. This is an absolute, non-resetting wall-clock ceiling; on trip, `generate()` rejects with a TERMINAL `TimeoutError` (`code: 'EDEADLINE'`, `context.bound: 'deadline'`, `retryable: false` — a hard ceiling meant to STOP, not re-spend). DISABLED by default (a deliberately long single call is legitimate); `0`/`Infinity` disable. When both are set with `timeoutMs < deadlineMs`, a silent socket trips the idle bound first. Overridable per call via `generate(..., { deadlineMs })`.
|
|
30
33
|
*/
|
|
31
34
|
|
|
32
35
|
class AnthropicProvider {
|
|
@@ -52,13 +55,17 @@ class AnthropicProvider {
|
|
|
52
55
|
this.thinking = options.thinking != null ? options.thinking : null;
|
|
53
56
|
// See OpenAIProvider: attach full upstream body to err.body only on opt-in.
|
|
54
57
|
this.exposeErrorBody = options.exposeErrorBody === true;
|
|
58
|
+
// BA-18: request/idle timeout (ms). Resolved at call time (default 600000; 0/Infinity disable).
|
|
59
|
+
this.timeoutMs = options.timeoutMs;
|
|
60
|
+
// BA-19: total call-duration deadline (ms). Resolved at call time (default 0 = disabled).
|
|
61
|
+
this.deadlineMs = options.deadlineMs;
|
|
55
62
|
}
|
|
56
63
|
|
|
57
64
|
/**
|
|
58
65
|
* Generate a response from the Anthropic API.
|
|
59
66
|
* @param {Message[]} messages - Conversation messages (OpenAI format, auto-converted).
|
|
60
67
|
* @param {ToolDef[]} [tools=[]] - Tool definitions.
|
|
61
|
-
* @param {Record<string, any>} [options={}] - Options (temperature, maxTokens, system).
|
|
68
|
+
* @param {Record<string, any>} [options={}] - Options (temperature, maxTokens, system, timeoutMs — a per-call override of the constructor's `timeoutMs`, see BA-18; deadlineMs — a per-call override of the constructor's `deadlineMs`, see BA-19).
|
|
62
69
|
* @returns {Promise<GenerateResult>}
|
|
63
70
|
* @throws {Error} `[AnthropicProvider] ...` — on HTTP errors (4xx/5xx) or invalid JSON response.
|
|
64
71
|
*/
|
|
@@ -142,8 +149,10 @@ class AnthropicProvider {
|
|
|
142
149
|
// BA-10: some models (e.g. claude-sonnet-5) reject a non-default `temperature` with a 400 — drop it
|
|
143
150
|
// and retry once rather than let the whole call fail. `temperatureDropped` flows back so an upstream
|
|
144
151
|
// receipt (recurse's refineLeaf) can report the effective temperature, not the one the model ignored.
|
|
152
|
+
const timeoutMs = resolveTimeoutMs(this.timeoutMs, options.timeoutMs);
|
|
153
|
+
const deadlineMs = resolveTimeoutMs(this.deadlineMs, options.deadlineMs, 0, 'deadlineMs');
|
|
145
154
|
const { data, temperatureDropped } = await requestWithTemperatureFallback({
|
|
146
|
-
request: () => this._request(body),
|
|
155
|
+
request: () => this._request(body, timeoutMs, deadlineMs),
|
|
147
156
|
hadTemperature: () => body.temperature != null,
|
|
148
157
|
stripTemperature: () => { delete body.temperature; },
|
|
149
158
|
warnOnce: () => this._warnTemperatureDropped(),
|
|
@@ -275,9 +284,11 @@ class AnthropicProvider {
|
|
|
275
284
|
|
|
276
285
|
/**
|
|
277
286
|
* @param {Record<string, any>} body
|
|
287
|
+
* @param {number} [timeoutMs=0] - Idle-socket timeout (ms); 0 disables. See BA-18 / provider-http.
|
|
288
|
+
* @param {number} [deadlineMs=0] - Total call-duration deadline (ms); 0 disables. See BA-19 / provider-http.
|
|
278
289
|
* @returns {Promise<any>}
|
|
279
290
|
*/
|
|
280
|
-
_request(body) {
|
|
291
|
+
_request(body, timeoutMs = 0, deadlineMs = 0) {
|
|
281
292
|
return new Promise((resolve, reject) => {
|
|
282
293
|
const payload = JSON.stringify(body);
|
|
283
294
|
const url = new URL(this.baseUrl + '/messages');
|
|
@@ -314,6 +325,7 @@ class AnthropicProvider {
|
|
|
314
325
|
}
|
|
315
326
|
});
|
|
316
327
|
});
|
|
328
|
+
applyRequestBounds(req, { timeoutMs, deadlineMs }, 'AnthropicProvider');
|
|
317
329
|
req.on('error', reject);
|
|
318
330
|
req.write(payload);
|
|
319
331
|
req.end();
|
package/src/provider-gemini.d.ts
CHANGED
|
@@ -19,6 +19,14 @@ export type GeminiOptions = {
|
|
|
19
19
|
* - Attach the full upstream response to `err.body` on HTTP errors (off by default; `err.message` still carries the API error).
|
|
20
20
|
*/
|
|
21
21
|
exposeErrorBody?: boolean | undefined;
|
|
22
|
+
/**
|
|
23
|
+
* - BA-18: request/idle timeout in ms. Bounds a silent or never-answering socket on inactivity so `generate()` rejects with a retryable `TimeoutError` (`code: 'ETIMEDOUT'`, `context.bound: 'idle'`) instead of hanging until the OS TCP timeout (~2h). `0`/`Infinity` disables it (pre-BA-18 behaviour). Overridable per call via `generate(..., { timeoutMs })`.
|
|
24
|
+
*/
|
|
25
|
+
timeoutMs?: number | undefined;
|
|
26
|
+
/**
|
|
27
|
+
* - BA-19: TOTAL call-duration deadline in ms, beside `timeoutMs`. The idle bound resets on any socket activity, so a response that trickles a byte forever never trips it and hangs for hours. This is an absolute, non-resetting ceiling; on trip, `generate()` rejects with a TERMINAL `TimeoutError` (`code: 'EDEADLINE'`, `context.bound: 'deadline'`, `retryable: false`). DISABLED by default; `0`/`Infinity` disable. Overridable per call via `generate(..., { deadlineMs })`.
|
|
28
|
+
*/
|
|
29
|
+
deadlineMs?: number | undefined;
|
|
22
30
|
};
|
|
23
31
|
/**
|
|
24
32
|
* @typedef {object} GeminiOptions
|
|
@@ -26,6 +34,8 @@ export type GeminiOptions = {
|
|
|
26
34
|
* @property {string} [model='gemini-2.5-flash'] - Model ID.
|
|
27
35
|
* @property {string} [baseUrl='https://generativelanguage.googleapis.com/v1beta'] - API base (override for proxies; posts to `${baseUrl}/models/${model}:generateContent`).
|
|
28
36
|
* @property {boolean} [exposeErrorBody=false] - Attach the full upstream response to `err.body` on HTTP errors (off by default; `err.message` still carries the API error).
|
|
37
|
+
* @property {number} [timeoutMs=600000] - BA-18: request/idle timeout in ms. Bounds a silent or never-answering socket on inactivity so `generate()` rejects with a retryable `TimeoutError` (`code: 'ETIMEDOUT'`, `context.bound: 'idle'`) instead of hanging until the OS TCP timeout (~2h). `0`/`Infinity` disables it (pre-BA-18 behaviour). Overridable per call via `generate(..., { timeoutMs })`.
|
|
38
|
+
* @property {number} [deadlineMs=0] - BA-19: TOTAL call-duration deadline in ms, beside `timeoutMs`. The idle bound resets on any socket activity, so a response that trickles a byte forever never trips it and hangs for hours. This is an absolute, non-resetting ceiling; on trip, `generate()` rejects with a TERMINAL `TimeoutError` (`code: 'EDEADLINE'`, `context.bound: 'deadline'`, `retryable: false`). DISABLED by default; `0`/`Infinity` disable. Overridable per call via `generate(..., { deadlineMs })`.
|
|
29
39
|
*/
|
|
30
40
|
/**
|
|
31
41
|
* Google Gemini provider (native `generateContent` API, NOT the OpenAI-compat endpoint — that endpoint
|
|
@@ -41,11 +51,13 @@ export class GeminiProvider {
|
|
|
41
51
|
model: string;
|
|
42
52
|
baseUrl: string;
|
|
43
53
|
exposeErrorBody: boolean;
|
|
54
|
+
timeoutMs: number | undefined;
|
|
55
|
+
deadlineMs: number | undefined;
|
|
44
56
|
/**
|
|
45
57
|
* Generate a response from the Gemini API.
|
|
46
58
|
* @param {Message[]} messages - Conversation messages (OpenAI format, auto-converted).
|
|
47
59
|
* @param {ToolDef[]} [tools=[]] - Tool definitions.
|
|
48
|
-
* @param {Record<string, any>} [options={}] - Options (temperature, maxTokens).
|
|
60
|
+
* @param {Record<string, any>} [options={}] - Options (temperature, maxTokens, timeoutMs — a per-call override of the constructor's `timeoutMs`, see BA-18; deadlineMs — a per-call override of the constructor's `deadlineMs`, see BA-19).
|
|
49
61
|
* @returns {Promise<GenerateResult>}
|
|
50
62
|
* @throws {Error} `[GeminiProvider] ...` — on HTTP errors (4xx/5xx) or invalid JSON response.
|
|
51
63
|
*/
|
|
@@ -66,8 +78,10 @@ export class GeminiProvider {
|
|
|
66
78
|
/**
|
|
67
79
|
* @param {string} path
|
|
68
80
|
* @param {Record<string, any>} body
|
|
81
|
+
* @param {number} [timeoutMs=0] - Idle-socket timeout (ms); 0 disables. See BA-18 / provider-http.
|
|
82
|
+
* @param {number} [deadlineMs=0] - Total call-duration deadline (ms); 0 disables. See BA-19 / provider-http.
|
|
69
83
|
* @returns {Promise<any>}
|
|
70
84
|
*/
|
|
71
|
-
_request(path: string, body: Record<string, any
|
|
85
|
+
_request(path: string, body: Record<string, any>, timeoutMs?: number, deadlineMs?: number): Promise<any>;
|
|
72
86
|
_warnedInsecure: boolean | undefined;
|
|
73
87
|
}
|
package/src/provider-gemini.js
CHANGED
|
@@ -5,6 +5,7 @@ const http = require('http');
|
|
|
5
5
|
const { ProviderError } = require('./errors');
|
|
6
6
|
const { requestWithTemperatureFallback } = require('./provider-temperature');
|
|
7
7
|
const { normalizeStopReason } = require('./provider-stop-reason');
|
|
8
|
+
const { resolveTimeoutMs, applyRequestBounds } = require('./provider-http');
|
|
8
9
|
|
|
9
10
|
/** @typedef {import('../types').Message} Message */
|
|
10
11
|
/** @typedef {import('../types').ToolDef} ToolDef */
|
|
@@ -23,6 +24,8 @@ function isLoopbackHost(hostname) {
|
|
|
23
24
|
* @property {string} [model='gemini-2.5-flash'] - Model ID.
|
|
24
25
|
* @property {string} [baseUrl='https://generativelanguage.googleapis.com/v1beta'] - API base (override for proxies; posts to `${baseUrl}/models/${model}:generateContent`).
|
|
25
26
|
* @property {boolean} [exposeErrorBody=false] - Attach the full upstream response to `err.body` on HTTP errors (off by default; `err.message` still carries the API error).
|
|
27
|
+
* @property {number} [timeoutMs=600000] - BA-18: request/idle timeout in ms. Bounds a silent or never-answering socket on inactivity so `generate()` rejects with a retryable `TimeoutError` (`code: 'ETIMEDOUT'`, `context.bound: 'idle'`) instead of hanging until the OS TCP timeout (~2h). `0`/`Infinity` disables it (pre-BA-18 behaviour). Overridable per call via `generate(..., { timeoutMs })`.
|
|
28
|
+
* @property {number} [deadlineMs=0] - BA-19: TOTAL call-duration deadline in ms, beside `timeoutMs`. The idle bound resets on any socket activity, so a response that trickles a byte forever never trips it and hangs for hours. This is an absolute, non-resetting ceiling; on trip, `generate()` rejects with a TERMINAL `TimeoutError` (`code: 'EDEADLINE'`, `context.bound: 'deadline'`, `retryable: false`). DISABLED by default; `0`/`Infinity` disable. Overridable per call via `generate(..., { deadlineMs })`.
|
|
26
29
|
*/
|
|
27
30
|
|
|
28
31
|
/**
|
|
@@ -39,13 +42,17 @@ class GeminiProvider {
|
|
|
39
42
|
this.model = options.model || 'gemini-2.5-flash';
|
|
40
43
|
this.baseUrl = options.baseUrl || 'https://generativelanguage.googleapis.com/v1beta';
|
|
41
44
|
this.exposeErrorBody = options.exposeErrorBody === true;
|
|
45
|
+
// BA-18: request/idle timeout (ms). Resolved at call time (default 600000; 0/Infinity disable).
|
|
46
|
+
this.timeoutMs = options.timeoutMs;
|
|
47
|
+
// BA-19: total call-duration deadline (ms). Resolved at call time (default 0 = disabled).
|
|
48
|
+
this.deadlineMs = options.deadlineMs;
|
|
42
49
|
}
|
|
43
50
|
|
|
44
51
|
/**
|
|
45
52
|
* Generate a response from the Gemini API.
|
|
46
53
|
* @param {Message[]} messages - Conversation messages (OpenAI format, auto-converted).
|
|
47
54
|
* @param {ToolDef[]} [tools=[]] - Tool definitions.
|
|
48
|
-
* @param {Record<string, any>} [options={}] - Options (temperature, maxTokens).
|
|
55
|
+
* @param {Record<string, any>} [options={}] - Options (temperature, maxTokens, timeoutMs — a per-call override of the constructor's `timeoutMs`, see BA-18; deadlineMs — a per-call override of the constructor's `deadlineMs`, see BA-19).
|
|
49
56
|
* @returns {Promise<GenerateResult>}
|
|
50
57
|
* @throws {Error} `[GeminiProvider] ...` — on HTTP errors (4xx/5xx) or invalid JSON response.
|
|
51
58
|
*/
|
|
@@ -107,8 +114,10 @@ class GeminiProvider {
|
|
|
107
114
|
|
|
108
115
|
// BA-10: graceful degrade if a model rejects a non-default `temperature` (Gemini nests it under
|
|
109
116
|
// generationConfig). Keyed off the API error text, so dormant on models that accept temperature.
|
|
117
|
+
const timeoutMs = resolveTimeoutMs(this.timeoutMs, options.timeoutMs);
|
|
118
|
+
const deadlineMs = resolveTimeoutMs(this.deadlineMs, options.deadlineMs, 0, 'deadlineMs');
|
|
110
119
|
const { data, temperatureDropped } = await requestWithTemperatureFallback({
|
|
111
|
-
request: () => this._request(`/models/${this.model}:generateContent`, body),
|
|
120
|
+
request: () => this._request(`/models/${this.model}:generateContent`, body, timeoutMs, deadlineMs),
|
|
112
121
|
hadTemperature: () => body.generationConfig?.temperature != null,
|
|
113
122
|
stripTemperature: () => { if (body.generationConfig) delete body.generationConfig.temperature; },
|
|
114
123
|
warnOnce: () => this._warnTemperatureDropped(),
|
|
@@ -174,9 +183,11 @@ class GeminiProvider {
|
|
|
174
183
|
/**
|
|
175
184
|
* @param {string} path
|
|
176
185
|
* @param {Record<string, any>} body
|
|
186
|
+
* @param {number} [timeoutMs=0] - Idle-socket timeout (ms); 0 disables. See BA-18 / provider-http.
|
|
187
|
+
* @param {number} [deadlineMs=0] - Total call-duration deadline (ms); 0 disables. See BA-19 / provider-http.
|
|
177
188
|
* @returns {Promise<any>}
|
|
178
189
|
*/
|
|
179
|
-
_request(path, body) {
|
|
190
|
+
_request(path, body, timeoutMs = 0, deadlineMs = 0) {
|
|
180
191
|
return new Promise((resolve, reject) => {
|
|
181
192
|
const url = new URL(this.baseUrl + path);
|
|
182
193
|
const transport = url.protocol === 'https:' ? https : http;
|
|
@@ -213,6 +224,7 @@ class GeminiProvider {
|
|
|
213
224
|
}
|
|
214
225
|
});
|
|
215
226
|
});
|
|
227
|
+
applyRequestBounds(req, { timeoutMs, deadlineMs }, 'GeminiProvider');
|
|
216
228
|
req.on('error', reject);
|
|
217
229
|
req.write(payload);
|
|
218
230
|
req.end();
|
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* BA-18 — shared request-timeout helper for the http(s)-based providers (Anthropic, OpenAI,
|
|
3
|
+
* Gemini, Ollama). They all build a `http.ClientRequest` with only `req.on('error')` wired, so a
|
|
4
|
+
* socket the server silently dropped — or a response that never starts — was bounded only by the
|
|
5
|
+
* OS TCP timeout (~2h on Linux). That presents to the caller as a hang, not a failure, so every
|
|
6
|
+
* retry/casualty policy above it is inert. This adds a finite, configurable idle bound in one
|
|
7
|
+
* place so the four providers cannot drift.
|
|
8
|
+
*/
|
|
9
|
+
export const DEFAULT_TIMEOUT_MS: 600000;
|
|
10
|
+
/**
|
|
11
|
+
* Resolve an effective timeout-shaped ms bound. A per-call value overrides the instance default, but
|
|
12
|
+
* `null` and `undefined` BOTH mean "inherit" — so a per-call `null` never shadows an instance-level
|
|
13
|
+
* disable. The explicit opt-out idiom is `0` or `Infinity` → returns 0 (no bound). When the knob is
|
|
14
|
+
* UNSET on both, returns `defaultMs` (the idle bound's 10 min, or 0/disabled for the deadline).
|
|
15
|
+
*
|
|
16
|
+
* The disable-edge (finding-3, and the BA-4/5/6 optimistic-rounding family) is the load-bearing part:
|
|
17
|
+
* a config MISTAKE — a NaN / negative / non-numeric value the caller EXPLICITLY set (e.g.
|
|
18
|
+
* `Number(process.env.X)` on an unset var, or a string `'30s'`) — must never SILENTLY remove a bound
|
|
19
|
+
* the caller evidently tried to set. Two cases, decided by whether the knob has a safe default:
|
|
20
|
+
* - `defaultMs > 0` (the BA-18 idle bound): fail SAFE to that real default (a garbage idle value
|
|
21
|
+
* keeps the 10-min safety net — byte-identical to the shipped BA-18 behaviour).
|
|
22
|
+
* - `defaultMs === 0` (the BA-19 deadline, disabled-by-design): there is NO safe bound to fall back
|
|
23
|
+
* to, so silently returning 0 would reintroduce the very hang the deadline exists to prevent
|
|
24
|
+
* (BA-19 review finding 1). Fail LOUD instead — throw a {@link ValidationError} so the config
|
|
25
|
+
* mistake surfaces immediately, rather than running unbounded for hours.
|
|
26
|
+
* NOTE: an UNSET deadline (null/undefined) is legitimate and still returns 0 — only an explicitly-set
|
|
27
|
+
* garbage value throws.
|
|
28
|
+
* @param {number|undefined|null} instanceTimeout
|
|
29
|
+
* @param {number|undefined|null} [callTimeout]
|
|
30
|
+
* @param {number} [defaultMs=DEFAULT_TIMEOUT_MS] - value returned when the knob is unset (or garbage, if >0)
|
|
31
|
+
* @param {string} [name='timeoutMs'] - knob name for the throw message
|
|
32
|
+
* @returns {number} a finite positive ms bound, or 0 to disable
|
|
33
|
+
* @throws {ValidationError} when the value is explicitly set but garbage AND `defaultMs` is 0 (no safe fallback)
|
|
34
|
+
*/
|
|
35
|
+
export function resolveTimeoutMs(instanceTimeout: number | undefined | null, callTimeout?: number | undefined | null, defaultMs?: number, name?: string): number;
|
|
36
|
+
/**
|
|
37
|
+
* Bound an in-flight ClientRequest on socket INACTIVITY. On timeout, destroy the request with a
|
|
38
|
+
* retryable {@link TimeoutError} (`code: 'ETIMEDOUT'`) — `DEFAULT_RETRY_ON` classifies that as
|
|
39
|
+
* transient, and the provider's own `req.on('error', reject)` turns it into a rejected promise, so
|
|
40
|
+
* the caller regains control instead of hanging. Idle semantics (via `req.setTimeout`): the timer
|
|
41
|
+
* resets on any socket activity, so a slow-but-streaming response is NOT killed — only a silent or
|
|
42
|
+
* never-answering socket trips it. A `timeoutMs` of 0 is a no-op (bound disabled).
|
|
43
|
+
* @param {import('http').ClientRequest} req
|
|
44
|
+
* @param {number} timeoutMs - resolved bound; 0 disables
|
|
45
|
+
* @param {string} providerName - for the error message (e.g. 'AnthropicProvider')
|
|
46
|
+
*/
|
|
47
|
+
export function applyRequestTimeout(req: import("http").ClientRequest, timeoutMs: number, providerName: string): void;
|
|
48
|
+
/**
|
|
49
|
+
* BA-19 — bound an in-flight ClientRequest on TOTAL call duration, beside {@link applyRequestTimeout}.
|
|
50
|
+
* The idle bound (`req.setTimeout`) resets on ANY socket activity, so a response that trickles a byte
|
|
51
|
+
* forever (a "zombie stream") never trips it — bytes keep arriving while the response never completes,
|
|
52
|
+
* and the call hangs until the OS TCP timeout (~4.5h observed). This adds an absolute, non-resetting
|
|
53
|
+
* wall-clock ceiling: a plain `setTimeout` that destroys the request whether or not the socket is
|
|
54
|
+
* active. On trip, the request is destroyed with a TERMINAL {@link TimeoutError} — distinct
|
|
55
|
+
* `code: 'EDEADLINE'` and `context.bound: 'deadline'` (so a consumer routing governance stops vs
|
|
56
|
+
* transport casualties can tell which timer fired), and `retryable: false` because a deadline is a
|
|
57
|
+
* HARD ceiling the caller set to STOP: auto-retrying would re-spend up to another full `deadlineMs`
|
|
58
|
+
* of tokens/budget. Disabled-by-design (a deliberately long single call is legitimate); a
|
|
59
|
+
* `deadlineMs` of 0 is a no-op. When both bounds are armed and `timeoutMs < deadlineMs`, a silent
|
|
60
|
+
* socket trips the idle bound first; only a still-active-but-never-completing stream reaches the
|
|
61
|
+
* deadline. The timer is unref'd (never keeps the event loop alive) and cleared when the request
|
|
62
|
+
* closes (no dangling handle, no late destroy of a settled request).
|
|
63
|
+
* @param {import('http').ClientRequest} req
|
|
64
|
+
* @param {number} deadlineMs - resolved bound; 0 disables
|
|
65
|
+
* @param {string} providerName - for the error message (e.g. 'AnthropicProvider')
|
|
66
|
+
*/
|
|
67
|
+
export function applyRequestDeadline(req: import("http").ClientRequest, deadlineMs: number, providerName: string): void;
|
|
68
|
+
/**
|
|
69
|
+
* Wire ALL request bounds onto a ClientRequest in one call — the single seam each provider's
|
|
70
|
+
* `_request` uses, so the idle (BA-18) + deadline (BA-19) wiring is not copy-pasted at four call
|
|
71
|
+
* sites (BA-19 review finding 2). A future third bound is added HERE once, not at every provider.
|
|
72
|
+
* Each individual bound is a no-op when its ms value is 0/absent, so an unset knob costs nothing.
|
|
73
|
+
* @param {import('http').ClientRequest} req
|
|
74
|
+
* @param {{ timeoutMs?: number, deadlineMs?: number }} bounds - resolved ms bounds; 0/absent disables each
|
|
75
|
+
* @param {string} providerName - for error messages (e.g. 'AnthropicProvider')
|
|
76
|
+
*/
|
|
77
|
+
export function applyRequestBounds(req: import("http").ClientRequest, bounds: {
|
|
78
|
+
timeoutMs?: number;
|
|
79
|
+
deadlineMs?: number;
|
|
80
|
+
}, providerName: string): void;
|
|
@@ -0,0 +1,126 @@
|
|
|
1
|
+
'use strict';
|
|
2
|
+
|
|
3
|
+
const { TimeoutError, ValidationError } = require('./errors');
|
|
4
|
+
|
|
5
|
+
/**
|
|
6
|
+
* BA-18 — shared request-timeout helper for the http(s)-based providers (Anthropic, OpenAI,
|
|
7
|
+
* Gemini, Ollama). They all build a `http.ClientRequest` with only `req.on('error')` wired, so a
|
|
8
|
+
* socket the server silently dropped — or a response that never starts — was bounded only by the
|
|
9
|
+
* OS TCP timeout (~2h on Linux). That presents to the caller as a hang, not a failure, so every
|
|
10
|
+
* retry/casualty policy above it is inert. This adds a finite, configurable idle bound in one
|
|
11
|
+
* place so the four providers cannot drift.
|
|
12
|
+
*/
|
|
13
|
+
|
|
14
|
+
// 10 minutes: safely above any single non-streaming completion (a big reasoning response is a few
|
|
15
|
+
// minutes at most), well below the ~2h OS TCP default. These requests are non-streaming, so a legit
|
|
16
|
+
// slow completion can have no socket activity until the whole body arrives (TTFB ≈ generation time)
|
|
17
|
+
// — the default must clear that, not a typical round-trip.
|
|
18
|
+
const DEFAULT_TIMEOUT_MS = 600000;
|
|
19
|
+
|
|
20
|
+
/**
|
|
21
|
+
* Resolve an effective timeout-shaped ms bound. A per-call value overrides the instance default, but
|
|
22
|
+
* `null` and `undefined` BOTH mean "inherit" — so a per-call `null` never shadows an instance-level
|
|
23
|
+
* disable. The explicit opt-out idiom is `0` or `Infinity` → returns 0 (no bound). When the knob is
|
|
24
|
+
* UNSET on both, returns `defaultMs` (the idle bound's 10 min, or 0/disabled for the deadline).
|
|
25
|
+
*
|
|
26
|
+
* The disable-edge (finding-3, and the BA-4/5/6 optimistic-rounding family) is the load-bearing part:
|
|
27
|
+
* a config MISTAKE — a NaN / negative / non-numeric value the caller EXPLICITLY set (e.g.
|
|
28
|
+
* `Number(process.env.X)` on an unset var, or a string `'30s'`) — must never SILENTLY remove a bound
|
|
29
|
+
* the caller evidently tried to set. Two cases, decided by whether the knob has a safe default:
|
|
30
|
+
* - `defaultMs > 0` (the BA-18 idle bound): fail SAFE to that real default (a garbage idle value
|
|
31
|
+
* keeps the 10-min safety net — byte-identical to the shipped BA-18 behaviour).
|
|
32
|
+
* - `defaultMs === 0` (the BA-19 deadline, disabled-by-design): there is NO safe bound to fall back
|
|
33
|
+
* to, so silently returning 0 would reintroduce the very hang the deadline exists to prevent
|
|
34
|
+
* (BA-19 review finding 1). Fail LOUD instead — throw a {@link ValidationError} so the config
|
|
35
|
+
* mistake surfaces immediately, rather than running unbounded for hours.
|
|
36
|
+
* NOTE: an UNSET deadline (null/undefined) is legitimate and still returns 0 — only an explicitly-set
|
|
37
|
+
* garbage value throws.
|
|
38
|
+
* @param {number|undefined|null} instanceTimeout
|
|
39
|
+
* @param {number|undefined|null} [callTimeout]
|
|
40
|
+
* @param {number} [defaultMs=DEFAULT_TIMEOUT_MS] - value returned when the knob is unset (or garbage, if >0)
|
|
41
|
+
* @param {string} [name='timeoutMs'] - knob name for the throw message
|
|
42
|
+
* @returns {number} a finite positive ms bound, or 0 to disable
|
|
43
|
+
* @throws {ValidationError} when the value is explicitly set but garbage AND `defaultMs` is 0 (no safe fallback)
|
|
44
|
+
*/
|
|
45
|
+
function resolveTimeoutMs(instanceTimeout, callTimeout, defaultMs = DEFAULT_TIMEOUT_MS, name = 'timeoutMs') {
|
|
46
|
+
const raw = callTimeout != null ? callTimeout : instanceTimeout; // null/undefined per-call → inherit
|
|
47
|
+
if (raw == null) return defaultMs; // absent on both → the knob's own default (0 = disabled)
|
|
48
|
+
const n = Number(raw);
|
|
49
|
+
if (n === 0 || n === Infinity) return 0; // the explicit opt-out idiom → no bound
|
|
50
|
+
if (Number.isFinite(n) && n > 0) return n; // finite positive bound
|
|
51
|
+
// Explicitly set but garbage. Never SILENTLY disable a bound the caller tried to set.
|
|
52
|
+
if (defaultMs > 0) return defaultMs; // a real default exists → fail safe to it (BA-18)
|
|
53
|
+
throw new ValidationError(
|
|
54
|
+
`[provider-http] invalid ${name}: ${String(raw).slice(0, 40)} — expected a positive number, 0, or Infinity`
|
|
55
|
+
);
|
|
56
|
+
}
|
|
57
|
+
|
|
58
|
+
/**
|
|
59
|
+
* Bound an in-flight ClientRequest on socket INACTIVITY. On timeout, destroy the request with a
|
|
60
|
+
* retryable {@link TimeoutError} (`code: 'ETIMEDOUT'`) — `DEFAULT_RETRY_ON` classifies that as
|
|
61
|
+
* transient, and the provider's own `req.on('error', reject)` turns it into a rejected promise, so
|
|
62
|
+
* the caller regains control instead of hanging. Idle semantics (via `req.setTimeout`): the timer
|
|
63
|
+
* resets on any socket activity, so a slow-but-streaming response is NOT killed — only a silent or
|
|
64
|
+
* never-answering socket trips it. A `timeoutMs` of 0 is a no-op (bound disabled).
|
|
65
|
+
* @param {import('http').ClientRequest} req
|
|
66
|
+
* @param {number} timeoutMs - resolved bound; 0 disables
|
|
67
|
+
* @param {string} providerName - for the error message (e.g. 'AnthropicProvider')
|
|
68
|
+
*/
|
|
69
|
+
function applyRequestTimeout(req, timeoutMs, providerName) {
|
|
70
|
+
if (!(timeoutMs > 0)) return;
|
|
71
|
+
req.setTimeout(timeoutMs, () => {
|
|
72
|
+
// `context.bound: 'idle'` mirrors the deadline trip's discriminator (BA-19) so a consumer can
|
|
73
|
+
// switch on one uniform field to tell which timer spoke, not just on the `code`.
|
|
74
|
+
req.destroy(new TimeoutError(
|
|
75
|
+
`[${providerName}] request timed out after ${timeoutMs}ms of socket inactivity`,
|
|
76
|
+
{ context: { bound: 'idle' } }
|
|
77
|
+
));
|
|
78
|
+
});
|
|
79
|
+
}
|
|
80
|
+
|
|
81
|
+
/**
|
|
82
|
+
* BA-19 — bound an in-flight ClientRequest on TOTAL call duration, beside {@link applyRequestTimeout}.
|
|
83
|
+
* The idle bound (`req.setTimeout`) resets on ANY socket activity, so a response that trickles a byte
|
|
84
|
+
* forever (a "zombie stream") never trips it — bytes keep arriving while the response never completes,
|
|
85
|
+
* and the call hangs until the OS TCP timeout (~4.5h observed). This adds an absolute, non-resetting
|
|
86
|
+
* wall-clock ceiling: a plain `setTimeout` that destroys the request whether or not the socket is
|
|
87
|
+
* active. On trip, the request is destroyed with a TERMINAL {@link TimeoutError} — distinct
|
|
88
|
+
* `code: 'EDEADLINE'` and `context.bound: 'deadline'` (so a consumer routing governance stops vs
|
|
89
|
+
* transport casualties can tell which timer fired), and `retryable: false` because a deadline is a
|
|
90
|
+
* HARD ceiling the caller set to STOP: auto-retrying would re-spend up to another full `deadlineMs`
|
|
91
|
+
* of tokens/budget. Disabled-by-design (a deliberately long single call is legitimate); a
|
|
92
|
+
* `deadlineMs` of 0 is a no-op. When both bounds are armed and `timeoutMs < deadlineMs`, a silent
|
|
93
|
+
* socket trips the idle bound first; only a still-active-but-never-completing stream reaches the
|
|
94
|
+
* deadline. The timer is unref'd (never keeps the event loop alive) and cleared when the request
|
|
95
|
+
* closes (no dangling handle, no late destroy of a settled request).
|
|
96
|
+
* @param {import('http').ClientRequest} req
|
|
97
|
+
* @param {number} deadlineMs - resolved bound; 0 disables
|
|
98
|
+
* @param {string} providerName - for the error message (e.g. 'AnthropicProvider')
|
|
99
|
+
*/
|
|
100
|
+
function applyRequestDeadline(req, deadlineMs, providerName) {
|
|
101
|
+
if (!(deadlineMs > 0)) return;
|
|
102
|
+
const timer = setTimeout(() => {
|
|
103
|
+
req.destroy(new TimeoutError(
|
|
104
|
+
`[${providerName}] request exceeded its total deadline of ${deadlineMs}ms`,
|
|
105
|
+
{ code: 'EDEADLINE', retryable: false, context: { bound: 'deadline' } }
|
|
106
|
+
));
|
|
107
|
+
}, deadlineMs);
|
|
108
|
+
if (timer.unref) timer.unref();
|
|
109
|
+
req.once('close', () => clearTimeout(timer));
|
|
110
|
+
}
|
|
111
|
+
|
|
112
|
+
/**
|
|
113
|
+
* Wire ALL request bounds onto a ClientRequest in one call — the single seam each provider's
|
|
114
|
+
* `_request` uses, so the idle (BA-18) + deadline (BA-19) wiring is not copy-pasted at four call
|
|
115
|
+
* sites (BA-19 review finding 2). A future third bound is added HERE once, not at every provider.
|
|
116
|
+
* Each individual bound is a no-op when its ms value is 0/absent, so an unset knob costs nothing.
|
|
117
|
+
* @param {import('http').ClientRequest} req
|
|
118
|
+
* @param {{ timeoutMs?: number, deadlineMs?: number }} bounds - resolved ms bounds; 0/absent disables each
|
|
119
|
+
* @param {string} providerName - for error messages (e.g. 'AnthropicProvider')
|
|
120
|
+
*/
|
|
121
|
+
function applyRequestBounds(req, bounds, providerName) {
|
|
122
|
+
applyRequestTimeout(req, (bounds && bounds.timeoutMs) || 0, providerName);
|
|
123
|
+
applyRequestDeadline(req, (bounds && bounds.deadlineMs) || 0, providerName);
|
|
124
|
+
}
|
|
125
|
+
|
|
126
|
+
module.exports = { DEFAULT_TIMEOUT_MS, resolveTimeoutMs, applyRequestTimeout, applyRequestDeadline, applyRequestBounds };
|
package/src/provider-ollama.d.ts
CHANGED
|
@@ -5,6 +5,14 @@ export type OllamaOptions = {
|
|
|
5
5
|
model?: string | undefined;
|
|
6
6
|
url?: string | undefined;
|
|
7
7
|
exposeErrorBody?: boolean | undefined;
|
|
8
|
+
/**
|
|
9
|
+
* - BA-18: request/idle timeout in ms. Bounds a silent or never-answering socket on inactivity so `generate()` rejects with a retryable `TimeoutError` (`code: 'ETIMEDOUT'`, `context.bound: 'idle'`) instead of hanging until the OS TCP timeout. `0`/`Infinity` disables it. Overridable per call via `generate(..., { timeoutMs })`. (A local Ollama that is loading a large model cold can be slow to first byte — raise this or disable it for very large local models.)
|
|
10
|
+
*/
|
|
11
|
+
timeoutMs?: number | undefined;
|
|
12
|
+
/**
|
|
13
|
+
* - BA-19: TOTAL call-duration deadline in ms, beside `timeoutMs`. The idle bound resets on any socket activity, so a response that trickles a byte forever never trips it and hangs for hours. This is an absolute, non-resetting ceiling; on trip, `generate()` rejects with a TERMINAL `TimeoutError` (`code: 'EDEADLINE'`, `context.bound: 'deadline'`, `retryable: false`). DISABLED by default; `0`/`Infinity` disable. Overridable per call via `generate(..., { deadlineMs })`. (A cold large-model load is a legitimate long single call — leave this disabled or set it generously for such models.)
|
|
14
|
+
*/
|
|
15
|
+
deadlineMs?: number | undefined;
|
|
8
16
|
};
|
|
9
17
|
/** @typedef {import('../types').Message} Message */
|
|
10
18
|
/** @typedef {import('../types').ToolDef} ToolDef */
|
|
@@ -14,6 +22,8 @@ export type OllamaOptions = {
|
|
|
14
22
|
* @property {string} [model='llama3.2']
|
|
15
23
|
* @property {string} [url='http://localhost:11434']
|
|
16
24
|
* @property {boolean} [exposeErrorBody=false]
|
|
25
|
+
* @property {number} [timeoutMs=600000] - BA-18: request/idle timeout in ms. Bounds a silent or never-answering socket on inactivity so `generate()` rejects with a retryable `TimeoutError` (`code: 'ETIMEDOUT'`, `context.bound: 'idle'`) instead of hanging until the OS TCP timeout. `0`/`Infinity` disables it. Overridable per call via `generate(..., { timeoutMs })`. (A local Ollama that is loading a large model cold can be slow to first byte — raise this or disable it for very large local models.)
|
|
26
|
+
* @property {number} [deadlineMs=0] - BA-19: TOTAL call-duration deadline in ms, beside `timeoutMs`. The idle bound resets on any socket activity, so a response that trickles a byte forever never trips it and hangs for hours. This is an absolute, non-resetting ceiling; on trip, `generate()` rejects with a TERMINAL `TimeoutError` (`code: 'EDEADLINE'`, `context.bound: 'deadline'`, `retryable: false`). DISABLED by default; `0`/`Infinity` disable. Overridable per call via `generate(..., { deadlineMs })`. (A cold large-model load is a legitimate long single call — leave this disabled or set it generously for such models.)
|
|
17
27
|
*/
|
|
18
28
|
export class OllamaProvider {
|
|
19
29
|
/**
|
|
@@ -23,11 +33,13 @@ export class OllamaProvider {
|
|
|
23
33
|
model: string;
|
|
24
34
|
url: string;
|
|
25
35
|
exposeErrorBody: boolean;
|
|
36
|
+
timeoutMs: number | undefined;
|
|
37
|
+
deadlineMs: number | undefined;
|
|
26
38
|
/**
|
|
27
39
|
* Generate a response from a local Ollama instance.
|
|
28
40
|
* @param {Message[]} messages - Conversation messages.
|
|
29
41
|
* @param {ToolDef[]} [tools=[]] - Tool definitions.
|
|
30
|
-
* @param {Record<string, any>} [options={}] - Options (`temperature`, `maxTokens`).
|
|
42
|
+
* @param {Record<string, any>} [options={}] - Options (`temperature`, `maxTokens`, `timeoutMs` — a per-call override of the constructor's `timeoutMs`, see BA-18; `deadlineMs` — a per-call override of the constructor's `deadlineMs`, see BA-19).
|
|
31
43
|
* @returns {Promise<GenerateResult>}
|
|
32
44
|
* @throws {Error} `[OllamaProvider] ...` — on HTTP errors or invalid JSON response.
|
|
33
45
|
*/
|
|
@@ -38,7 +50,9 @@ export class OllamaProvider {
|
|
|
38
50
|
/**
|
|
39
51
|
* @param {string} path
|
|
40
52
|
* @param {Record<string, any>} body
|
|
53
|
+
* @param {number} [timeoutMs=0] - Idle-socket timeout (ms); 0 disables. See BA-18 / provider-http.
|
|
54
|
+
* @param {number} [deadlineMs=0] - Total call-duration deadline (ms); 0 disables. See BA-19 / provider-http.
|
|
41
55
|
* @returns {Promise<any>}
|
|
42
56
|
*/
|
|
43
|
-
_request(path: string, body: Record<string, any
|
|
57
|
+
_request(path: string, body: Record<string, any>, timeoutMs?: number, deadlineMs?: number): Promise<any>;
|
|
44
58
|
}
|
package/src/provider-ollama.js
CHANGED
|
@@ -4,6 +4,7 @@ const http = require('http');
|
|
|
4
4
|
const { ProviderError } = require('./errors');
|
|
5
5
|
const { requestWithTemperatureFallback } = require('./provider-temperature');
|
|
6
6
|
const { normalizeStopReason } = require('./provider-stop-reason');
|
|
7
|
+
const { resolveTimeoutMs, applyRequestBounds } = require('./provider-http');
|
|
7
8
|
|
|
8
9
|
/** @typedef {import('../types').Message} Message */
|
|
9
10
|
/** @typedef {import('../types').ToolDef} ToolDef */
|
|
@@ -14,6 +15,8 @@ const { normalizeStopReason } = require('./provider-stop-reason');
|
|
|
14
15
|
* @property {string} [model='llama3.2']
|
|
15
16
|
* @property {string} [url='http://localhost:11434']
|
|
16
17
|
* @property {boolean} [exposeErrorBody=false]
|
|
18
|
+
* @property {number} [timeoutMs=600000] - BA-18: request/idle timeout in ms. Bounds a silent or never-answering socket on inactivity so `generate()` rejects with a retryable `TimeoutError` (`code: 'ETIMEDOUT'`, `context.bound: 'idle'`) instead of hanging until the OS TCP timeout. `0`/`Infinity` disables it. Overridable per call via `generate(..., { timeoutMs })`. (A local Ollama that is loading a large model cold can be slow to first byte — raise this or disable it for very large local models.)
|
|
19
|
+
* @property {number} [deadlineMs=0] - BA-19: TOTAL call-duration deadline in ms, beside `timeoutMs`. The idle bound resets on any socket activity, so a response that trickles a byte forever never trips it and hangs for hours. This is an absolute, non-resetting ceiling; on trip, `generate()` rejects with a TERMINAL `TimeoutError` (`code: 'EDEADLINE'`, `context.bound: 'deadline'`, `retryable: false`). DISABLED by default; `0`/`Infinity` disable. Overridable per call via `generate(..., { deadlineMs })`. (A cold large-model load is a legitimate long single call — leave this disabled or set it generously for such models.)
|
|
17
20
|
*/
|
|
18
21
|
|
|
19
22
|
class OllamaProvider {
|
|
@@ -25,13 +28,17 @@ class OllamaProvider {
|
|
|
25
28
|
this.url = options.url || 'http://localhost:11434';
|
|
26
29
|
// See OpenAIProvider: attach full upstream body to err.body only on opt-in.
|
|
27
30
|
this.exposeErrorBody = options.exposeErrorBody === true;
|
|
31
|
+
// BA-18: request/idle timeout (ms). Resolved at call time (default 600000; 0/Infinity disable).
|
|
32
|
+
this.timeoutMs = options.timeoutMs;
|
|
33
|
+
// BA-19: total call-duration deadline (ms). Resolved at call time (default 0 = disabled).
|
|
34
|
+
this.deadlineMs = options.deadlineMs;
|
|
28
35
|
}
|
|
29
36
|
|
|
30
37
|
/**
|
|
31
38
|
* Generate a response from a local Ollama instance.
|
|
32
39
|
* @param {Message[]} messages - Conversation messages.
|
|
33
40
|
* @param {ToolDef[]} [tools=[]] - Tool definitions.
|
|
34
|
-
* @param {Record<string, any>} [options={}] - Options (`temperature`, `maxTokens`).
|
|
41
|
+
* @param {Record<string, any>} [options={}] - Options (`temperature`, `maxTokens`, `timeoutMs` — a per-call override of the constructor's `timeoutMs`, see BA-18; `deadlineMs` — a per-call override of the constructor's `deadlineMs`, see BA-19).
|
|
35
42
|
* @returns {Promise<GenerateResult>}
|
|
36
43
|
* @throws {Error} `[OllamaProvider] ...` — on HTTP errors or invalid JSON response.
|
|
37
44
|
*/
|
|
@@ -64,8 +71,10 @@ class OllamaProvider {
|
|
|
64
71
|
|
|
65
72
|
// BA-10: graceful degrade if a model rejects a non-default `temperature` (Ollama nests it under
|
|
66
73
|
// `options`). Keyed off the API error text, so dormant on models that accept temperature.
|
|
74
|
+
const timeoutMs = resolveTimeoutMs(this.timeoutMs, options.timeoutMs);
|
|
75
|
+
const deadlineMs = resolveTimeoutMs(this.deadlineMs, options.deadlineMs, 0, 'deadlineMs');
|
|
67
76
|
const { data, temperatureDropped } = await requestWithTemperatureFallback({
|
|
68
|
-
request: () => this._request('/api/chat', body),
|
|
77
|
+
request: () => this._request('/api/chat', body, timeoutMs, deadlineMs),
|
|
69
78
|
hadTemperature: () => body.options?.temperature != null,
|
|
70
79
|
stripTemperature: () => { if (body.options) delete body.options.temperature; },
|
|
71
80
|
warnOnce: () => this._warnTemperatureDropped(),
|
|
@@ -110,9 +119,11 @@ class OllamaProvider {
|
|
|
110
119
|
/**
|
|
111
120
|
* @param {string} path
|
|
112
121
|
* @param {Record<string, any>} body
|
|
122
|
+
* @param {number} [timeoutMs=0] - Idle-socket timeout (ms); 0 disables. See BA-18 / provider-http.
|
|
123
|
+
* @param {number} [deadlineMs=0] - Total call-duration deadline (ms); 0 disables. See BA-19 / provider-http.
|
|
113
124
|
* @returns {Promise<any>}
|
|
114
125
|
*/
|
|
115
|
-
_request(path, body) {
|
|
126
|
+
_request(path, body, timeoutMs = 0, deadlineMs = 0) {
|
|
116
127
|
return new Promise((resolve, reject) => {
|
|
117
128
|
const url = new URL(this.url + path);
|
|
118
129
|
const payload = JSON.stringify(body);
|
|
@@ -141,6 +152,7 @@ class OllamaProvider {
|
|
|
141
152
|
}
|
|
142
153
|
});
|
|
143
154
|
});
|
|
155
|
+
applyRequestBounds(req, { timeoutMs, deadlineMs }, 'OllamaProvider');
|
|
144
156
|
req.on('error', reject);
|
|
145
157
|
req.write(payload);
|
|
146
158
|
req.end();
|
package/src/provider-openai.d.ts
CHANGED
|
@@ -14,6 +14,22 @@ export type OpenAIOptions = {
|
|
|
14
14
|
* debugging only.
|
|
15
15
|
*/
|
|
16
16
|
exposeErrorBody?: boolean | undefined;
|
|
17
|
+
/**
|
|
18
|
+
* - BA-18: request/idle timeout in ms. Bounds a silent or
|
|
19
|
+
* never-answering socket on inactivity so `generate()` rejects with a retryable `TimeoutError`
|
|
20
|
+
* (`code: 'ETIMEDOUT'`, `context.bound: 'idle'`) instead of hanging until the OS TCP timeout (~2h).
|
|
21
|
+
* `0`/`Infinity` disables it (pre-BA-18 behaviour). Overridable per call via `generate(..., { timeoutMs })`.
|
|
22
|
+
*/
|
|
23
|
+
timeoutMs?: number | undefined;
|
|
24
|
+
/**
|
|
25
|
+
* - BA-19: TOTAL call-duration deadline in ms, beside `timeoutMs`.
|
|
26
|
+
* The idle bound resets on any socket activity, so a response that trickles a byte forever never
|
|
27
|
+
* trips it and hangs for hours. This is an absolute, non-resetting ceiling; on trip, `generate()`
|
|
28
|
+
* rejects with a TERMINAL `TimeoutError` (`code: 'EDEADLINE'`, `context.bound: 'deadline'`,
|
|
29
|
+
* `retryable: false`). DISABLED by default; `0`/`Infinity` disable. Overridable per call via
|
|
30
|
+
* `generate(..., { deadlineMs })`.
|
|
31
|
+
*/
|
|
32
|
+
deadlineMs?: number | undefined;
|
|
17
33
|
};
|
|
18
34
|
/**
|
|
19
35
|
* @typedef {object} OpenAIOptions
|
|
@@ -25,6 +41,16 @@ export type OpenAIOptions = {
|
|
|
25
41
|
* field in an error payload can't leak through logs that dump the error
|
|
26
42
|
* object; `err.message` still carries the API's error message. Turn on for
|
|
27
43
|
* debugging only.
|
|
44
|
+
* @property {number} [timeoutMs=600000] - BA-18: request/idle timeout in ms. Bounds a silent or
|
|
45
|
+
* never-answering socket on inactivity so `generate()` rejects with a retryable `TimeoutError`
|
|
46
|
+
* (`code: 'ETIMEDOUT'`, `context.bound: 'idle'`) instead of hanging until the OS TCP timeout (~2h).
|
|
47
|
+
* `0`/`Infinity` disables it (pre-BA-18 behaviour). Overridable per call via `generate(..., { timeoutMs })`.
|
|
48
|
+
* @property {number} [deadlineMs=0] - BA-19: TOTAL call-duration deadline in ms, beside `timeoutMs`.
|
|
49
|
+
* The idle bound resets on any socket activity, so a response that trickles a byte forever never
|
|
50
|
+
* trips it and hangs for hours. This is an absolute, non-resetting ceiling; on trip, `generate()`
|
|
51
|
+
* rejects with a TERMINAL `TimeoutError` (`code: 'EDEADLINE'`, `context.bound: 'deadline'`,
|
|
52
|
+
* `retryable: false`). DISABLED by default; `0`/`Infinity` disable. Overridable per call via
|
|
53
|
+
* `generate(..., { deadlineMs })`.
|
|
28
54
|
*/
|
|
29
55
|
export class OpenAIProvider {
|
|
30
56
|
/**
|
|
@@ -35,11 +61,13 @@ export class OpenAIProvider {
|
|
|
35
61
|
model: string;
|
|
36
62
|
baseUrl: string;
|
|
37
63
|
exposeErrorBody: boolean;
|
|
64
|
+
timeoutMs: number | undefined;
|
|
65
|
+
deadlineMs: number | undefined;
|
|
38
66
|
/**
|
|
39
67
|
* Generate a response from the OpenAI API.
|
|
40
68
|
* @param {Message[]} messages - Conversation messages.
|
|
41
69
|
* @param {ToolDef[]} [tools=[]] - Tool definitions.
|
|
42
|
-
* @param {Record<string, any>} [options={}] - Options (temperature, maxTokens).
|
|
70
|
+
* @param {Record<string, any>} [options={}] - Options (temperature, maxTokens, timeoutMs — a per-call override of the constructor's `timeoutMs`, see BA-18; deadlineMs — a per-call override of the constructor's `deadlineMs`, see BA-19).
|
|
43
71
|
* @returns {Promise<GenerateResult>}
|
|
44
72
|
* @throws {Error} `[OpenAIProvider] ...` — on HTTP errors (4xx/5xx) or invalid JSON response.
|
|
45
73
|
*/
|
|
@@ -60,8 +88,10 @@ export class OpenAIProvider {
|
|
|
60
88
|
/**
|
|
61
89
|
* @param {string} path
|
|
62
90
|
* @param {Record<string, any>} body
|
|
91
|
+
* @param {number} [timeoutMs=0] - Idle-socket timeout (ms); 0 disables. See BA-18 / provider-http.
|
|
92
|
+
* @param {number} [deadlineMs=0] - Total call-duration deadline (ms); 0 disables. See BA-19 / provider-http.
|
|
63
93
|
* @returns {Promise<any>}
|
|
64
94
|
*/
|
|
65
|
-
_request(path: string, body: Record<string, any
|
|
95
|
+
_request(path: string, body: Record<string, any>, timeoutMs?: number, deadlineMs?: number): Promise<any>;
|
|
66
96
|
_warnedInsecure: boolean | undefined;
|
|
67
97
|
}
|
package/src/provider-openai.js
CHANGED
|
@@ -5,6 +5,7 @@ const http = require('http');
|
|
|
5
5
|
const { ProviderError } = require('./errors');
|
|
6
6
|
const { requestWithTemperatureFallback } = require('./provider-temperature');
|
|
7
7
|
const { normalizeStopReason } = require('./provider-stop-reason');
|
|
8
|
+
const { resolveTimeoutMs, applyRequestBounds } = require('./provider-http');
|
|
8
9
|
|
|
9
10
|
/** @typedef {import('../types').Message} Message */
|
|
10
11
|
/** @typedef {import('../types').ToolDef} ToolDef */
|
|
@@ -27,6 +28,16 @@ function isLoopbackHost(hostname) {
|
|
|
27
28
|
* field in an error payload can't leak through logs that dump the error
|
|
28
29
|
* object; `err.message` still carries the API's error message. Turn on for
|
|
29
30
|
* debugging only.
|
|
31
|
+
* @property {number} [timeoutMs=600000] - BA-18: request/idle timeout in ms. Bounds a silent or
|
|
32
|
+
* never-answering socket on inactivity so `generate()` rejects with a retryable `TimeoutError`
|
|
33
|
+
* (`code: 'ETIMEDOUT'`, `context.bound: 'idle'`) instead of hanging until the OS TCP timeout (~2h).
|
|
34
|
+
* `0`/`Infinity` disables it (pre-BA-18 behaviour). Overridable per call via `generate(..., { timeoutMs })`.
|
|
35
|
+
* @property {number} [deadlineMs=0] - BA-19: TOTAL call-duration deadline in ms, beside `timeoutMs`.
|
|
36
|
+
* The idle bound resets on any socket activity, so a response that trickles a byte forever never
|
|
37
|
+
* trips it and hangs for hours. This is an absolute, non-resetting ceiling; on trip, `generate()`
|
|
38
|
+
* rejects with a TERMINAL `TimeoutError` (`code: 'EDEADLINE'`, `context.bound: 'deadline'`,
|
|
39
|
+
* `retryable: false`). DISABLED by default; `0`/`Infinity` disable. Overridable per call via
|
|
40
|
+
* `generate(..., { deadlineMs })`.
|
|
30
41
|
*/
|
|
31
42
|
|
|
32
43
|
class OpenAIProvider {
|
|
@@ -38,13 +49,17 @@ class OpenAIProvider {
|
|
|
38
49
|
this.model = options.model || 'gpt-4o-mini';
|
|
39
50
|
this.baseUrl = options.baseUrl || 'https://api.openai.com/v1';
|
|
40
51
|
this.exposeErrorBody = options.exposeErrorBody === true;
|
|
52
|
+
// BA-18: request/idle timeout (ms). Resolved at call time (default 600000; 0/Infinity disable).
|
|
53
|
+
this.timeoutMs = options.timeoutMs;
|
|
54
|
+
// BA-19: total call-duration deadline (ms). Resolved at call time (default 0 = disabled).
|
|
55
|
+
this.deadlineMs = options.deadlineMs;
|
|
41
56
|
}
|
|
42
57
|
|
|
43
58
|
/**
|
|
44
59
|
* Generate a response from the OpenAI API.
|
|
45
60
|
* @param {Message[]} messages - Conversation messages.
|
|
46
61
|
* @param {ToolDef[]} [tools=[]] - Tool definitions.
|
|
47
|
-
* @param {Record<string, any>} [options={}] - Options (temperature, maxTokens).
|
|
62
|
+
* @param {Record<string, any>} [options={}] - Options (temperature, maxTokens, timeoutMs — a per-call override of the constructor's `timeoutMs`, see BA-18; deadlineMs — a per-call override of the constructor's `deadlineMs`, see BA-19).
|
|
48
63
|
* @returns {Promise<GenerateResult>}
|
|
49
64
|
* @throws {Error} `[OpenAIProvider] ...` — on HTTP errors (4xx/5xx) or invalid JSON response.
|
|
50
65
|
*/
|
|
@@ -65,8 +80,10 @@ class OpenAIProvider {
|
|
|
65
80
|
|
|
66
81
|
// BA-10: newer models (o1/gpt-5-class) reject a non-default `temperature` with a 400 — drop it and
|
|
67
82
|
// retry once. `temperatureDropped` flows back so an upstream receipt can report the effective value.
|
|
83
|
+
const timeoutMs = resolveTimeoutMs(this.timeoutMs, options.timeoutMs);
|
|
84
|
+
const deadlineMs = resolveTimeoutMs(this.deadlineMs, options.deadlineMs, 0, 'deadlineMs');
|
|
68
85
|
const { data, temperatureDropped } = await requestWithTemperatureFallback({
|
|
69
|
-
request: () => this._request('/chat/completions', body),
|
|
86
|
+
request: () => this._request('/chat/completions', body, timeoutMs, deadlineMs),
|
|
70
87
|
hadTemperature: () => body.temperature != null,
|
|
71
88
|
stripTemperature: () => { delete body.temperature; },
|
|
72
89
|
warnOnce: () => this._warnTemperatureDropped(),
|
|
@@ -124,9 +141,11 @@ class OpenAIProvider {
|
|
|
124
141
|
/**
|
|
125
142
|
* @param {string} path
|
|
126
143
|
* @param {Record<string, any>} body
|
|
144
|
+
* @param {number} [timeoutMs=0] - Idle-socket timeout (ms); 0 disables. See BA-18 / provider-http.
|
|
145
|
+
* @param {number} [deadlineMs=0] - Total call-duration deadline (ms); 0 disables. See BA-19 / provider-http.
|
|
127
146
|
* @returns {Promise<any>}
|
|
128
147
|
*/
|
|
129
|
-
_request(path, body) {
|
|
148
|
+
_request(path, body, timeoutMs = 0, deadlineMs = 0) {
|
|
130
149
|
return new Promise((resolve, reject) => {
|
|
131
150
|
const url = new URL(this.baseUrl + path);
|
|
132
151
|
const transport = url.protocol === 'https:' ? https : http;
|
|
@@ -168,6 +187,7 @@ class OpenAIProvider {
|
|
|
168
187
|
}
|
|
169
188
|
});
|
|
170
189
|
});
|
|
190
|
+
applyRequestBounds(req, { timeoutMs, deadlineMs }, 'OpenAIProvider');
|
|
171
191
|
req.on('error', reject);
|
|
172
192
|
req.write(payload);
|
|
173
193
|
req.end();
|