metergraph 0.3.2 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +44 -2
- package/dist/batch-first.d.ts +112 -0
- package/dist/batch-first.js +207 -0
- package/dist/capture.d.ts +2 -0
- package/dist/capture.js +10 -4
- package/dist/index.d.ts +10 -0
- package/dist/index.js +24 -1
- package/dist/provider-batch.d.ts +160 -0
- package/dist/provider-batch.js +240 -0
- package/dist/repo-config.d.ts +22 -0
- package/dist/repo-config.js +214 -0
- package/dist/session.d.ts +23 -0
- package/dist/session.js +109 -0
- package/dist/transport.d.ts +3 -0
- package/dist/transport.js +13 -1
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -86,7 +86,12 @@ required and throws if missing so a caller error surfaces immediately
|
|
|
86
86
|
instead of an `undefined` model reaching a provider call.
|
|
87
87
|
|
|
88
88
|
Set `METERGRAPH_APP_TOKEN`; `METERGRAPH_INGEST_URL` is only needed to override
|
|
89
|
-
the hosted HTTPS endpoint. SDK 0.
|
|
89
|
+
the hosted HTTPS endpoint. SDK 0.4 automatically reads the GitHub `origin` on
|
|
90
|
+
first initialization and creates `.metergraph/config.json` at the repository
|
|
91
|
+
root if absent. Commit this non-secret file so production can use
|
|
92
|
+
repository-aware ingest without Git metadata. Existing config is authoritative
|
|
93
|
+
and never overwritten; when discovery or creation is unavailable, the SDK
|
|
94
|
+
continues with protocol v1. SDK 0.4 captures the scrubbed provider request and a
|
|
90
95
|
normalized response envelope, including assistant content and tool calls, by
|
|
91
96
|
default. Provider credentials and transport headers are removed. Request and
|
|
92
97
|
response are each limited to 100 KiB of UTF-8 with an explicit truncation
|
|
@@ -120,6 +125,43 @@ inference while iterating a wrapped `client.messages.batches.results()` result.
|
|
|
120
125
|
Consume results inside `route()` so the asynchronous batch retains its product
|
|
121
126
|
route. Job-management polls are deliberately not counted as model calls.
|
|
122
127
|
|
|
128
|
+
## Batch-first execution (opt-in)
|
|
129
|
+
|
|
130
|
+
`batchFirst()` is a separate, explicitly opt-in code path from `wrap()`/capture: submit one request through a provider's Batch API, wait up to a caller-chosen deadline, and fall back to exactly one direct call if the batch hasn't finished in time.
|
|
131
|
+
|
|
132
|
+
```ts
|
|
133
|
+
import { batchFirst } from "metergraph";
|
|
134
|
+
import OpenAI from "openai";
|
|
135
|
+
|
|
136
|
+
const client = new OpenAI(); // unwrapped — batchFirst() drives it directly, not through wrap()
|
|
137
|
+
|
|
138
|
+
const outcome = await batchFirst(
|
|
139
|
+
client,
|
|
140
|
+
"openai",
|
|
141
|
+
{ model: "gpt-5-mini", input: "Summarize this document." },
|
|
142
|
+
{
|
|
143
|
+
deadlineMs: 60_000,
|
|
144
|
+
acceptDuplicateProviderExecution: true, // required: a missed deadline can execute the request twice
|
|
145
|
+
onLateBatchSettled: (info) => {
|
|
146
|
+
// Fires later, asynchronously, only if a losing batch eventually
|
|
147
|
+
// settles. Never blocks batchFirst()'s own return, and never exposes
|
|
148
|
+
// the late result's content — only whether it contained a
|
|
149
|
+
// tool-call plan.
|
|
150
|
+
},
|
|
151
|
+
},
|
|
152
|
+
);
|
|
153
|
+
|
|
154
|
+
outcome.source; // "batch" | "direct"
|
|
155
|
+
outcome.result; // the provider response
|
|
156
|
+
outcome.metadata.batch_outcome; // "completed" | "failed" | "expired" | "pending_at_deadline"
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
`provider` is `"openai" | "anthropic" | "google"`, matching `wrap()`'s own explicit-provider option — never inferred from the client instance. A request with `stream: true` is rejected before any provider call. A request carrying `tools` is rejected unless `allowDuplicateToolCallPlans: true` is also set, acknowledging that the batch result and the direct fallback are independent provider executions that may each choose a different tool-call plan. `acceptDuplicateProviderExecution` must be exactly `true` — there is no default and no environment-variable override, and a missed deadline can execute (and bill) the same request twice. Neither the batch nor the direct path executes a tool call automatically; the caller receives the tool-call plan in `outcome.result` and is responsible for executing it, exactly as with a normal (non-batch-first) provider response.
|
|
160
|
+
|
|
161
|
+
`batchFirst()` is not integrated with `wrap()`'s capture/telemetry pipeline; its result and metadata are returned directly to the caller, never enqueued for delivery. The background poll that watches a losing batch for late telemetry uses an ordinary `setTimeout`, not an unref'd one — after a deadline-triggered direct fallback it keeps the Node process alive until the batch reaches a terminal state, which can take up to 24 hours; this is currently unmitigated, so a short-lived script or CLI process should not assume it exits promptly after a fallback.
|
|
162
|
+
|
|
163
|
+
OpenAI, Anthropic, and Google Gemini all have adapters (`createOpenAIBatchAdapter`, `createAnthropicBatchAdapter`, `createGoogleBatchAdapter`), built against each provider's documented Batch API shape and covered by fake-client contract tests, but **none has been exercised against a live provider Batch API from this SDK** — treat this as a beta-quality, code-reviewed-but-not-live-verified surface.
|
|
164
|
+
|
|
123
165
|
## Set up with an AI coding agent
|
|
124
166
|
|
|
125
167
|
Paste this into Claude Code, Codex, Cursor, or any coding agent inside the
|
|
@@ -134,7 +176,7 @@ import * as mg from "metergraph". wrap() returns the same client and
|
|
|
134
176
|
initializes itself from the environment: METERGRAPH_APP_TOKEN is required
|
|
135
177
|
(capture is silently off without it) and METERGRAPH_INGEST_URL is only for
|
|
136
178
|
self-hosted servers. Add both to .env.example, and never commit a real token.
|
|
137
|
-
SDK 0.
|
|
179
|
+
SDK 0.4 captures scrubbed provider requests and normalized responses by
|
|
138
180
|
default for the hosted dashboard; use METERGRAPH_CAPTURE_TEXT=0 or
|
|
139
181
|
captureText: false around sensitive operations. Provider credentials and
|
|
140
182
|
transport headers must never be captured. Capture is fail-open, so do not
|
|
@@ -0,0 +1,112 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Explicit, opt-in batch-first execution: submit one request through a
|
|
3
|
+
* provider's Batch API, wait a caller-selected deadline, and fall back to
|
|
4
|
+
* a single direct call if the batch hasn't finished in time. Never enabled
|
|
5
|
+
* by wrap()/capture defaults or by an environment variable — a caller
|
|
6
|
+
* reaches this only by importing and calling batchFirst() directly, and
|
|
7
|
+
* only after explicitly acknowledging the duplicate-execution semantic
|
|
8
|
+
* below.
|
|
9
|
+
*
|
|
10
|
+
* On a missed deadline, the request may execute twice against the
|
|
11
|
+
* provider (once via batch, once via direct) — an accepted, deliberate
|
|
12
|
+
* semantic, never silently avoided. What this module guarantees instead:
|
|
13
|
+
* exactly one direct fallback is ever issued, and a batch result that
|
|
14
|
+
* arrives after the fallback already won is never returned, never
|
|
15
|
+
* executed, and never mutates an already-resolved result.
|
|
16
|
+
*/
|
|
17
|
+
import { type BatchFirstRequest, type ProviderBatchAdapter } from "./provider-batch.js";
|
|
18
|
+
export type { BatchFirstRequest } from "./provider-batch.js";
|
|
19
|
+
/** An injectable time source. Real timers by default; tests can supply a
|
|
20
|
+
* fake to drive the deadline deterministically, with no real waiting. */
|
|
21
|
+
export interface BatchFirstClock {
|
|
22
|
+
setTimeout(handler: () => void, ms: number): unknown;
|
|
23
|
+
clearTimeout(handle: unknown): void;
|
|
24
|
+
}
|
|
25
|
+
/** Reported asynchronously, after batchFirst() has already returned via the
|
|
26
|
+
* direct fallback, if the losing batch eventually reaches a terminal
|
|
27
|
+
* state. Its actual result content is never included here and never
|
|
28
|
+
* returned from batchFirst() — only whether it happened to contain a
|
|
29
|
+
* tool-call plan, which may differ from the one the direct fallback
|
|
30
|
+
* produced (see allowDuplicateToolCallPlans on BatchFirstPolicy). */
|
|
31
|
+
export interface LateBatchInfo {
|
|
32
|
+
outcome: "completed" | "failed" | "expired";
|
|
33
|
+
containedToolCallPlan: boolean;
|
|
34
|
+
}
|
|
35
|
+
export interface BatchFirstPolicy {
|
|
36
|
+
/** How long to wait for the batch before issuing the one direct
|
|
37
|
+
* fallback. Required — there is no default deadline. */
|
|
38
|
+
deadlineMs: number;
|
|
39
|
+
/**
|
|
40
|
+
* Required, explicit acknowledgement that a missed deadline can result
|
|
41
|
+
* in the same request executing twice against the provider. Must be
|
|
42
|
+
* exactly `true`; never read from an environment variable, and never
|
|
43
|
+
* defaulted.
|
|
44
|
+
*/
|
|
45
|
+
acceptDuplicateProviderExecution: true;
|
|
46
|
+
/**
|
|
47
|
+
* Required, explicit acknowledgement for a request that includes
|
|
48
|
+
* `tools`: the batch result and the direct fallback are two independent
|
|
49
|
+
* provider executions of the same prompt, and may each choose a
|
|
50
|
+
* DIFFERENT tool-call plan (different tool, different arguments). A
|
|
51
|
+
* caller whose tools have side effects must not assume the two plans
|
|
52
|
+
* agree. Without this flag, a request carrying `tools` is rejected
|
|
53
|
+
* before any provider call is made.
|
|
54
|
+
*/
|
|
55
|
+
allowDuplicateToolCallPlans?: boolean;
|
|
56
|
+
/** Fired once, later, only when the losing batch eventually settles.
|
|
57
|
+
* Never blocks batchFirst()'s own return. */
|
|
58
|
+
onLateBatchSettled?: (info: LateBatchInfo) => void;
|
|
59
|
+
}
|
|
60
|
+
/** Non-public knobs for driving runBatchFirst()'s poll interval and time
|
|
61
|
+
* source from tests — deliberately kept off BatchFirstPolicy so a real
|
|
62
|
+
* caller of batchFirst() has no way to set them. */
|
|
63
|
+
interface RunBatchFirstTestControls {
|
|
64
|
+
/** How often to re-check batch status while waiting. Default 2000ms. */
|
|
65
|
+
pollIntervalMs?: number;
|
|
66
|
+
/** Injectable time source — see BatchFirstClock. */
|
|
67
|
+
clock?: BatchFirstClock;
|
|
68
|
+
}
|
|
69
|
+
export type BatchFirstSource = "batch" | "direct";
|
|
70
|
+
export interface BatchFirstMetadata {
|
|
71
|
+
execution_mode: "batch_first";
|
|
72
|
+
deadline_ms: number;
|
|
73
|
+
/** Wall-clock time from submission to the canonical result settling. */
|
|
74
|
+
batch_wait_ms: number;
|
|
75
|
+
/** The batch's own status as of when batchFirst() returned — not its
|
|
76
|
+
* eventual status if that differs (see onLateBatchSettled). */
|
|
77
|
+
batch_outcome: "completed" | "failed" | "expired" | "pending_at_deadline";
|
|
78
|
+
canonical_result: BatchFirstSource;
|
|
79
|
+
duplicate_provider_execution: boolean;
|
|
80
|
+
/** Always false when canonical_result is "batch" (no lateness is
|
|
81
|
+
* possible — the batch IS the canonical result). When
|
|
82
|
+
* canonical_result is "direct", this reflects what was known AT THE
|
|
83
|
+
* MOMENT batchFirst() returned, which is always false: a completion
|
|
84
|
+
* confirmed later only reaches the caller through
|
|
85
|
+
* onLateBatchSettled, never by mutating this object. */
|
|
86
|
+
late_batch_completed: boolean;
|
|
87
|
+
late_batch_contained_tool_call_plan: boolean;
|
|
88
|
+
}
|
|
89
|
+
export interface BatchFirstResult<TResult = unknown> {
|
|
90
|
+
source: BatchFirstSource;
|
|
91
|
+
result: TResult;
|
|
92
|
+
metadata: BatchFirstMetadata;
|
|
93
|
+
}
|
|
94
|
+
/** Thrown before any provider call when a request/policy combination is
|
|
95
|
+
* not eligible for batch-first execution — streaming, tools without
|
|
96
|
+
* acknowledgement, a missing/false acceptDuplicateProviderExecution, or
|
|
97
|
+
* an adapter-specific ineligibility. */
|
|
98
|
+
export declare class BatchFirstIneligibleError extends Error {
|
|
99
|
+
}
|
|
100
|
+
/**
|
|
101
|
+
* The adapter-injected core state machine — exported so fake-adapter,
|
|
102
|
+
* fake-clock tests can drive it directly, and used internally by the
|
|
103
|
+
* public, provider-explicit batchFirst() below.
|
|
104
|
+
*/
|
|
105
|
+
export declare function runBatchFirst<TResult>(adapter: ProviderBatchAdapter<TResult>, request: BatchFirstRequest, policy: BatchFirstPolicy, testControls?: RunBatchFirstTestControls): Promise<BatchFirstResult<TResult>>;
|
|
106
|
+
export type BatchFirstProvider = "openai" | "anthropic" | "google";
|
|
107
|
+
/**
|
|
108
|
+
* Explicit, provider-specific batch-first execution. Never inferred from
|
|
109
|
+
* the client instance — the caller states the provider, matching wrap()'s
|
|
110
|
+
* own explicit-provider option.
|
|
111
|
+
*/
|
|
112
|
+
export declare function batchFirst<TResult = unknown>(client: unknown, provider: BatchFirstProvider, request: BatchFirstRequest, policy: BatchFirstPolicy): Promise<BatchFirstResult<TResult>>;
|
|
@@ -0,0 +1,207 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Explicit, opt-in batch-first execution: submit one request through a
|
|
3
|
+
* provider's Batch API, wait a caller-selected deadline, and fall back to
|
|
4
|
+
* a single direct call if the batch hasn't finished in time. Never enabled
|
|
5
|
+
* by wrap()/capture defaults or by an environment variable — a caller
|
|
6
|
+
* reaches this only by importing and calling batchFirst() directly, and
|
|
7
|
+
* only after explicitly acknowledging the duplicate-execution semantic
|
|
8
|
+
* below.
|
|
9
|
+
*
|
|
10
|
+
* On a missed deadline, the request may execute twice against the
|
|
11
|
+
* provider (once via batch, once via direct) — an accepted, deliberate
|
|
12
|
+
* semantic, never silently avoided. What this module guarantees instead:
|
|
13
|
+
* exactly one direct fallback is ever issued, and a batch result that
|
|
14
|
+
* arrives after the fallback already won is never returned, never
|
|
15
|
+
* executed, and never mutates an already-resolved result.
|
|
16
|
+
*/
|
|
17
|
+
import { createAnthropicBatchAdapter, createGoogleBatchAdapter, createOpenAIBatchAdapter, } from "./provider-batch.js";
|
|
18
|
+
function realClock() {
|
|
19
|
+
return {
|
|
20
|
+
setTimeout: (handler, ms) => setTimeout(handler, ms),
|
|
21
|
+
clearTimeout: (handle) => clearTimeout(handle),
|
|
22
|
+
};
|
|
23
|
+
}
|
|
24
|
+
/** Thrown before any provider call when a request/policy combination is
|
|
25
|
+
* not eligible for batch-first execution — streaming, tools without
|
|
26
|
+
* acknowledgement, a missing/false acceptDuplicateProviderExecution, or
|
|
27
|
+
* an adapter-specific ineligibility. */
|
|
28
|
+
export class BatchFirstIneligibleError extends Error {
|
|
29
|
+
}
|
|
30
|
+
function hasTools(request) {
|
|
31
|
+
return Array.isArray(request.tools) && request.tools.length > 0;
|
|
32
|
+
}
|
|
33
|
+
function validate(request, policy) {
|
|
34
|
+
if (policy.acceptDuplicateProviderExecution !== true) {
|
|
35
|
+
throw new BatchFirstIneligibleError("batchFirst() requires policy.acceptDuplicateProviderExecution: true — "
|
|
36
|
+
+ "a missed deadline can execute the request twice against the provider");
|
|
37
|
+
}
|
|
38
|
+
if (!Number.isFinite(policy.deadlineMs) || policy.deadlineMs <= 0) {
|
|
39
|
+
throw new BatchFirstIneligibleError("batchFirst() requires a positive policy.deadlineMs");
|
|
40
|
+
}
|
|
41
|
+
if (request.stream === true) {
|
|
42
|
+
throw new BatchFirstIneligibleError("batchFirst() does not support streaming requests — streaming is direct-only");
|
|
43
|
+
}
|
|
44
|
+
if (hasTools(request) && policy.allowDuplicateToolCallPlans !== true) {
|
|
45
|
+
throw new BatchFirstIneligibleError("batchFirst() requires policy.allowDuplicateToolCallPlans: true for requests with "
|
|
46
|
+
+ "tools — the batch result and the direct fallback are independent provider "
|
|
47
|
+
+ "executions and may each choose a different tool call plan");
|
|
48
|
+
}
|
|
49
|
+
}
|
|
50
|
+
const DEFAULT_POLL_INTERVAL_MS = 2_000;
|
|
51
|
+
/**
|
|
52
|
+
* The adapter-injected core state machine — exported so fake-adapter,
|
|
53
|
+
* fake-clock tests can drive it directly, and used internally by the
|
|
54
|
+
* public, provider-explicit batchFirst() below.
|
|
55
|
+
*/
|
|
56
|
+
export async function runBatchFirst(adapter, request, policy, testControls) {
|
|
57
|
+
validate(request, policy);
|
|
58
|
+
const eligibility = adapter.eligibility(request);
|
|
59
|
+
if (!eligibility.eligible) {
|
|
60
|
+
throw new BatchFirstIneligibleError(`request is not eligible for batch-first execution: ${eligibility.reason ?? "unsupported by this adapter"}`);
|
|
61
|
+
}
|
|
62
|
+
const clock = testControls?.clock ?? realClock();
|
|
63
|
+
const pollIntervalMs = testControls?.pollIntervalMs ?? DEFAULT_POLL_INTERVAL_MS;
|
|
64
|
+
const startedAt = Date.now();
|
|
65
|
+
const handle = await adapter.submitOne(request);
|
|
66
|
+
let settleDeadline;
|
|
67
|
+
const deadlineSettled = new Promise((resolve) => {
|
|
68
|
+
settleDeadline = resolve;
|
|
69
|
+
});
|
|
70
|
+
const deadlineTimer = clock.setTimeout(() => settleDeadline?.(), policy.deadlineMs);
|
|
71
|
+
// Resolves at most once, and only from inside the poll loop below, the
|
|
72
|
+
// moment a real terminal status is observed — never resolves at all if
|
|
73
|
+
// the deadline wins first. Racing this directly (rather than racing the
|
|
74
|
+
// loop's own completion promise) means there is no "loop stopped without
|
|
75
|
+
// a terminal status" case to fake a result for: that case simply never
|
|
76
|
+
// resolves this promise, which is exactly the semantics Promise.race
|
|
77
|
+
// needs.
|
|
78
|
+
let resolveBatchTerminal;
|
|
79
|
+
const batchTerminalSettled = new Promise((resolve) => {
|
|
80
|
+
resolveBatchTerminal = resolve;
|
|
81
|
+
});
|
|
82
|
+
let stopPolling = false;
|
|
83
|
+
const pollUntilTerminal = (async () => {
|
|
84
|
+
while (!stopPolling) {
|
|
85
|
+
const outcome = await adapter.poll(handle);
|
|
86
|
+
if (outcome.status !== "pending") {
|
|
87
|
+
resolveBatchTerminal?.(outcome.status);
|
|
88
|
+
return outcome.status;
|
|
89
|
+
}
|
|
90
|
+
if (stopPolling)
|
|
91
|
+
return undefined;
|
|
92
|
+
await new Promise((resolve) => clock.setTimeout(resolve, pollIntervalMs));
|
|
93
|
+
}
|
|
94
|
+
return undefined;
|
|
95
|
+
})();
|
|
96
|
+
const winner = await Promise.race([
|
|
97
|
+
batchTerminalSettled.then((status) => ({ kind: "batch", status })),
|
|
98
|
+
deadlineSettled.then(() => ({ kind: "deadline" })),
|
|
99
|
+
]);
|
|
100
|
+
clock.clearTimeout(deadlineTimer);
|
|
101
|
+
// A batch reported "completed" but whose result cannot be read (a
|
|
102
|
+
// missing output file, an item-level provider error, a malformed or
|
|
103
|
+
// missing matching line, a transient read failure) is neither a valid
|
|
104
|
+
// canonical batch result nor grounds to throw out of batchFirst()
|
|
105
|
+
// instead of the promised fallback — it is treated exactly like a batch
|
|
106
|
+
// that reported "failed": exactly one direct fallback, never a second
|
|
107
|
+
// read attempt, never surfaced as a rejection.
|
|
108
|
+
let unreadableCompletedBatch = false;
|
|
109
|
+
if (winner.kind === "batch" && winner.status === "completed") {
|
|
110
|
+
stopPolling = true;
|
|
111
|
+
try {
|
|
112
|
+
const { result } = await adapter.readResult(handle);
|
|
113
|
+
return {
|
|
114
|
+
source: "batch",
|
|
115
|
+
result,
|
|
116
|
+
metadata: {
|
|
117
|
+
execution_mode: "batch_first",
|
|
118
|
+
deadline_ms: policy.deadlineMs,
|
|
119
|
+
batch_wait_ms: Date.now() - startedAt,
|
|
120
|
+
batch_outcome: "completed",
|
|
121
|
+
canonical_result: "batch",
|
|
122
|
+
duplicate_provider_execution: false,
|
|
123
|
+
late_batch_completed: false,
|
|
124
|
+
late_batch_contained_tool_call_plan: false,
|
|
125
|
+
},
|
|
126
|
+
};
|
|
127
|
+
}
|
|
128
|
+
catch {
|
|
129
|
+
unreadableCompletedBatch = true;
|
|
130
|
+
}
|
|
131
|
+
}
|
|
132
|
+
// Either the deadline fired first, the batch reached a non-completed
|
|
133
|
+
// terminal status before the deadline, or the batch completed but its
|
|
134
|
+
// result could not be read — either way, issue exactly one direct
|
|
135
|
+
// fallback now, and never wait further (or retry a read) on the batch
|
|
136
|
+
// for the canonical result. Only stop the poll loop when the batch
|
|
137
|
+
// itself already produced a terminal status (it has nothing left to do,
|
|
138
|
+
// so this is a no-op) — when the DEADLINE won, deliberately leave
|
|
139
|
+
// polling running in the background: "keep polling only to write
|
|
140
|
+
// terminal telemetry" requires the loop to keep going, not stop here.
|
|
141
|
+
const batchOutcomeAtFallback = unreadableCompletedBatch
|
|
142
|
+
? "failed"
|
|
143
|
+
: winner.kind === "batch"
|
|
144
|
+
? winner.status
|
|
145
|
+
: "pending_at_deadline";
|
|
146
|
+
if (winner.kind === "batch")
|
|
147
|
+
stopPolling = true;
|
|
148
|
+
const directResultPromise = adapter.direct(request);
|
|
149
|
+
if (winner.kind === "deadline") {
|
|
150
|
+
// Observe the batch purely for telemetry — never read for its content
|
|
151
|
+
// to be returned, executed, or used to mutate the already-in-flight
|
|
152
|
+
// direct result.
|
|
153
|
+
pollUntilTerminal
|
|
154
|
+
.then(async (status) => {
|
|
155
|
+
if (status !== "completed" && status !== "failed" && status !== "expired")
|
|
156
|
+
return;
|
|
157
|
+
let containedToolCallPlan = false;
|
|
158
|
+
if (status === "completed") {
|
|
159
|
+
try {
|
|
160
|
+
const late = await adapter.readResult(handle);
|
|
161
|
+
containedToolCallPlan = late.containedToolCallPlan;
|
|
162
|
+
}
|
|
163
|
+
catch {
|
|
164
|
+
/* telemetry only — never thrown, never surfaced */
|
|
165
|
+
}
|
|
166
|
+
}
|
|
167
|
+
policy.onLateBatchSettled?.({ outcome: status, containedToolCallPlan });
|
|
168
|
+
})
|
|
169
|
+
.catch(() => { });
|
|
170
|
+
}
|
|
171
|
+
const { result } = await directResultPromise;
|
|
172
|
+
return {
|
|
173
|
+
source: "direct",
|
|
174
|
+
result,
|
|
175
|
+
metadata: {
|
|
176
|
+
execution_mode: "batch_first",
|
|
177
|
+
deadline_ms: policy.deadlineMs,
|
|
178
|
+
batch_wait_ms: Date.now() - startedAt,
|
|
179
|
+
batch_outcome: batchOutcomeAtFallback,
|
|
180
|
+
canonical_result: "direct",
|
|
181
|
+
duplicate_provider_execution: true,
|
|
182
|
+
late_batch_completed: false,
|
|
183
|
+
late_batch_contained_tool_call_plan: false,
|
|
184
|
+
},
|
|
185
|
+
};
|
|
186
|
+
}
|
|
187
|
+
function resolveAdapter(client, provider) {
|
|
188
|
+
if (provider === "openai") {
|
|
189
|
+
return createOpenAIBatchAdapter(client);
|
|
190
|
+
}
|
|
191
|
+
if (provider === "anthropic") {
|
|
192
|
+
return createAnthropicBatchAdapter(client);
|
|
193
|
+
}
|
|
194
|
+
if (provider === "google") {
|
|
195
|
+
return createGoogleBatchAdapter(client);
|
|
196
|
+
}
|
|
197
|
+
throw new BatchFirstIneligibleError(`batchFirst() has no adapter for provider "${provider}" yet`);
|
|
198
|
+
}
|
|
199
|
+
/**
|
|
200
|
+
* Explicit, provider-specific batch-first execution. Never inferred from
|
|
201
|
+
* the client instance — the caller states the provider, matching wrap()'s
|
|
202
|
+
* own explicit-provider option.
|
|
203
|
+
*/
|
|
204
|
+
export async function batchFirst(client, provider, request, policy) {
|
|
205
|
+
const adapter = resolveAdapter(client, provider);
|
|
206
|
+
return runBatchFirst(adapter, request, policy);
|
|
207
|
+
}
|
package/dist/capture.d.ts
CHANGED
|
@@ -4,6 +4,7 @@ export interface RuntimeOptions {
|
|
|
4
4
|
captureText: boolean;
|
|
5
5
|
redact?: (text: string, kind: "request" | "response") => string;
|
|
6
6
|
appRoot: string;
|
|
7
|
+
repoRoot?: string;
|
|
7
8
|
skipFrames: string[];
|
|
8
9
|
environment?: string;
|
|
9
10
|
textMaxBytes: number;
|
|
@@ -12,6 +13,7 @@ interface Frame {
|
|
|
12
13
|
m: string;
|
|
13
14
|
f: string;
|
|
14
15
|
l: number;
|
|
16
|
+
p?: string;
|
|
15
17
|
}
|
|
16
18
|
export interface CallState {
|
|
17
19
|
provider: string;
|
package/dist/capture.js
CHANGED
|
@@ -376,7 +376,7 @@ const SDK_DIR = (() => {
|
|
|
376
376
|
return "metergraph-sdk-dir-unavailable";
|
|
377
377
|
}
|
|
378
378
|
})();
|
|
379
|
-
function frames(stack, appRoot, skip) {
|
|
379
|
+
function frames(stack, appRoot, skip, repoRoot) {
|
|
380
380
|
if (!stack)
|
|
381
381
|
return [];
|
|
382
382
|
const result = [];
|
|
@@ -389,11 +389,17 @@ function frames(stack, appRoot, skip) {
|
|
|
389
389
|
continue;
|
|
390
390
|
if (["node_modules", "node:internal", SDK_DIR, ...skip].some((value) => file.includes(value)))
|
|
391
391
|
continue;
|
|
392
|
-
|
|
392
|
+
const entry = {
|
|
393
393
|
m: file.slice(file.indexOf(appRoot) + appRoot.length).replace(/^\//, ""),
|
|
394
394
|
f: fn,
|
|
395
395
|
l: Number(lineNo),
|
|
396
|
-
}
|
|
396
|
+
};
|
|
397
|
+
const normalizedFile = file.replaceAll("\\", "/").replace(/^file:\/\//, "");
|
|
398
|
+
const normalizedRepoRoot = repoRoot?.replaceAll("\\", "/").replace(/\/$/, "");
|
|
399
|
+
if (normalizedRepoRoot && normalizedFile.startsWith(`${normalizedRepoRoot}/`)) {
|
|
400
|
+
entry.p = normalizedFile.slice(normalizedRepoRoot.length + 1);
|
|
401
|
+
}
|
|
402
|
+
result.push(entry);
|
|
397
403
|
if (result.length === 5)
|
|
398
404
|
break;
|
|
399
405
|
}
|
|
@@ -414,7 +420,7 @@ export class CaptureRuntime {
|
|
|
414
420
|
context,
|
|
415
421
|
started: performance.now(),
|
|
416
422
|
ts: new Date().toISOString(),
|
|
417
|
-
frames: frames(stack, this.options.appRoot, this.options.skipFrames),
|
|
423
|
+
frames: frames(stack, this.options.appRoot, this.options.skipFrames, this.options.repoRoot),
|
|
418
424
|
traceId: context.traceId ?? randomBytes(16).toString("hex"),
|
|
419
425
|
spanId: randomBytes(8).toString("hex"),
|
|
420
426
|
parentSpanId: context.parentSpanId,
|
package/dist/index.d.ts
CHANGED
|
@@ -1,4 +1,5 @@
|
|
|
1
1
|
import { route, setSession, setTags, trace, type RouteOptions, type TraceOptions } from "./context.js";
|
|
2
|
+
import { BatchFirstIneligibleError, batchFirst, type BatchFirstMetadata, type BatchFirstPolicy, type BatchFirstProvider, type BatchFirstRequest, type BatchFirstResult, type BatchFirstSource, type LateBatchInfo } from "./batch-first.js";
|
|
2
3
|
import { track } from "./track.js";
|
|
3
4
|
import { type TransportMode, type WaitUntil } from "./transport.js";
|
|
4
5
|
import { type VercelAISDKMiddleware, type VercelAISDKMiddlewareOptions, type VercelAISDKMiddlewareSpecificationOptions, type VercelAISDKMiddlewareVersionOptions, type VercelAISDKSpecificationVersion } from "./vercel-ai.js";
|
|
@@ -60,3 +61,12 @@ export declare function vercelAISDKMiddleware(options: VercelAISDKMiddlewareVers
|
|
|
60
61
|
export declare function vercelAISDKMiddleware<TVersion extends VercelAISDKSpecificationVersion = "v3">(options?: VercelAISDKMiddlewareSpecificationOptions<TVersion>): VercelAISDKMiddleware<TVersion>;
|
|
61
62
|
export { route, setSession, setTags, trace, track };
|
|
62
63
|
export type { RouteOptions, TraceOptions, TransportMode, VercelAISDKMiddleware, VercelAISDKMiddlewareOptions, VercelAISDKSpecificationVersion, };
|
|
64
|
+
/**
|
|
65
|
+
* Explicit, opt-in batch-first execution over a provider's Batch API —
|
|
66
|
+
* never enabled by wrap()/capture defaults or by an environment variable.
|
|
67
|
+
* See BatchFirstPolicy for the required, explicit acknowledgements
|
|
68
|
+
* (acceptDuplicateProviderExecution, and allowDuplicateToolCallPlans for
|
|
69
|
+
* requests that include tools).
|
|
70
|
+
*/
|
|
71
|
+
export { batchFirst, BatchFirstIneligibleError };
|
|
72
|
+
export type { BatchFirstMetadata, BatchFirstPolicy, BatchFirstProvider, BatchFirstRequest, BatchFirstResult, BatchFirstSource, LateBatchInfo, };
|
package/dist/index.js
CHANGED
|
@@ -1,14 +1,19 @@
|
|
|
1
1
|
import { CaptureRuntime } from "./capture.js";
|
|
2
2
|
import { ConfigPoller } from "./config.js";
|
|
3
3
|
import { contextSnapshot, route, setSession, setTags, trace, } from "./context.js";
|
|
4
|
+
import { BatchFirstIneligibleError, batchFirst, } from "./batch-first.js";
|
|
5
|
+
import { ensureRepoConfig } from "./repo-config.js";
|
|
6
|
+
import { SessionManager } from "./session.js";
|
|
4
7
|
import { track } from "./track.js";
|
|
5
8
|
import { Transport } from "./transport.js";
|
|
9
|
+
import { SDK_VERSION } from "./version.js";
|
|
6
10
|
import { setCaptureRuntime, wrap as wrapProvider } from "./wrap.js";
|
|
7
11
|
import { createVercelAISDKMiddleware, } from "./vercel-ai.js";
|
|
8
12
|
let initialized = false;
|
|
9
13
|
let warnedNoToken = false;
|
|
10
14
|
let transport;
|
|
11
15
|
let config;
|
|
16
|
+
let sessionManager;
|
|
12
17
|
export const DEFAULT_INGEST_URL = "https://d2xus7mp8zdv6t.cloudfront.net";
|
|
13
18
|
function env(name) {
|
|
14
19
|
return typeof process === "undefined" ? undefined : process.env[name];
|
|
@@ -38,16 +43,23 @@ export function init(options = {}) {
|
|
|
38
43
|
}
|
|
39
44
|
initialized = true;
|
|
40
45
|
try {
|
|
46
|
+
const appRoot = options.appRoot ?? (typeof process === "undefined" ? "" : process.cwd());
|
|
47
|
+
const repoConfig = ensureRepoConfig(appRoot);
|
|
48
|
+
sessionManager = repoConfig
|
|
49
|
+
? new SessionManager(token, ingestUrl, repoConfig.repository, SDK_VERSION)
|
|
50
|
+
: undefined;
|
|
41
51
|
transport = new Transport(token, ingestUrl, {
|
|
42
52
|
queueSize: options.queueSize ?? Number(env("METERGRAPH_QUEUE_SIZE") ?? 2_000),
|
|
43
53
|
batchSize: options.batchSize ?? Number(env("METERGRAPH_BATCH_SIZE") ?? 100),
|
|
44
54
|
flushMs: options.flushMs ?? Number(env("METERGRAPH_FLUSH_MS") ?? 5_000),
|
|
45
55
|
mode: options.transport ?? "auto",
|
|
56
|
+
session: sessionManager,
|
|
46
57
|
});
|
|
47
58
|
setCaptureRuntime(new CaptureRuntime(transport, {
|
|
48
59
|
captureText: options.captureText ?? envBool("METERGRAPH_CAPTURE_TEXT", true),
|
|
49
60
|
redact: options.redact,
|
|
50
|
-
appRoot
|
|
61
|
+
appRoot,
|
|
62
|
+
repoRoot: repoConfig?.repoRoot,
|
|
51
63
|
skipFrames: options.skipFrames ?? [],
|
|
52
64
|
environment: options.environment ?? env("METERGRAPH_ENV"),
|
|
53
65
|
textMaxBytes: Math.min(100 * 1024, Math.max(1, Number(env("METERGRAPH_TEXT_MAX_BYTES") ?? 100 * 1024))),
|
|
@@ -57,6 +69,7 @@ export function init(options = {}) {
|
|
|
57
69
|
catch {
|
|
58
70
|
transport = undefined;
|
|
59
71
|
config = undefined;
|
|
72
|
+
sessionManager = undefined;
|
|
60
73
|
setCaptureRuntime();
|
|
61
74
|
console.warn("Metergraph initialization failed; application is running uninstrumented");
|
|
62
75
|
}
|
|
@@ -116,6 +129,8 @@ export async function shutdown() {
|
|
|
116
129
|
config = undefined;
|
|
117
130
|
await transport?.shutdown();
|
|
118
131
|
transport = undefined;
|
|
132
|
+
sessionManager?.stop();
|
|
133
|
+
sessionManager = undefined;
|
|
119
134
|
setCaptureRuntime();
|
|
120
135
|
}
|
|
121
136
|
export function wrapHandler(handler) {
|
|
@@ -144,3 +159,11 @@ export function vercelAISDKMiddleware(options = {}) {
|
|
|
144
159
|
return createVercelAISDKMiddleware(options);
|
|
145
160
|
}
|
|
146
161
|
export { route, setSession, setTags, trace, track };
|
|
162
|
+
/**
|
|
163
|
+
* Explicit, opt-in batch-first execution over a provider's Batch API —
|
|
164
|
+
* never enabled by wrap()/capture defaults or by an environment variable.
|
|
165
|
+
* See BatchFirstPolicy for the required, explicit acknowledgements
|
|
166
|
+
* (acceptDuplicateProviderExecution, and allowDuplicateToolCallPlans for
|
|
167
|
+
* requests that include tools).
|
|
168
|
+
*/
|
|
169
|
+
export { batchFirst, BatchFirstIneligibleError };
|