margin-meter 0.2.0 → 0.4.0

package/README.md

@@ -1,5 +1,11 @@
 # margin-meter (TypeScript SDK)
 
+> **You probably do not need this to start.** Margin's first step installs nothing: sign in at
+> [trymargin.io/console](https://trymargin.io/console), connect the repository your agents live
+> in (or drop the folder), and press Execute. Each agent runs on the evals beside it and every
+> call is metered. This SDK is the **optional** route for metering your live production traffic,
+> once you want that as well ([`/start#production`](https://trymargin.io/start#production)).
+
 The tiny client a **TypeScript/JavaScript** project imports to connect to
 Margin — the TS twin of the Python [`margin-meter`](../python) package. It wraps
 your LLM calls and records their outcomes, emitting each one **over HTTP** to a
@@ -7,6 +13,10 @@ Margin **ingest API** (`POST /api/ingest/calls` / `/api/ingest/outcomes`),
 authenticated with a per-project ingest key.
 
 - **Dependency-free.** Uses the global `fetch` — nothing to install at runtime.
+- **Off your hot path.** Calls are buffered and flushed in batches, so your
+  agent's latency never depends on Margin being up. See
+  [Batching](#batching-what-it-does-to-your-process) — read it before you
+  integrate, because the guarantees and their costs are both stated there.
 - **Fail-safe.** A failed emit resolves to `{ ok: false, … }` instead of throwing.
   Pass `raiseOnError: true` for strict/CI behaviour.
 - **Provenance-honest.** `isSimulated` is carried through untouched; the written
@@ -39,7 +49,7 @@ build via `prepublishOnly`).
 Two environment variables — the deployed API base and your project's key:
 
 ```bash
-export MARGIN_INGEST_URL="https://margin-ai-rho.vercel.app"
+export MARGIN_INGEST_URL="https://trymargin.io"
 export MARGIN_INGEST_KEY="mgk_…"   # issued by the Margin owner
 ```
 
@@ -86,6 +96,115 @@ const res = await meter.recordCall({
 if (!res.ok) console.warn(`margin ingest failed: ${res.error} (${res.statusCode})`);
 ```
 
+## Say what each call was FOR (`taskKey`)
+
+`workflowId` names the pipeline and `operation` names the step inside it. Neither
+says what the work actually WAS — a billing question and a password reset look
+identical on both. `taskKey` is that third axis, in your own vocabulary:
+
+```ts
+await meter.recordCall({
+  workflowId: "support-triage",
+  provider: "openai",
+  model: "gpt-4o-mini",
+  inputTokens: 812,
+  outputTokens: 96,
+  taskKey: ticket.topic, // "billing" | "password-reset" | whatever you already use
+});
+```
+
+Why it earns a field. Margin measures a cheaper route's parity on a sample and
+records what that sample was made of. If the distribution of arriving work moves —
+new ticket topics, a mix that shifts as you grow — **parity decays with no model
+change at all**, and nothing that watches for a route getting worse will fire,
+because nothing got worse. With `taskKey` flowing, Margin compares the measured mix
+against the arriving one and says `steady` or `moved`. Without it the answer is
+`unknown`, which is what the console shows and is the honest state, not a bug.
+
+It is never priced and never rolled into cost. Send it on more than 60% of a
+workflow's calls and the comparison becomes answerable; below that Margin declines
+to rule rather than describing your instrumented slice as if it were your traffic.
+
+## Say where each call RAN (`environment`)
+
+`environment` splits the work you sell from the work you did to decide what to
+sell. A spike, an eval pass and your production traffic are three different kinds
+of spend, and only one of them belongs in the number you report upward.
+
+```ts
+await meter.recordCall({
+  workflowId: "support-triage",
+  provider: "openai",
+  model: "gpt-4o-mini",
+  inputTokens: 812,
+  outputTokens: 96,
+  environment: "prod", // "prod" | "dev" | "eval"
+});
+```
+
+| value | send it for |
+| --- | --- |
+| `prod` | traffic your customers cause |
+| `dev` | a spike, a local run, a branch deploy |
+| `eval` | a graded pass, in CI or from an eval harness |
+
+Why it earns a field. `costPerOutcome` is total call spend divided by passed
+outcomes. With `environment` unset, your R&D and your operations land in that one
+number, so the headline describes neither. Margin already keeps its own face-off
+spend out of your number; this is how you get the same split on your side.
+
+It does not default. Leave it off and the row is `unknown`, which is honest and
+costs you nothing, where guessing `prod` would book your spike as production. It
+is never priced and never rolled into cost. It is a label, like `team`.
+
+## Multi-agent pipelines (name each stage)
+
+A crew of agents — a planner→coder hand-off, a LangChain pipeline — runs under one
+workflow. Auto-instrumentation binds a single `operation` at wire-up, so every
+agent would meter under the same label and the run reads as one blob. `stage()`
+splits it: wrap the current step and every metered call inside — however many
+`await`s deep — is attributed to that stage, under a shared run id. It is the twin
+of the Python SDK's `stage()`; because TS has no `with`, it takes the block as a
+callback and returns whatever the callback returns.
+
+```ts
+import { MarginMeter, stage } from "margin-meter";
+
+const meter = new MarginMeter();
+const runId = "run-42";
+
+await stage("classify", { taskClass: "signal-check", sessionId: runId }, () =>
+  classifier.run(ticket),          // its calls meter as operation="classify"
+);
+await stage("implement", { taskClass: "algorithm", sessionId: runId }, () =>
+  implementer.run(plan),           // ... and these as operation="implement"
+);
+```
+
+The two stages share `sessionId` but carry distinct operations, so the run is a
+real multi-stage pipeline the Money Map decomposes — and `taskClass`, declared per
+step, is what lets `margin_stage_plan` recommend a model and effort for each. It
+uses `AsyncLocalStorage` (the direct analogue of Python's contextvars), so it stays
+correct across `await` and concurrent runs; it re-labels and links only — tokens,
+cost and the outcome are untouched.
+
+Nest `stage()`/`span()` to build the tree, and reach for `span()` when a block fans
+out and you want to record the orchestration shape:
+
+```ts
+import { span } from "margin-meter";
+
+await span({ pattern: "panel", branchWidth: 3, stoppingRule: "majority of 3" }, () =>
+  Promise.all(candidates.map((c) => judge(c))),   // each judge nests its own node
+);
+```
+
+One caveat carried over from the Python side: `AsyncLocalStorage` does not cross a
+worker boundary (as a contextvar does not cross a bare thread). A branch you run on
+a worker must pass an explicit `parentSpanId` on the call or in the options, or it
+records as a root — which reads as "no chain happened" rather than the missing edge
+it is.
+
 ## Auto-instrument (a few lines for a whole repo)
 
 To instrument an existing agent you didn't write, don't edit every call site.
@@ -112,6 +231,17 @@ undo(); // restore the original method
 (recovering the model id off the instance). Each returns an `undo()` and throws
 `MarginInstrumentError` if the client shape has moved.
 
+To flow `taskKey` (above) through the auto-instrument path, pass a **resolver, not
+a value** — one wrapper meters a whole run, so the unit of work is read at call
+time, never baked as a constant (a constant would collapse the run into one class):
+
+```ts
+const undo = instrumentOpenAI(client, meter, {
+  workflowId: "support-triage",
+  resolveTaskKey: () => currentTicket()?.topic, // read per call; undefined ⇒ unknown
+});
+```
+
 Two lower-level primitives when you want tighter control:
 
 ```ts
@@ -131,6 +261,97 @@ cache reads are billed separately, never double-counted as fresh input — and a
 metering failure is fail-safe (your call still returns / still throws). A failed
 provider call is still recorded with `status:"error"` so it appears in the chain.
 
+## Report your evals (promptfoo)
+
+Your eval suite already grades pass/fail — that is the outcome signal Margin's
+cost-/value-per-outcome math divides by. If you run [promptfoo](https://promptfoo.dev),
+point this adapter at the file it already writes; no hand-written POST:
+
+```bash
+promptfoo eval -o results.json
+```
+
+```ts
+import { readFile } from "node:fs/promises";
+import { MarginMeter, promptfoo } from "margin-meter";
+
+const meter = new MarginMeter();                       // MARGIN_INGEST_URL/KEY from env
+const data = JSON.parse(await readFile("results.json", "utf8"));
+const { recorded, skipped, errors } = await promptfoo.recordResults(meter, data, "my-eval-suite");
+```
+
+One promptfoo result becomes one outcome. `success` is the pass/fail; `score` rides
+through when it is in `[0, 1]`. The grade's provenance is **derived from the assert
+types, never assumed** — a code assert (`contains`, `equals`, `regex`, …) records
+`ground_truth`/`code`, any model-graded assert (`llm-rubric`, `factuality`,
+`similar`, …) records `llm_judge`/`judge`, and the weaker honest claim always wins,
+so a rubric grade is never reported as ground truth. Tag a case with
+`vars.margin_value: <usd>` (or `testCase.metadata.margin_value`) to report the
+outcome's worth — the value-per-outcome numerator; absent is the honest
+cost-per-outcome default, never `0`. A row with no boolean `success` is not a
+verdict and is skipped rather than invented. Same contract, field for field, as the
+Python SDK's `margin_meter.promptfoo`.
+
+Run [Braintrust](https://braintrust.dev) instead? Same shape, one difference — Braintrust
+scores are continuous, so a pass is derived rather than intrinsic:
+
+```ts
+import { MarginMeter, braintrust } from "margin-meter";
+
+const meter = new MarginMeter();
+const data = JSON.parse(await readFile("results.json", "utf8"));
+// a case passes when its mean score clears the threshold …
+await braintrust.recordResults(meter, data, "my-suite", { threshold: 0.7 });
+// … or, with no threshold, only cases whose scorers are all binary (0/1) are
+// recorded; a continuous-score case is skipped, never given a fabricated verdict.
+```
+
+`qualityScore` is the mean of the case's per-scorer scores; the grade's provenance is
+derived from the scorer names (a model autoeval like `Factuality` → `llm_judge`/`judge`,
+a code scorer like `ExactMatch` → `ground_truth`/`code`). Tag `metadata.margin_value`
+for rung 8, and `metadata.margin_judge_model` to record which model judged (a bare
+`model` field is the generation model under test, not the judge, so it is never used).
+
+Your unit tests are already an eval harness — a passing test is a code-graded, ground-truth
+pass/fail. Point the adapter at the JSON your test runner already writes and every test
+becomes an outcome, no hand-written POST. One adapter covers both **vitest** and **jest**
+(they emit the same jest-compatible JSON):
+
+```ts
+import { MarginMeter, vitest } from "margin-meter";
+// vitest run --reporter=json --outputFile=results.json   (or: jest --json --outputFile=results.json)
+
+const meter = new MarginMeter();
+const data = JSON.parse(await readFile("results.json", "utf8"));
+const { recorded, skipped } = await vitest.recordResults(meter, data, "my-test-suite");
+```
+
+A `passed` assertion records as `ground_truth`/`code` with `qualityScore` 1.0, a `failed`
+one 0.0; a `skipped`/`todo`/`pending` test is **not a verdict** and is skipped, never
+fabricated. The `link` traces back to the test's `fullName`.
+
+The JSON file carries no per-test value, so that path is cost-per-outcome only. To attach a
+**value** (value-per-outcome, the way pytest uses `@pytest.mark.margin_value`), use the runtime
+reporter instead — it runs inside vitest and reads `task.meta.marginValue`:
+
+```ts
+// vitest.config.ts
+import { vitest as m } from "margin-meter";
+export default {
+  test: { reporters: ["default", new m.MarginVitestReporter({ workflowId: "my-test-suite" })] },
+};
+
+// in a test — tag its business value (optional; absent = cost-per-outcome)
+import { getCurrentTest } from "vitest";
+test("resolves the ticket", () => {
+  getCurrentTest().meta.marginValue = 120; // this outcome is worth $120
+  // ...
+});
+```
+
+The reporter is a no-op unless a `workflowId` (or `MARGIN_VITEST_WORKFLOW`) is set, and it never
+throws into your test run — a delivery fault warns and is counted.
+
 ## API
 
 | Method | Emits to | Notes |
@@ -141,9 +362,78 @@ provider call is still recorded with `status:"error"` so it appears in the chain
 | `recordResponse(input)` | `recordCall` | meters straight off a provider response — no manual tokens |
 | `wrap(fn, opts)` | `recordCall` per call | returns an auto-metered drop-in for any LLM call |
 | `instrumentOpenAI` / `instrumentAnthropic` / `instrumentGemini` | — | monkeypatch a client instance's create method |
+| `stage(op, opts?, fn)` | — | attributes every call inside `fn` to one pipeline step under a shared run id; opens a chain node; returns `fn`'s value |
+| `span(opts, fn)` | — | marks the calls inside `fn` as one node of an agent tree, with the orchestration shape; nest to build the tree |
+| `currentStage()` / `currentSpan()` | — | the active step / tree node, or `undefined` outside any |
+
+| `deliveryStats()` | — | what this meter delivered against what it tried to (see below) |
 
-Every method resolves to an `IngestResult { ok, statusCode, body?, error? }`.
+Every method resolves to an `IngestResult { ok, statusCode, body?, error?, reason? }`.
 `ok` is true only on HTTP 200; `body` holds `call_id`/`outcome_id` + `source`.
+`reason` is the failure as a stable bucket (`no_key`, `transport`,
+`http_<status>`) — read that rather than parsing `error`, whose wording is prose
+and will move.
+
+## Batching (what it does to your process)
+
+Calls are **buffered and flushed in batches** — 100 records or 10 seconds,
+whichever comes first. `recordCall()` pushes onto a bounded queue and resolves;
+the POST happens off your await chain, driven by an `unref`'d timer that never
+holds your process open.
+
+Why it works this way: at 1M metered calls/day, one call per HTTP request is 30M
+serverless invocations a month and the same traffic batched at 100 is 300K. That
+100x is Margin's cost, not yours, which is exactly why it belongs in the SDK
+rather than the server — only the client can decide not to make the request.
+
+| knob | default | env override |
+| --- | ---: | --- |
+| records per flush | 100 | `MARGIN_BATCH_MAX_RECORDS` |
+| seconds between flushes | 10 | `MARGIN_BATCH_FLUSH_SECONDS` |
+| queued records (memory ceiling) | 10,000 | `MARGIN_BATCH_MAX_QUEUE` |
+
+**What you are guaranteed:** metering never blocks your call path, never throws
+into it, and cannot grow without bound in your process.
+
+**What it costs you, stated plainly:**
+
+- `recordCall()` resolves to `{ ok: true, body: { queued: true } }` — accepted
+  into the buffer, *not* written to the ledger. `deliveryStats()` is where you
+  learn what landed.
+- A hard kill loses at most 100 records or 10 seconds of them. `beforeExit`
+  covers a normal exit; no in-memory buffer survives `SIGKILL`.
+- If Margin is unreachable long enough to fill the queue, the **newest** records
+  are dropped, and every drop is counted under `queue_full`.
+- Delivery is at-least-once. Every record carries an `event_id` and the server is
+  idempotent on it, so a redelivery never double-counts your spend.
+
+`await meter.flush()` resolves once everything queued has been delivered or
+counted. `await meter.close()` drains and stops the timer. `raiseOnError: true`
+skips the buffer entirely and posts at the call site, because strict mode's whole
+meaning is "throw where I called you".
+
+Full contract, including the arithmetic behind every default:
+[`docs/reference/SDK_BATCHING.md`](../../docs/reference/SDK_BATCHING.md).
+
+## Knowing what you dropped
+
+Metering is fail-safe: a failed emit resolves to `ok: false` and your call carries
+on. That is the right trade — telemetry should never take your agent down — but it
+means a lost row looks, from the ledger, exactly like a call that never happened.
+Cost per outcome then reads cheaper than your bill.
+
+So the meter counts its own losses:
+
+```ts
+const s = meter.deliveryStats();
+console.log(s.attempted, s.delivered, s.dropped, s.dropRate);
+console.log(s.droppedByReason); // { http_502: 3, transport: 1 }
+```
+
+The Python SDK's `delivery_stats()` reports two more fields, `deduped` and
+`unmeasured`. Neither applies here: this client has no file sink to lose buffered
+rows in and no cross-hook de-dup to suppress a second hook, so every emit it makes
+goes through one door and is counted there.
 
 ## Testing against the ingest contract (no network)
 

package/dist/batching.d.ts

@@ -0,0 +1,94 @@
+/**
+ * The buffer that decides WHEN a metered row goes over the wire.
+ *
+ * The full contract, with the arithmetic behind every default, is
+ * `docs/reference/SDK_BATCHING.md`, and this file is the TS twin of
+ * `margin_meter/batching.py` — the two must agree, because they are one contract
+ * a customer integrates against, not two implementations.
+ *
+ * The short version: at 1M metered calls/day, one call per HTTP request is 30M
+ * Vercel invocations a month and the same traffic batched at 100 is 300K. 100x on
+ * a line item Margin pays. It lives in the SDK rather than the server because
+ * only the client can decide not to make the request.
+ *
+ * ⛔ Two properties are not negotiable, and both are about the customer's process
+ * rather than our data:
+ *
+ * 1. **Nothing blocks the caller.** `submit` pushes onto an array and returns.
+ *    There is no thread here and none is needed — the flush runs off the caller's
+ *    await chain, driven by an `unref`'d timer so it never holds a process open.
+ * 2. **The queue is BOUNDED and drops the NEWEST.** An unbounded buffer behind a
+ *    dead endpoint is an OOM with our name on it, and taking down a customer's
+ *    production agent because our meter was unavailable is the worst thing this
+ *    SDK could do. Drop-newest leaves a contiguous prefix and a gap at the tail,
+ *    which reads as "we stopped recording at 14:20"; drop-oldest leaves a hole in
+ *    the middle of history, which reads as "they spent less that hour" — an
+ *    under-count in the flattering direction wearing the costume of a measurement.
+ */
+export declare const BATCH_PATH = "/api/ingest/batch";
+/** Flush on 100 records OR 10 seconds, whichever comes first. The crossover is
+ *  100 ÷ 10 = 10 calls/sec: above it size fires first and invocations scale as
+ *  volume ÷ 100; below it the timer governs and invocations cap at 8,640/day
+ *  however quiet the customer is. Each trigger covers the other's failure — size
+ *  alone stalls forever on a low-volume customer, time alone spends a fixed
+ *  ~518K invocations/month whatever the traffic. */
+export declare const DEFAULT_MAX_RECORDS = 100;
+export declare const DEFAULT_FLUSH_MS = 10000;
+/** ~14 minutes of buffered traffic at 11.6 calls/sec — enough to ride out a
+ *  deploy or a short outage of ours. A row is 400–1500 bytes, so the ceiling on
+ *  the host's memory is 4–15 MB, which is the number to quote a customer. */
+export declare const DEFAULT_MAX_QUEUE = 10000;
+/** Bounded so a shutdown flush can never hang the host's exit. */
+export declare const SHUTDOWN_TIMEOUT_MS = 5000;
+/** One buffered emit: which ingest leg it belongs to, and its built payload. */
+export interface BatchRecord {
+    path: string;
+    payload: Record<string, unknown>;
+}
+export interface BatcherOptions {
+    maxRecords?: number;
+    flushMs?: number;
+    maxQueue?: number;
+}
+/**
+ * A bounded queue plus a timer that flushes it on size or time.
+ *
+ * `send` is awaited off the caller's path and is expected never to reject — it
+ * owns the POST, the retries and the accounting.
+ */
+export declare class Batcher {
+    readonly maxRecords: number;
+    readonly flushMs: number;
+    readonly maxQueue: number;
+    private queue;
+    private timer;
+    private inFlight;
+    /** True once `close()` has run. Public so a caller can report a refused emit
+     *  under the reason that actually caused it: "the meter was shut down" and "the
+     *  buffer is full" have opposite fixes, and filing one as the other sends the
+     *  reader hunting for a network problem that is not there. */
+    closed: boolean;
+    private readonly send;
+    constructor(send: (batch: BatchRecord[]) => Promise<void>, options?: BatcherOptions);
+    /** Queue one record. `false` when the queue was full and it was DROPPED.
+     *
+     *  ⛔ A bounded push and a size check are the entire cost on the caller's path.
+     *  Nothing here is awaited. */
+    submit(record: BatchRecord): boolean;
+    /** Start the interval timer if it is not already running.
+     *
+     *  ⭐ `unref`'d, which is what lets a short-lived script exit on its own
+     *  schedule instead of waiting out our timer — and what makes `beforeExit`
+     *  fire at all, since a ref'd timer would keep the event loop alive forever. */
+    private arm;
+    /** Send everything queued when this was called, and resolve when it has been
+     *  delivered or counted as dropped. Never rejects. */
+    flush(): Promise<void>;
+    /** Stop accepting, drain what is queued, and return anything left behind.
+     *
+     *  The stranded records are handed BACK rather than swallowed so the caller can
+     *  count them: an abandoned buffer nobody counts is exactly the invisible loss
+     *  the delivery accounting exists to make visible. */
+    close(timeoutMs?: number): Promise<BatchRecord[]>;
+}
+//# sourceMappingURL=batching.d.ts.map

package/dist/batching.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"batching.d.ts","sourceRoot":"","sources":["../src/batching.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;GA0BG;AAEH,eAAO,MAAM,UAAU,sBAAsB,CAAC;AAE9C;;;;;oDAKoD;AACpD,eAAO,MAAM,mBAAmB,MAAM,CAAC;AACvC,eAAO,MAAM,gBAAgB,QAAS,CAAC;AAEvC;;6EAE6E;AAC7E,eAAO,MAAM,iBAAiB,QAAS,CAAC;AAExC,kEAAkE;AAClE,eAAO,MAAM,mBAAmB,OAAQ,CAAC;AAEzC,gFAAgF;AAChF,MAAM,WAAW,WAAW;IAC1B,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;CAClC;AAED,MAAM,WAAW,cAAc;IAC7B,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,QAAQ,CAAC,EAAE,MAAM,CAAC;CACnB;AAgBD;;;;;GAKG;AACH,qBAAa,OAAO;IAClB,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAC;IAC5B,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;IACzB,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,OAAO,CAAC,KAAK,CAAqB;IAClC,OAAO,CAAC,KAAK,CAA8C;IAC3D,OAAO,CAAC,QAAQ,CAAoC;IACpD;;;kEAG8D;IAC9D,MAAM,UAAS;IAKf,OAAO,CAAC,QAAQ,CAAC,IAAI,CAA0C;gBAG7D,IAAI,EAAE,CAAC,KAAK,EAAE,WAAW,EAAE,KAAK,OAAO,CAAC,IAAI,CAAC,EAC7C,OAAO,GAAE,cAAmB;IAgB9B;;;mCAG+B;IAC/B,MAAM,CAAC,MAAM,EAAE,WAAW,GAAG,OAAO;IAiBpC;;;;oFAIgF;IAChF,OAAO,CAAC,GAAG;IAYX;0DACsD;IAChD,KAAK,IAAI,OAAO,CAAC,IAAI,CAAC;IAsB5B;;;;0DAIsD;IAChD,KAAK,CAAC,SAAS,GAAE,MAA4B,GAAG,OAAO,CAAC,WAAW,EAAE,CAAC;CAqB7E"}

package/dist/batching.js

@@ -0,0 +1,181 @@
+/**
+ * The buffer that decides WHEN a metered row goes over the wire.
+ *
+ * The full contract, with the arithmetic behind every default, is
+ * `docs/reference/SDK_BATCHING.md`, and this file is the TS twin of
+ * `margin_meter/batching.py` — the two must agree, because they are one contract
+ * a customer integrates against, not two implementations.
+ *
+ * The short version: at 1M metered calls/day, one call per HTTP request is 30M
+ * Vercel invocations a month and the same traffic batched at 100 is 300K. 100x on
+ * a line item Margin pays. It lives in the SDK rather than the server because
+ * only the client can decide not to make the request.
+ *
+ * ⛔ Two properties are not negotiable, and both are about the customer's process
+ * rather than our data:
+ *
+ * 1. **Nothing blocks the caller.** `submit` pushes onto an array and returns.
+ *    There is no thread here and none is needed — the flush runs off the caller's
+ *    await chain, driven by an `unref`'d timer so it never holds a process open.
+ * 2. **The queue is BOUNDED and drops the NEWEST.** An unbounded buffer behind a
+ *    dead endpoint is an OOM with our name on it, and taking down a customer's
+ *    production agent because our meter was unavailable is the worst thing this
+ *    SDK could do. Drop-newest leaves a contiguous prefix and a gap at the tail,
+ *    which reads as "we stopped recording at 14:20"; drop-oldest leaves a hole in
+ *    the middle of history, which reads as "they spent less that hour" — an
+ *    under-count in the flattering direction wearing the costume of a measurement.
+ */
+export const BATCH_PATH = "/api/ingest/batch";
+/** Flush on 100 records OR 10 seconds, whichever comes first. The crossover is
+ *  100 ÷ 10 = 10 calls/sec: above it size fires first and invocations scale as
+ *  volume ÷ 100; below it the timer governs and invocations cap at 8,640/day
+ *  however quiet the customer is. Each trigger covers the other's failure — size
+ *  alone stalls forever on a low-volume customer, time alone spends a fixed
+ *  ~518K invocations/month whatever the traffic. */
+export const DEFAULT_MAX_RECORDS = 100;
+export const DEFAULT_FLUSH_MS = 10_000;
+/** ~14 minutes of buffered traffic at 11.6 calls/sec — enough to ride out a
+ *  deploy or a short outage of ours. A row is 400–1500 bytes, so the ceiling on
+ *  the host's memory is 4–15 MB, which is the number to quote a customer. */
+export const DEFAULT_MAX_QUEUE = 10_000;
+/** Bounded so a shutdown flush can never hang the host's exit. */
+export const SHUTDOWN_TIMEOUT_MS = 5_000;
+/** Read a positive tunable from the environment, ignoring nonsense.
+ *
+ *  A malformed or non-positive value falls back to the default rather than
+ *  throwing: a typo in a deployment variable must never stop a customer's process
+ *  from starting, and a zero flush interval would be a busy loop. */
+function envNumber(name, fallback, minimum) {
+    const env = (typeof process !== "undefined" && process.env) || {};
+    const raw = env[name];
+    if (raw === undefined)
+        return fallback;
+    const value = Number(raw);
+    if (!Number.isFinite(value) || value < minimum)
+        return fallback;
+    return value;
+}
+/**
+ * A bounded queue plus a timer that flushes it on size or time.
+ *
+ * `send` is awaited off the caller's path and is expected never to reject — it
+ * owns the POST, the retries and the accounting.
+ */
+export class Batcher {
+    maxRecords;
+    flushMs;
+    maxQueue;
+    queue = [];
+    timer = null;
+    inFlight = Promise.resolve();
+    /** True once `close()` has run. Public so a caller can report a refused emit
+     *  under the reason that actually caused it: "the meter was shut down" and "the
+     *  buffer is full" have opposite fixes, and filing one as the other sends the
+     *  reader hunting for a network problem that is not there. */
+    closed = false;
+    // ⚠️ Declared rather than written as a constructor parameter property: Node's
+    // strip-only TypeScript mode (which this SDK's own test runner uses) rejects
+    // parameter properties outright, and a package that cannot be loaded by the
+    // runtime its tests run on is not testable.
+    send;
+    constructor(send, options = {}) {
+        this.send = send;
+        this.maxRecords = Math.floor(options.maxRecords ?? envNumber("MARGIN_BATCH_MAX_RECORDS", DEFAULT_MAX_RECORDS, 1));
+        // The env var is in SECONDS so the two SDKs read the same variable the same
+        // way; a deployment should not have to remember which one wants milliseconds.
+        this.flushMs =
+            options.flushMs ??
+                envNumber("MARGIN_BATCH_FLUSH_SECONDS", DEFAULT_FLUSH_MS / 1000, 0.001) * 1000;
+        this.maxQueue = Math.floor(options.maxQueue ?? envNumber("MARGIN_BATCH_MAX_QUEUE", DEFAULT_MAX_QUEUE, 1));
+    }
+    /** Queue one record. `false` when the queue was full and it was DROPPED.
+     *
+     *  ⛔ A bounded push and a size check are the entire cost on the caller's path.
+     *  Nothing here is awaited. */
+    submit(record) {
+        if (this.closed)
+            return false;
+        if (this.queue.length >= this.maxQueue) {
+            // Drop the NEWEST — see the module comment. The rows already queued have
+            // waited longer and, under a sustained outage, are the ones whose loss
+            // would put a hole in the middle of the ledger rather than at its end.
+            return false;
+        }
+        this.queue.push(record);
+        if (this.queue.length >= this.maxRecords) {
+            void this.flush();
+            return true;
+        }
+        this.arm();
+        return true;
+    }
+    /** Start the interval timer if it is not already running.
+     *
+     *  ⭐ `unref`'d, which is what lets a short-lived script exit on its own
+     *  schedule instead of waiting out our timer — and what makes `beforeExit`
+     *  fire at all, since a ref'd timer would keep the event loop alive forever. */
+    arm() {
+        if (this.timer !== null || this.closed)
+            return;
+        const timer = setTimeout(() => {
+            this.timer = null;
+            void this.flush();
+        }, this.flushMs);
+        if (typeof timer.unref === "function") {
+            timer.unref();
+        }
+        this.timer = timer;
+    }
+    /** Send everything queued when this was called, and resolve when it has been
+     *  delivered or counted as dropped. Never rejects. */
+    async flush() {
+        if (this.timer !== null) {
+            clearTimeout(this.timer);
+            this.timer = null;
+        }
+        // Chained rather than parallel: two overlapping flushes would interleave
+        // batches and make "everything queued when I called" unanswerable.
+        const previous = this.inFlight;
+        this.inFlight = (async () => {
+            await previous;
+            while (this.queue.length > 0) {
+                const batch = this.queue.splice(0, this.maxRecords);
+                try {
+                    await this.send(batch);
+                }
+                catch {
+                    /* `send` owns its own accounting; a throw here must not stop the drain */
+                }
+            }
+        })();
+        await this.inFlight;
+    }
+    /** Stop accepting, drain what is queued, and return anything left behind.
+     *
+     *  The stranded records are handed BACK rather than swallowed so the caller can
+     *  count them: an abandoned buffer nobody counts is exactly the invisible loss
+     *  the delivery accounting exists to make visible. */
+    async close(timeoutMs = SHUTDOWN_TIMEOUT_MS) {
+        this.closed = true;
+        if (this.timer !== null) {
+            clearTimeout(this.timer);
+            this.timer = null;
+        }
+        let timer;
+        const bounded = new Promise((resolve) => {
+            timer = setTimeout(resolve, timeoutMs);
+            if (typeof timer.unref === "function") {
+                timer.unref();
+            }
+        });
+        // ⛔ Raced against a timeout so a wedged endpoint can never hang the host's
+        // exit. Whatever is still queued when the race ends is returned, not lost.
+        await Promise.race([this.flush(), bounded]);
+        if (timer !== undefined)
+            clearTimeout(timer);
+        const stranded = this.queue;
+        this.queue = [];
+        return stranded;
+    }
+}
+//# sourceMappingURL=batching.js.map

package/dist/batching.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"batching.js","sourceRoot":"","sources":["../src/batching.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;GA0BG;AAEH,MAAM,CAAC,MAAM,UAAU,GAAG,mBAAmB,CAAC;AAE9C;;;;;oDAKoD;AACpD,MAAM,CAAC,MAAM,mBAAmB,GAAG,GAAG,CAAC;AACvC,MAAM,CAAC,MAAM,gBAAgB,GAAG,MAAM,CAAC;AAEvC;;6EAE6E;AAC7E,MAAM,CAAC,MAAM,iBAAiB,GAAG,MAAM,CAAC;AAExC,kEAAkE;AAClE,MAAM,CAAC,MAAM,mBAAmB,GAAG,KAAK,CAAC;AAczC;;;;qEAIqE;AACrE,SAAS,SAAS,CAAC,IAAY,EAAE,QAAgB,EAAE,OAAe;IAChE,MAAM,GAAG,GAAG,CAAC,OAAO,OAAO,KAAK,WAAW,IAAI,OAAO,CAAC,GAAG,CAAC,IAAI,EAAE,CAAC;IAClE,MAAM,GAAG,GAAG,GAAG,CAAC,IAAI,CAAC,CAAC;IACtB,IAAI,GAAG,KAAK,SAAS;QAAE,OAAO,QAAQ,CAAC;IACvC,MAAM,KAAK,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC;IAC1B,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,KAAK,CAAC,IAAI,KAAK,GAAG,OAAO;QAAE,OAAO,QAAQ,CAAC;IAChE,OAAO,KAAK,CAAC;AACf,CAAC;AAED;;;;;GAKG;AACH,MAAM,OAAO,OAAO;IACT,UAAU,CAAS;IACnB,OAAO,CAAS;IAChB,QAAQ,CAAS;IAClB,KAAK,GAAkB,EAAE,CAAC;IAC1B,KAAK,GAAyC,IAAI,CAAC;IACnD,QAAQ,GAAkB,OAAO,CAAC,OAAO,EAAE,CAAC;IACpD;;;kEAG8D;IAC9D,MAAM,GAAG,KAAK,CAAC;IACf,8EAA8E;IAC9E,6EAA6E;IAC7E,4EAA4E;IAC5E,4CAA4C;IAC3B,IAAI,CAA0C;IAE/D,YACE,IAA6C,EAC7C,UAA0B,EAAE;QAE5B,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;QACjB,IAAI,CAAC,UAAU,GAAG,IAAI,CAAC,KAAK,CAC1B,OAAO,CAAC,UAAU,IAAI,SAAS,CAAC,0BAA0B,EAAE,mBAAmB,EAAE,CAAC,CAAC,CACpF,CAAC;QACF,4EAA4E;QAC5E,8EAA8E;QAC9E,IAAI,CAAC,OAAO;YACV,OAAO,CAAC,OAAO;gBACf,SAAS,CAAC,4BAA4B,EAAE,gBAAgB,GAAG,IAAI,EAAE,KAAK,CAAC,GAAG,IAAI,CAAC;QACjF,IAAI,CAAC,QAAQ,GAAG,IAAI,CAAC,KAAK,CACxB,OAAO,CAAC,QAAQ,IAAI,SAAS,CAAC,wBAAwB,EAAE,iBAAiB,EAAE,CAAC,CAAC,CAC9E,CAAC;IACJ,CAAC;IAED;;;mCAG+B;IAC/B,MAAM,CAAC,MAAmB;QACxB,IAAI,IAAI,CAAC,MAAM;YAAE,OAAO,KAAK,CAAC;QAC9B,IAAI,IAAI,CAAC,KAAK,CAAC,MAAM,IAAI,IAAI,CAAC,QAAQ,EAAE,CAAC;YACvC,yEAAyE;YACzE,uEAAuE;YACvE,uEAAuE;YACvE,OAAO,KAAK,CAAC;QACf,CAAC;QACD,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACxB,IAAI,IAAI,CAAC,KAAK,CAAC,MAAM,IAAI,IAAI,CAAC,UAAU,EAAE,CAAC;YACzC,KAAK,IAAI,CAAC,KAAK,EAAE,CAAC;YAClB,OAAO,IAAI,CAAC;QACd,CAAC;QACD,IAAI,CAAC,GAAG,EAAE,CAAC;QACX,OAAO,IAAI,CAAC;IACd,CAAC;IAED;;;;oFAIgF;IACxE,GAAG;QACT,IAAI,IAAI,CAAC,KAAK,KAAK,IAAI,IAAI,IAAI,CAAC,MAAM;YAAE,OAAO;QAC/C,MAAM,KAAK,GAAG,UAAU,CAAC,GAAG,EAAE;YAC5B,IAAI,CAAC,KAAK,GAAG,IAAI,CAAC;YAClB,KAAK,IAAI,CAAC,KAAK,EAAE,CAAC;QACpB,CAAC,EAAE,IAAI,CAAC,OAAO,CAAC,CAAC;QACjB,IAAI,OAAQ,KAAgC,CAAC,KAAK,KAAK,UAAU,EAAE,CAAC;YACjE,KAA+B,CAAC,KAAK,EAAE,CAAC;QAC3C,CAAC;QACD,IAAI,CAAC,KAAK,GAAG,KAAK,CAAC;IACrB,CAAC;IAED;0DACsD;IACtD,KAAK,CAAC,KAAK;QACT,IAAI,IAAI,CAAC,KAAK,KAAK,IAAI,EAAE,CAAC;YACxB,YAAY,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;YACzB,IAAI,CAAC,KAAK,GAAG,IAAI,CAAC;QACpB,CAAC;QACD,yEAAyE;QACzE,mEAAmE;QACnE,MAAM,QAAQ,GAAG,IAAI,CAAC,QAAQ,CAAC;QAC/B,IAAI,CAAC,QAAQ,GAAG,CAAC,KAAK,IAAI,EAAE;YAC1B,MAAM,QAAQ,CAAC;YACf,OAAO,IAAI,CAAC,KAAK,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;gBAC7B,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,EAAE,IAAI,CAAC,UAAU,CAAC,CAAC;gBACpD,IAAI,CAAC;oBACH,MAAM,IAAI,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;gBACzB,CAAC;gBAAC,MAAM,CAAC;oBACP,0EAA0E;gBAC5E,CAAC;YACH,CAAC;QACH,CAAC,CAAC,EAAE,CAAC;QACL,MAAM,IAAI,CAAC,QAAQ,CAAC;IACtB,CAAC;IAED;;;;0DAIsD;IACtD,KAAK,CAAC,KAAK,CAAC,YAAoB,mBAAmB;QACjD,IAAI,CAAC,MAAM,GAAG,IAAI,CAAC;QACnB,IAAI,IAAI,CAAC,KAAK,KAAK,IAAI,EAAE,CAAC;YACxB,YAAY,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;YACzB,IAAI,CAAC,KAAK,GAAG,IAAI,CAAC;QACpB,CAAC;QACD,IAAI,KAAgD,CAAC;QACrD,MAAM,OAAO,GAAG,IAAI,OAAO,CAAO,CAAC,OAAO,EAAE,EAAE;YAC5C,KAAK,GAAG,UAAU,CAAC,OAAO,EAAE,SAAS,CAAC,CAAC;YACvC,IAAI,OAAQ,KAAgC,CAAC,KAAK,KAAK,UAAU,EAAE,CAAC;gBACjE,KAA+B,CAAC,KAAK,EAAE,CAAC;YAC3C,CAAC;QACH,CAAC,CAAC,CAAC;QACH,2EAA2E;QAC3E,2EAA2E;QAC3E,MAAM,OAAO,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,KAAK,EAAE,EAAE,OAAO,CAAC,CAAC,CAAC;QAC5C,IAAI,KAAK,KAAK,SAAS;YAAE,YAAY,CAAC,KAAK,CAAC,CAAC;QAC7C,MAAM,QAAQ,GAAG,IAAI,CAAC,KAAK,CAAC;QAC5B,IAAI,CAAC,KAAK,GAAG,EAAE,CAAC;QAChB,OAAO,QAAQ,CAAC;IAClB,CAAC;CACF"}