@imbingox/acex 0.4.0-beta.16 → 0.4.0-beta.18

package/CHANGELOG.md

@@ -1,5 +1,17 @@
 # @imbingox/acex
 
+## 0.4.0-beta.18
+
+### Minor Changes
+
+- 3edefc1: Extend the public rate limiter SPI with optional topology plans, bucket reserve headroom, request priority, opaque reservations, and bucket-level snapshots. The default limiter now supports Binance REST topology registration, fixed-window bucket budget admission, cancel-priority reserve for Binance PAPI request weight, usage-header reconciliation, request-not-sent refunds, jittered bucket-level 429 fallback, and bucket-level 429/418 blocking while remaining backward compatible with existing custom `RateLimiter` implementations.
+
+## 0.4.0-beta.17
+
+### Minor Changes
+
+- 35b8163: 事件流新增 `conflate` / `buffer` 与 `maxBuffer` 订阅选项：L1 Book 与 Funding Rate 默认改为 latest-wins，慢消费者只保留同一 `venue:symbol` 的最新事件；market status 事件按 activity/ready/freshness/reason 去重发布；buffer 溢出会丢弃最旧事件并通过 `EVENT_BUFFER_OVERFLOW` runtime error 告警。
+
 ## 0.4.0-beta.16
 
 ### Minor Changes

package/docs/api.md

@@ -253,7 +253,7 @@ const client = createClient({
 });
 ```
 
-`clock` 只用于 outbound request / signing timestamp，不驱动 WebSocket freshness 的 received-at 时钟。需要自定义 REST 限流行为时可传 `rateLimiter`，否则使用默认 reactive limiter。Binance `riskPollIntervalMs` 默认 5s，用于风险和 mark-to-market 仓位刷新；`privateReconcileIntervalMs` 默认 60s，用于账户余额、仓位和订单状态 REST 对账，显式传 `0` 可关闭 private reconcile，但不关闭 risk polling。`sandbox`、`logger`、`logLevel` 目前是预留位。
+`clock` 只用于 outbound request / signing timestamp，不驱动 WebSocket freshness 的 received-at 时钟。需要自定义 REST 限流行为时可传 `rateLimiter`，否则使用默认 bucket-aware budget limiter：它会注册 Binance REST topology，在 `beforeRequest` 中按固定窗口和 `rateLimit.utilizationTarget`（默认 0.9）主动预扣预算，接近上限时 sleep 到下一窗口；Binance PAPI request-weight 桶为 `priority:"cancel"` 保留 headroom，撤单请求仍计入真实 weight 但可使用保留区；响应后的 Binance usage header 会回填校正 bucket 用量，429/418 block 也会落到对应 bucket，缺少 `Retry-After` 的 429 会冷却到窗口结束并带小 jitter。Binance `riskPollIntervalMs` 默认 5s，用于风险和 mark-to-market 仓位刷新；`privateReconcileIntervalMs` 默认 60s，用于账户余额、仓位和订单状态 REST 对账，显式传 `0` 可关闭 private reconcile，但不关闭 risk polling。`sandbox`、`logger`、`logLevel` 目前是预留位。
 
 ### 4.2 `start()` / `stop()`
 
@@ -372,6 +372,26 @@ console.log(time.serverTime, time.roundTripMs, time.estimatedOffsetMs);
 
 Funding Rate 当前通过 Binance mark price websocket 更新，仅支持永续合约（`MarketDefinition.type === "swap"`，包括 Binance TradFi Perps）。spot 或 future 订阅会抛 `MARKET_FUNDING_RATE_UNSUPPORTED`。
 
+### 5.5 事件流 options
+
+Market 事件流支持可选第二参：
+
+```ts
+type EventStreamOptions = {
+  mode?: "conflate" | "buffer";
+  maxBuffer?: number;
+};
+
+client.market.events.l1BookUpdates(
+  { venue: "binance", symbol: "BTC/USDT:USDT" },
+  { mode: "buffer", maxBuffer: 50_000 },
+);
+```
+
+`l1BookUpdates()` 与 `fundingRateUpdates()` 默认使用 `conflate`，同一 `venue:symbol` 慢消费者只保留最新事件，适合策略热路径。需要录制每个 tick 时显式传 `{ mode: "buffer" }`。`market.events.all()` 与 `market.events.status()` 默认使用 `buffer`；显式传 `{ mode: "conflate" }` 时，`all()` 按 `type:venue:symbol` 合并，`status()` 按 `venue:symbol` 合并。
+
+`buffer` 模式默认每个订阅者最多积压 `10_000` 条事件，超过后丢弃最旧事件。每次积压 episode 只会向 `client.events.errors()` 发布一次 `EVENT_BUFFER_OVERFLOW` runtime error，事件 metadata 包含 `stream` 与 `maxBuffer`；队列排空后再次溢出会再次告警。`conflate` 模式天然有界，不使用 `maxBuffer`。
+
 ## 6. AccountManager
 
 ```ts
@@ -400,7 +420,7 @@ Account 事件用于消费余额、仓位、风险或全量快照替换：
 ```ts
 for await (const event of client.account.events.updates({
   accountId: "main-binance",
-})) {
+}, { maxBuffer: 20_000 })) {
   if (event.type === "risk.updated") {
     console.log(event.snapshot.riskRatio);
   }
@@ -408,6 +428,8 @@ for await (const event of client.account.events.updates({
 }
 ```
 
+Account 事件流只支持 `{ maxBuffer?: number }`，不提供 conflate；余额、仓位、风险和状态事件默认按 buffer 语义保留顺序。
+
 ## 7. OrderManager
 
 ```ts
@@ -456,7 +478,7 @@ Order 事件用于消费订单状态变化和 open orders 快照校准。Binance
 for await (const event of client.order.events.updates({
   accountId: "main-binance",
   symbol: "BTC/USDT:USDT",
-})) {
+}, { maxBuffer: 20_000 })) {
   if (event.type === "order.filled") {
     console.log(event.snapshot.filled);
   }
@@ -464,23 +486,28 @@ for await (const event of client.order.events.updates({
 }
 ```
 
+Order 事件流只支持 `{ maxBuffer?: number }`，不提供 conflate；订单中间状态和错误恢复信号默认按 buffer 语义保留顺序。
+
 ## 8. 健康与错误事件
 
 ```ts
 const health = client.getHealth();
 
-for await (const event of client.events.health({ venue: "binance" })) {
+for await (const event of client.events.health(
+  { venue: "binance" },
+  { maxBuffer: 20_000 },
+)) {
   console.log(event.type);
   break;
 }
 
-for await (const error of client.events.errors()) {
+for await (const error of client.events.errors({ maxBuffer: 20_000 })) {
   console.error(error.source, error.error);
   break;
 }
 ```
 
-`getHealth()` 聚合 client、market、account、order 的当前状态。`events.health(filter)` 只返回满足 filter 的事件；如果事件没有 filter 请求的字段，会被过滤掉。
+`getHealth()` 聚合 client、market、account、order 的当前状态。`events.health(filter, options?)` 只返回满足 filter 的事件；如果事件没有 filter 请求的字段，会被过滤掉。`events.health()` 与 `events.errors()` 只支持 `{ maxBuffer?: number }`，默认 buffer 上限同样是 `10_000`；`errors()` 自身溢出时只丢弃最旧错误事件，不再发布新的 overflow 错误，避免递归。
 
 ## 9. 数据类型速查
 
@@ -572,6 +599,9 @@ interface CreateClientOptions {
   sandbox?: boolean;
   clock?: { now(): number };
   rateLimiter?: RateLimiter;
+  rateLimit?: {
+    utilizationTarget?: number;
+  };
   logger?: Logger;
   logLevel?: "debug" | "info" | "warn" | "error";
   market?: {
@@ -614,14 +644,77 @@ interface RateLimitUsage {
   orderCount?: Record<string, number>;
 }
 
+type RateLimitPriority = "normal" | "cancel" | "risk" | (string & {});
+type RateLimitBucketKind = "request_weight" | "orders" | (string & {});
+type RateLimitScopeDimension = "venue" | "account" | "endpoint";
+
+interface RateLimitBucketReserve {
+  priority: RateLimitPriority;
+  units: number;
+}
+
+interface RateLimitBucketDescriptor {
+  id: string;
+  kind: RateLimitBucketKind;
+  limit: number;
+  intervalMs: number;
+  scope: readonly RateLimitScopeDimension[];
+  utilizationTarget?: number;
+  reserve?: RateLimitBucketReserve;
+}
+
+interface RateLimitCost {
+  bucketId: string;
+  cost: number;
+}
+
+interface RateLimitPlan {
+  id: string;
+  costs: readonly RateLimitCost[];
+  priority?: RateLimitPriority;
+}
+
+interface RateLimitTopology {
+  id: string;
+  buckets: readonly RateLimitBucketDescriptor[];
+  plans: readonly RateLimitPlan[];
+}
+
+interface RateLimitReservation {
+  readonly __opaqueRateLimitReservation?: never;
+}
+
+interface RateLimitTopologyRegistry {
+  registerRateLimitTopology(topology: RateLimitTopology): void;
+}
+
+interface RateLimitBucketSnapshot {
+  bucketId: string;
+  kind: RateLimitBucketKind;
+  limit: number;
+  intervalMs: number;
+  utilizationTarget?: number;
+  reserve?: RateLimitBucketReserve;
+  used?: number;
+  windowStartMs?: number;
+  windowEndMs?: number;
+  blockedUntil?: number;
+  retryAfterMs?: number;
+  state: "ok" | "rate_limited" | "banned";
+  updatedAt?: number;
+}
+
 interface RateLimitRequestContext {
   scope: RateLimitScope;
+  planId?: string;
+  priority?: RateLimitPriority;
 }
 
 interface RateLimitResponseContext {
   status: number;
   headers?: Headers;
   usage?: RateLimitUsage;
+  reservation?: RateLimitReservation;
 }
 
 interface RateLimitTransportErrorContext {
@@ -629,6 +722,8 @@ interface RateLimitTransportErrorContext {
   headers?: Headers;
   retryAfterMs?: number;
   usage?: RateLimitUsage;
+  reservation?: RateLimitReservation;
+  requestNotSent?: boolean;
 }
 
 interface RateLimitSnapshot {
@@ -638,10 +733,13 @@ interface RateLimitSnapshot {
   retryAfterMs?: number;
   state: "ok" | "rate_limited" | "banned";
   updatedAt?: number;
+  buckets?: RateLimitBucketSnapshot[];
 }
 
 interface RateLimiter {
-  beforeRequest(ctx: RateLimitRequestContext): Promise<void> | void;
+  beforeRequest(
+    ctx: RateLimitRequestContext,
+  ): Promise<RateLimitReservation | void> | RateLimitReservation | void;
   afterResponse(
     ctx: RateLimitRequestContext,
     response: RateLimitResponseContext,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@imbingox/acex",
-  "version": "0.4.0-beta.16",
+  "version": "0.4.0-beta.18",
   "description": "Multi-exchange trading SDK for market data, account, and order management",
   "repository": {
     "type": "git",

package/src/adapters/binance/adapter.ts

@@ -17,6 +17,7 @@ import {
   type BinanceMarketDefinition,
   loadBinanceMarkets,
 } from "./market-catalog.ts";
+import { registerBinanceRateLimitTopology } from "./rate-limit-topology.ts";
 import { fetchBinanceServerTime } from "./server-time.ts";
 import {
   type BinanceStreamDescriptor,
@@ -61,7 +62,9 @@ export class BinanceMarketAdapter implements MarketAdapter {
     private readonly options: {
       readonly rateLimiter?: RateLimiter;
     } = {},
-  ) {}
+  ) {
+    registerBinanceRateLimitTopology(this.options.rateLimiter);
+  }
 
   async loadMarkets(): Promise<MarketDefinition[]> {
     const markets = await loadBinanceMarkets(fetch, {

package/src/adapters/binance/market-catalog.ts

@@ -11,8 +11,12 @@ import type {
   RateLimitScope,
 } from "../../types/index.ts";
 import { parseBinanceRateLimitUsage } from "./rate-limit.ts";
+import { getBinanceCatalogRateLimitPlanId } from "./rate-limit-topology.ts";
 
-type FetchLike = typeof fetch;
+type FetchLike = (
+  input: string | URL | Request,
+  init?: RequestInit,
+) => Promise<Response>;
 
 export type BinanceMarketFamily = "spot" | "usdm" | "coinm";
 
@@ -240,8 +244,13 @@ async function requestCatalogJson<T>(
     venue: "binance",
     endpointKey,
   };
+  const requestContext = {
+    scope,
+    planId: getBinanceCatalogRateLimitPlanId(endpointKey),
+  };
 
-  await rateLimiter?.beforeRequest({ scope });
+  const reservation =
+    (await rateLimiter?.beforeRequest(requestContext)) ?? undefined;
 
   try {
     const response = await httpRequest<T>({
@@ -252,32 +261,28 @@ async function requestCatalogJson<T>(
       jsonParseMode: "response",
       retryPolicy: {
         idempotent: true,
-        maxAttempts: 3,
+        maxAttempts: 1,
       },
       messages: BINANCE_CATALOG_HTTP_MESSAGES,
     });
 
-    await rateLimiter?.afterResponse(
-      { scope },
-      {
-        status: response.status,
-        headers: response.headers,
-        usage: parseBinanceRateLimitUsage(response.headers),
-      },
-    );
+    await rateLimiter?.afterResponse(requestContext, {
+      status: response.status,
+      headers: response.headers,
+      usage: parseBinanceRateLimitUsage(response.headers),
+      reservation,
+    });
 
     return response.body;
   } catch (error) {
     if (isTransportError(error)) {
-      await rateLimiter?.onTransportError(
-        { scope },
-        {
-          status: error.status,
-          headers: error.headers,
-          retryAfterMs: error.retryAfterMs,
-          usage: parseBinanceRateLimitUsage(error.headers),
-        },
-      );
+      await rateLimiter?.onTransportError(requestContext, {
+        status: error.status,
+        headers: error.headers,
+        retryAfterMs: error.retryAfterMs,
+        usage: parseBinanceRateLimitUsage(error.headers),
+        reservation,
+      });
     }
 
     throw error;

package/src/adapters/binance/private-adapter.ts

@@ -11,6 +11,7 @@ import type {
   AccountCredentials,
   PositionSide,
   RateLimiter,
+  RateLimitPriority,
   RateLimitScope,
   TimeProvider,
   VenueAccountCapabilities,
@@ -35,10 +36,17 @@ import type {
 } from "../types.ts";
 import { normalizeBinanceErrorCode } from "./error-codes.ts";
 import { parseBinanceRateLimitUsage } from "./rate-limit.ts";
+import {
+  getBinancePapiRateLimitPlanId,
+  registerBinanceRateLimitTopology,
+} from "./rate-limit-topology.ts";
 
 type TimerHandle = ReturnType<typeof setInterval>;
 type SignedRequestMethod = "GET" | "POST" | "DELETE";
-type FetchLike = typeof fetch;
+type FetchLike = (
+  input: string | URL | Request,
+  init?: RequestInit,
+) => Promise<Response>;
 
 interface BinancePapiBalance {
   asset?: string;
@@ -172,18 +180,14 @@ const BINANCE_PAPI_WS_BASE_URL = "wss://fstream.binance.com/pm/ws";
 const DEFAULT_RECV_WINDOW = 5_000;
 const DEFAULT_HTTP_TIMEOUT_MS = 10_000;
 const USDM_QUOTE_ASSETS = ["FDUSD", "USDC", "BUSD", "USDT"];
-const SAFE_READ_RETRY_POLICY: HttpRetryPolicy = {
+const SINGLE_ATTEMPT_IDEMPOTENT_POLICY: HttpRetryPolicy = {
   idempotent: true,
-  maxAttempts: 3,
+  maxAttempts: 1,
 };
 const NO_RETRY_POLICY: HttpRetryPolicy = {
   idempotent: false,
   maxAttempts: 1,
 };
-const LISTEN_KEY_KEEPALIVE_RETRY_POLICY: HttpRetryPolicy = {
-  idempotent: true,
-  maxAttempts: 3,
-};
 function getBinancePapiHttpMessages(timeoutMs: number): HttpClientMessages {
   return {
     http: ({ status, statusText, url, rawBody }) =>
@@ -674,7 +678,9 @@ export class BinancePrivateAdapter implements PrivateUserDataAdapter {
       readonly signingClock?: TimeProvider;
       readonly rateLimiter?: RateLimiter;
     } = {},
-  ) {}
+  ) {
+    registerBinanceRateLimitTopology(this.options.rateLimiter);
+  }
 
   normalizeVenueErrorCode(code: string) {
     return normalizeBinanceErrorCode(code);
@@ -692,7 +698,7 @@ export class BinancePrivateAdapter implements PrivateUserDataAdapter {
         credentials,
         accountOptions,
         undefined,
-        SAFE_READ_RETRY_POLICY,
+        SINGLE_ATTEMPT_IDEMPOTENT_POLICY,
       ),
       this.signedRequest<BinancePapiAccount>(
         "GET",
@@ -700,7 +706,7 @@ export class BinancePrivateAdapter implements PrivateUserDataAdapter {
         credentials,
         accountOptions,
         undefined,
-        SAFE_READ_RETRY_POLICY,
+        SINGLE_ATTEMPT_IDEMPOTENT_POLICY,
       ),
       this.signedRequest<BinancePapiUmPosition[]>(
         "GET",
@@ -708,7 +714,7 @@ export class BinancePrivateAdapter implements PrivateUserDataAdapter {
         credentials,
         accountOptions,
         undefined,
-        SAFE_READ_RETRY_POLICY,
+        SINGLE_ATTEMPT_IDEMPOTENT_POLICY,
       ),
     ]);
 
@@ -734,7 +740,7 @@ export class BinancePrivateAdapter implements PrivateUserDataAdapter {
         credentials,
         accountOptions,
         undefined,
-        SAFE_READ_RETRY_POLICY,
+        SINGLE_ATTEMPT_IDEMPOTENT_POLICY,
       ),
       this.signedRequest<BinancePapiUmPosition[]>(
         "GET",
@@ -742,7 +748,7 @@ export class BinancePrivateAdapter implements PrivateUserDataAdapter {
         credentials,
         accountOptions,
         undefined,
-        SAFE_READ_RETRY_POLICY,
+        SINGLE_ATTEMPT_IDEMPOTENT_POLICY,
       ),
     ]);
 
@@ -768,7 +774,7 @@ export class BinancePrivateAdapter implements PrivateUserDataAdapter {
       credentials,
       accountOptions,
       undefined,
-      SAFE_READ_RETRY_POLICY,
+      SINGLE_ATTEMPT_IDEMPOTENT_POLICY,
     );
 
     return {
@@ -797,7 +803,7 @@ export class BinancePrivateAdapter implements PrivateUserDataAdapter {
           orderId: request.orderId,
           origClientOrderId: request.clientOrderId,
         },
-        SAFE_READ_RETRY_POLICY,
+        SINGLE_ATTEMPT_IDEMPOTENT_POLICY,
       );
 
       return mapOpenOrder(response, receivedAt);
@@ -870,6 +876,7 @@ export class BinancePrivateAdapter implements PrivateUserDataAdapter {
         origClientOrderId: request.clientOrderId,
       },
       NO_RETRY_POLICY,
+      "cancel",
     );
 
     const mapped = mapOpenOrder(response, receivedAt);
@@ -896,7 +903,8 @@ export class BinancePrivateAdapter implements PrivateUserDataAdapter {
       {
         symbol,
       },
-      SAFE_READ_RETRY_POLICY,
+      SINGLE_ATTEMPT_IDEMPOTENT_POLICY,
+      "cancel",
     );
 
     // Venue responds {code,msg}; returned updates are synthesized from the
@@ -911,6 +919,7 @@ export class BinancePrivateAdapter implements PrivateUserDataAdapter {
         symbol,
       },
       NO_RETRY_POLICY,
+      "cancel",
     );
 
     if (response.code !== undefined && `${response.code}` !== "200") {
@@ -1216,10 +1225,18 @@ export class BinancePrivateAdapter implements PrivateUserDataAdapter {
     accountOptions?: Record<string, unknown>,
     queryParams?: Record<string, string | undefined>,
     retryPolicy?: HttpRetryPolicy,
+    priority?: RateLimitPriority,
   ): Promise<T> {
     const { apiKey, secret } = requirePrivateCredentials(credentials);
     const scope = this.rateLimitScope(method, path, accountOptions);
-    await this.options.rateLimiter?.beforeRequest({ scope });
+    const requestContext = {
+      scope,
+      planId: getBinancePapiRateLimitPlanId(method, path, queryParams),
+      priority,
+    };
+    const reservation =
+      (await this.options.rateLimiter?.beforeRequest(requestContext)) ??
+      undefined;
 
     const params = new URLSearchParams();
     for (const [key, value] of Object.entries(queryParams ?? {})) {
@@ -1258,27 +1275,23 @@ export class BinancePrivateAdapter implements PrivateUserDataAdapter {
         messages: getBinancePapiHttpMessages(timeoutMs),
       });
 
-      await this.options.rateLimiter?.afterResponse(
-        { scope },
-        {
-          status: response.status,
-          headers: response.headers,
-          usage: parseBinanceRateLimitUsage(response.headers),
-        },
-      );
+      await this.options.rateLimiter?.afterResponse(requestContext, {
+        status: response.status,
+        headers: response.headers,
+        usage: parseBinanceRateLimitUsage(response.headers),
+        reservation,
+      });
 
       return response.body;
     } catch (error) {
       if (isTransportError(error)) {
-        await this.options.rateLimiter?.onTransportError(
-          { scope },
-          {
-            status: error.status,
-            headers: error.headers,
-            retryAfterMs: error.retryAfterMs,
-            usage: parseBinanceRateLimitUsage(error.headers),
-          },
-        );
+        await this.options.rateLimiter?.onTransportError(requestContext, {
+          status: error.status,
+          headers: error.headers,
+          retryAfterMs: error.retryAfterMs,
+          usage: parseBinanceRateLimitUsage(error.headers),
+          reservation,
+        });
       }
 
       throw error;
@@ -1312,7 +1325,7 @@ export class BinancePrivateAdapter implements PrivateUserDataAdapter {
       "PUT",
       credentials,
       listenKey,
-      LISTEN_KEY_KEEPALIVE_RETRY_POLICY,
+      SINGLE_ATTEMPT_IDEMPOTENT_POLICY,
       accountOptions,
     );
   }
@@ -1344,7 +1357,13 @@ export class BinancePrivateAdapter implements PrivateUserDataAdapter {
       "/papi/v1/listenKey",
       accountOptions,
     );
-    await this.options.rateLimiter?.beforeRequest({ scope });
+    const requestContext = {
+      scope,
+      planId: getBinancePapiRateLimitPlanId(method, "/papi/v1/listenKey"),
+    };
+    const reservation =
+      (await this.options.rateLimiter?.beforeRequest(requestContext)) ??
+      undefined;
 
     const params = new URLSearchParams();
     if (listenKey) {
@@ -1371,27 +1390,23 @@ export class BinancePrivateAdapter implements PrivateUserDataAdapter {
         messages: getBinancePapiHttpMessages(timeoutMs),
       });
 
-      await this.options.rateLimiter?.afterResponse(
-        { scope },
-        {
-          status: response.status,
-          headers: response.headers,
-          usage: parseBinanceRateLimitUsage(response.headers),
-        },
-      );
+      await this.options.rateLimiter?.afterResponse(requestContext, {
+        status: response.status,
+        headers: response.headers,
+        usage: parseBinanceRateLimitUsage(response.headers),
+        reservation,
+      });
 
       return response.body;
     } catch (error) {
       if (isTransportError(error)) {
-        await this.options.rateLimiter?.onTransportError(
-          { scope },
-          {
-            status: error.status,
-            headers: error.headers,
-            retryAfterMs: error.retryAfterMs,
-            usage: parseBinanceRateLimitUsage(error.headers),
-          },
-        );
+        await this.options.rateLimiter?.onTransportError(requestContext, {
+          status: error.status,
+          headers: error.headers,
+          retryAfterMs: error.retryAfterMs,
+          usage: parseBinanceRateLimitUsage(error.headers),
+          reservation,
+        });
       }
 
       throw error;