@x12i/ai-gateway 10.0.5 → 10.0.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +19 -8
- package/dist/gateway-config.js +2 -11
- package/dist/gateway-utils.d.ts +5 -14
- package/dist/gateway-utils.js +22 -28
- package/dist/gateway.js +1 -3
- package/dist/index.d.ts +1 -1
- package/dist/index.js +1 -1
- package/dist-cjs/gateway-config.cjs +2 -11
- package/dist-cjs/gateway-utils.cjs +22 -28
- package/dist-cjs/gateway-utils.d.ts +5 -14
- package/dist-cjs/gateway.cjs +1 -3
- package/dist-cjs/index.cjs +1 -1
- package/dist-cjs/index.d.ts +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -9,9 +9,9 @@ Unified gateway for LLM provider routing, structured logging, optional Activix a
|
|
|
9
9
|
| **Routing** | Registers providers (or lazy-registers from env), invokes the router with merged model config, retries, and optional fallback chain. |
|
|
10
10
|
| **`invoke()`** | Builds messages from instructions + prompt templates + `workingMemory`; requires runtime **identity** and **actionType** / **actionRef**. |
|
|
11
11
|
| **`invokeChat()`** | Raw chat-style requests; no instruction builder or action classification. |
|
|
12
|
-
| **Cost** |
|
|
13
|
-
| **Activix** | Optional Mongo-backed activity rows
|
|
14
|
-
| **Trace mode** | `diagnostics.mode === 'trace'` adds `metadata.attempts[]`, `metadata.usage`, and per-attempt
|
|
12
|
+
| **Cost** | Steps A→D on every successful **`invoke()`** / **`invokeChat()`**: router cost first, then **`@x12i/ai-tools`** catalog via **`calculateFromRecord`** when still unpriced. Single path — **`resolveCostCompletionWithAiTools`**. |
|
|
13
|
+
| **Activix** | Optional Mongo-backed activity rows; billing written from gateway-computed slice on **`completeRecord`** (`outer.cost` + root fields). No Activix **`autoCost`** re-pricing. |
|
|
14
|
+
| **Trace mode** | `diagnostics.mode === 'trace'` adds `metadata.attempts[]`, `metadata.usage`, and per-attempt **`costUsd`** / **`costStatus`**. |
|
|
15
15
|
|
|
16
16
|
Pinned dependency versions are in `package.json` (currently **Activix ^8.5**, **ai-tools ^2.5**, **ai-providers-router ^4.9**).
|
|
17
17
|
|
|
@@ -171,6 +171,7 @@ Hosts wrapping the gateway should expose on **their** public API:
|
|
|
171
171
|
| `temperature`, `topP`, `frequencyPenalty`, `presencePenalty`, `maxTokens` | Optional | Document defaults from `GATEWAY_DEFAULT_*` |
|
|
172
172
|
| `retry` | Optional | Same shape as `RetryConfig`; defaults from `GATEWAY_DEFAULT_RETRY` |
|
|
173
173
|
| `mode` | Optional | `'dev'` \| `'debug'` \| `'prod'` — pass through to `GatewayConfig.mode` |
|
|
174
|
+
| Billing | Read-only on response | **`response.metadata.costUsd`**, **`costStatus`**, **`tokens`** — gateway-owned; do not re-price |
|
|
174
175
|
| `templateRenderOptions` / `smartInput` | Optional | Rendrix overrides |
|
|
175
176
|
|
|
176
177
|
Instructions must be **complete caller text** — the gateway no longer injects packaged instruction blocks.
|
|
@@ -223,7 +224,7 @@ Engine-owned catalog bootstrap and post-call billing. Consumers read **`metadata
|
|
|
223
224
|
|
|
224
225
|
Step A always wins; explicit router **`costStatus: "unpriced"`** is never overridden by catalog.
|
|
225
226
|
|
|
226
|
-
Implemented in **`resolveCostCompletionWithAiTools`** (
|
|
227
|
+
Implemented in **`resolveCostCompletionWithAiTools`** only ( **`CostCalculator.calculateFromRecord`** via **`buildGatewayPricingRecord`** for Step B). Upstream target: **`resolveInvokeBilling`** in ai-tools — [AI_TOOLS_INVOKE_BILLING_ORCHESTRATOR_SPEC.md](./docs/upstream-reports/AI_TOOLS_INVOKE_BILLING_ORCHESTRATOR_SPEC.md).
|
|
227
228
|
|
|
228
229
|
### `aiTools` config (aligned with funcx / generic engine contract)
|
|
229
230
|
|
|
@@ -242,11 +243,11 @@ Implemented in **`resolveCostCompletionWithAiTools`** (delegates to **`CostCalcu
|
|
|
242
243
|
|
|
243
244
|
Gateway exports the model orchestrator from `@x12i/ai-tools` ≥ **2.5.0** (`resolveInvokeModel`, …) — see [AI_TOOLS_INVOKE_MODEL_RESOLUTION_ORCHESTRATOR_SPEC.md](./docs/upstream-reports/AI_TOOLS_INVOKE_MODEL_RESOLUTION_ORCHESTRATOR_SPEC.md).
|
|
244
245
|
|
|
245
|
-
Gateway billing helpers (
|
|
246
|
+
Gateway billing helpers (exported for tests/integrators): `resolveCostCompletionWithAiTools`, `buildGatewayPricingRecord`, `catalogPricingSucceeded`, `buildTraceUsageSummary`, `enrichTraceAttemptsWithBilling`.
|
|
246
247
|
|
|
247
248
|
---
|
|
248
249
|
|
|
249
|
-
## Activity tracking (@x12i/activix
|
|
250
|
+
## Activity tracking (@x12i/activix 8.x)
|
|
250
251
|
|
|
251
252
|
When tracking is enabled and no custom tracker is supplied, the gateway constructs Activix with fixed collection names (see `src/config/activity-tracking-config.ts`):
|
|
252
253
|
|
|
@@ -265,7 +266,7 @@ When tracking is enabled and no custom tracker is supplied, the gateway construc
|
|
|
265
266
|
- `outer.cost`: Activix cost shape (`usd`, `tokens`, `provider`, `model`, `details`)
|
|
266
267
|
- `response.metadata`: same billing slice as returned to callers
|
|
267
268
|
|
|
268
|
-
|
|
269
|
+
Gateway resolves billing **before** `completeRecord` and sets **`outer.cost`** from that slice. Activix **`autoCost`** is **not** used on the default activity manager (no second pricing path).
|
|
269
270
|
|
|
270
271
|
Mongo env: `MONGO_URI` + `MONGO_LOGS_DB` or `MONGO_DB`.
|
|
271
272
|
|
|
@@ -276,7 +277,17 @@ Mongo env: `MONGO_URI` + `MONGO_LOGS_DB` or `MONGO_DB`.
|
|
|
276
277
|
On every successful **`invoke()`** and **`invokeChat()`**:
|
|
277
278
|
|
|
278
279
|
- **`metadata.provider`**, **`modelUsed`**, **`maxTokensRequested`**, **`effectiveModelConfig`** (invoke only)
|
|
279
|
-
- **`metadata.tokens`**, **`costStatus`**, **`costUsd
|
|
280
|
+
- **`metadata.tokens`**, **`costStatus`**, **`costUsd`**, optional **`costBreakdown`**; **`cost`** mirrors **`costUsd`** when priced
|
|
281
|
+
|
|
282
|
+
### Client rules (ai-skills, graph-engine, etc.)
|
|
283
|
+
|
|
284
|
+
| `metadata.costStatus` | Meaning | Client action |
|
|
285
|
+
|------------------------|---------|---------------|
|
|
286
|
+
| **`priced`** | Gateway resolved a billable USD amount | Use **`metadata.costUsd`** (or **`cost`**) |
|
|
287
|
+
| **`unpriced`** | Tokens recorded; no authoritative price | Do **not** call ai-tools or re-price |
|
|
288
|
+
| *(absent)* | No token usage | No billing signal |
|
|
289
|
+
|
|
290
|
+
Do **not** add a direct **`@x12i/ai-tools`** dependency for post-call cost. For Activix rows you write yourself, use **`normalizeToActivixCostShape`** (re-exported from `@x12i/activix`) from **`costUsd`** + **`metadata.tokens`**.
|
|
280
291
|
|
|
281
292
|
Full contract: [AI Gateway invoke execution metadata](./docs/AI_GATEWAY_INVOKE_EXECUTION_METADATA.md).
|
|
282
293
|
|
package/dist/gateway-config.js
CHANGED
|
@@ -176,17 +176,8 @@ export function initializeGatewayComponents(config) {
|
|
|
176
176
|
enableActivityTracking: config.enableActivityTracking ?? true,
|
|
177
177
|
customTracker: config.activityTracker,
|
|
178
178
|
logger,
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
: {
|
|
182
|
-
autoCost: config.aiTools?.enabled === false || config.aiTools?.calculateCost === false
|
|
183
|
-
? false
|
|
184
|
-
: {
|
|
185
|
-
enabled: true,
|
|
186
|
-
overwriteOuterCost: false,
|
|
187
|
-
...(config.aiTools?.bundledOnly ? { bundledOnly: true } : {})
|
|
188
|
-
}
|
|
189
|
-
})
|
|
179
|
+
// Billing is resolved in gateway before logSuccess; Activix gets outer.cost from that slice only.
|
|
180
|
+
...(config.activityTracker ? {} : { autoCost: false })
|
|
190
181
|
});
|
|
191
182
|
const templateRendering = mergeTemplateRenderOptions(defaultTemplateRendering, config.templateRendering);
|
|
192
183
|
const messageBuilderConfig = {
|
package/dist/gateway-utils.d.ts
CHANGED
|
@@ -72,8 +72,8 @@ export declare function hasNonZeroTokenUsage(tokens: {
|
|
|
72
72
|
total: number;
|
|
73
73
|
}): boolean;
|
|
74
74
|
/**
|
|
75
|
-
*
|
|
76
|
-
* Prefer {@link
|
|
75
|
+
* Step A/C/D cost slice when the router omits explicit `metadata.costStatus`.
|
|
76
|
+
* Prefer {@link resolveCostCompletionWithAiTools} at invoke boundaries.
|
|
77
77
|
*/
|
|
78
78
|
export declare function resolveActivityCostCompletion(tokens: {
|
|
79
79
|
prompt: number;
|
|
@@ -81,8 +81,7 @@ export declare function resolveActivityCostCompletion(tokens: {
|
|
|
81
81
|
total: number;
|
|
82
82
|
}, costUsd: number | undefined): ResolvedActivityCost;
|
|
83
83
|
/**
|
|
84
|
-
*
|
|
85
|
-
* otherwise gateway applies the G8 fallback (usage + no price → `unpriced`).
|
|
84
|
+
* Step A router passthrough + Step C when the router omits `metadata.costStatus`.
|
|
86
85
|
*/
|
|
87
86
|
export declare function resolveCostCompletionForActivity(routerResponse: unknown, tokens: {
|
|
88
87
|
prompt: number;
|
|
@@ -118,16 +117,8 @@ export declare function buildGatewayPricingRecord(routerResponse: unknown, token
|
|
|
118
117
|
}, mergedConfig?: unknown): Record<string, unknown>;
|
|
119
118
|
export declare function mapAiCostResultToResolvedActivityCost(base: ResolvedActivityCost, result: AiCostResult): ResolvedActivityCost;
|
|
120
119
|
/**
|
|
121
|
-
*
|
|
122
|
-
*
|
|
123
|
-
*/
|
|
124
|
-
export declare function ensureInvokeBillingCostStatus(billing: ResolvedActivityCost, tokens: {
|
|
125
|
-
prompt: number;
|
|
126
|
-
completion: number;
|
|
127
|
-
total: number;
|
|
128
|
-
}): ResolvedActivityCost;
|
|
129
|
-
/**
|
|
130
|
-
* Router cost passthrough, then optional @x12i/ai-tools catalog pricing when still unpriced.
|
|
120
|
+
* Post-invoke billing (Steps A→D): router cost, then catalog via ai-tools when still unpriced.
|
|
121
|
+
* Single entry point for `invoke()` / `invokeChat()` and trace enrichment.
|
|
131
122
|
*/
|
|
132
123
|
export declare function resolveCostCompletionWithAiTools(routerResponse: unknown, tokens: {
|
|
133
124
|
prompt: number;
|
package/dist/gateway-utils.js
CHANGED
|
@@ -340,8 +340,8 @@ function pickRouterCostStatus(routerResponse) {
|
|
|
340
340
|
return status === 'priced' || status === 'unpriced' ? status : undefined;
|
|
341
341
|
}
|
|
342
342
|
/**
|
|
343
|
-
*
|
|
344
|
-
* Prefer {@link
|
|
343
|
+
* Step A/C/D cost slice when the router omits explicit `metadata.costStatus`.
|
|
344
|
+
* Prefer {@link resolveCostCompletionWithAiTools} at invoke boundaries.
|
|
345
345
|
*/
|
|
346
346
|
export function resolveActivityCostCompletion(tokens, costUsd) {
|
|
347
347
|
if (typeof costUsd === 'number' && Number.isFinite(costUsd)) {
|
|
@@ -353,8 +353,7 @@ export function resolveActivityCostCompletion(tokens, costUsd) {
|
|
|
353
353
|
return {};
|
|
354
354
|
}
|
|
355
355
|
/**
|
|
356
|
-
*
|
|
357
|
-
* otherwise gateway applies the G8 fallback (usage + no price → `unpriced`).
|
|
356
|
+
* Step A router passthrough + Step C when the router omits `metadata.costStatus`.
|
|
358
357
|
*/
|
|
359
358
|
export function resolveCostCompletionForActivity(routerResponse, tokens) {
|
|
360
359
|
const routerStatus = pickRouterCostStatus(routerResponse);
|
|
@@ -492,41 +491,36 @@ export function mapAiCostResultToResolvedActivityCost(base, result) {
|
|
|
492
491
|
};
|
|
493
492
|
}
|
|
494
493
|
/**
|
|
495
|
-
*
|
|
496
|
-
* Used at invoke boundaries after {@link resolveCostCompletionWithAiTools}.
|
|
494
|
+
* Step C/D: token usage without billing signal → `unpriced`; no usage → omit status.
|
|
497
495
|
*/
|
|
498
|
-
|
|
496
|
+
function finalizeInvokeBillingCost(billing, tokens) {
|
|
499
497
|
if (!billing.costStatus && hasNonZeroTokenUsage(tokens)) {
|
|
500
498
|
return { ...billing, costStatus: 'unpriced' };
|
|
501
499
|
}
|
|
502
500
|
return billing;
|
|
503
501
|
}
|
|
504
502
|
/**
|
|
505
|
-
*
|
|
503
|
+
* Post-invoke billing (Steps A→D): router cost, then catalog via ai-tools when still unpriced.
|
|
504
|
+
* Single entry point for `invoke()` / `invokeChat()` and trace enrichment.
|
|
506
505
|
*/
|
|
507
506
|
export async function resolveCostCompletionWithAiTools(routerResponse, tokens, options) {
|
|
508
507
|
const routerStatus = pickRouterCostStatus(routerResponse);
|
|
509
|
-
|
|
510
|
-
if (
|
|
511
|
-
|
|
512
|
-
|
|
513
|
-
|
|
514
|
-
|
|
515
|
-
|
|
516
|
-
|
|
517
|
-
|
|
518
|
-
|
|
519
|
-
|
|
520
|
-
|
|
521
|
-
|
|
522
|
-
|
|
523
|
-
const record = buildGatewayPricingRecord(routerResponse, tokens, options.mergedConfig);
|
|
524
|
-
const result = await options.calculator.calculateFromRecord(record);
|
|
525
|
-
return mapAiCostResultToResolvedActivityCost(base, result);
|
|
526
|
-
}
|
|
527
|
-
catch {
|
|
528
|
-
return ensureInvokeBillingCostStatus(base, tokens);
|
|
508
|
+
let billing = resolveCostCompletionForActivity(routerResponse, tokens);
|
|
509
|
+
if (billing.costStatus !== 'priced' &&
|
|
510
|
+
routerStatus !== 'unpriced' &&
|
|
511
|
+
options?.calculateCost !== false &&
|
|
512
|
+
options?.calculator &&
|
|
513
|
+
hasNonZeroTokenUsage(tokens)) {
|
|
514
|
+
try {
|
|
515
|
+
const record = buildGatewayPricingRecord(routerResponse, tokens, options.mergedConfig);
|
|
516
|
+
const result = await options.calculator.calculateFromRecord(record);
|
|
517
|
+
billing = mapAiCostResultToResolvedActivityCost(billing, result);
|
|
518
|
+
}
|
|
519
|
+
catch {
|
|
520
|
+
// Step B unavailable — Step C applies below.
|
|
521
|
+
}
|
|
529
522
|
}
|
|
523
|
+
return finalizeInvokeBillingCost(billing, tokens);
|
|
530
524
|
}
|
|
531
525
|
function applyBillingToTraceAttempt(attempt, billing) {
|
|
532
526
|
if (billing.costStatus === 'priced' || billing.costStatus === 'unpriced') {
|
package/dist/gateway.js
CHANGED
|
@@ -11,7 +11,7 @@ import { resolveRetryConfig } from './gateway-defaults.js';
|
|
|
11
11
|
import { buildMessages } from './message-builder.js';
|
|
12
12
|
import { extractJsonFromFlexMd } from './flex-md-loader.js';
|
|
13
13
|
import { enrichParsedContentForOutputContract, resolveOutputContractFieldKeys } from './output-contract-normalizer.js';
|
|
14
|
-
import { attachGatewayInvokeRejectionMetadata, buildGatewayFallbackAttemptsFromTrace, buildInvokeRejectionMetadata, capActivityFullResponsePayload, formatFallbackExhaustionMessage, logResolvedModelRouting, mapGatewayFallbackAttemptsToRouter, DEFAULT_ACTIVITY_FULL_RESPONSE_MAX_CHARS, extractCostUsdFromRouterResponse, extractTokenUsageFromRouterResponse, mergeConfig, pickEffectiveModelConfigForMetadata, pickInvokeRoutingMetadataSlice, pickTraceMergedRouterConfig, resolveCostCompletionWithAiTools,
|
|
14
|
+
import { attachGatewayInvokeRejectionMetadata, buildGatewayFallbackAttemptsFromTrace, buildInvokeRejectionMetadata, capActivityFullResponsePayload, formatFallbackExhaustionMessage, logResolvedModelRouting, mapGatewayFallbackAttemptsToRouter, DEFAULT_ACTIVITY_FULL_RESPONSE_MAX_CHARS, extractCostUsdFromRouterResponse, extractTokenUsageFromRouterResponse, mergeConfig, pickEffectiveModelConfigForMetadata, pickInvokeRoutingMetadataSlice, pickTraceMergedRouterConfig, resolveCostCompletionWithAiTools, buildTraceUsageSummary, enrichTraceAttemptsWithBilling, tryExtractRouterLikePayloadFromErrorChain } from './gateway-utils.js';
|
|
15
15
|
import { getAiToolsClient } from './ai-tools-client.js';
|
|
16
16
|
import { autoRegisterProviders } from './gateway-provider-auto-register.js';
|
|
17
17
|
import { setGatewayLastJobId, setGatewayRuntimeClients } from './runtime-objects.js';
|
|
@@ -140,7 +140,6 @@ export class AIGateway {
|
|
|
140
140
|
calculator: aiTools?.calculator ?? null,
|
|
141
141
|
calculateCost: this.config.aiTools?.calculateCost
|
|
142
142
|
});
|
|
143
|
-
costCompletionChat = ensureInvokeBillingCostStatus(costCompletionChat, tokensChat);
|
|
144
143
|
// Create enhanced response
|
|
145
144
|
const enhancedResponse = {
|
|
146
145
|
content: response.content || '',
|
|
@@ -615,7 +614,6 @@ export class AIGateway {
|
|
|
615
614
|
calculator: aiTools?.calculator ?? null,
|
|
616
615
|
calculateCost: this.config.aiTools?.calculateCost
|
|
617
616
|
});
|
|
618
|
-
costCompletion = ensureInvokeBillingCostStatus(costCompletion, tokens);
|
|
619
617
|
const routerMetaForCost = routerResponse?.metadata || {};
|
|
620
618
|
const routingMetadataSlice = pickInvokeRoutingMetadataSlice(routerResponse, mergedConfig);
|
|
621
619
|
const effectiveModelConfig = pickEffectiveModelConfigForMetadata(mergedConfig);
|
package/dist/index.d.ts
CHANGED
|
@@ -17,7 +17,7 @@ export { AIGateway } from './gateway.js';
|
|
|
17
17
|
export { InstructionNotFoundError, InstructionBackendError, ModelRequiredError, MaxTokensRequiredError } from './instruction-errors.js';
|
|
18
18
|
export { autoRegisterProviders } from './gateway-provider-auto-register.js';
|
|
19
19
|
export type { GatewayConfig, ProviderModelRef, ModelConfig, RetryConfig, ChatRequest, AIInvokeRequest, AIRequest, GatewayActionType, GatewayInvokeRejectionMetadata, GatewayFallbackAttempt, GatewayTraceRequestIds, GatewayTraceAttempt, GatewayTraceUsageSummary, GatewayTraceMergedConfig, EnhancedLLMResponse, InstructionMetadata, ValidationRule, TemplateRenderOptions, SmartInputConfig, SmartInputRenderOptions } from './types.js';
|
|
20
|
-
export { attachGatewayInvokeRejectionMetadata, buildInvokeRejectionMetadata, tryExtractRouterLikePayloadFromErrorChain, tryExtractFallbackAttemptsFromErrorChain, pickRequestIdsFromRouterLike, resolveActivityCostCompletion, resolveCostCompletionForActivity, resolveCostCompletionWithAiTools, buildGatewayPricingRecord, mapAiCostResultToResolvedActivityCost, catalogPricingSucceeded,
|
|
20
|
+
export { attachGatewayInvokeRejectionMetadata, buildInvokeRejectionMetadata, tryExtractRouterLikePayloadFromErrorChain, tryExtractFallbackAttemptsFromErrorChain, pickRequestIdsFromRouterLike, resolveActivityCostCompletion, resolveCostCompletionForActivity, resolveCostCompletionWithAiTools, buildGatewayPricingRecord, mapAiCostResultToResolvedActivityCost, catalogPricingSucceeded, extractUsageExtrasFromRouterResponse, buildTraceUsageSummary, enrichTraceAttemptsWithBilling, hasNonZeroTokenUsage, MODEL_PROFILE_UNROUTABLE, ModelProfileUnroutableError, ModelProfileInputRejectedError, buildGatewayFallbackAttemptsFromTrace, formatFallbackExhaustionMessage, logResolvedModelRouting, mapGatewayFallbackAttemptsToRouter } from './gateway-utils.js';
|
|
21
21
|
export { getGatewayOperationalMode, isProdGatewayMode, parseModelProviderSpec } from './gateway-mode.js';
|
|
22
22
|
export type { GatewayOperationalMode } from './gateway-mode.js';
|
|
23
23
|
export { DEFAULT_ACTIVITY_FULL_RESPONSE_MAX_CHARS, GATEWAY_DEFAULT_FREQUENCY_PENALTY, GATEWAY_DEFAULT_PRESENCE_PENALTY, GATEWAY_DEFAULT_RETRY, GATEWAY_DEFAULT_TEMPERATURE, GATEWAY_DEFAULT_TOP_P, resolveRetryConfig } from './gateway-defaults.js';
|
package/dist/index.js
CHANGED
|
@@ -17,7 +17,7 @@ export * from '@x12i/ai-providers-router';
|
|
|
17
17
|
export { AIGateway } from './gateway.js';
|
|
18
18
|
export { InstructionNotFoundError, InstructionBackendError, ModelRequiredError, MaxTokensRequiredError } from './instruction-errors.js';
|
|
19
19
|
export { autoRegisterProviders } from './gateway-provider-auto-register.js';
|
|
20
|
-
export { attachGatewayInvokeRejectionMetadata, buildInvokeRejectionMetadata, tryExtractRouterLikePayloadFromErrorChain, tryExtractFallbackAttemptsFromErrorChain, pickRequestIdsFromRouterLike, resolveActivityCostCompletion, resolveCostCompletionForActivity, resolveCostCompletionWithAiTools, buildGatewayPricingRecord, mapAiCostResultToResolvedActivityCost, catalogPricingSucceeded,
|
|
20
|
+
export { attachGatewayInvokeRejectionMetadata, buildInvokeRejectionMetadata, tryExtractRouterLikePayloadFromErrorChain, tryExtractFallbackAttemptsFromErrorChain, pickRequestIdsFromRouterLike, resolveActivityCostCompletion, resolveCostCompletionForActivity, resolveCostCompletionWithAiTools, buildGatewayPricingRecord, mapAiCostResultToResolvedActivityCost, catalogPricingSucceeded, extractUsageExtrasFromRouterResponse, buildTraceUsageSummary, enrichTraceAttemptsWithBilling, hasNonZeroTokenUsage, MODEL_PROFILE_UNROUTABLE, ModelProfileUnroutableError, ModelProfileInputRejectedError, buildGatewayFallbackAttemptsFromTrace, formatFallbackExhaustionMessage, logResolvedModelRouting, mapGatewayFallbackAttemptsToRouter } from './gateway-utils.js';
|
|
21
21
|
export { getGatewayOperationalMode, isProdGatewayMode, parseModelProviderSpec } from './gateway-mode.js';
|
|
22
22
|
export { DEFAULT_ACTIVITY_FULL_RESPONSE_MAX_CHARS, GATEWAY_DEFAULT_FREQUENCY_PENALTY, GATEWAY_DEFAULT_PRESENCE_PENALTY, GATEWAY_DEFAULT_RETRY, GATEWAY_DEFAULT_TEMPERATURE, GATEWAY_DEFAULT_TOP_P, resolveRetryConfig } from './gateway-defaults.js';
|
|
23
23
|
export { contractSpecToFieldKeys, enrichParsedContentForOutputContract, resolveOutputContractFieldKeys } from './output-contract-normalizer.js';
|
|
@@ -176,17 +176,8 @@ export function initializeGatewayComponents(config) {
|
|
|
176
176
|
enableActivityTracking: config.enableActivityTracking ?? true,
|
|
177
177
|
customTracker: config.activityTracker,
|
|
178
178
|
logger,
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
: {
|
|
182
|
-
autoCost: config.aiTools?.enabled === false || config.aiTools?.calculateCost === false
|
|
183
|
-
? false
|
|
184
|
-
: {
|
|
185
|
-
enabled: true,
|
|
186
|
-
overwriteOuterCost: false,
|
|
187
|
-
...(config.aiTools?.bundledOnly ? { bundledOnly: true } : {})
|
|
188
|
-
}
|
|
189
|
-
})
|
|
179
|
+
// Billing is resolved in gateway before logSuccess; Activix gets outer.cost from that slice only.
|
|
180
|
+
...(config.activityTracker ? {} : { autoCost: false })
|
|
190
181
|
});
|
|
191
182
|
const templateRendering = mergeTemplateRenderOptions(defaultTemplateRendering, config.templateRendering);
|
|
192
183
|
const messageBuilderConfig = {
|
|
@@ -340,8 +340,8 @@ function pickRouterCostStatus(routerResponse) {
|
|
|
340
340
|
return status === 'priced' || status === 'unpriced' ? status : undefined;
|
|
341
341
|
}
|
|
342
342
|
/**
|
|
343
|
-
*
|
|
344
|
-
* Prefer {@link
|
|
343
|
+
* Step A/C/D cost slice when the router omits explicit `metadata.costStatus`.
|
|
344
|
+
* Prefer {@link resolveCostCompletionWithAiTools} at invoke boundaries.
|
|
345
345
|
*/
|
|
346
346
|
export function resolveActivityCostCompletion(tokens, costUsd) {
|
|
347
347
|
if (typeof costUsd === 'number' && Number.isFinite(costUsd)) {
|
|
@@ -353,8 +353,7 @@ export function resolveActivityCostCompletion(tokens, costUsd) {
|
|
|
353
353
|
return {};
|
|
354
354
|
}
|
|
355
355
|
/**
|
|
356
|
-
*
|
|
357
|
-
* otherwise gateway applies the G8 fallback (usage + no price → `unpriced`).
|
|
356
|
+
* Step A router passthrough + Step C when the router omits `metadata.costStatus`.
|
|
358
357
|
*/
|
|
359
358
|
export function resolveCostCompletionForActivity(routerResponse, tokens) {
|
|
360
359
|
const routerStatus = pickRouterCostStatus(routerResponse);
|
|
@@ -492,41 +491,36 @@ export function mapAiCostResultToResolvedActivityCost(base, result) {
|
|
|
492
491
|
};
|
|
493
492
|
}
|
|
494
493
|
/**
|
|
495
|
-
*
|
|
496
|
-
* Used at invoke boundaries after {@link resolveCostCompletionWithAiTools}.
|
|
494
|
+
* Step C/D: token usage without billing signal → `unpriced`; no usage → omit status.
|
|
497
495
|
*/
|
|
498
|
-
|
|
496
|
+
function finalizeInvokeBillingCost(billing, tokens) {
|
|
499
497
|
if (!billing.costStatus && hasNonZeroTokenUsage(tokens)) {
|
|
500
498
|
return { ...billing, costStatus: 'unpriced' };
|
|
501
499
|
}
|
|
502
500
|
return billing;
|
|
503
501
|
}
|
|
504
502
|
/**
|
|
505
|
-
*
|
|
503
|
+
* Post-invoke billing (Steps A→D): router cost, then catalog via ai-tools when still unpriced.
|
|
504
|
+
* Single entry point for `invoke()` / `invokeChat()` and trace enrichment.
|
|
506
505
|
*/
|
|
507
506
|
export async function resolveCostCompletionWithAiTools(routerResponse, tokens, options) {
|
|
508
507
|
const routerStatus = pickRouterCostStatus(routerResponse);
|
|
509
|
-
|
|
510
|
-
if (
|
|
511
|
-
|
|
512
|
-
|
|
513
|
-
|
|
514
|
-
|
|
515
|
-
|
|
516
|
-
|
|
517
|
-
|
|
518
|
-
|
|
519
|
-
|
|
520
|
-
|
|
521
|
-
|
|
522
|
-
|
|
523
|
-
const record = buildGatewayPricingRecord(routerResponse, tokens, options.mergedConfig);
|
|
524
|
-
const result = await options.calculator.calculateFromRecord(record);
|
|
525
|
-
return mapAiCostResultToResolvedActivityCost(base, result);
|
|
526
|
-
}
|
|
527
|
-
catch {
|
|
528
|
-
return ensureInvokeBillingCostStatus(base, tokens);
|
|
508
|
+
let billing = resolveCostCompletionForActivity(routerResponse, tokens);
|
|
509
|
+
if (billing.costStatus !== 'priced' &&
|
|
510
|
+
routerStatus !== 'unpriced' &&
|
|
511
|
+
options?.calculateCost !== false &&
|
|
512
|
+
options?.calculator &&
|
|
513
|
+
hasNonZeroTokenUsage(tokens)) {
|
|
514
|
+
try {
|
|
515
|
+
const record = buildGatewayPricingRecord(routerResponse, tokens, options.mergedConfig);
|
|
516
|
+
const result = await options.calculator.calculateFromRecord(record);
|
|
517
|
+
billing = mapAiCostResultToResolvedActivityCost(billing, result);
|
|
518
|
+
}
|
|
519
|
+
catch {
|
|
520
|
+
// Step B unavailable — Step C applies below.
|
|
521
|
+
}
|
|
529
522
|
}
|
|
523
|
+
return finalizeInvokeBillingCost(billing, tokens);
|
|
530
524
|
}
|
|
531
525
|
function applyBillingToTraceAttempt(attempt, billing) {
|
|
532
526
|
if (billing.costStatus === 'priced' || billing.costStatus === 'unpriced') {
|
|
@@ -72,8 +72,8 @@ export declare function hasNonZeroTokenUsage(tokens: {
|
|
|
72
72
|
total: number;
|
|
73
73
|
}): boolean;
|
|
74
74
|
/**
|
|
75
|
-
*
|
|
76
|
-
* Prefer {@link
|
|
75
|
+
* Step A/C/D cost slice when the router omits explicit `metadata.costStatus`.
|
|
76
|
+
* Prefer {@link resolveCostCompletionWithAiTools} at invoke boundaries.
|
|
77
77
|
*/
|
|
78
78
|
export declare function resolveActivityCostCompletion(tokens: {
|
|
79
79
|
prompt: number;
|
|
@@ -81,8 +81,7 @@ export declare function resolveActivityCostCompletion(tokens: {
|
|
|
81
81
|
total: number;
|
|
82
82
|
}, costUsd: number | undefined): ResolvedActivityCost;
|
|
83
83
|
/**
|
|
84
|
-
*
|
|
85
|
-
* otherwise gateway applies the G8 fallback (usage + no price → `unpriced`).
|
|
84
|
+
* Step A router passthrough + Step C when the router omits `metadata.costStatus`.
|
|
86
85
|
*/
|
|
87
86
|
export declare function resolveCostCompletionForActivity(routerResponse: unknown, tokens: {
|
|
88
87
|
prompt: number;
|
|
@@ -118,16 +117,8 @@ export declare function buildGatewayPricingRecord(routerResponse: unknown, token
|
|
|
118
117
|
}, mergedConfig?: unknown): Record<string, unknown>;
|
|
119
118
|
export declare function mapAiCostResultToResolvedActivityCost(base: ResolvedActivityCost, result: AiCostResult): ResolvedActivityCost;
|
|
120
119
|
/**
|
|
121
|
-
*
|
|
122
|
-
*
|
|
123
|
-
*/
|
|
124
|
-
export declare function ensureInvokeBillingCostStatus(billing: ResolvedActivityCost, tokens: {
|
|
125
|
-
prompt: number;
|
|
126
|
-
completion: number;
|
|
127
|
-
total: number;
|
|
128
|
-
}): ResolvedActivityCost;
|
|
129
|
-
/**
|
|
130
|
-
* Router cost passthrough, then optional @x12i/ai-tools catalog pricing when still unpriced.
|
|
120
|
+
* Post-invoke billing (Steps A→D): router cost, then catalog via ai-tools when still unpriced.
|
|
121
|
+
* Single entry point for `invoke()` / `invokeChat()` and trace enrichment.
|
|
131
122
|
*/
|
|
132
123
|
export declare function resolveCostCompletionWithAiTools(routerResponse: unknown, tokens: {
|
|
133
124
|
prompt: number;
|
package/dist-cjs/gateway.cjs
CHANGED
|
@@ -11,7 +11,7 @@ import { resolveRetryConfig } from './gateway-defaults.js';
|
|
|
11
11
|
import { buildMessages } from './message-builder.js';
|
|
12
12
|
import { extractJsonFromFlexMd } from './flex-md-loader.js';
|
|
13
13
|
import { enrichParsedContentForOutputContract, resolveOutputContractFieldKeys } from './output-contract-normalizer.js';
|
|
14
|
-
import { attachGatewayInvokeRejectionMetadata, buildGatewayFallbackAttemptsFromTrace, buildInvokeRejectionMetadata, capActivityFullResponsePayload, formatFallbackExhaustionMessage, logResolvedModelRouting, mapGatewayFallbackAttemptsToRouter, DEFAULT_ACTIVITY_FULL_RESPONSE_MAX_CHARS, extractCostUsdFromRouterResponse, extractTokenUsageFromRouterResponse, mergeConfig, pickEffectiveModelConfigForMetadata, pickInvokeRoutingMetadataSlice, pickTraceMergedRouterConfig, resolveCostCompletionWithAiTools,
|
|
14
|
+
import { attachGatewayInvokeRejectionMetadata, buildGatewayFallbackAttemptsFromTrace, buildInvokeRejectionMetadata, capActivityFullResponsePayload, formatFallbackExhaustionMessage, logResolvedModelRouting, mapGatewayFallbackAttemptsToRouter, DEFAULT_ACTIVITY_FULL_RESPONSE_MAX_CHARS, extractCostUsdFromRouterResponse, extractTokenUsageFromRouterResponse, mergeConfig, pickEffectiveModelConfigForMetadata, pickInvokeRoutingMetadataSlice, pickTraceMergedRouterConfig, resolveCostCompletionWithAiTools, buildTraceUsageSummary, enrichTraceAttemptsWithBilling, tryExtractRouterLikePayloadFromErrorChain } from './gateway-utils.js';
|
|
15
15
|
import { getAiToolsClient } from './ai-tools-client.js';
|
|
16
16
|
import { autoRegisterProviders } from './gateway-provider-auto-register.js';
|
|
17
17
|
import { setGatewayLastJobId, setGatewayRuntimeClients } from './runtime-objects.js';
|
|
@@ -140,7 +140,6 @@ export class AIGateway {
|
|
|
140
140
|
calculator: aiTools?.calculator ?? null,
|
|
141
141
|
calculateCost: this.config.aiTools?.calculateCost
|
|
142
142
|
});
|
|
143
|
-
costCompletionChat = ensureInvokeBillingCostStatus(costCompletionChat, tokensChat);
|
|
144
143
|
// Create enhanced response
|
|
145
144
|
const enhancedResponse = {
|
|
146
145
|
content: response.content || '',
|
|
@@ -615,7 +614,6 @@ export class AIGateway {
|
|
|
615
614
|
calculator: aiTools?.calculator ?? null,
|
|
616
615
|
calculateCost: this.config.aiTools?.calculateCost
|
|
617
616
|
});
|
|
618
|
-
costCompletion = ensureInvokeBillingCostStatus(costCompletion, tokens);
|
|
619
617
|
const routerMetaForCost = routerResponse?.metadata || {};
|
|
620
618
|
const routingMetadataSlice = pickInvokeRoutingMetadataSlice(routerResponse, mergedConfig);
|
|
621
619
|
const effectiveModelConfig = pickEffectiveModelConfigForMetadata(mergedConfig);
|
package/dist-cjs/index.cjs
CHANGED
|
@@ -17,7 +17,7 @@ export * from '@x12i/ai-providers-router';
|
|
|
17
17
|
export { AIGateway } from './gateway.js';
|
|
18
18
|
export { InstructionNotFoundError, InstructionBackendError, ModelRequiredError, MaxTokensRequiredError } from './instruction-errors.js';
|
|
19
19
|
export { autoRegisterProviders } from './gateway-provider-auto-register.js';
|
|
20
|
-
export { attachGatewayInvokeRejectionMetadata, buildInvokeRejectionMetadata, tryExtractRouterLikePayloadFromErrorChain, tryExtractFallbackAttemptsFromErrorChain, pickRequestIdsFromRouterLike, resolveActivityCostCompletion, resolveCostCompletionForActivity, resolveCostCompletionWithAiTools, buildGatewayPricingRecord, mapAiCostResultToResolvedActivityCost, catalogPricingSucceeded,
|
|
20
|
+
export { attachGatewayInvokeRejectionMetadata, buildInvokeRejectionMetadata, tryExtractRouterLikePayloadFromErrorChain, tryExtractFallbackAttemptsFromErrorChain, pickRequestIdsFromRouterLike, resolveActivityCostCompletion, resolveCostCompletionForActivity, resolveCostCompletionWithAiTools, buildGatewayPricingRecord, mapAiCostResultToResolvedActivityCost, catalogPricingSucceeded, extractUsageExtrasFromRouterResponse, buildTraceUsageSummary, enrichTraceAttemptsWithBilling, hasNonZeroTokenUsage, MODEL_PROFILE_UNROUTABLE, ModelProfileUnroutableError, ModelProfileInputRejectedError, buildGatewayFallbackAttemptsFromTrace, formatFallbackExhaustionMessage, logResolvedModelRouting, mapGatewayFallbackAttemptsToRouter } from './gateway-utils.js';
|
|
21
21
|
export { getGatewayOperationalMode, isProdGatewayMode, parseModelProviderSpec } from './gateway-mode.js';
|
|
22
22
|
export { DEFAULT_ACTIVITY_FULL_RESPONSE_MAX_CHARS, GATEWAY_DEFAULT_FREQUENCY_PENALTY, GATEWAY_DEFAULT_PRESENCE_PENALTY, GATEWAY_DEFAULT_RETRY, GATEWAY_DEFAULT_TEMPERATURE, GATEWAY_DEFAULT_TOP_P, resolveRetryConfig } from './gateway-defaults.js';
|
|
23
23
|
export { contractSpecToFieldKeys, enrichParsedContentForOutputContract, resolveOutputContractFieldKeys } from './output-contract-normalizer.js';
|
package/dist-cjs/index.d.ts
CHANGED
|
@@ -17,7 +17,7 @@ export { AIGateway } from './gateway.js';
|
|
|
17
17
|
export { InstructionNotFoundError, InstructionBackendError, ModelRequiredError, MaxTokensRequiredError } from './instruction-errors.js';
|
|
18
18
|
export { autoRegisterProviders } from './gateway-provider-auto-register.js';
|
|
19
19
|
export type { GatewayConfig, ProviderModelRef, ModelConfig, RetryConfig, ChatRequest, AIInvokeRequest, AIRequest, GatewayActionType, GatewayInvokeRejectionMetadata, GatewayFallbackAttempt, GatewayTraceRequestIds, GatewayTraceAttempt, GatewayTraceUsageSummary, GatewayTraceMergedConfig, EnhancedLLMResponse, InstructionMetadata, ValidationRule, TemplateRenderOptions, SmartInputConfig, SmartInputRenderOptions } from './types.js';
|
|
20
|
-
export { attachGatewayInvokeRejectionMetadata, buildInvokeRejectionMetadata, tryExtractRouterLikePayloadFromErrorChain, tryExtractFallbackAttemptsFromErrorChain, pickRequestIdsFromRouterLike, resolveActivityCostCompletion, resolveCostCompletionForActivity, resolveCostCompletionWithAiTools, buildGatewayPricingRecord, mapAiCostResultToResolvedActivityCost, catalogPricingSucceeded,
|
|
20
|
+
export { attachGatewayInvokeRejectionMetadata, buildInvokeRejectionMetadata, tryExtractRouterLikePayloadFromErrorChain, tryExtractFallbackAttemptsFromErrorChain, pickRequestIdsFromRouterLike, resolveActivityCostCompletion, resolveCostCompletionForActivity, resolveCostCompletionWithAiTools, buildGatewayPricingRecord, mapAiCostResultToResolvedActivityCost, catalogPricingSucceeded, extractUsageExtrasFromRouterResponse, buildTraceUsageSummary, enrichTraceAttemptsWithBilling, hasNonZeroTokenUsage, MODEL_PROFILE_UNROUTABLE, ModelProfileUnroutableError, ModelProfileInputRejectedError, buildGatewayFallbackAttemptsFromTrace, formatFallbackExhaustionMessage, logResolvedModelRouting, mapGatewayFallbackAttemptsToRouter } from './gateway-utils.js';
|
|
21
21
|
export { getGatewayOperationalMode, isProdGatewayMode, parseModelProviderSpec } from './gateway-mode.js';
|
|
22
22
|
export type { GatewayOperationalMode } from './gateway-mode.js';
|
|
23
23
|
export { DEFAULT_ACTIVITY_FULL_RESPONSE_MAX_CHARS, GATEWAY_DEFAULT_FREQUENCY_PENALTY, GATEWAY_DEFAULT_PRESENCE_PENALTY, GATEWAY_DEFAULT_RETRY, GATEWAY_DEFAULT_TEMPERATURE, GATEWAY_DEFAULT_TOP_P, resolveRetryConfig } from './gateway-defaults.js';
|