@x12i/ai-gateway 9.0.9 → 9.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +897 -998
- package/dist/activity-manager.js +46 -6
- package/dist/config/activity-tracking-config.d.ts +2 -1
- package/dist/config/activity-tracking-config.js +3 -2
- package/dist/gateway-memory.d.ts +1 -2
- package/dist/gateway-memory.js +1 -15
- package/dist/gateway-meta.js +3 -0
- package/dist/gateway-utils.d.ts +9 -1
- package/dist/gateway-utils.js +79 -18
- package/dist/gateway-validation.d.ts +3 -3
- package/dist/gateway-validation.js +10 -1
- package/dist/gateway.d.ts +2 -2
- package/dist/gateway.js +20 -3
- package/dist/index.d.ts +2 -2
- package/dist/instruction-optimizer.js +3 -0
- package/dist/runtime-objects.d.ts +2 -13
- package/dist/troubleshooting-helper.d.ts +0 -3
- package/dist/troubleshooting-helper.js +99 -20
- package/dist/types.d.ts +39 -89
- package/dist-cjs/activity-manager.cjs +45 -5
- package/dist-cjs/config/activity-tracking-config.cjs +3 -2
- package/dist-cjs/config/activity-tracking-config.d.ts +2 -1
- package/dist-cjs/gateway-memory.cjs +1 -15
- package/dist-cjs/gateway-memory.d.ts +1 -2
- package/dist-cjs/gateway-meta.cjs +3 -0
- package/dist-cjs/gateway-utils.cjs +81 -18
- package/dist-cjs/gateway-utils.d.ts +9 -1
- package/dist-cjs/gateway-validation.cjs +10 -1
- package/dist-cjs/gateway-validation.d.ts +3 -3
- package/dist-cjs/gateway.cjs +19 -2
- package/dist-cjs/gateway.d.ts +2 -2
- package/dist-cjs/index.d.ts +2 -2
- package/dist-cjs/instruction-optimizer.cjs +3 -0
- package/dist-cjs/runtime-objects.d.ts +2 -13
- package/dist-cjs/troubleshooting-helper.cjs +99 -20
- package/dist-cjs/troubleshooting-helper.d.ts +0 -3
- package/dist-cjs/types.d.ts +39 -89
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -12,12 +12,21 @@ Every **`invoke`** / **`invokeChat`** request **must** include **`identity`**: t
|
|
|
12
12
|
|
|
13
13
|
See [Identity contract](./docs/IDENTITY_OBJECT_CONTRACT.md) and [Logger initialization](./docs/LOGGER_INITIALIZATION.md).
|
|
14
14
|
|
|
15
|
+
### AI invoke payload (`gateway.invoke`)
|
|
16
|
+
|
|
17
|
+
Use a single object typed as **`AIInvokeRequest`** (exported alias: **`AIRequest`**). Besides **`identity`** / **`aiRequestId`** / **`agentId`** / **`instructions`**, every **`invoke()`** call **must** include:
|
|
18
|
+
|
|
19
|
+
- **`actionType`**: **`'skill'`** | **`'preSkill'`** | **`'postSkill'`** — whether this call is the main skill or a pre/post hook.
|
|
20
|
+
- **`actionRef`**: non-empty string — stable reference for the action (for example the skill id).
|
|
21
|
+
|
|
22
|
+
The gateway copies these onto **`request.identity`** for **`runContext`** (Activix) and onto activity documents when tracking is enabled. **`invokeChat()`** does **not** use these fields.
|
|
23
|
+
|
|
15
24
|
## Features
|
|
16
25
|
|
|
17
26
|
- **🔀 Provider Routing**: Dynamic provider registration and automatic routing with fallback support
|
|
18
27
|
- **📊 Context Propagation**: `aiRequestId` and identity propagation for distributed tracing (see [Identity contract](./docs/IDENTITY_OBJECT_CONTRACT.md))
|
|
19
28
|
- **⚡ Usage Tier Tracking**: RPM/TPM limit enforcement via `@x12i/x-models`
|
|
20
|
-
- **📈 Activity Tracking**: Comprehensive activity logging via `@x12i/activix`
|
|
29
|
+
- **📈 Activity Tracking**: Comprehensive activity logging via `@x12i/activix` v7 (xronox-activitix), fixed Mongo collections `ai-actions` / `bad-requests`, validated root-level **`outer` / `inner`** I/O plus **`runContext`** for Activix 7
|
|
21
30
|
- **📝 Structured Logging**: Production-ready logging via **`@x12i/logxer`** (LogMeta `jobId` / `sessionId` / `correlationId`, optional `debugKind`, default `runtimeIdentity` from env) with diagnostic tracing for instruction resolution and propagation debugging
|
|
22
31
|
- **📋 Rich Metadata**: Detailed execution metadata (latency, tokens, model, cost, `aiRequestId`, `identity`)
|
|
23
32
|
- **🏥 Health Checks**: Monitor provider health and availability
|
|
@@ -32,7 +41,7 @@ See [Identity contract](./docs/IDENTITY_OBJECT_CONTRACT.md) and [Logger initiali
|
|
|
32
41
|
- **🔄 Automatic Retry**: Intelligent retry logic for network errors, server errors (5xx), and throttling (429) with exponential backoff
|
|
33
42
|
- **📚 Content Resolver (nx-content)**: Resolve instruction keys, prompt keys, and instructions blocks from local folder or git repo via **nx-content**. See [Content Resolver — Upstream Guide](./CONTENT_RESOLVER_UPSTREAM_GUIDE.md).
|
|
34
43
|
- **📋 Instruction Metadata API**: Fetch structured metadata (outputType, schema, validation rules) for metadata-driven inference systems (v1.6.9+)
|
|
35
|
-
-
|
|
44
|
+
- **Post-processing & interceptors**: Request-level `transformations` hooks were **removed**; transform outputs in your app or use router interceptors (see [Response transformation hooks](#7-response-transformation-hooks-removed-from-request))
|
|
36
45
|
- **🔧 Custom/Dynamic Instructions Mode**: Use instructions that already contain full JSON schema - no schema formatting added, instructions used exactly as provided (v3.0.4+)
|
|
37
46
|
- **🤖 Response Repair Fallback**: In `mode=prod`, performs a minimal in-gateway repair attempt for malformed JSON/Markdown responses (logs a warning when used). In `mode=debug`, parsing failures hard-fail for maximum visibility.
|
|
38
47
|
- **📊 Response Fix Metadata**: Track when and how responses were fixed, including fix strategy, confidence, and warnings (v3.0.4+)
|
|
@@ -43,7 +52,7 @@ See [Identity contract](./docs/IDENTITY_OBJECT_CONTRACT.md) and [Logger initiali
|
|
|
43
52
|
- **📚 Standard Object Types**: Reference standard object types by name (e.g., `'sentiment-analysis'`) instead of defining schemas manually - includes examples, validation, and structured text instructions (v3.0.6+)
|
|
44
53
|
- **🔍 Auto-Extraction of Output Formats**: Automatically extract output format specifications from instruction templates when using structured-text mode - no need to manually specify `flexMdFormat` or `primaryObjectType` (v3.3.3+)
|
|
45
54
|
- **✅ Output Format Validation**: Validates output format specifications using flex-md SDK before sending to LLM, with configurable minimum compliance level (L0-L3)
|
|
46
|
-
- **📋 Contract
|
|
55
|
+
- **📋 Contract hints (`StructuredTextSpec`)**: Optional field on **`ChatRequest`** for typing / forward compatibility; dedicated activity **`contractOutput`** persistence is **not** implemented in the gateway runtime — validate **`parsedContent`** / **`content`** in your app or use interceptors (see **section 9** under Setup Guide)
|
|
47
56
|
|
|
48
57
|
## Installation
|
|
49
58
|
|
|
@@ -72,7 +81,7 @@ export AI_GATEWAY_DEBUG_REQUEST=true # Detailed request structure
|
|
|
72
81
|
export FLEX_MD_MIN_COMPLIANCE_LEVEL=L0 # Output format validation level (L0/L1/L2/L3, default: L0)
|
|
73
82
|
```
|
|
74
83
|
|
|
75
|
-
This logs the exact request structure received by `invoke()`, including property descriptors, which is critical for debugging validation errors
|
|
84
|
+
This logs the exact request structure received by `invoke()`, including property descriptors, which is critical for debugging validation errors (for example missing **`actionType`**, **`actionRef`**, or **`identity.jobId`** / **`identity.taskId`**).
|
|
76
85
|
|
|
77
86
|
### 🔍 Advanced Diagnostic Logging
|
|
78
87
|
|
|
@@ -132,11 +141,14 @@ gateway.register(new GrokProvider({
|
|
|
132
141
|
apiKey: process.env.GROK_API_KEY
|
|
133
142
|
}));
|
|
134
143
|
|
|
135
|
-
// Invoke with mandatory runtime identity (
|
|
144
|
+
// Invoke with mandatory runtime identity + action classification (Activix / tracing)
|
|
136
145
|
const response = await gateway.invoke({
|
|
137
146
|
aiRequestId: 'call-001',
|
|
138
147
|
agentId: 'agent-456',
|
|
148
|
+
actionType: 'skill',
|
|
149
|
+
actionRef: 'skills/quick-reply',
|
|
139
150
|
instructions: 'Reply briefly.',
|
|
151
|
+
prompt: '{{input}}',
|
|
140
152
|
identity: {
|
|
141
153
|
sessionId: 'run-1',
|
|
142
154
|
instance: { instanceId: 'agent-456', type: 'ai-reasoner' },
|
|
@@ -145,8 +157,9 @@ const response = await gateway.invoke({
|
|
|
145
157
|
taskId: 'task-789',
|
|
146
158
|
agentId: 'agent-456'
|
|
147
159
|
},
|
|
148
|
-
workingMemory: { input: 'Hello!' }
|
|
149
|
-
|
|
160
|
+
workingMemory: { input: 'Hello!' },
|
|
161
|
+
config: { model: 'gpt-4o-mini', provider: 'openai' }
|
|
162
|
+
// … primaryObjectType / objectTypes as needed for your flow (`invoke()` does not use client `messages`; use `invokeChat()` for raw transcripts)
|
|
150
163
|
});
|
|
151
164
|
|
|
152
165
|
// Response includes comprehensive metadata (including `identity`)
|
|
@@ -271,9 +284,9 @@ DEBUG=my-app # elevates verbose/debug when package is not
|
|
|
271
284
|
|
|
272
285
|
**What happens if `logger` is not provided:** The gateway creates a default **`Logxer`** via **`createLogxer`**. For production, prefer passing your app’s logger so levels, transports, and **`runtimeIdentity`** stay aligned with the rest of your stack.
|
|
273
286
|
|
|
274
|
-
### 2. Activity Tracking Configuration (xronox-activitix via @x12i/activix
|
|
287
|
+
### 2. Activity Tracking Configuration (xronox-activitix via @x12i/activix v7)
|
|
275
288
|
|
|
276
|
-
**Activix version:** This gateway targets **`@x12i/activix`
|
|
289
|
+
**Activix version:** This gateway targets **`@x12i/activix` v7.x** (built on `@x12i/xronox-store`). The dependency range is declared in `package.json` (currently `^7.1.0`). Activity I/O is stored at the **document root** as **`outer`** (and optional **`inner`**); the deprecated nested **`structure`** wrapper is not used.
|
|
277
290
|
|
|
278
291
|
**Where to configure:**
|
|
279
292
|
- Gateway constructor: `enableActivityTracking`, `activityTracker`
|
|
@@ -283,11 +296,39 @@ DEBUG=my-app # elevates verbose/debug when package is not
|
|
|
283
296
|
|
|
284
297
|
**✅ Centralized configuration**
|
|
285
298
|
|
|
286
|
-
The gateway
|
|
299
|
+
The gateway reads **Mongo connection** settings from the environment, but **collection names are package constants**: they are defined only in `src/config/activity-tracking-config.ts` and **cannot** be overridden by env vars. If you pass your own **`Activix`** instance, register collections whose **`name`** values match these strings exactly or persistence and dashboards will disagree.
|
|
300
|
+
|
|
301
|
+
| Mongo collection | Typical `activityType` | What gets stored |
|
|
302
|
+
|------------------|------------------------|------------------|
|
|
303
|
+
| **`ai-actions`** | `gateway-invocation` | Normal LLM runs after validation: full request/config snapshots, lifecycle, **`runContext`**, and Activix **`outer`** I/O for each gateway invocation. |
|
|
304
|
+
| **`skill-executions`** | `skill-execution` | Skill-specific lifecycle rows when skill execution tracking is used (separate from the main gateway row). |
|
|
305
|
+
| **`bad-requests`** | `bad-request` | Failures **before** `startActivity` (validation, configuration, format extraction, etc.). |
|
|
306
|
+
|
|
307
|
+
**What the gateway sends into Activix (lifecycle)**
|
|
308
|
+
|
|
309
|
+
`ActivityManager` drives **`@x12i/activix` v7** with a **two-phase** API:
|
|
310
|
+
|
|
311
|
+
1. **`startRecord`** — Inserts a new document with **`status: 'started'`**, **`startTime`**, **`runContext`** (same object as **`request.identity`**), root **`request`** / **`config`** snapshots, gateway metadata (e.g. **`activityType`**, **`aiRequestId`**), and the initial **`outer`** fragment (see below). Activix returns **`activityId`** (prefix **`act-`**, configured as the collection **`primaryKey`**); that id is used for all later updates — **not** `jobId`.
|
|
312
|
+
2. **`completeRecord`** or **`failRecord`** — Patches the **same** document by **`activityId`**. Success adds **`response`**, **`endTime`**, **`duration`**, **`cost`**, refreshed **`status`**, and sets **`outer.output`** to the completion payload. Failure adds error details (and may attach **`outer.output`** for certain failure modes such as response parsing).
|
|
287
313
|
|
|
288
|
-
**
|
|
289
|
-
|
|
290
|
-
-
|
|
314
|
+
**How a document is shaped (reading `ai-actions` in Mongo)**
|
|
315
|
+
|
|
316
|
+
- **`runContext`**: Canonical correlation BSON object — the merged gateway **`identity`** (`jobId`, `taskId`, `sessionId`, `instance`, `aiRequestId`, optional graph/skill linkage, `actionType` / `actionRef` on **`invoke()`**, plus any extra keys your upstream put on **`identity`**).
|
|
317
|
+
- **Root-level copies** of common identity fields may appear beside **`runContext`** for convenient indexing; treat **`runContext`** as the full envelope when in doubt.
|
|
318
|
+
- **`request`**: Structured snapshot only — **`raw`** / **`parsed`** instructions, context, prompt; **`messages`**; **`workingMemory`** (template/user payload). There is **no** separate legacy **`input`** field on this object; use **`workingMemory`**.
|
|
319
|
+
- **`config`**: `model`, `provider`, `temperature`, `maxTokens`, **`rawConfig`** (exact router config).
|
|
320
|
+
- **`outer`**: Activix v7 **validated I/O** at the document root. At **start**, **`outer.input`** contains **`activityType`** and the same **`request`** snapshot as root **`request`** when a body exists (`{ activityType, request }`). At **success**, **`outer.output`** matches the **`response`** object written on completion. Root **`request`** / **`response`** support querying and older tooling; **`outer`** satisfies Activix’s envelope — so the same logical request snapshot can appear both at **`request`** and under **`outer.input.request`** by design. Large provider blobs (**`response.content.fullResponse`**) and size limits are described in [Activities outer duplication & payload controls](./docs/ACTIVITIES_OUTER_DUPLICATION.md).
|
|
321
|
+
|
|
322
|
+
**Environment variable priority (Activix / Mongo — implemented in `@x12i/activix`, not in `activity-tracking-config.ts`):**
|
|
323
|
+
- **Mongo URI**: `MONGO_LOGS_URI` if set, otherwise **`MONGO_URI`**. If neither is set, Activix cannot use the database.
|
|
324
|
+
- **Mongo database name** (where collections such as **`ai-actions`** live): `ACTIVIX_DB_NAME` → `MONGO_AI_LOGS_DB` → **`MONGO_LOGS_DB`** → **`MONGO_DB`** → default **`activitix`** if none are set.
|
|
325
|
+
|
|
326
|
+
**Why you might not see `ai-actions` in Mongo**
|
|
327
|
+
|
|
328
|
+
1. **`storageMode: 'automatic'` (gateway default)** — On startup Activix **pings** Mongo with the URI above. If the URI is missing or the ping fails, it **silently falls back** to filesystem storage under **`./playground/`** (see `playground/collections/ai-actions/`). No Mongo collection is created in that mode. Check logs for **`Activity tracking persistence backend ready`**: `storageBackend` must be **`database`** for Mongo.
|
|
329
|
+
2. **Lazy collections** — MongoDB normally creates a collection on the **first insert**. Until at least one activity **`startRecord`** succeeds against Mongo, **`ai-actions`** may not exist.
|
|
330
|
+
3. **Wrong database in Compass / shell** — If you did not set `MONGO_LOGS_DB` / `MONGO_DB`, Activix uses the default database **`activitix`**, not your application’s primary DB.
|
|
331
|
+
4. **`.env` load order** — Environment variables must be available **before** `new AIGateway(...)`. If another package constructs the gateway before `dotenv.config()`, Activix may never see `MONGO_URI`.
|
|
291
332
|
|
|
292
333
|
**⚠️ CRITICAL: correlation and identity**
|
|
293
334
|
|
|
@@ -296,6 +337,7 @@ The gateway resolves activity tracking from environment variables via `src/confi
|
|
|
296
337
|
- **`jobTypeId`**, **`taskTypeId`**: Optional aggregation / grouping fields (unchanged semantics).
|
|
297
338
|
- **Each activity**: Gets its own **unique database record** with unique `_id` (MongoDB ObjectId).
|
|
298
339
|
- **Two-phase tracking**: `startActivity()` creates a new record; `logSuccess()` / `logFailure()` update the same record by that record’s id.
|
|
340
|
+
- **Payload shape**: Root **`request`** and **`outer.input.request`** both snapshot the same gateway request (Activix `outer` envelope). Large provider blobs on completion live under **`content.fullResponse`** unless you opt out — see [Activities outer duplication & payload controls](./docs/ACTIVITIES_OUTER_DUPLICATION.md).
|
|
299
341
|
|
|
300
342
|
**Runtime objects observability (debug only):**
|
|
301
343
|
|
|
@@ -332,13 +374,13 @@ See [Runtime Objects Observability Methodology](./docs/RUNTIME_OBJECTS_OBSERVABI
|
|
|
332
374
|
import { AIGateway } from '@x12i/ai-gateway';
|
|
333
375
|
import { OpenAIProvider } from '@x12i/ai-provider-openai';
|
|
334
376
|
|
|
335
|
-
// Set environment variables:
|
|
377
|
+
// Set environment variables (before creating the gateway):
|
|
336
378
|
// MONGO_URI=mongodb://localhost:27017
|
|
337
|
-
// MONGO_LOGS_DB=logs-db
|
|
379
|
+
// MONGO_LOGS_DB=my-logs-db # optional; default DB name for Activix is "activitix"
|
|
338
380
|
|
|
339
381
|
const gateway = new AIGateway({
|
|
340
382
|
enableActivityTracking: true, // default: true
|
|
341
|
-
// Activix is auto-configured; writes use collections ai-
|
|
383
|
+
// Activix is auto-configured; writes use collections ai-actions / bad-requests / skill-executions
|
|
342
384
|
});
|
|
343
385
|
|
|
344
386
|
gateway.register(new OpenAIProvider({
|
|
@@ -346,7 +388,7 @@ gateway.register(new OpenAIProvider({
|
|
|
346
388
|
}));
|
|
347
389
|
```
|
|
348
390
|
|
|
349
|
-
**Advanced (custom Activix
|
|
391
|
+
**Advanced (custom Activix v7 instance):**
|
|
350
392
|
|
|
351
393
|
If you pass your own `Activix`, configure **the same collection names** the gateway expects so routing matches persistence:
|
|
352
394
|
|
|
@@ -364,7 +406,7 @@ const statusValues = {
|
|
|
364
406
|
|
|
365
407
|
const activityTracker = new Activix({
|
|
366
408
|
collections: [
|
|
367
|
-
{ name: 'ai-
|
|
409
|
+
{ name: 'ai-actions', statusValues },
|
|
368
410
|
{ name: 'skill-executions', statusValues },
|
|
369
411
|
{ name: 'bad-requests', statusValues }
|
|
370
412
|
]
|
|
@@ -376,13 +418,16 @@ const gateway = new AIGateway({
|
|
|
376
418
|
});
|
|
377
419
|
```
|
|
378
420
|
|
|
421
|
+
When the gateway constructs Activix internally, each collection uses **`primaryKey: 'activityId'`** and **`primaryKeyPrefix: 'act-'`**. If you supply a custom **`Activix`**, align with that (or your **`activityId`** values will not match operational tooling).
|
|
422
|
+
|
|
379
423
|
**What gets tracked (persisted when DB is configured):**
|
|
380
424
|
- **Identity**: Fields aligned with **`request.identity`** / Activix **`runContext`**: **`aiRequestId`**, upstream **`jobId`** and **`taskId`**, `sessionId`, `instance`, plus optional `jobTypeId`, `agentId`, `taskTypeId`, etc., as provided
|
|
381
425
|
- **Timing**: `startTime`, `endTime`, `duration`, `status` (`started|success|failed`)
|
|
382
|
-
- **Request data**: Stored in
|
|
383
|
-
- **Config data**: Stored in
|
|
384
|
-
- **Response data**: Stored in
|
|
385
|
-
- **
|
|
426
|
+
- **Request data**: Stored in **`request`** (raw/parsed prompts, **`messages`**, **`workingMemory`**) and mirrored under **`outer.input.request`** when Activix **`outer`** is populated — see table above
|
|
427
|
+
- **Config data**: Stored in **`config`** (model, provider, temperature, maxTokens, **`rawConfig`**)
|
|
428
|
+
- **Response data**: Stored in **`response`** on completion (content, metadata, optional **`fullResponse`** per diagnostics)
|
|
429
|
+
- **Activix I/O**: Root **`outer`** — **`outer.input`** at start, **`outer.output`** on success (and some failure paths)
|
|
430
|
+
- **Cost**: Calculated and stored per activity on success
|
|
386
431
|
|
|
387
432
|
**Best Practices for Type IDs:**
|
|
388
433
|
- **`jobTypeId`**: Use MD5 hash of your job type string (e.g., `MD5('data-processing-job')`) for consistent job-level aggregation
|
|
@@ -398,12 +443,12 @@ const gateway = new AIGateway({
|
|
|
398
443
|
|
|
399
444
|
**Default:** Activity tracking is enabled by default; without DB config it will log but not persist.
|
|
400
445
|
|
|
401
|
-
**✅ Activix
|
|
446
|
+
**✅ Activix v7 integration**
|
|
402
447
|
|
|
403
448
|
1. **Configuration** (`activity-tracking-config.ts`):
|
|
404
|
-
- Mongo connection from env; **collection names** `ai-
|
|
449
|
+
- Mongo connection from env; **collection names** `ai-actions`, `bad-requests`, and `skill-executions` are **fixed literals** (not env-driven) for consistency across deployments.
|
|
405
450
|
|
|
406
|
-
2. **Lifecycle** (`@x12i/activix`
|
|
451
|
+
2. **Lifecycle** (`@x12i/activix` v7):
|
|
407
452
|
- ✅ `startRecord` / `completeRecord` / `failRecord` (two-phase lifecycle)
|
|
408
453
|
- ✅ Status transitions: `started` → `success` or `failed` (per your `statusValues` mapping)
|
|
409
454
|
- ✅ Persistence via xronox-store queue semantics (see Activix package docs)
|
|
@@ -431,14 +476,27 @@ When executing skills (instruction keys starting with `skills/`), the gateway au
|
|
|
431
476
|
**Example: Basic Skill Execution**
|
|
432
477
|
|
|
433
478
|
```typescript
|
|
479
|
+
const aiRequestId = 'skill-pa-1';
|
|
434
480
|
const response = await gateway.invoke({
|
|
435
|
-
|
|
481
|
+
aiRequestId,
|
|
436
482
|
agentId: 'my-agent',
|
|
483
|
+
actionType: 'skill',
|
|
484
|
+
actionRef: 'skills/professional-answer',
|
|
437
485
|
instructions: 'skills/professional-answer',
|
|
486
|
+
prompt: '{{question}}',
|
|
487
|
+
workingMemory: { question: 'What is...' },
|
|
438
488
|
inferenceType: 'question-answer', // Recommended
|
|
439
489
|
skillId: 'skills/professional-answer', // Optional, auto-detected from instructions
|
|
440
|
-
|
|
441
|
-
|
|
490
|
+
primaryObjectType: 'professional-answer',
|
|
491
|
+
identity: {
|
|
492
|
+
sessionId: 's1',
|
|
493
|
+
instance: { instanceId: 'my-agent', type: 'test' },
|
|
494
|
+
aiRequestId,
|
|
495
|
+
jobId: 'job-123',
|
|
496
|
+
taskId: 'task-1',
|
|
497
|
+
agentId: 'my-agent'
|
|
498
|
+
},
|
|
499
|
+
config: { model: 'gpt-4o', provider: 'openai' }
|
|
442
500
|
});
|
|
443
501
|
|
|
444
502
|
// Get the activity ID for linking child skills
|
|
@@ -448,24 +506,52 @@ const activityId = response.metadata?.activityId;
|
|
|
448
506
|
**Example: Nested Skill Execution (Skill Calling Another Skill)**
|
|
449
507
|
|
|
450
508
|
```typescript
|
|
509
|
+
const parentAiRequestId = 'skill-parent-1';
|
|
451
510
|
// Parent skill execution
|
|
452
511
|
const parentResponse = await gateway.invoke({
|
|
453
|
-
|
|
512
|
+
aiRequestId: parentAiRequestId,
|
|
454
513
|
agentId: 'my-agent',
|
|
514
|
+
actionType: 'skill',
|
|
515
|
+
actionRef: 'skills/parent-skill',
|
|
455
516
|
instructions: 'skills/parent-skill',
|
|
517
|
+
prompt: '{{input}}',
|
|
518
|
+
workingMemory: { input: '…' },
|
|
456
519
|
inferenceType: 'analysis',
|
|
457
|
-
skillId: 'skills/parent-skill'
|
|
520
|
+
skillId: 'skills/parent-skill',
|
|
521
|
+
identity: {
|
|
522
|
+
sessionId: 's1',
|
|
523
|
+
instance: { instanceId: 'my-agent', type: 'test' },
|
|
524
|
+
aiRequestId: parentAiRequestId,
|
|
525
|
+
jobId: 'job-123',
|
|
526
|
+
taskId: 'task-parent',
|
|
527
|
+
agentId: 'my-agent'
|
|
528
|
+
},
|
|
529
|
+
config: { model: 'gpt-4o', provider: 'openai' }
|
|
458
530
|
});
|
|
459
531
|
|
|
532
|
+
const childAiRequestId = 'skill-child-1';
|
|
460
533
|
// When parent skill calls child skill, pass parent's activityId and skillId
|
|
461
534
|
const childResponse = await gateway.invoke({
|
|
462
|
-
|
|
535
|
+
aiRequestId: childAiRequestId,
|
|
463
536
|
agentId: 'my-agent',
|
|
537
|
+
actionType: 'skill',
|
|
538
|
+
actionRef: 'skills/child-skill',
|
|
464
539
|
instructions: 'skills/child-skill',
|
|
540
|
+
prompt: '{{input}}',
|
|
541
|
+
workingMemory: { input: '…' },
|
|
465
542
|
inferenceType: 'question-answer',
|
|
466
543
|
skillId: 'skills/child-skill',
|
|
467
544
|
masterSkillActivityId: parentResponse.metadata?.activityId, // ✅ Parent's activity ID
|
|
468
|
-
masterSkillId: 'skills/parent-skill' // ✅ Parent's skill ID
|
|
545
|
+
masterSkillId: 'skills/parent-skill', // ✅ Parent's skill ID
|
|
546
|
+
identity: {
|
|
547
|
+
sessionId: 's1',
|
|
548
|
+
instance: { instanceId: 'my-agent', type: 'test' },
|
|
549
|
+
aiRequestId: childAiRequestId,
|
|
550
|
+
jobId: 'job-123', // ✅ Same jobId (links activities)
|
|
551
|
+
taskId: 'task-child',
|
|
552
|
+
agentId: 'my-agent'
|
|
553
|
+
},
|
|
554
|
+
config: { model: 'gpt-4o', provider: 'openai' }
|
|
469
555
|
});
|
|
470
556
|
```
|
|
471
557
|
|
|
@@ -536,11 +622,23 @@ const gateway = new AIGateway({
|
|
|
536
622
|
|
|
537
623
|
**Step 3: Request-level override:**
|
|
538
624
|
```typescript
|
|
625
|
+
const aiRequestId = 'cfg-1';
|
|
539
626
|
const response = await gateway.invoke({
|
|
540
|
-
|
|
627
|
+
aiRequestId,
|
|
541
628
|
agentId: 'agent-456',
|
|
629
|
+
actionType: 'skill',
|
|
630
|
+
actionRef: 'skills/helpful',
|
|
542
631
|
instructions: 'You are helpful',
|
|
543
|
-
|
|
632
|
+
prompt: '{{input}}',
|
|
633
|
+
workingMemory: { input: 'Hello' },
|
|
634
|
+
identity: {
|
|
635
|
+
sessionId: 's1',
|
|
636
|
+
instance: { instanceId: 'agent-456', type: 'test' },
|
|
637
|
+
aiRequestId,
|
|
638
|
+
jobId: 'job-123',
|
|
639
|
+
taskId: 'task-1',
|
|
640
|
+
agentId: 'agent-456'
|
|
641
|
+
},
|
|
544
642
|
config: {
|
|
545
643
|
model: 'gpt-4o', // Overrides gateway default
|
|
546
644
|
provider: 'openai', // Overrides gateway default
|
|
@@ -642,9 +740,12 @@ const gateway = new AIGateway({
|
|
|
642
740
|
**How to configure:**
|
|
643
741
|
|
|
644
742
|
```typescript
|
|
743
|
+
const aiRequestId = 'tpl-1';
|
|
645
744
|
const response = await gateway.invoke({
|
|
646
|
-
|
|
745
|
+
aiRequestId,
|
|
647
746
|
agentId: 'agent-456',
|
|
747
|
+
actionType: 'skill',
|
|
748
|
+
actionRef: 'skills/professional-answer',
|
|
648
749
|
// Instructions can be a key (resolved from content resolver) or text (parsed as template)
|
|
649
750
|
instructions: 'professional-answer.instructions', // Key with suffix
|
|
650
751
|
// OR: instructions: 'You are a {{role}} assistant.', // Text with template variables
|
|
@@ -652,13 +753,21 @@ const response = await gateway.invoke({
|
|
|
652
753
|
// Prompts can be a key (resolved from content resolver) or text (parsed as template)
|
|
653
754
|
prompt: 'professional-answer.prompt', // Key with suffix
|
|
654
755
|
// OR: prompt: 'Analyze this {{type}}: {{input}}', // Text with template variables
|
|
655
|
-
input: 'This is a review',
|
|
656
756
|
workingMemory: {
|
|
657
757
|
role: 'helpful',
|
|
658
758
|
project: 'AI Gateway',
|
|
659
759
|
type: 'product review',
|
|
660
760
|
input: 'This is a review'
|
|
661
|
-
}
|
|
761
|
+
},
|
|
762
|
+
identity: {
|
|
763
|
+
sessionId: 's1',
|
|
764
|
+
instance: { instanceId: 'agent-456', type: 'test' },
|
|
765
|
+
aiRequestId,
|
|
766
|
+
jobId: 'job-123',
|
|
767
|
+
taskId: 'task-1',
|
|
768
|
+
agentId: 'agent-456'
|
|
769
|
+
},
|
|
770
|
+
config: { model: 'gpt-4o', provider: 'openai' }
|
|
662
771
|
});
|
|
663
772
|
```
|
|
664
773
|
|
|
@@ -679,7 +788,7 @@ const response = await gateway.invoke({
|
|
|
679
788
|
**Gateway template options (passthrough)**
|
|
680
789
|
|
|
681
790
|
- **`GatewayConfig.templateRendering`** — default `TemplateRenderOptions` for every `invoke()` render path (merged after packaged **`src/defaults/template-rendering.json`**, which ships with **`subPathSearch.enabled: false`**). Your gateway config overrides that JSON.
|
|
682
|
-
- **`templateRenderOptions` on the request** (`ChatRequest` / `
|
|
791
|
+
- **`templateRenderOptions` on the request** (`ChatRequest` / `AIInvokeRequest`) — merged on top of the gateway default for that call only (per-field override; `subPathSearch` fields merge with request winning).
|
|
683
792
|
- Supported fields match the parser: **`templateId`**, **`subPathSearch`** (`enabled`, `roots`), **`silentMissingMustTokens`** (legacy Handlebars-style silence for missing MUST paths).
|
|
684
793
|
- **Sub-path root priority:** `subPathSearch.roots` is an **ordered** list. The parser tries roots in **array order**; **the first root that resolves the leaf path wins** (see ISSUE-005). There is no separate “priority” field—the order of `roots` *is* the priority. Omit `roots` when `enabled` is true to use **`@x12i/rendrix`** packaged defaults.
|
|
685
794
|
|
|
@@ -694,7 +803,7 @@ new AIGateway({
|
|
|
694
803
|
}
|
|
695
804
|
});
|
|
696
805
|
```
|
|
697
|
-
- **Memory overlay priority** for
|
|
806
|
+
- **Memory overlay priority** for Rendrix resolution (when memories are supplied): **`shortTermMemory`** → **`workingMemory`** → **`experienceMemory`** → **`knowledgeMemory`**. (Request-level **`templateTokens`** was removed; put overrides in **`workingMemory`** or resolver-backed memories.)
|
|
698
807
|
- Root **`config.defaults.json`** may include a **`templateRendering`** block for apps that merge this file into `GatewayConfig`. Packaged **`template-rendering.json`** includes a sample **`roots`** order (used when you turn **`enabled`** on; while **`enabled`** is **`false`**, roots are ignored by the parser).
|
|
699
808
|
|
|
700
809
|
**Template-Based Prompts:**
|
|
@@ -811,7 +920,7 @@ const statusValues = {
|
|
|
811
920
|
|
|
812
921
|
const activityTracker = new Activix({
|
|
813
922
|
collections: [
|
|
814
|
-
{ name: 'ai-
|
|
923
|
+
{ name: 'ai-actions', statusValues },
|
|
815
924
|
{ name: 'skill-executions', statusValues },
|
|
816
925
|
{ name: 'bad-requests', statusValues }
|
|
817
926
|
]
|
|
@@ -870,60 +979,44 @@ For each configuration option, priority is (highest to lowest):
|
|
|
870
979
|
|
|
871
980
|
## Enhanced Gateway Features
|
|
872
981
|
|
|
873
|
-
|
|
982
|
+
Throughout this section, **`gateway.invoke()`** examples must include **`aiRequestId`**, **`identity`** (with **`jobId`** / **`taskId`**), **`actionType`**, **`actionRef`**, and usually **`prompt`** + **`workingMemory`** unless noted. Older snippets that show only **`jobId`** / **`input`** / **`parseOptions`** / **`validateOutputSchema`** / **`transformations`** are outdated relative to the current [`AIInvokeRequest`](#chatrequest-and-airequest--aiinvokerequest) type.
|
|
983
|
+
|
|
984
|
+
### 1. Context Propagation (identity / job / task)
|
|
874
985
|
|
|
875
|
-
|
|
986
|
+
Correlation flows through **`request.identity`**: **`identity.jobId`** and **`identity.taskId`** are supplied by the upstream client, forwarded to the router, echoed in **`response.metadata.identity`**, and persisted as Activix **`runContext`**. For **`invoke()`**, also set **`actionType`** and **`actionRef`** (see [AI invoke payload](#ai-invoke-payload-gatewayinvoke)).
|
|
876
987
|
|
|
877
988
|
```typescript
|
|
989
|
+
const aiRequestId = 'call-001';
|
|
878
990
|
const response = await gateway.invoke({
|
|
879
|
-
|
|
880
|
-
|
|
881
|
-
|
|
882
|
-
|
|
883
|
-
|
|
884
|
-
|
|
885
|
-
|
|
886
|
-
|
|
887
|
-
|
|
888
|
-
|
|
889
|
-
|
|
890
|
-
|
|
891
|
-
|
|
892
|
-
|
|
893
|
-
|
|
894
|
-
|
|
895
|
-
graphId: 'graph-123', // Optional: Graph execution context
|
|
896
|
-
nodeId: 'node-456', // Optional: Node execution context
|
|
897
|
-
config: { model: 'gpt-5-nono' }
|
|
991
|
+
aiRequestId,
|
|
992
|
+
agentId: 'agent-456',
|
|
993
|
+
actionType: 'skill',
|
|
994
|
+
actionRef: 'skills/example',
|
|
995
|
+
instructions: 'You are a helpful assistant.',
|
|
996
|
+
prompt: '{{input}}',
|
|
997
|
+
workingMemory: { input: 'What is AI?' },
|
|
998
|
+
identity: {
|
|
999
|
+
sessionId: 'run-1',
|
|
1000
|
+
instance: { instanceId: 'agent-456', type: 'ai-reasoner' },
|
|
1001
|
+
aiRequestId,
|
|
1002
|
+
jobId: 'job-123',
|
|
1003
|
+
taskId: 'task-789',
|
|
1004
|
+
agentId: 'agent-456'
|
|
1005
|
+
},
|
|
1006
|
+
config: { model: 'gpt-5-nano', provider: 'openai' }
|
|
898
1007
|
});
|
|
899
|
-
|
|
900
|
-
// jobId is automatically:
|
|
901
|
-
// - Attached to request metadata
|
|
902
|
-
// - Included in response metadata
|
|
903
|
-
// - Logged in all log entries
|
|
904
|
-
// - Tracked in activity logs
|
|
905
1008
|
```
|
|
906
1009
|
|
|
907
|
-
**
|
|
908
|
-
|
|
909
|
-
|
|
910
|
-
|
|
911
|
-
|
|
912
|
-
|
|
913
|
-
- **Optional fields**: `context` (inserted as system message between instructions and user content), `workingMemory` (for template variables), `graphId`/`nodeId`/`coreSkillId` (for graph execution context - see [Graph Execution Support](./docs/GRAPH_EXECUTION_SUPPORT.md))
|
|
914
|
-
- **Model requirement**: `config.model` must be supplied per request (no default model)
|
|
915
|
-
|
|
916
|
-
**Important Notes:**
|
|
917
|
-
- `instructions` is **always required**, even when using `messages` array
|
|
918
|
-
- When `messages` is provided, the gateway constructs system messages from `instructions` (and `context` if provided), then appends your `messages` array
|
|
919
|
-
- `context` is inserted as a separate system message between instructions and the user prompt/input
|
|
920
|
-
- **Template-Based Instructions and Prompts**: Both `instructions` and `prompt` can be resolved from the content resolver (nx-content) using keys (e.g., `professional-answer.instructions` and `professional-answer.prompt`). Both receive the same memory context for template rendering. See [Content Resolver — Upstream Guide](./CONTENT_RESOLVER_UPSTREAM_GUIDE.md) and [Prompt Template Usage Guide](./docs/PROMPT_TEMPLATE_USAGE.md).
|
|
921
|
-
- All template fields (`instructions`, `context`, `prompt`) support template variables via `workingMemory`
|
|
1010
|
+
**Note:** On **`invoke()`**, provider **`messages`** are produced by the internal message builder from **`instructions`** / **`prompt`** / **`context`** (not from a client **`messages`** array). Use **`invokeChat()`** if you need to pass a raw **`messages`** array.
|
|
1011
|
+
|
|
1012
|
+
**Request requirements (`invoke`):**
|
|
1013
|
+
- **Required**: `aiRequestId`, `agentId`, `instructions`, `identity` (with upstream `jobId` / `taskId`), `actionType`, `actionRef`; structured flows typically need `prompt` + `workingMemory` for templates (see message builder).
|
|
1014
|
+
- **Optional**: `context`, `messages`, graph/skill linkage fields, `config` / `modelConfig`.
|
|
1015
|
+
- Do **not** use top-level **`input`** — use **`workingMemory.input`** (and **`{{input}}`** in templates).
|
|
922
1016
|
|
|
923
1017
|
**Benefits:**
|
|
924
|
-
-
|
|
925
|
-
-
|
|
926
|
-
- Debug complex multi-step AI workflows
|
|
1018
|
+
- Stable tracing across logs and Activix
|
|
1019
|
+
- Clear separation between chat (`invokeChat`) and structured invoke (`invoke`)
|
|
927
1020
|
|
|
928
1021
|
### 2. Usage Tier Tracking (RPM/TPM Limits)
|
|
929
1022
|
|
|
@@ -956,9 +1049,9 @@ console.log(`RPM Limit: ${tierInfo?.rpm}, TPM Limit: ${tierInfo?.tpm}`);
|
|
|
956
1049
|
- `tier-4`: 10,000 RPM, 4M TPM
|
|
957
1050
|
- `tier-5`: 15,000 RPM, 40M TPM
|
|
958
1051
|
|
|
959
|
-
### 3. Activity Tracking (xronox-activitix via @x12i/activix
|
|
1052
|
+
### 3. Activity Tracking (xronox-activitix via @x12i/activix v7)
|
|
960
1053
|
|
|
961
|
-
The gateway uses **`@x12i/activix`
|
|
1054
|
+
The gateway uses **`@x12i/activix` v7** (xronox-activitix) for full lifecycle logging. Recommended: enable MongoDB persistence so tracking is automatic. Writes use the **fixed** Mongo collection names **`ai-actions`**, **`bad-requests`**, and **`skill-executions`** (literal strings from `activity-tracking-config.ts`; see **section 2** for what lands in each collection and how documents are shaped).
|
|
962
1055
|
|
|
963
1056
|
#### ⚠️ CRITICAL: correlation, identity, and unique record ids
|
|
964
1057
|
|
|
@@ -970,23 +1063,21 @@ The gateway uses **`@x12i/activix` v6** (xronox-activitix) for full lifecycle lo
|
|
|
970
1063
|
- **`jobTypeId`**, **`taskTypeId`**: Optional aggregation fields (same ideas as before).
|
|
971
1064
|
- **Activity**: Each individual LLM request is a separate **activity** with its own unique record.
|
|
972
1065
|
|
|
973
|
-
2.
|
|
974
|
-
- **`
|
|
975
|
-
-
|
|
1066
|
+
2. **`activityId` + Mongo `_id` (not `jobId`)**
|
|
1067
|
+
- Each row gets Mongo **`_id`** and an Activix **`activityId`** (e.g. **`act-…`**, the collection **`primaryKey`**). **`completeRecord` / `failRecord`** address the row by **`activityId`** returned from **`startRecord`**.
|
|
1068
|
+
- **`jobId`** / **`taskId`** mirror upstream **`identity`** for correlation only; many rows may share a **`jobId`**.
|
|
976
1069
|
|
|
977
|
-
3. **Two-phase tracking (Activix
|
|
978
|
-
- **Phase 1 (start)**: Creates a NEW database
|
|
979
|
-
- Sends request
|
|
980
|
-
- Returns
|
|
981
|
-
- **Phase 2 (complete / fail)**: Updates the SAME
|
|
982
|
-
-
|
|
983
|
-
-
|
|
1070
|
+
3. **Two-phase tracking (Activix v7)**
|
|
1071
|
+
- **Phase 1 (start)**: Creates a NEW database document
|
|
1072
|
+
- Sends **`runContext`**, **`request`**, **`config`**, **`startTime`**, **`status: 'started'`**, plus Activix **`outer.input`** (wraps **`activityType`** and the same **`request`** snapshot when present — see section 2).
|
|
1073
|
+
- Returns **`activityId`** (and record payload) for phase 2.
|
|
1074
|
+
- **Phase 2 (complete / fail)**: Updates the SAME document by **`activityId`**
|
|
1075
|
+
- Success: **`response`**, **`cost`**, **`endTime`**, **`duration`**, **`status`**, and **`outer.output`** set to the completion **`response`** payload (request/config are **not** re-sent).
|
|
1076
|
+
- Failure: error payload and timing; optional **`response`** / **`outer.output`** only for specific failure kinds.
|
|
984
1077
|
|
|
985
|
-
4. **
|
|
986
|
-
-
|
|
987
|
-
-
|
|
988
|
-
- Response fields (`content`, `metadata`) → **ONLY in `response` object**
|
|
989
|
-
- **NO duplication**: Fields are NOT at root level, only in their structured objects
|
|
1078
|
+
4. **Structured fields vs Activix `outer` (v2.6.0+):**
|
|
1079
|
+
- LLM request fields live under root **`request`** (not as loose keys on the document root). Config under **`config`**; completion payload under **`response`**.
|
|
1080
|
+
- **`outer`** duplicates the logical **request** snapshot under **`outer.input.request`** when applicable — required for Activix v7 validation — so root **`request`** and **`outer.input.request`** align by design ([details](./docs/ACTIVITIES_OUTER_DUPLICATION.md)).
|
|
990
1081
|
|
|
991
1082
|
**Example: same logical job, three LLM calls**
|
|
992
1083
|
|
|
@@ -1001,31 +1092,49 @@ function md5(text: string): string {
|
|
|
1001
1092
|
|
|
1002
1093
|
const jobTypeId = md5('data-processing-job');
|
|
1003
1094
|
|
|
1004
|
-
const identityBase = {
|
|
1005
|
-
jobId: 'job-123',
|
|
1006
|
-
sessionId: 'sess-1',
|
|
1007
|
-
instance: { instanceId: 'inst-1', type: 'gateway' }
|
|
1008
|
-
};
|
|
1009
|
-
|
|
1010
1095
|
await gateway.invoke({
|
|
1011
1096
|
aiRequestId: 'req-001',
|
|
1012
|
-
identity: { ...identityBase, taskId: 'task-001' },
|
|
1013
|
-
jobTypeId,
|
|
1014
1097
|
agentId: 'agent-1',
|
|
1015
|
-
|
|
1098
|
+
actionType: 'skill',
|
|
1099
|
+
actionRef: 'skills/example',
|
|
1100
|
+
instructions: '…',
|
|
1101
|
+
prompt: '{{input}}',
|
|
1102
|
+
workingMemory: { input: '…' },
|
|
1103
|
+
jobTypeId,
|
|
1104
|
+
identity: {
|
|
1105
|
+
sessionId: 'sess-1',
|
|
1106
|
+
instance: { instanceId: 'inst-1', type: 'gateway' },
|
|
1107
|
+
aiRequestId: 'req-001',
|
|
1108
|
+
jobId: 'job-123',
|
|
1109
|
+
taskId: 'task-001',
|
|
1110
|
+
agentId: 'agent-1'
|
|
1111
|
+
},
|
|
1112
|
+
config: { model: 'gpt-5-nano', provider: 'openai' }
|
|
1016
1113
|
});
|
|
1017
1114
|
|
|
1018
1115
|
await gateway.invoke({
|
|
1019
1116
|
aiRequestId: 'req-002',
|
|
1020
|
-
identity: { ...identityBase, taskId: 'task-002' },
|
|
1021
|
-
jobTypeId,
|
|
1022
1117
|
agentId: 'agent-1',
|
|
1023
|
-
|
|
1118
|
+
actionType: 'skill',
|
|
1119
|
+
actionRef: 'skills/example',
|
|
1120
|
+
instructions: '…',
|
|
1121
|
+
prompt: '{{input}}',
|
|
1122
|
+
workingMemory: { input: '…' },
|
|
1123
|
+
jobTypeId,
|
|
1124
|
+
identity: {
|
|
1125
|
+
sessionId: 'sess-1',
|
|
1126
|
+
instance: { instanceId: 'inst-1', type: 'gateway' },
|
|
1127
|
+
aiRequestId: 'req-002',
|
|
1128
|
+
jobId: 'job-123',
|
|
1129
|
+
taskId: 'task-002',
|
|
1130
|
+
agentId: 'agent-1'
|
|
1131
|
+
},
|
|
1132
|
+
config: { model: 'gpt-5-nano', provider: 'openai' }
|
|
1024
1133
|
});
|
|
1025
1134
|
|
|
1026
|
-
// Query in Mongo (main collection name is ai-
|
|
1027
|
-
// db.getCollection('ai-
|
|
1028
|
-
// db.getCollection('ai-
|
|
1135
|
+
// Query in Mongo (main collection name is ai-actions):
|
|
1136
|
+
// db.getCollection('ai-actions').find({ 'runContext.aiRequestId': 'req-001' })
|
|
1137
|
+
// db.getCollection('ai-actions').find({ 'runContext.jobId': 'job-123' })
|
|
1029
1138
|
```
|
|
1030
1139
|
|
|
1031
1140
|
#### Configuration
|
|
@@ -1043,7 +1152,7 @@ const statusValues = {
|
|
|
1043
1152
|
|
|
1044
1153
|
const activityTracker = new Activix({
|
|
1045
1154
|
collections: [
|
|
1046
|
-
{ name: 'ai-
|
|
1155
|
+
{ name: 'ai-actions', statusValues },
|
|
1047
1156
|
{ name: 'skill-executions', statusValues },
|
|
1048
1157
|
{ name: 'bad-requests', statusValues }
|
|
1049
1158
|
]
|
|
@@ -1064,12 +1173,19 @@ const gateway = new AIGateway({
|
|
|
1064
1173
|
|
|
1065
1174
|
#### Database Record Structure
|
|
1066
1175
|
|
|
1176
|
+
Example shape for a completed row in **`ai-actions`** (`activityType: 'gateway-invocation'`). **`skill-executions`** / **`bad-requests`** share the same Activix lifecycle pattern but different **`activityType`** and fields.
|
|
1177
|
+
|
|
1067
1178
|
```typescript
|
|
1068
1179
|
{
|
|
1069
|
-
//
|
|
1070
|
-
_id: ObjectId('693970636e8d0f171e4aa528'),
|
|
1071
|
-
|
|
1072
|
-
// Activix
|
|
1180
|
+
// Mongo primary key
|
|
1181
|
+
_id: ObjectId('693970636e8d0f171e4aa528'),
|
|
1182
|
+
|
|
1183
|
+
// Activix logical row key (returned from startRecord; used by completeRecord / failRecord)
|
|
1184
|
+
activityId: 'act-63f0357e-b2fc-4038-8f94-a9c7fa8fb892',
|
|
1185
|
+
|
|
1186
|
+
activityType: 'gateway-invocation',
|
|
1187
|
+
|
|
1188
|
+
// Activix v7: canonical correlation BSON object `runContext` (same reference as `request.identity`, merged with gateway fields)
|
|
1073
1189
|
runContext: {
|
|
1074
1190
|
sessionId: 'sess-1',
|
|
1075
1191
|
instance: { instanceId: 'gw-1', type: 'gateway' },
|
|
@@ -1096,9 +1212,15 @@ const gateway = new AIGateway({
|
|
|
1096
1212
|
graphId: 'graph-456',
|
|
1097
1213
|
nodeId: 'node-789',
|
|
1098
1214
|
|
|
1099
|
-
//
|
|
1100
|
-
|
|
1101
|
-
//
|
|
1215
|
+
// Activix v7 root-level I/O tier — see section 2 for semantics
|
|
1216
|
+
// startRecord: outer.input ≈ { activityType, request } (request matches root `request` when present)
|
|
1217
|
+
// completeRecord: outer.output ← same object as root `response` on success
|
|
1218
|
+
outer: {
|
|
1219
|
+
input: { activityType: 'gateway-invocation', request: { /* same snapshot as root request */ } },
|
|
1220
|
+
output: { /* success: normalized gateway response object */ },
|
|
1221
|
+
metadata: { /* tier metadata / aiRequestId routing — see @x12i/activix */ }
|
|
1222
|
+
},
|
|
1223
|
+
// inner: optional step array for multi-step flows (see @x12i/activix docs)
|
|
1102
1224
|
|
|
1103
1225
|
// Timing
|
|
1104
1226
|
startTime: 1765372020804,
|
|
@@ -1116,11 +1238,10 @@ const gateway = new AIGateway({
|
|
|
1116
1238
|
parsed: {
|
|
1117
1239
|
instructions, // Parsed instructions (after template parsing with workingMemory)
|
|
1118
1240
|
context, // Parsed context (after template parsing with workingMemory)
|
|
1119
|
-
prompt // Parsed prompt (after template parsing with workingMemory
|
|
1241
|
+
prompt // Parsed prompt (after template parsing with workingMemory)
|
|
1120
1242
|
},
|
|
1121
|
-
input: "...", // Original input text
|
|
1122
1243
|
messages: [...], // Final constructed messages array
|
|
1123
|
-
workingMemory: {...} //
|
|
1244
|
+
workingMemory: {...} // Template/user payload (e.g. { input: '…' } for '{{input}}'); not a separate root `input` field
|
|
1124
1245
|
},
|
|
1125
1246
|
|
|
1126
1247
|
// Config data (from startActivity - ONLY in config object)
|
|
@@ -1148,13 +1269,13 @@ const gateway = new AIGateway({
|
|
|
1148
1269
|
```
|
|
1149
1270
|
|
|
1150
1271
|
**Key points:**
|
|
1151
|
-
- ✅ Each activity = separate
|
|
1272
|
+
- ✅ Each activity = separate Mongo document (**`_id`**) with stable **`activityId`** (`act-…`) for Activix APIs
|
|
1152
1273
|
- ✅ **`aiRequestId`** = per-request correlation (required on invoke)
|
|
1153
1274
|
- ✅ **`runContext.jobId`** / **`runContext.taskId`** = upstream identity (required on invoke since v9+)
|
|
1154
|
-
- ✅ Request
|
|
1155
|
-
- ✅ Updates
|
|
1275
|
+
- ✅ Request/config sent at **start**; response/timing/cost at **complete**
|
|
1276
|
+
- ✅ Updates target **`activityId`** from **`startRecord`**, not **`jobId`**
|
|
1156
1277
|
|
|
1157
|
-
#### Retry Tracking (@x12i/activix
|
|
1278
|
+
#### Retry Tracking (@x12i/activix v7)
|
|
1158
1279
|
|
|
1159
1280
|
The gateway automatically retries network errors, server errors (5xx), and throttling (429) with exponential backoff. Retry attempts are tracked and stored in activity records.
|
|
1160
1281
|
|
|
@@ -1232,12 +1353,24 @@ The gateway returns a comprehensive response structure that captures the full li
|
|
|
1232
1353
|
#### Complete Response Structure
|
|
1233
1354
|
|
|
1234
1355
|
```typescript
|
|
1356
|
+
const aiRequestId = 'resp-shape-1';
|
|
1235
1357
|
const response = await gateway.invoke({
|
|
1236
|
-
|
|
1358
|
+
aiRequestId,
|
|
1237
1359
|
agentId: 'agent-456',
|
|
1360
|
+
actionType: 'skill',
|
|
1361
|
+
actionRef: 'skills/qa',
|
|
1238
1362
|
instructions: 'You are a helpful assistant.',
|
|
1239
|
-
|
|
1240
|
-
|
|
1363
|
+
prompt: '{{input}}',
|
|
1364
|
+
workingMemory: { input: 'What is AI?' },
|
|
1365
|
+
identity: {
|
|
1366
|
+
sessionId: 's1',
|
|
1367
|
+
instance: { instanceId: 'agent-456', type: 'test' },
|
|
1368
|
+
aiRequestId,
|
|
1369
|
+
jobId: 'job-123',
|
|
1370
|
+
taskId: 'task-1',
|
|
1371
|
+
agentId: 'agent-456'
|
|
1372
|
+
},
|
|
1373
|
+
config: { model: 'gpt-5-nano', provider: 'openai' }
|
|
1241
1374
|
});
|
|
1242
1375
|
|
|
1243
1376
|
// Response structure:
|
|
@@ -1330,14 +1463,25 @@ const response = await gateway.invoke({
|
|
|
1330
1463
|
#### Example: Full Response
|
|
1331
1464
|
|
|
1332
1465
|
```typescript
|
|
1466
|
+
const aiRequestId = 'cls-1';
|
|
1333
1467
|
const response = await gateway.invoke({
|
|
1334
|
-
|
|
1468
|
+
aiRequestId,
|
|
1335
1469
|
agentId: 'agent-456',
|
|
1470
|
+
actionType: 'skill',
|
|
1471
|
+
actionRef: 'skills/sentiment',
|
|
1336
1472
|
instructions: 'Classify sentiment',
|
|
1337
|
-
|
|
1473
|
+
prompt: '{{input}}',
|
|
1474
|
+
workingMemory: { input: 'I love this product!' },
|
|
1338
1475
|
inferenceType: 'classification',
|
|
1339
|
-
|
|
1340
|
-
|
|
1476
|
+
identity: {
|
|
1477
|
+
sessionId: 's1',
|
|
1478
|
+
instance: { instanceId: 'agent-456', type: 'test' },
|
|
1479
|
+
aiRequestId,
|
|
1480
|
+
jobId: 'job-123',
|
|
1481
|
+
taskId: 'task-1',
|
|
1482
|
+
agentId: 'agent-456'
|
|
1483
|
+
},
|
|
1484
|
+
config: { model: 'gpt-5-nano', provider: 'openai' }
|
|
1341
1485
|
});
|
|
1342
1486
|
|
|
1343
1487
|
// Complete response structure:
|
|
@@ -1411,12 +1555,12 @@ const gateway = new AIGateway({
|
|
|
1411
1555
|
|
|
1412
1556
|
The gateway integrates with `@x12i/outputs-library` to parse LLM responses into typed inference outputs (classification, question-answer, extraction, etc.).
|
|
1413
1557
|
|
|
1558
|
+
> **Current request model:** **`parseOptions`**, **`validateOutputSchema`**, and **`strictValidation`** are **not** fields on **`ChatRequest`** / **`AIInvokeRequest`** anymore (removed). **`inferenceType`** may still be recorded for **Activix** / activity metadata. The primary **`invoke()`** response path uses **flex-md extraction** into **`parsedContent`**; for **`@x12i/outputs-library`** workflows, parse **`response.content`** / **`parsedContent`** in your app or follow instruction metadata (**`InstructionMetadata.parseOptions`** applies to catalog entries, not the invoke payload).
|
|
1559
|
+
> The **code samples** in subsections below still show legacy request shapes for illustration — adjust them to include **`aiRequestId`**, **`identity`**, **`actionType`**, **`actionRef`**, **`prompt`**, **`workingMemory`**, and omit removed fields.
|
|
1560
|
+
|
|
1414
1561
|
#### Overview
|
|
1415
1562
|
|
|
1416
|
-
When
|
|
1417
|
-
1. Parses the response into a typed output object
|
|
1418
|
-
2. Validates against JSON Schema (optional)
|
|
1419
|
-
3. Provides type-safe access to structured data
|
|
1563
|
+
When **`@x12i/outputs-library`** is used (directly or via future gateway wiring), specifying an **`inferenceType`** can classify parsing intent. Response typing and validation should align with your integration layer and **`EnhancedLLMResponse.metadata`** (e.g. **`parsedContent`**, **`parsingMethod`**).
|
|
1420
1564
|
|
|
1421
1565
|
#### Installation
|
|
1422
1566
|
|
|
@@ -1446,89 +1590,37 @@ npm install --legacy-peer-deps
|
|
|
1446
1590
|
- `recommendation` - Generate recommendations with priorities
|
|
1447
1591
|
- `transformation` - Transform data between formats
|
|
1448
1592
|
|
|
1449
|
-
#### Basic
|
|
1450
|
-
|
|
1451
|
-
```typescript
|
|
1452
|
-
import { AIGateway } from '@x12i/ai-gateway';
|
|
1453
|
-
import type { ClassificationOutput } from '@x12i/ai-gateway';
|
|
1454
|
-
|
|
1455
|
-
const gateway = new AIGateway({
|
|
1456
|
-
defaultProvider: 'openai'
|
|
1457
|
-
});
|
|
1458
|
-
|
|
1459
|
-
// Request with inference type
|
|
1460
|
-
const response = await gateway.invoke({
|
|
1461
|
-
jobId: 'job-123',
|
|
1462
|
-
agentId: 'agent-456',
|
|
1463
|
-
instructions: 'Classify the sentiment of the text.',
|
|
1464
|
-
input: 'This product is amazing!',
|
|
1465
|
-
inferenceType: 'classification',
|
|
1466
|
-
parseOptions: {
|
|
1467
|
-
classes: ['positive', 'negative', 'neutral']
|
|
1468
|
-
}
|
|
1469
|
-
});
|
|
1470
|
-
|
|
1471
|
-
// Access parsed output
|
|
1472
|
-
const classification = response.metadata.parsedOutput as ClassificationOutput;
|
|
1473
|
-
console.log(classification.classes); // ['positive', 'negative', 'neutral']
|
|
1474
|
-
console.log(classification.confidence); // { positive: 0.85, negative: 0.1, neutral: 0.05 }
|
|
1475
|
-
```
|
|
1593
|
+
#### Basic usage (`invoke` + `inferenceType`)
|
|
1476
1594
|
|
|
1477
|
-
|
|
1595
|
+
`parseOptions` / `validateOutputSchema` / `strictValidation` are **not** request fields. Supply **`aiRequestId`**, **`identity`**, **`actionType`**, **`actionRef`**, **`prompt`**, **`workingMemory`**, and optionally **`inferenceType`** for activity metadata.
|
|
1478
1596
|
|
|
1479
1597
|
```typescript
|
|
1480
|
-
const
|
|
1598
|
+
const aiRequestId = 'cls-1';
|
|
1599
|
+
const identity = {
|
|
1600
|
+
sessionId: 's1',
|
|
1601
|
+
instance: { instanceId: 'agent-456', type: 'test' },
|
|
1602
|
+
aiRequestId,
|
|
1481
1603
|
jobId: 'job-123',
|
|
1482
|
-
|
|
1483
|
-
|
|
1484
|
-
|
|
1485
|
-
inferenceType: 'extraction',
|
|
1486
|
-
validateOutputSchema: true // Enable validation
|
|
1487
|
-
});
|
|
1488
|
-
|
|
1489
|
-
// Check validation errors (if any)
|
|
1490
|
-
if (response.metadata.outputValidationErrors) {
|
|
1491
|
-
console.warn('Validation errors:', response.metadata.outputValidationErrors);
|
|
1492
|
-
}
|
|
1493
|
-
|
|
1494
|
-
// Access parsed output
|
|
1495
|
-
const extraction = response.metadata.parsedOutput as ExtractionOutput;
|
|
1496
|
-
console.log(extraction.extracted); // { name: 'John Doe', email: 'john@example.com' }
|
|
1497
|
-
```
|
|
1498
|
-
|
|
1499
|
-
#### Question-Answer Example
|
|
1604
|
+
taskId: 'task-1',
|
|
1605
|
+
agentId: 'agent-456'
|
|
1606
|
+
};
|
|
1500
1607
|
|
|
1501
|
-
```typescript
|
|
1502
1608
|
const response = await gateway.invoke({
|
|
1503
|
-
|
|
1609
|
+
aiRequestId,
|
|
1504
1610
|
agentId: 'agent-456',
|
|
1505
|
-
|
|
1506
|
-
|
|
1507
|
-
|
|
1508
|
-
|
|
1509
|
-
|
|
1510
|
-
|
|
1611
|
+
actionType: 'skill',
|
|
1612
|
+
actionRef: 'skills/classification',
|
|
1613
|
+
instructions: 'Classify the sentiment of the text.',
|
|
1614
|
+
prompt: '{{input}}',
|
|
1615
|
+
workingMemory: { input: 'This product is amazing!' },
|
|
1616
|
+
inferenceType: 'classification',
|
|
1617
|
+
identity,
|
|
1618
|
+
config: { model: 'gpt-4o', provider: 'openai' }
|
|
1511
1619
|
});
|
|
1512
|
-
|
|
1513
|
-
const qa = response.metadata.parsedOutput as QuestionAnswerOutput;
|
|
1514
|
-
console.log(qa.answer); // 'The capital of France is Paris.'
|
|
1515
|
-
console.log(qa.confidence); // 0.95
|
|
1516
1620
|
```
|
|
1517
1621
|
|
|
1518
|
-
|
|
1519
|
-
|
|
1520
|
-
```typescript
|
|
1521
|
-
const response = await gateway.invoke({
|
|
1522
|
-
jobId: 'job-123',
|
|
1523
|
-
agentId: 'agent-456',
|
|
1524
|
-
instructions: 'Extract structured data from the text.',
|
|
1525
|
-
input: 'User: John Doe, Age: 30, Location: New York',
|
|
1526
|
-
inferenceType: 'extraction'
|
|
1527
|
-
});
|
|
1622
|
+
Use the same shape for **`question-answer`**, **`extraction`**, etc., changing **`inferenceType`**, **`instructions`**, and **`workingMemory`**. Parse or validate structured output with **`@x12i/outputs-library`** in your process if needed.
|
|
1528
1623
|
|
|
1529
|
-
const extraction = response.metadata.parsedOutput as ExtractionOutput;
|
|
1530
|
-
console.log(extraction.extracted); // { user: 'John Doe', age: 30, location: 'New York' }
|
|
1531
|
-
```
|
|
1532
1624
|
|
|
1533
1625
|
#### Response Structure
|
|
1534
1626
|
|
|
@@ -1653,12 +1745,25 @@ The gateway will:
|
|
|
1653
1745
|
// }
|
|
1654
1746
|
// }
|
|
1655
1747
|
|
|
1748
|
+
const aiRequestId = 'schema-guide-1';
|
|
1656
1749
|
const response = await gateway.invoke({
|
|
1657
|
-
|
|
1750
|
+
aiRequestId,
|
|
1658
1751
|
agentId: 'agent-456',
|
|
1659
|
-
|
|
1660
|
-
|
|
1661
|
-
|
|
1752
|
+
actionType: 'skill',
|
|
1753
|
+
actionRef: 'skills/extraction-user-data',
|
|
1754
|
+
instructions: 'extraction/user-data', // Instruction key with outputSchema
|
|
1755
|
+
prompt: '{{input}}',
|
|
1756
|
+
workingMemory: { input: 'John Doe, john@example.com, 30 years old' },
|
|
1757
|
+
inferenceType: 'extraction',
|
|
1758
|
+
identity: {
|
|
1759
|
+
sessionId: 's1',
|
|
1760
|
+
instance: { instanceId: 'agent-456', type: 'test' },
|
|
1761
|
+
aiRequestId,
|
|
1762
|
+
jobId: 'job-123',
|
|
1763
|
+
taskId: 'task-1',
|
|
1764
|
+
agentId: 'agent-456'
|
|
1765
|
+
},
|
|
1766
|
+
config: { model: 'gpt-4o', provider: 'openai' }
|
|
1662
1767
|
});
|
|
1663
1768
|
|
|
1664
1769
|
// Instructions automatically extended with:
|
|
@@ -1687,71 +1792,12 @@ Add `outputObjectPrefix` to `instructions-blocks.json`:
|
|
|
1687
1792
|
|
|
1688
1793
|
This prefix is automatically appended to instructions when `outputType` or `outputSchema` is available.
|
|
1689
1794
|
|
|
1690
|
-
#### Schema
|
|
1691
|
-
|
|
1692
|
-
The gateway supports enhanced schema validation with strict/non-strict modes and automatic schema resolution from instruction metadata.
|
|
1693
|
-
|
|
1694
|
-
**Validation Options:**
|
|
1695
|
-
|
|
1696
|
-
```typescript
|
|
1697
|
-
const response = await gateway.invoke({
|
|
1698
|
-
jobId: 'job-123',
|
|
1699
|
-
agentId: 'agent-456',
|
|
1700
|
-
instructions: 'classification/vulnerability-severity',
|
|
1701
|
-
input: 'CVE-2024-1234: Critical RCE',
|
|
1702
|
-
inferenceType: 'classification',
|
|
1703
|
-
validateOutputSchema: true, // Enable validation
|
|
1704
|
-
strictValidation: false // Non-strict: log warnings, don't throw (default)
|
|
1705
|
-
});
|
|
1706
|
-
```
|
|
1707
|
-
|
|
1708
|
-
**Strict Validation Mode:**
|
|
1709
|
-
|
|
1710
|
-
```typescript
|
|
1711
|
-
const response = await gateway.invoke({
|
|
1712
|
-
jobId: 'job-123',
|
|
1713
|
-
agentId: 'agent-456',
|
|
1714
|
-
instructions: 'classification/vulnerability-severity',
|
|
1715
|
-
input: 'CVE-2024-1234: Critical RCE',
|
|
1716
|
-
inferenceType: 'classification',
|
|
1717
|
-
validateOutputSchema: true,
|
|
1718
|
-
strictValidation: true // Strict: throw error if validation fails
|
|
1719
|
-
});
|
|
1720
|
-
|
|
1721
|
-
// If validation fails, throws error:
|
|
1722
|
-
// Error: Schema validation failed for inference type 'classification': ...
|
|
1723
|
-
```
|
|
1795
|
+
#### Schema validation (request flags removed)
|
|
1724
1796
|
|
|
1725
|
-
**
|
|
1726
|
-
|
|
1727
|
-
When `inferenceType` is provided and instruction metadata is available, the gateway automatically:
|
|
1728
|
-
1. Checks instruction metadata for `outputSchema`
|
|
1729
|
-
2. Uses it for validation (overrides default schema from outputs-library)
|
|
1730
|
-
3. Falls back to outputs-library schema registry if not found
|
|
1731
|
-
|
|
1732
|
-
```typescript
|
|
1733
|
-
// Instruction metadata has:
|
|
1734
|
-
// {
|
|
1735
|
-
// instructionKey: 'classification/vulnerability-severity',
|
|
1736
|
-
// outputSchema: { /* custom schema */ }
|
|
1737
|
-
// }
|
|
1738
|
-
|
|
1739
|
-
const response = await gateway.invoke({
|
|
1740
|
-
jobId: 'job-123',
|
|
1741
|
-
agentId: 'agent-456',
|
|
1742
|
-
instructions: 'classification/vulnerability-severity', // Instruction key
|
|
1743
|
-
input: 'CVE-2024-1234: Critical RCE',
|
|
1744
|
-
inferenceType: 'classification',
|
|
1745
|
-
validateOutputSchema: true
|
|
1746
|
-
// Gateway automatically uses outputSchema from metadata
|
|
1747
|
-
});
|
|
1748
|
-
```
|
|
1797
|
+
**`validateOutputSchema`** and **`strictValidation`** are **not** on **`AIInvokeRequest`** / **`ChatRequest`** anymore. Validate **`response.parsedContent`** or **`response.content`** in your application (for example **`SchemaValidator`** from **`@x12i/outputs-library`**).
|
|
1749
1798
|
|
|
1750
|
-
**
|
|
1799
|
+
Instruction metadata can still carry **`outputSchema`** for documentation and instruction rendering via the content registry; that does **not** restore the old request-level validation switches.
|
|
1751
1800
|
|
|
1752
|
-
- **Non-strict mode (default)**: Validation failures log warnings but continue
|
|
1753
|
-
- **Strict mode**: Validation failures throw errors and stop execution
|
|
1754
|
-
- **Validation results** are always available in `response.metadata.outputValidationErrors` and `response.metadata.outputValidationPassed`
|
|
1755
1801
|
|
|
1756
1802
|
#### Output Structure Audit (v2.1.1+)
|
|
1757
1803
|
|
|
@@ -1769,16 +1815,28 @@ When `outputSchema` is available (from instruction metadata or provided directly
|
|
|
1769
1815
|
**Audit Results:**
|
|
1770
1816
|
|
|
1771
1817
|
```typescript
|
|
1818
|
+
const aiRequestId = 'audit-1';
|
|
1772
1819
|
const response = await gateway.invoke({
|
|
1773
|
-
|
|
1820
|
+
aiRequestId,
|
|
1774
1821
|
agentId: 'agent-456',
|
|
1822
|
+
actionType: 'skill',
|
|
1823
|
+
actionRef: 'skills/extraction-user-data',
|
|
1775
1824
|
instructions: 'extraction/user-data',
|
|
1776
|
-
|
|
1825
|
+
prompt: '{{input}}',
|
|
1826
|
+
workingMemory: { input: 'John Doe, john@example.com' },
|
|
1777
1827
|
inferenceType: 'extraction',
|
|
1778
|
-
|
|
1828
|
+
identity: {
|
|
1829
|
+
sessionId: 's1',
|
|
1830
|
+
instance: { instanceId: 'agent-456', type: 'test' },
|
|
1831
|
+
aiRequestId,
|
|
1832
|
+
jobId: 'job-123',
|
|
1833
|
+
taskId: 'task-1',
|
|
1834
|
+
agentId: 'agent-456'
|
|
1835
|
+
},
|
|
1836
|
+
config: { model: 'gpt-4o', provider: 'openai' }
|
|
1779
1837
|
});
|
|
1780
1838
|
|
|
1781
|
-
//
|
|
1839
|
+
// When present, `outputAudit` is informational (see types / runtime behavior)
|
|
1782
1840
|
console.log(response.metadata.outputAudit);
|
|
1783
1841
|
// {
|
|
1784
1842
|
// hasAllRequiredFields: true,
|
|
@@ -1924,342 +1982,99 @@ All errors are logged with clear messages indicating the issue and the fallback
|
|
|
1924
1982
|
- **Integration**: Seamlessly integrated into gateway responses
|
|
1925
1983
|
- **Graceful Degradation**: Automatic fallback when outputs library unavailable (v1.7.0+)
|
|
1926
1984
|
|
|
1927
|
-
### 7. Response
|
|
1985
|
+
### 7. Response transformation hooks (removed from request)
|
|
1928
1986
|
|
|
1929
|
-
|
|
1987
|
+
**Current behavior:** Request-level `transformations` and the `ResponseTransformationConfig` type are **no longer** part of the gateway request model. They were removed in favor of a smaller public surface.
|
|
1930
1988
|
|
|
1931
|
-
|
|
1989
|
+
**What to use instead:**
|
|
1932
1990
|
|
|
1933
|
-
|
|
1991
|
+
- Post-process `EnhancedLLMResponse` in your application (e.g. map `parsedContent` / `content`).
|
|
1992
|
+
- Use router **request/response interceptors** from `@x12i/ai-providers-router` where appropriate.
|
|
1993
|
+
- For instruction-catalog defaults, `InstructionMetadata` may still carry its own `parseOptions`; that is metadata about an instruction definition, **not** a field on `ChatRequest` / `AIInvokeRequest`.
|
|
1934
1994
|
|
|
1935
|
-
|
|
1995
|
+
Historical examples in older docs or forks that show `transformations` on `invoke()` payloads should be ignored or migrated.
|
|
1936
1996
|
|
|
1937
|
-
|
|
1938
|
-
2. **`postParse`**: Transform parsed content after parsing (applied to `parsedOutput` or `parsedContent`)
|
|
1939
|
-
3. **`preValidate`**: Transform parsed content before schema validation
|
|
1940
|
-
4. **`postValidate`**: Transform validated content after validation
|
|
1997
|
+
### 8. Content Registry Integration (Instruction Keys)
|
|
1941
1998
|
|
|
1942
|
-
|
|
1999
|
+
The gateway integrates with `@xronoces/content-registry` to fetch instructions by key instead of hardcoding them. This enables centralized content management, versioning, and zero-deploy updates.
|
|
1943
2000
|
|
|
1944
|
-
|
|
1945
|
-
interface ResponseTransformationConfig {
|
|
1946
|
-
preParse?: (rawText: string, metadata: any) => string;
|
|
1947
|
-
postParse?: (parsed: any, metadata: any) => any;
|
|
1948
|
-
preValidate?: (parsed: any, schema: any) => any;
|
|
1949
|
-
postValidate?: (validated: any, metadata: any) => any;
|
|
1950
|
-
}
|
|
1951
|
-
```
|
|
2001
|
+
#### Overview: Two Modes for Instructions
|
|
1952
2002
|
|
|
1953
|
-
**
|
|
2003
|
+
The gateway supports **two modes** for providing instructions to LLMs:
|
|
1954
2004
|
|
|
1955
|
-
|
|
2005
|
+
1. **Text Mode (Default)**: Instructions are actual text strings embedded in code
|
|
2006
|
+
2. **Key Mode (New)**: Instructions are keys that reference content stored in `@xronoces/content-registry`
|
|
1956
2007
|
|
|
1957
|
-
|
|
1958
|
-
{
|
|
1959
|
-
jobId: string;
|
|
1960
|
-
agentId: string;
|
|
1961
|
-
taskId?: string;
|
|
1962
|
-
taskTypeId?: string;
|
|
1963
|
-
instructionKey: string; // The instruction key or text
|
|
1964
|
-
inferenceType?: string;
|
|
1965
|
-
}
|
|
1966
|
-
```
|
|
2008
|
+
Both modes work seamlessly - the gateway automatically detects which mode you're using based on the instruction format.
|
|
1967
2009
|
|
|
1968
|
-
####
|
|
2010
|
+
#### Installation
|
|
1969
2011
|
|
|
1970
|
-
```
|
|
1971
|
-
|
|
1972
|
-
jobId: 'job-1',
|
|
1973
|
-
agentId: 'agent-1',
|
|
1974
|
-
instructions: 'classification/basic',
|
|
1975
|
-
input: 'This is great!',
|
|
1976
|
-
config: { model: 'gpt-4o' },
|
|
1977
|
-
inferenceType: 'classification',
|
|
1978
|
-
transformations: {
|
|
1979
|
-
// Clean raw text before parsing
|
|
1980
|
-
preParse: (rawText, metadata) => {
|
|
1981
|
-
return rawText.trim().replace(/\s+/g, ' ');
|
|
1982
|
-
},
|
|
1983
|
-
|
|
1984
|
-
// Transform parsed output
|
|
1985
|
-
postParse: (parsed, metadata) => {
|
|
1986
|
-
return {
|
|
1987
|
-
...parsed,
|
|
1988
|
-
transformed: true,
|
|
1989
|
-
timestamp: Date.now()
|
|
1990
|
-
};
|
|
1991
|
-
},
|
|
1992
|
-
|
|
1993
|
-
// Apply output mapping before validation
|
|
1994
|
-
preValidate: (parsed, schema) => {
|
|
1995
|
-
// Map fields to standardized names
|
|
1996
|
-
return {
|
|
1997
|
-
label: parsed.class,
|
|
1998
|
-
score: parsed.confidence,
|
|
1999
|
-
...parsed
|
|
2000
|
-
};
|
|
2001
|
-
},
|
|
2002
|
-
|
|
2003
|
-
// Add computed fields after validation
|
|
2004
|
-
postValidate: (validated, metadata) => {
|
|
2005
|
-
return {
|
|
2006
|
-
...validated,
|
|
2007
|
-
computed: {
|
|
2008
|
-
isHighConfidence: validated.confidence > 0.8,
|
|
2009
|
-
category: validated.class === 'positive' ? 'positive' : 'negative'
|
|
2010
|
-
}
|
|
2011
|
-
};
|
|
2012
|
-
}
|
|
2013
|
-
}
|
|
2014
|
-
});
|
|
2012
|
+
```bash
|
|
2013
|
+
npm install @xronoces/content-registry
|
|
2015
2014
|
```
|
|
2016
2015
|
|
|
2017
|
-
|
|
2018
|
-
|
|
2019
|
-
**1. Output Field Mapping**
|
|
2016
|
+
**Note**: `@xronoces/content-registry` (v2.14.0+) is automatically installed as a dependency. This version includes simplified APIs for easier integration.
|
|
2020
2017
|
|
|
2021
|
-
|
|
2018
|
+
**Configuration**: Content registry is used when `contentRegistryConfig` is provided in the gateway configuration.
|
|
2022
2019
|
|
|
2023
|
-
|
|
2024
|
-
transformations: {
|
|
2025
|
-
postParse: (parsed, metadata) => {
|
|
2026
|
-
// Map from LLM output to standardized format
|
|
2027
|
-
const mapping = {
|
|
2028
|
-
'class': 'label',
|
|
2029
|
-
'confidence': 'score',
|
|
2030
|
-
'reason': 'explanation'
|
|
2031
|
-
};
|
|
2032
|
-
|
|
2033
|
-
const mapped: Record<string, any> = {};
|
|
2034
|
-
for (const [key, value] of Object.entries(parsed)) {
|
|
2035
|
-
const mappedKey = mapping[key] || key;
|
|
2036
|
-
mapped[mappedKey] = value;
|
|
2037
|
-
}
|
|
2038
|
-
|
|
2039
|
-
return { ...parsed, ...mapped };
|
|
2040
|
-
}
|
|
2041
|
-
}
|
|
2042
|
-
```
|
|
2020
|
+
**For Internal Use**: Content-registry auto-initializes from environment variables if available, enabling instructionsBlocks resolution without requiring explicit configuration.
|
|
2043
2021
|
|
|
2044
|
-
**
|
|
2022
|
+
**See**: [Content Resolver — Upstream Guide](./CONTENT_RESOLVER_UPSTREAM_GUIDE.md) for configuration, environment variables, content layout, and upstream checklist.
|
|
2045
2023
|
|
|
2046
|
-
|
|
2024
|
+
#### Configuration
|
|
2047
2025
|
|
|
2048
2026
|
```typescript
|
|
2049
|
-
|
|
2050
|
-
postParse: (parsed, metadata) => {
|
|
2051
|
-
// Normalize confidence scores to 0-1 range
|
|
2052
|
-
if (parsed.confidence && parsed.confidence > 1) {
|
|
2053
|
-
parsed.confidence = parsed.confidence / 100;
|
|
2054
|
-
}
|
|
2055
|
-
|
|
2056
|
-
// Normalize class names to lowercase
|
|
2057
|
-
if (parsed.class) {
|
|
2058
|
-
parsed.class = parsed.class.toLowerCase();
|
|
2059
|
-
}
|
|
2060
|
-
|
|
2061
|
-
return parsed;
|
|
2062
|
-
}
|
|
2063
|
-
}
|
|
2064
|
-
```
|
|
2065
|
-
|
|
2066
|
-
**3. Add Computed Fields**
|
|
2067
|
-
|
|
2068
|
-
Add derived/computed fields:
|
|
2027
|
+
import { AIGateway } from '@x12i/ai-gateway';
|
|
2069
2028
|
|
|
2070
|
-
|
|
2071
|
-
|
|
2072
|
-
|
|
2073
|
-
|
|
2074
|
-
|
|
2075
|
-
|
|
2076
|
-
|
|
2077
|
-
|
|
2078
|
-
|
|
2079
|
-
|
|
2080
|
-
|
|
2081
|
-
|
|
2082
|
-
|
|
2083
|
-
|
|
2029
|
+
const gateway = new AIGateway({
|
|
2030
|
+
defaultProvider: 'openai',
|
|
2031
|
+
// Enable content-registry mode
|
|
2032
|
+
enableContentRegistry: true,
|
|
2033
|
+
// Configure content registry
|
|
2034
|
+
contentRegistryConfig: {
|
|
2035
|
+
s3Bucket: 'my-content-registry-bucket',
|
|
2036
|
+
cacheTTL: 3600, // Cache content for 1 hour
|
|
2037
|
+
redis: {
|
|
2038
|
+
host: 'localhost',
|
|
2039
|
+
port: 6379,
|
|
2040
|
+
password: process.env.REDIS_PASSWORD
|
|
2041
|
+
},
|
|
2042
|
+
// ... other content-registry config options
|
|
2084
2043
|
}
|
|
2085
|
-
}
|
|
2044
|
+
});
|
|
2086
2045
|
```
|
|
2087
2046
|
|
|
2088
|
-
|
|
2047
|
+
#### Mode 1: Text Mode (Automatic - Has Spaces)
|
|
2048
|
+
|
|
2049
|
+
**How it works**: Automatically detected when instructions contain whitespace. Used as literal text without any resolution.
|
|
2089
2050
|
|
|
2090
|
-
|
|
2051
|
+
**When used automatically**:
|
|
2052
|
+
- Instructions contain spaces, tabs, or newlines
|
|
2053
|
+
- No content registry lookup performed
|
|
2054
|
+
- Fast processing with no external dependencies
|
|
2091
2055
|
|
|
2056
|
+
**Example**:
|
|
2092
2057
|
```typescript
|
|
2093
|
-
|
|
2094
|
-
|
|
2095
|
-
// Transform flat structure to nested
|
|
2096
|
-
return {
|
|
2097
|
-
result: {
|
|
2098
|
-
classification: {
|
|
2099
|
-
label: parsed.class,
|
|
2100
|
-
confidence: parsed.confidence
|
|
2101
|
-
},
|
|
2102
|
-
metadata: {
|
|
2103
|
-
jobId: metadata.jobId,
|
|
2104
|
-
agentId: metadata.agentId
|
|
2105
|
-
}
|
|
2106
|
-
}
|
|
2107
|
-
};
|
|
2108
|
-
}
|
|
2109
|
-
}
|
|
2110
|
-
```
|
|
2111
|
-
|
|
2112
|
-
**5. Clean Raw Text**
|
|
2113
|
-
|
|
2114
|
-
Clean and normalize raw text before parsing:
|
|
2115
|
-
|
|
2116
|
-
```typescript
|
|
2117
|
-
transformations: {
|
|
2118
|
-
preParse: (rawText, metadata) => {
|
|
2119
|
-
// Remove markdown code blocks if present
|
|
2120
|
-
let cleaned = rawText.replace(/```json\n?/g, '').replace(/```\n?/g, '');
|
|
2121
|
-
|
|
2122
|
-
// Remove leading/trailing whitespace
|
|
2123
|
-
cleaned = cleaned.trim();
|
|
2124
|
-
|
|
2125
|
-
// Fix common JSON issues
|
|
2126
|
-
cleaned = cleaned.replace(/,(\s*[}\]])/g, '$1'); // Remove trailing commas
|
|
2127
|
-
|
|
2128
|
-
return cleaned;
|
|
2129
|
-
}
|
|
2130
|
-
}
|
|
2131
|
-
```
|
|
2132
|
-
|
|
2133
|
-
#### Error Handling
|
|
2134
|
-
|
|
2135
|
-
Transformation hooks have built-in error handling - if a transformation fails, the gateway logs a warning and continues with the original (untransformed) data. This ensures that transformation errors don't break the request.
|
|
2136
|
-
|
|
2137
|
-
```typescript
|
|
2138
|
-
// If transformation throws an error, gateway logs:
|
|
2139
|
-
// "postParse transformation failed, using original parsed output"
|
|
2140
|
-
// and continues with the original data
|
|
2141
|
-
```
|
|
2142
|
-
|
|
2143
|
-
#### Integration with Instruction Metadata
|
|
2144
|
-
|
|
2145
|
-
Combine transformation hooks with instruction metadata for fully metadata-driven systems:
|
|
2146
|
-
|
|
2147
|
-
```typescript
|
|
2148
|
-
// Fetch metadata
|
|
2149
|
-
const metadata = await gateway.getInstructionMetadata('classification/basic');
|
|
2150
|
-
|
|
2151
|
-
if (metadata?.defaultOutputMapping) {
|
|
2152
|
-
// Use metadata to configure transformation
|
|
2153
|
-
const response = await gateway.invoke({
|
|
2154
|
-
jobId: 'job-1',
|
|
2155
|
-
agentId: 'agent-1',
|
|
2156
|
-
instructions: 'classification/basic',
|
|
2157
|
-
input: 'This is great!',
|
|
2158
|
-
config: { model: 'gpt-4o' },
|
|
2159
|
-
inferenceType: metadata.outputType,
|
|
2160
|
-
transformations: {
|
|
2161
|
-
postParse: (parsed, reqMetadata) => {
|
|
2162
|
-
// Apply output mapping from metadata
|
|
2163
|
-
const mapped: Record<string, any> = {};
|
|
2164
|
-
for (const [key, mappedKey] of Object.entries(metadata.defaultOutputMapping!)) {
|
|
2165
|
-
if (key in parsed) {
|
|
2166
|
-
mapped[mappedKey] = (parsed as any)[key];
|
|
2167
|
-
}
|
|
2168
|
-
}
|
|
2169
|
-
return { ...parsed, ...mapped };
|
|
2170
|
-
}
|
|
2171
|
-
}
|
|
2172
|
-
});
|
|
2173
|
-
}
|
|
2174
|
-
```
|
|
2175
|
-
|
|
2176
|
-
#### Processing Order
|
|
2177
|
-
|
|
2178
|
-
Transformations are applied in this order:
|
|
2179
|
-
|
|
2180
|
-
1. **preParse**: Raw text → Transformed raw text
|
|
2181
|
-
2. **Parsing**: Transformed raw text → Parsed output (if `inferenceType` provided)
|
|
2182
|
-
3. **postParse**: Parsed output → Transformed parsed output
|
|
2183
|
-
4. **preValidate**: Transformed parsed output → Pre-validated output (if validation enabled)
|
|
2184
|
-
5. **Validation**: Pre-validated output → Validated output (if validation enabled)
|
|
2185
|
-
6. **postValidate**: Validated output → Final transformed output
|
|
2186
|
-
|
|
2187
|
-
If no `inferenceType` is provided, `postParse` is applied to `parsedContent` instead.
|
|
2188
|
-
|
|
2189
|
-
### 8. Content Registry Integration (Instruction Keys)
|
|
2190
|
-
|
|
2191
|
-
The gateway integrates with `@xronoces/content-registry` to fetch instructions by key instead of hardcoding them. This enables centralized content management, versioning, and zero-deploy updates.
|
|
2192
|
-
|
|
2193
|
-
#### Overview: Two Modes for Instructions
|
|
2194
|
-
|
|
2195
|
-
The gateway supports **two modes** for providing instructions to LLMs:
|
|
2196
|
-
|
|
2197
|
-
1. **Text Mode (Default)**: Instructions are actual text strings embedded in code
|
|
2198
|
-
2. **Key Mode (New)**: Instructions are keys that reference content stored in `@xronoces/content-registry`
|
|
2199
|
-
|
|
2200
|
-
Both modes work seamlessly - the gateway automatically detects which mode you're using based on the instruction format.
|
|
2201
|
-
|
|
2202
|
-
#### Installation
|
|
2203
|
-
|
|
2204
|
-
```bash
|
|
2205
|
-
npm install @xronoces/content-registry
|
|
2206
|
-
```
|
|
2207
|
-
|
|
2208
|
-
**Note**: `@xronoces/content-registry` (v2.14.0+) is automatically installed as a dependency. This version includes simplified APIs for easier integration.
|
|
2209
|
-
|
|
2210
|
-
**Configuration**: Content registry is used when `contentRegistryConfig` is provided in the gateway configuration.
|
|
2211
|
-
|
|
2212
|
-
**For Internal Use**: Content-registry auto-initializes from environment variables if available, enabling instructionsBlocks resolution without requiring explicit configuration.
|
|
2213
|
-
|
|
2214
|
-
**See**: [Content Resolver — Upstream Guide](./CONTENT_RESOLVER_UPSTREAM_GUIDE.md) for configuration, environment variables, content layout, and upstream checklist.
|
|
2215
|
-
|
|
2216
|
-
#### Configuration
|
|
2217
|
-
|
|
2218
|
-
```typescript
|
|
2219
|
-
import { AIGateway } from '@x12i/ai-gateway';
|
|
2220
|
-
|
|
2221
|
-
const gateway = new AIGateway({
|
|
2222
|
-
defaultProvider: 'openai',
|
|
2223
|
-
// Enable content-registry mode
|
|
2224
|
-
enableContentRegistry: true,
|
|
2225
|
-
// Configure content registry
|
|
2226
|
-
contentRegistryConfig: {
|
|
2227
|
-
s3Bucket: 'my-content-registry-bucket',
|
|
2228
|
-
cacheTTL: 3600, // Cache content for 1 hour
|
|
2229
|
-
redis: {
|
|
2230
|
-
host: 'localhost',
|
|
2231
|
-
port: 6379,
|
|
2232
|
-
password: process.env.REDIS_PASSWORD
|
|
2233
|
-
},
|
|
2234
|
-
// ... other content-registry config options
|
|
2235
|
-
}
|
|
2236
|
-
});
|
|
2237
|
-
```
|
|
2238
|
-
|
|
2239
|
-
#### Mode 1: Text Mode (Automatic - Has Spaces)
|
|
2240
|
-
|
|
2241
|
-
**How it works**: Automatically detected when instructions contain whitespace. Used as literal text without any resolution.
|
|
2242
|
-
|
|
2243
|
-
**When used automatically**:
|
|
2244
|
-
- Instructions contain spaces, tabs, or newlines
|
|
2245
|
-
- No content registry lookup performed
|
|
2246
|
-
- Fast processing with no external dependencies
|
|
2247
|
-
|
|
2248
|
-
**Example**:
|
|
2249
|
-
```typescript
|
|
2250
|
-
// Text mode - instructions as plain text
|
|
2058
|
+
// Text mode - instructions as plain text (structured invoke path resolves templates from workingMemory)
|
|
2059
|
+
const aiRequestId = 'text-mode-1';
|
|
2251
2060
|
const response = await gateway.invoke({
|
|
2252
|
-
|
|
2253
|
-
|
|
2254
|
-
|
|
2255
|
-
|
|
2256
|
-
|
|
2257
|
-
|
|
2258
|
-
|
|
2259
|
-
|
|
2260
|
-
|
|
2261
|
-
|
|
2262
|
-
|
|
2061
|
+
aiRequestId,
|
|
2062
|
+
agentId: 'agent-1',
|
|
2063
|
+
actionType: 'skill',
|
|
2064
|
+
actionRef: 'skills/classification',
|
|
2065
|
+
instructions:
|
|
2066
|
+
'You are a helpful assistant that classifies text into categories: positive, negative, or neutral.',
|
|
2067
|
+
prompt: '{{userLine}}',
|
|
2068
|
+
workingMemory: { userLine: 'Classify: "This product is amazing!"' },
|
|
2069
|
+
identity: {
|
|
2070
|
+
sessionId: 's1',
|
|
2071
|
+
instance: { instanceId: 'agent-1', type: 'test' },
|
|
2072
|
+
aiRequestId,
|
|
2073
|
+
jobId: 'job-123',
|
|
2074
|
+
taskId: 'task-1',
|
|
2075
|
+
agentId: 'agent-1'
|
|
2076
|
+
},
|
|
2077
|
+
config: { model: 'gpt-4o', provider: 'openai' }
|
|
2263
2078
|
});
|
|
2264
2079
|
```
|
|
2265
2080
|
|
|
@@ -2280,24 +2095,29 @@ const response = await gateway.invoke({
|
|
|
2280
2095
|
|
|
2281
2096
|
**Example**:
|
|
2282
2097
|
```typescript
|
|
2283
|
-
// Key mode
|
|
2098
|
+
// Key mode — pass the instruction key on `instructions`; user text via `prompt` + `workingMemory`
|
|
2099
|
+
const aiRequestId = 'key-mode-1';
|
|
2284
2100
|
const response = await gateway.invoke({
|
|
2285
|
-
|
|
2286
|
-
|
|
2287
|
-
|
|
2288
|
-
|
|
2289
|
-
|
|
2290
|
-
|
|
2291
|
-
|
|
2292
|
-
|
|
2293
|
-
}
|
|
2294
|
-
],
|
|
2295
|
-
jobId: 'job-123',
|
|
2296
|
-
// Optional: variables for template rendering
|
|
2297
|
-
contentVariables: {
|
|
2101
|
+
aiRequestId,
|
|
2102
|
+
agentId: 'agent-456',
|
|
2103
|
+
actionType: 'skill',
|
|
2104
|
+
actionRef: 'skills/classification',
|
|
2105
|
+
instructions: 'classification/basic', // ✅ Key - resolved from content-registry
|
|
2106
|
+
prompt: '{{userLine}}',
|
|
2107
|
+
workingMemory: {
|
|
2108
|
+
userLine: 'Classify: "This product is amazing!"',
|
|
2298
2109
|
task: 'classification',
|
|
2299
2110
|
context: 'product reviews'
|
|
2300
|
-
}
|
|
2111
|
+
},
|
|
2112
|
+
identity: {
|
|
2113
|
+
sessionId: 's1',
|
|
2114
|
+
instance: { instanceId: 'agent-456', type: 'test' },
|
|
2115
|
+
aiRequestId,
|
|
2116
|
+
jobId: 'job-123',
|
|
2117
|
+
taskId: 'task-1',
|
|
2118
|
+
agentId: 'agent-456'
|
|
2119
|
+
},
|
|
2120
|
+
config: { model: 'gpt-4o', provider: 'openai' }
|
|
2301
2121
|
});
|
|
2302
2122
|
```
|
|
2303
2123
|
|
|
@@ -2308,23 +2128,37 @@ const response = await gateway.invoke({
|
|
|
2308
2128
|
- `content/instructions/{contentId}` (primary pattern)
|
|
2309
2129
|
- Handles consolidated vs individual file resolution
|
|
2310
2130
|
4. Content-registry returns raw template content with full metadata envelope
|
|
2311
|
-
5. Instructions parser renders template with
|
|
2131
|
+
5. Instructions parser renders template with **`workingMemory`** (and tier memories) using Rendrix
|
|
2312
2132
|
6. Gateway returns rendered instruction text with metadata
|
|
2313
2133
|
7. Sends the resolved instruction to the LLM
|
|
2314
2134
|
|
|
2135
|
+
**Note:** **`gateway.invoke()`** builds provider **`messages`** from **`instructions`** / **`prompt`** / **`context`** via the message builder; it does **not** append a raw **`messages`** array from the request. For a pre-built transcript, use **`gateway.invokeChat()`** (see below).
|
|
2136
|
+
|
|
2315
2137
|
**Template-Based Prompts (Same as Instructions)**:
|
|
2316
2138
|
Prompts work exactly like instructions - both can be resolved from content-registry using explicit keys with suffixes:
|
|
2317
2139
|
|
|
2318
2140
|
```typescript
|
|
2141
|
+
const aiRequestId = 'tpl-key-1';
|
|
2319
2142
|
const response = await gateway.invoke({
|
|
2320
|
-
|
|
2143
|
+
aiRequestId,
|
|
2321
2144
|
agentId: 'agent-456',
|
|
2322
|
-
|
|
2323
|
-
|
|
2324
|
-
|
|
2145
|
+
actionType: 'skill',
|
|
2146
|
+
actionRef: 'skills/professional-answer',
|
|
2147
|
+
instructions: 'professional-answer.instructions', // Key with suffix
|
|
2148
|
+
prompt: 'professional-answer.prompt', // Key with suffix
|
|
2325
2149
|
workingMemory: {
|
|
2150
|
+
input: 'What is the capital of France?',
|
|
2326
2151
|
taskDescription: 'Answer questions professionally'
|
|
2327
|
-
}
|
|
2152
|
+
},
|
|
2153
|
+
identity: {
|
|
2154
|
+
sessionId: 's1',
|
|
2155
|
+
instance: { instanceId: 'agent-456', type: 'test' },
|
|
2156
|
+
aiRequestId,
|
|
2157
|
+
jobId: 'job-123',
|
|
2158
|
+
taskId: 'task-1',
|
|
2159
|
+
agentId: 'agent-456'
|
|
2160
|
+
},
|
|
2161
|
+
config: { model: 'gpt-4o', provider: 'openai' }
|
|
2328
2162
|
});
|
|
2329
2163
|
```
|
|
2330
2164
|
|
|
@@ -2419,21 +2253,28 @@ Provide confidence scores and reasoning.
|
|
|
2419
2253
|
|
|
2420
2254
|
**Code**:
|
|
2421
2255
|
```typescript
|
|
2256
|
+
const aiRequestId = 'tpl-vars-1';
|
|
2422
2257
|
const response = await gateway.invoke({
|
|
2423
|
-
|
|
2424
|
-
|
|
2425
|
-
|
|
2426
|
-
|
|
2427
|
-
|
|
2428
|
-
|
|
2429
|
-
|
|
2430
|
-
|
|
2431
|
-
}
|
|
2432
|
-
],
|
|
2433
|
-
contentVariables: {
|
|
2258
|
+
aiRequestId,
|
|
2259
|
+
agentId: 'agent-456',
|
|
2260
|
+
actionType: 'skill',
|
|
2261
|
+
actionRef: 'skills/classification',
|
|
2262
|
+
instructions: 'classification/basic',
|
|
2263
|
+
prompt: '{{userLine}}',
|
|
2264
|
+
workingMemory: {
|
|
2265
|
+
userLine: 'Classify: "This product is amazing!"',
|
|
2434
2266
|
task: 'classification',
|
|
2435
2267
|
classes: 'positive, negative, neutral'
|
|
2436
|
-
}
|
|
2268
|
+
},
|
|
2269
|
+
identity: {
|
|
2270
|
+
sessionId: 's1',
|
|
2271
|
+
instance: { instanceId: 'agent-456', type: 'test' },
|
|
2272
|
+
aiRequestId,
|
|
2273
|
+
jobId: 'job-123',
|
|
2274
|
+
taskId: 'task-1',
|
|
2275
|
+
agentId: 'agent-456'
|
|
2276
|
+
},
|
|
2277
|
+
config: { model: 'gpt-4o', provider: 'openai' }
|
|
2437
2278
|
});
|
|
2438
2279
|
```
|
|
2439
2280
|
|
|
@@ -2450,46 +2291,44 @@ Provide confidence scores and reasoning.
|
|
|
2450
2291
|
- A/B testing with different variable values
|
|
2451
2292
|
- Centralized template management
|
|
2452
2293
|
|
|
2453
|
-
#### Mixed Mode (
|
|
2294
|
+
#### Mixed Mode (multi-turn chat vs registry keys)
|
|
2295
|
+
|
|
2296
|
+
**Registry keys** are resolved on **`gateway.invoke()`** when you pass the key on **`instructions`** / **`prompt`** (see key-mode example above). **`gateway.invoke()`** does **not** consume a client-supplied **`messages`** array for the provider call.
|
|
2454
2297
|
|
|
2455
|
-
|
|
2298
|
+
For **multi-turn** transcripts (system/user/assistant/user, arbitrary order), use **`gateway.invokeChat()`**. Pass **`instructions: ''`** when your **`messages`** array already includes the system prompt, so the gateway does not prepend a second system message. Message bodies are sent **literally**—if you need a registry key as the system prompt, resolve it first (e.g. **`InstructionResolver`** / **`getInstructionMetadata`**) and put the rendered text in **`messages`**.
|
|
2456
2299
|
|
|
2457
2300
|
```typescript
|
|
2458
|
-
const
|
|
2301
|
+
const aiRequestId = 'mix-chat-1';
|
|
2302
|
+
const response = await gateway.invokeChat({
|
|
2303
|
+
aiRequestId,
|
|
2304
|
+
agentId: 'agent-1',
|
|
2305
|
+
instructions: '',
|
|
2459
2306
|
messages: [
|
|
2460
2307
|
{
|
|
2461
2308
|
role: 'system',
|
|
2462
|
-
content:
|
|
2463
|
-
|
|
2464
|
-
{
|
|
2465
|
-
role: 'user',
|
|
2466
|
-
content: 'You are analyzing product reviews.' // ✅ Text - used as-is
|
|
2467
|
-
},
|
|
2468
|
-
{
|
|
2469
|
-
role: 'assistant',
|
|
2470
|
-
content: 'I understand.'
|
|
2309
|
+
content:
|
|
2310
|
+
'…system prompt text (resolve content-registry keys beforehand if needed)…'
|
|
2471
2311
|
},
|
|
2472
|
-
{
|
|
2473
|
-
|
|
2474
|
-
|
|
2475
|
-
}
|
|
2312
|
+
{ role: 'user', content: 'You are analyzing product reviews.' },
|
|
2313
|
+
{ role: 'assistant', content: 'I understand.' },
|
|
2314
|
+
{ role: 'user', content: 'Classify: "This product is amazing!"' }
|
|
2476
2315
|
],
|
|
2477
|
-
|
|
2478
|
-
|
|
2479
|
-
|
|
2480
|
-
|
|
2316
|
+
workingMemory: { task: 'classification' },
|
|
2317
|
+
identity: {
|
|
2318
|
+
sessionId: 's1',
|
|
2319
|
+
instance: { instanceId: 'agent-1', type: 'test' },
|
|
2320
|
+
aiRequestId,
|
|
2321
|
+
jobId: 'job-123',
|
|
2322
|
+
taskId: 'task-1',
|
|
2323
|
+
agentId: 'agent-1'
|
|
2324
|
+
},
|
|
2325
|
+
config: { model: 'gpt-4o', provider: 'openai' }
|
|
2481
2326
|
});
|
|
2482
2327
|
```
|
|
2483
2328
|
|
|
2484
|
-
**What happens**:
|
|
2485
|
-
- `'classification/basic'` → Detected as key → Resolved from content-registry
|
|
2486
|
-
- `'You are analyzing product reviews.'` → Detected as text → Used as-is
|
|
2487
|
-
- Other messages → Used as-is
|
|
2488
|
-
|
|
2489
2329
|
**Use cases**:
|
|
2490
|
-
-
|
|
2491
|
-
-
|
|
2492
|
-
- Use keys for complex instructions, text for simple ones
|
|
2330
|
+
- Multi-turn chat while keeping **`workingMemory`** / **`templateRenderOptions`** available on the chat path
|
|
2331
|
+
- Single-shot registry-backed flows → prefer **`invoke()`** with **`instructions`** as the key
|
|
2493
2332
|
|
|
2494
2333
|
#### Content Registry Accessor Methods
|
|
2495
2334
|
|
|
@@ -2599,20 +2438,26 @@ if (metadata) {
|
|
|
2599
2438
|
throw new Error(`Missing required variables: ${missingVars.join(', ')}`);
|
|
2600
2439
|
}
|
|
2601
2440
|
|
|
2602
|
-
// Use metadata to configure inference
|
|
2441
|
+
// Use metadata to configure inference (parsing options live on metadata for your app, not on the request)
|
|
2442
|
+
const aiRequestId = `md-${Date.now()}`;
|
|
2603
2443
|
const response = await gateway.invoke({
|
|
2604
|
-
|
|
2444
|
+
aiRequestId,
|
|
2605
2445
|
agentId: 'agent-1',
|
|
2446
|
+
actionType: 'skill',
|
|
2447
|
+
actionRef: metadata.instructionKey || 'skills/classification',
|
|
2606
2448
|
instructions: 'classification/basic',
|
|
2607
|
-
|
|
2608
|
-
|
|
2609
|
-
// Use metadata to configure parsing
|
|
2449
|
+
prompt: '{{input}}',
|
|
2450
|
+
workingMemory: { input: 'This is great!', ...variables },
|
|
2610
2451
|
inferenceType: metadata.outputType,
|
|
2611
|
-
|
|
2612
|
-
|
|
2613
|
-
|
|
2452
|
+
identity: {
|
|
2453
|
+
sessionId: 's1',
|
|
2454
|
+
instance: { instanceId: 'agent-1', type: 'test' },
|
|
2455
|
+
aiRequestId,
|
|
2456
|
+
jobId: 'job-1',
|
|
2457
|
+
taskId: 'task-1',
|
|
2458
|
+
agentId: 'agent-1'
|
|
2614
2459
|
},
|
|
2615
|
-
|
|
2460
|
+
config: { model: 'gpt-4o', provider: 'openai' }
|
|
2616
2461
|
});
|
|
2617
2462
|
|
|
2618
2463
|
// Apply output mapping if provided
|
|
@@ -2715,22 +2560,29 @@ const gateway = new AIGateway({
|
|
|
2715
2560
|
}
|
|
2716
2561
|
});
|
|
2717
2562
|
|
|
2718
|
-
// Use key
|
|
2563
|
+
// Use instruction key on invoke() (resolved via content-registry)
|
|
2564
|
+
const aiRequestId = 'registry-step2-1';
|
|
2719
2565
|
const response = await gateway.invoke({
|
|
2720
|
-
|
|
2721
|
-
|
|
2722
|
-
|
|
2723
|
-
|
|
2724
|
-
|
|
2725
|
-
|
|
2726
|
-
|
|
2727
|
-
|
|
2728
|
-
}
|
|
2729
|
-
],
|
|
2730
|
-
contentVariables: {
|
|
2566
|
+
aiRequestId,
|
|
2567
|
+
agentId: 'agent-1',
|
|
2568
|
+
actionType: 'skill',
|
|
2569
|
+
actionRef: 'skills/classification',
|
|
2570
|
+
instructions: 'classification/basic',
|
|
2571
|
+
prompt: '{{userLine}}',
|
|
2572
|
+
workingMemory: {
|
|
2573
|
+
userLine: 'Classify: "This is great!"',
|
|
2731
2574
|
task: 'classification',
|
|
2732
2575
|
classes: 'positive, negative, neutral'
|
|
2733
|
-
}
|
|
2576
|
+
},
|
|
2577
|
+
identity: {
|
|
2578
|
+
sessionId: 's1',
|
|
2579
|
+
instance: { instanceId: 'agent-1', type: 'test' },
|
|
2580
|
+
aiRequestId,
|
|
2581
|
+
jobId: 'job-123',
|
|
2582
|
+
taskId: 'task-1',
|
|
2583
|
+
agentId: 'agent-1'
|
|
2584
|
+
},
|
|
2585
|
+
config: { model: 'gpt-4o', provider: 'openai' }
|
|
2734
2586
|
});
|
|
2735
2587
|
```
|
|
2736
2588
|
|
|
@@ -2761,7 +2613,7 @@ await registry.createPrompt({
|
|
|
2761
2613
|
- **A/B Testing**: Test different instruction versions
|
|
2762
2614
|
- **Template Support**: Use Handlebars templates with variables
|
|
2763
2615
|
- **Caching**: Instructions are cached for performance (Redis via content-registry)
|
|
2764
|
-
- **
|
|
2616
|
+
- **Migration**: Prefer **`invoke()`** + **`instructions`** keys for registry-backed single-shot flows; multi-turn **`messages`** arrays belong on **`invokeChat()`** (see Mixed Mode). Request payloads require **`identity`**, **`aiRequestId`**, and (for **`invoke()`**) **`actionType`** / **`actionRef`** — top-level **`input`** is rejected.
|
|
2765
2617
|
- **Automatic Detection**: Gateway automatically detects keys vs text - no manual mode switching
|
|
2766
2618
|
|
|
2767
2619
|
#### Error Handling
|
|
@@ -2769,19 +2621,31 @@ await registry.createPrompt({
|
|
|
2769
2621
|
If a key cannot be resolved, the gateway falls back to using the key as text and logs a warning:
|
|
2770
2622
|
|
|
2771
2623
|
```typescript
|
|
2772
|
-
// If key 'invalid/key' doesn't exist in registry
|
|
2773
|
-
|
|
2774
|
-
|
|
2775
|
-
|
|
2776
|
-
|
|
2777
|
-
|
|
2778
|
-
|
|
2779
|
-
|
|
2780
|
-
|
|
2781
|
-
|
|
2782
|
-
|
|
2783
|
-
|
|
2784
|
-
|
|
2624
|
+
// If key 'invalid/key' doesn't exist in registry, invoke() fails closed during message build
|
|
2625
|
+
// (bad-request logging when activity tracking is enabled)—there is no silent fallback to raw text.
|
|
2626
|
+
const aiRequestId = 'registry-err-1';
|
|
2627
|
+
try {
|
|
2628
|
+
await gateway.invoke({
|
|
2629
|
+
aiRequestId,
|
|
2630
|
+
agentId: 'agent-1',
|
|
2631
|
+
actionType: 'skill',
|
|
2632
|
+
actionRef: 'skills/invalid',
|
|
2633
|
+
instructions: 'invalid/key',
|
|
2634
|
+
prompt: '{{input}}',
|
|
2635
|
+
workingMemory: { input: 'hello' },
|
|
2636
|
+
identity: {
|
|
2637
|
+
sessionId: 's1',
|
|
2638
|
+
instance: { instanceId: 'agent-1', type: 'test' },
|
|
2639
|
+
aiRequestId,
|
|
2640
|
+
jobId: 'job-123',
|
|
2641
|
+
taskId: 'task-1',
|
|
2642
|
+
agentId: 'agent-1'
|
|
2643
|
+
},
|
|
2644
|
+
config: { model: 'gpt-4o', provider: 'openai' }
|
|
2645
|
+
});
|
|
2646
|
+
} catch (e) {
|
|
2647
|
+
// Expect resolution / instruction errors; see TROUBLESHOOTING.md
|
|
2648
|
+
}
|
|
2785
2649
|
```
|
|
2786
2650
|
|
|
2787
2651
|
#### Advanced: Custom Instruction Resolver
|
|
@@ -2808,145 +2672,24 @@ const instructions = await resolver.resolveInstructions(
|
|
|
2808
2672
|
console.log(instructions); // Resolved instruction text
|
|
2809
2673
|
```
|
|
2810
2674
|
|
|
2811
|
-
### 9. Contract
|
|
2812
|
-
|
|
2813
|
-
The gateway supports contract output parsing and validation, where AI responses are parsed against expected schemas and the results are stored in activity records for compliance monitoring and debugging.
|
|
2814
|
-
|
|
2815
|
-
#### How It Works
|
|
2816
|
-
|
|
2817
|
-
1. **Schema Input**: Provide an optional `expectedSchema` parameter in your gateway calls
|
|
2818
|
-
2. **Response Parsing**: AI responses are automatically parsed using flex-md against the expected schema
|
|
2819
|
-
3. **Activity Storage**: Parsed results are stored in activity records under `content.contractOutput`
|
|
2820
|
-
4. **Status Tracking**: Compliance status is tracked in `content.outputStatus`
|
|
2821
|
-
|
|
2822
|
-
#### Basic Usage
|
|
2823
|
-
|
|
2824
|
-
```typescript
|
|
2825
|
-
import { AIGateway } from '@x12i/ai-gateway';
|
|
2826
|
-
|
|
2827
|
-
const gateway = new AIGateway({ /* config */ });
|
|
2828
|
-
|
|
2829
|
-
// Call with expected schema for contract validation
|
|
2830
|
-
const response = await gateway.invoke({
|
|
2831
|
-
jobId: 'job-123',
|
|
2832
|
-
agentId: 'agent-456',
|
|
2833
|
-
instructions: 'Analyze the following text and provide structured output.',
|
|
2834
|
-
|
|
2835
|
-
// Provide expected schema for contract output parsing
|
|
2836
|
-
expectedSchema: {
|
|
2837
|
-
description: "Analysis results with key insights and recommendations",
|
|
2838
|
-
formatHint: "### Key Insights\n[insights]\n\n### Recommendations\n[recommendations]"
|
|
2839
|
-
},
|
|
2840
|
-
|
|
2841
|
-
messages: [{ role: 'user', content: 'Analyze this product review...' }]
|
|
2842
|
-
});
|
|
2843
|
-
|
|
2844
|
-
// Activity record will contain:
|
|
2845
|
-
// content.contractOutput: { keyInsights: "...", recommendations: "..." }
|
|
2846
|
-
// content.outputStatus: "different" (parsed successfully)
|
|
2847
|
-
```
|
|
2848
|
-
|
|
2849
|
-
#### Schema Format
|
|
2675
|
+
### 9. Contract hints (`StructuredTextSpec`) vs gateway parsing
|
|
2850
2676
|
|
|
2851
|
-
|
|
2677
|
+
**`ChatRequest`** still includes an optional **`expectedSchema?: StructuredTextSpec`** field for typing and forward compatibility:
|
|
2852
2678
|
|
|
2853
2679
|
```typescript
|
|
2854
2680
|
interface StructuredTextSpec {
|
|
2855
|
-
description: string;
|
|
2856
|
-
formatHint?: string;
|
|
2857
|
-
}
|
|
2858
|
-
```
|
|
2859
|
-
|
|
2860
|
-
#### Activity Record Structure
|
|
2861
|
-
|
|
2862
|
-
When contract output parsing is enabled, activity records include additional fields:
|
|
2863
|
-
|
|
2864
|
-
```json
|
|
2865
|
-
{
|
|
2866
|
-
"content": {
|
|
2867
|
-
"contractOutput": {
|
|
2868
|
-
// Parsed structured data from AI response
|
|
2869
|
-
// Empty object {} if parsing fails
|
|
2870
|
-
},
|
|
2871
|
-
"outputStatus": "ok" | "different" | undefined
|
|
2872
|
-
}
|
|
2681
|
+
description: string;
|
|
2682
|
+
formatHint?: string;
|
|
2873
2683
|
}
|
|
2874
2684
|
```
|
|
2875
2685
|
|
|
2876
|
-
|
|
2686
|
+
**Current gateway behavior:** dedicated **contract-output** processing (persisting **`content.contractOutput`** / **`content.outputStatus`** on activities from **`expectedSchema`**) is **not** implemented on **`invoke()`** / **`invokeChat()`** — see comment in `gateway.ts` (“Contract output processing removed”). Passing **`expectedSchema`** does not, by itself, trigger extra persistence or validation inside this package.
|
|
2877
2687
|
|
|
2878
|
-
|
|
2879
|
-
- `"different"`: Contract output parsed successfully but differs from expected schema
|
|
2880
|
-
- `undefined`: No schema validation performed (when `expectedSchema` not provided)
|
|
2688
|
+
**Recommended patterns:**
|
|
2881
2689
|
|
|
2882
|
-
|
|
2883
|
-
|
|
2884
|
-
**
|
|
2885
|
-
```typescript
|
|
2886
|
-
// Ensure AI responses follow expected structure
|
|
2887
|
-
const result = await gateway.call({
|
|
2888
|
-
messages: [...],
|
|
2889
|
-
expectedSchema: {
|
|
2890
|
-
description: "Professional answer with short summary and detailed analysis",
|
|
2891
|
-
formatHint: "### Short Answer\n[summary]\n\n### Full Answer\n[analysis]"
|
|
2892
|
-
}
|
|
2893
|
-
});
|
|
2894
|
-
```
|
|
2895
|
-
|
|
2896
|
-
**Debugging Response Parsing:**
|
|
2897
|
-
```typescript
|
|
2898
|
-
// When AI doesn't follow expected format, contractOutput will be {}
|
|
2899
|
-
// Check activity records to debug parsing issues
|
|
2900
|
-
const result = await gateway.call({
|
|
2901
|
-
messages: [...],
|
|
2902
|
-
expectedSchema: { /* schema */ }
|
|
2903
|
-
});
|
|
2904
|
-
// If parsing fails: content.contractOutput = {}
|
|
2905
|
-
// If parsing succeeds: content.contractOutput = { parsed: "data" }
|
|
2906
|
-
```
|
|
2907
|
-
|
|
2908
|
-
#### Integration with Activity Tracking
|
|
2909
|
-
|
|
2910
|
-
Contract output parsing integrates seamlessly with `@x12i/activix` v6 (xronox-activitix):
|
|
2911
|
-
|
|
2912
|
-
- **Automatic Processing**: Parsing happens automatically when `expectedSchema` is provided
|
|
2913
|
-
- **Error Resilience**: Parsing failures don't break activity recording
|
|
2914
|
-
- **Performance**: Minimal overhead when schema is provided
|
|
2915
|
-
- **Backward Compatibility**: Existing calls work unchanged
|
|
2916
|
-
|
|
2917
|
-
#### Advanced Configuration
|
|
2918
|
-
|
|
2919
|
-
```typescript
|
|
2920
|
-
// Detailed schema specification
|
|
2921
|
-
const schema = {
|
|
2922
|
-
description: "Structured analysis with multiple sections",
|
|
2923
|
-
formatHint: `### Executive Summary
|
|
2924
|
-
[summary]
|
|
2925
|
-
|
|
2926
|
-
### Detailed Analysis
|
|
2927
|
-
[analysis]
|
|
2928
|
-
|
|
2929
|
-
### Recommendations
|
|
2930
|
-
[recommendations]
|
|
2931
|
-
|
|
2932
|
-
### Confidence Level
|
|
2933
|
-
[level]`
|
|
2934
|
-
};
|
|
2935
|
-
|
|
2936
|
-
const response = await gateway.invoke({
|
|
2937
|
-
aiRequestId: 'analysis-req-001',
|
|
2938
|
-
identity: {
|
|
2939
|
-
jobId: 'analysis-123',
|
|
2940
|
-
taskId: 'analysis-task-001',
|
|
2941
|
-
sessionId: 'sess-analyzer',
|
|
2942
|
-
instance: { instanceId: 'analyzer-1', type: 'gateway' }
|
|
2943
|
-
},
|
|
2944
|
-
agentId: 'analyzer-bot',
|
|
2945
|
-
instructions: 'Analyze the provided data...',
|
|
2946
|
-
expectedSchema: schema,
|
|
2947
|
-
messages: [{ role: 'user', content: data }]
|
|
2948
|
-
});
|
|
2949
|
-
```
|
|
2690
|
+
- Rely on **flex-md extraction** into **`response.parsedContent`** / **`response.content`** and validate in your app (e.g. **`@x12i/outputs-library`**, JSON Schema, or custom checks).
|
|
2691
|
+
- Use **response interceptors** if you need centralized post-processing or compliance logging.
|
|
2692
|
+
- Encode output shape in **instructions** / instruction metadata (**`outputSchema`**) so the model sees the contract; enforce it application-side.
|
|
2950
2693
|
|
|
2951
2694
|
## API Reference
|
|
2952
2695
|
|
|
@@ -2970,7 +2713,7 @@ interface GatewayConfig extends RouterConfig {
|
|
|
2970
2713
|
enableLogging?: boolean; // Default: true
|
|
2971
2714
|
contentRegistryConfig?: any; // Content registry config for instruction key resolution
|
|
2972
2715
|
logger?: import('@x12i/logxer').Logxer; // Custom Logxer (from createLogxer)
|
|
2973
|
-
activityTracker?: Activix; // Custom Activix
|
|
2716
|
+
activityTracker?: Activix; // Custom Activix v7 instance (match collection names: ai-actions, bad-requests, skill-executions)
|
|
2974
2717
|
packageName?: string; // Default: 'AI_GATEWAY'
|
|
2975
2718
|
// ... all RouterConfig options
|
|
2976
2719
|
}
|
|
@@ -2982,8 +2725,8 @@ interface GatewayConfig extends RouterConfig {
|
|
|
2982
2725
|
- `unregister(providerName: LLMProvider): void` - Unregister a provider
|
|
2983
2726
|
- `getProvider(providerName: LLMProvider)` - Get a provider instance
|
|
2984
2727
|
- `listProviders(): LLMProvider[]` - List all registered providers
|
|
2985
|
-
- `invoke(request:
|
|
2986
|
-
- `invokeChat(request: ChatRequest): Promise<EnhancedLLMResponse>` -
|
|
2728
|
+
- `invoke(request: AIInvokeRequest): Promise<EnhancedLLMResponse>` - Structured / skill path; requires **`actionType`**, **`actionRef`**, **`identity`**, **`aiRequestId`**, **`instructions`**, etc. (`AIRequest` is a type alias)
|
|
2729
|
+
- `invokeChat(request: ChatRequest): Promise<EnhancedLLMResponse>` - Chat / conversational path (no **`actionType`** / **`actionRef`**)
|
|
2987
2730
|
- `setDefaultProvider(provider: LLMProvider): void` - Set the default provider
|
|
2988
2731
|
- `setFallbackChain(chain: LLMProvider[]): void` - Configure fallback chain
|
|
2989
2732
|
- `addRequestInterceptor(interceptor: RequestInterceptor): void` - Add request interceptor
|
|
@@ -3003,109 +2746,68 @@ interface GatewayConfig extends RouterConfig {
|
|
|
3003
2746
|
- `testInstructions(instructions: string, testInput: string, expectedSchema?: Record<string, unknown>, options?: TestInstructionsOptions): Promise<TestInstructionsResult>` - Test instructions by running them and analyzing responses (v3.0.4+)
|
|
3004
2747
|
- `generateRequestReport(request: AIRequest): Promise<RequestReport>` - Generate comprehensive request report with validation, examples, and structured text information (v3.0.6+)
|
|
3005
2748
|
|
|
3006
|
-
### ChatRequest and AIRequest
|
|
2749
|
+
### ChatRequest and AIRequest / AIInvokeRequest
|
|
3007
2750
|
|
|
3008
|
-
|
|
2751
|
+
**Naming**
|
|
3009
2752
|
|
|
3010
|
-
|
|
2753
|
+
- **`ChatRequest`** — pass to **`gateway.invokeChat()`**. Conversational path; optional **`expectedSchema`** (`StructuredTextSpec`) for contract-style validation when enabled.
|
|
2754
|
+
- **`AIInvokeRequest`** — pass to **`gateway.invoke()`**. Canonical structured/skill path.
|
|
2755
|
+
- **`AIRequest`** — **type alias** for **`AIInvokeRequest`** (keeps existing imports working).
|
|
3011
2756
|
|
|
3012
|
-
|
|
3013
|
-
2. **`AIRequest`** - For structured output requests where `objectTypes` is required
|
|
2757
|
+
**Mandatory on `invoke()` (`AIInvokeRequest`)**
|
|
3014
2758
|
|
|
3015
|
-
|
|
2759
|
+
| Field | Meaning |
|
|
2760
|
+
|-------|--------|
|
|
2761
|
+
| **`actionType`** | **`'skill'`** \| **`'preSkill'`** \| **`'postSkill'`** — classifies the invocation for tracing and Activix. |
|
|
2762
|
+
| **`actionRef`** | Non-empty string reference for the action (e.g. **`skills/my-skill`**). Recorded with the activity. |
|
|
3016
2763
|
|
|
3017
|
-
|
|
2764
|
+
These are validated by **`validateAIRequest`**, merged into **`request.identity`** ( **`runContext`** for Activix ), and duplicated on activity root fields **`actionType`** / **`actionRef`** when activity tracking runs.
|
|
3018
2765
|
|
|
3019
|
-
|
|
3020
|
-
|
|
3021
|
-
|
|
3022
|
-
|
|
3023
|
-
|
|
3024
|
-
|
|
2766
|
+
**Mandatory on both paths**
|
|
2767
|
+
|
|
2768
|
+
- **`aiRequestId`**, **`agentId`**, **`instructions`**, **`identity`** (upstream **`identity.jobId`** and **`identity.taskId`** — see [Mandatory runtime identity](#mandatory-runtime-identity-v9)).
|
|
2769
|
+
- Top-level **`input`** is **not** supported; use **`workingMemory`** (e.g. **`workingMemory.input`**) for template data.
|
|
2770
|
+
|
|
2771
|
+
**Removed from the shared request model (`BaseLLMRequest`)**
|
|
2772
|
+
|
|
2773
|
+
Not accepted on gateway requests anymore:
|
|
3025
2774
|
|
|
3026
|
-
|
|
2775
|
+
- **`taskConfig`**, **`templateTokens`**, **`validateOutputSchema`**, **`strictValidation`**, **`transformations`**, **`parseOptions`** (request-level).
|
|
3027
2776
|
|
|
3028
|
-
|
|
2777
|
+
Use **`InstructionMetadata.parseOptions`** only as **instruction-catalog metadata**, not as invoke payload fields.
|
|
2778
|
+
|
|
2779
|
+
**Exports**
|
|
3029
2780
|
|
|
3030
2781
|
```typescript
|
|
3031
|
-
|
|
3032
|
-
|
|
3033
|
-
|
|
3034
|
-
|
|
3035
|
-
|
|
3036
|
-
|
|
3037
|
-
|
|
3038
|
-
|
|
3039
|
-
coreSkillId?: string; // Optional alternative node identifier for graph execution context
|
|
3040
|
-
prompt?: string; // Optional prompt key or text (resolved from content-registry if key, parsed as template if text)
|
|
3041
|
-
input?: string; // Optional input text
|
|
3042
|
-
context?: string; // Optional context text (template with workingMemory)
|
|
3043
|
-
workingMemory?: unknown; // Optional template variables for Rendrix
|
|
3044
|
-
expectedSchema?: StructuredTextSpec; // Optional schema for contract output parsing (v6.3.1+)
|
|
3045
|
-
inferenceType?: string; // Optional inference type for outputs library parsing
|
|
3046
|
-
parseOptions?: Record<string, unknown>; // Optional parse options for outputs library
|
|
3047
|
-
validateOutputSchema?: boolean; // Optional schema validation flag
|
|
3048
|
-
transformations?: ResponseTransformationConfig; // Optional response transformation hooks (v1.6.9+)
|
|
3049
|
-
config?: any; // LLM config (model, temperature, etc.)
|
|
3050
|
-
modelConfig?: ModelConfig; // Model configuration (modelConfig > config > gateway defaults)
|
|
3051
|
-
objectTypes?: Array<{ // Optional object types for structured output
|
|
3052
|
-
type: string;
|
|
3053
|
-
schema?: Record<string, unknown>;
|
|
3054
|
-
whenToUse: string;
|
|
3055
|
-
description?: string;
|
|
3056
|
-
}>;
|
|
3057
|
-
}
|
|
2782
|
+
import type {
|
|
2783
|
+
ChatRequest,
|
|
2784
|
+
AIInvokeRequest,
|
|
2785
|
+
AIRequest, // alias of AIInvokeRequest
|
|
2786
|
+
GatewayActionType, // 'skill' | 'preSkill' | 'postSkill'
|
|
2787
|
+
ActivityIdentity,
|
|
2788
|
+
EnhancedLLMResponse
|
|
2789
|
+
} from '@x12i/ai-gateway';
|
|
3058
2790
|
```
|
|
3059
2791
|
|
|
3060
|
-
|
|
2792
|
+
**Structured output helpers**
|
|
3061
2793
|
|
|
3062
|
-
|
|
2794
|
+
Structured flows often use **`primaryObjectType`** / **`objectTypes`** from the router **`LLMRequest`** intersection (type guards in message building). They are **not** duplicated here verbatim — see **`src/types.ts`** and **`LLMRequest`** from **`@x12i/ai-providers-router`**.
|
|
3063
2795
|
|
|
3064
|
-
|
|
3065
|
-
interface AIRequest extends BaseLLMRequest {
|
|
3066
|
-
// REQUIRED - main expected output structure
|
|
3067
|
-
// Can be string (standard type name) or object (custom type with schema)
|
|
3068
|
-
primaryObjectType: string | {
|
|
3069
|
-
type: string; // Required: object type identifier
|
|
3070
|
-
schema?: Record<string, unknown>; // Optional: JSON schema (required for custom types)
|
|
3071
|
-
whenToUse: string; // Required: guidance on when to use this type
|
|
3072
|
-
description?: string; // Optional: description
|
|
3073
|
-
};
|
|
2796
|
+
**Key differences**
|
|
3074
2797
|
|
|
3075
|
-
|
|
3076
|
-
|
|
3077
|
-
// Optional - alternative output structures
|
|
3078
|
-
// Each item can be string (standard type name) or object (custom type with schema)
|
|
3079
|
-
secondaryObjectTypes?: Array<string | {
|
|
3080
|
-
type: string;
|
|
3081
|
-
schema?: Record<string, unknown>;
|
|
3082
|
-
whenToUse: string;
|
|
3083
|
-
description?: string;
|
|
3084
|
-
}>;
|
|
3085
|
-
|
|
3086
|
-
// Output mode configuration (v3.0.5+)
|
|
3087
|
-
outputMode?: 'json' | 'structured-text' | 'two-step'; // Default: 'json'
|
|
3088
|
-
instructionFormat?: 'json-schema' | 'structured-text-spec'; // Default: 'json-schema'
|
|
3089
|
-
structuredTextSpec?: { // Required when instructionFormat is 'structured-text-spec'
|
|
3090
|
-
description: string; // Free-form description of output structure
|
|
3091
|
-
formatHint?: string; // Optional format hint (markdown, yaml, etc.)
|
|
3092
|
-
};
|
|
3093
|
-
conversionInstructions?: string; // Optional custom conversion instructions for two-step mode
|
|
3094
|
-
|
|
3095
|
-
// Optional graph execution fields (also available via BaseLLMRequest)
|
|
3096
|
-
masterSkillId?: string; // Optional graph identifier (maps to runContext.graphId when persisted)
|
|
3097
|
-
skillId?: string; // Optional node identifier (maps to runContext.nodeId when persisted)
|
|
3098
|
-
masterSkillActivityId?: string; // Optional parent skill activity ID (preserved in identity)
|
|
3099
|
-
}
|
|
3100
|
-
```
|
|
2798
|
+
- **`invokeChat(request)`** — `ChatRequest`; does **not** require **`actionType`** / **`actionRef`**.
|
|
2799
|
+
- **`invoke(request)`** — `AIInvokeRequest`; **requires** **`actionType`** and **`actionRef`**.
|
|
3101
2800
|
|
|
3102
|
-
|
|
2801
|
+
#### StructuredTextSpec (chat / contract hints)
|
|
3103
2802
|
|
|
3104
|
-
|
|
3105
|
-
- `ChatRequest`: `objectTypes` is **NOT supported** - use with `invokeChat()` for conversational requests
|
|
3106
|
-
- `AIRequest`: `objectTypes` is **REQUIRED** - use with `invoke()` for structured output
|
|
2803
|
+
Used by **`ChatRequest.expectedSchema`** when you attach optional contract structured-text hints:
|
|
3107
2804
|
|
|
3108
|
-
|
|
2805
|
+
```typescript
|
|
2806
|
+
interface StructuredTextSpec {
|
|
2807
|
+
description: string;
|
|
2808
|
+
formatHint?: string;
|
|
2809
|
+
}
|
|
2810
|
+
```
|
|
3109
2811
|
|
|
3110
2812
|
**Output Modes (v3.0.5+):**
|
|
3111
2813
|
|
|
@@ -3139,17 +2841,28 @@ The gateway supports three output modes, each with two instruction formats:
|
|
|
3139
2841
|
- Example: "a story with character, setting, and conflict"
|
|
3140
2842
|
- Requires `structuredTextSpec` field
|
|
3141
2843
|
|
|
3142
|
-
**Minimal
|
|
2844
|
+
**Minimal AIInvokeRequest example (custom `primaryObjectType`):**
|
|
3143
2845
|
```typescript
|
|
2846
|
+
const aiRequestId = 'req-1';
|
|
3144
2847
|
const response = await gateway.invoke({
|
|
3145
|
-
|
|
2848
|
+
aiRequestId,
|
|
3146
2849
|
agentId: 'agent-1',
|
|
2850
|
+
actionType: 'skill',
|
|
2851
|
+
actionRef: 'skills/professional-answer',
|
|
3147
2852
|
instructions: 'professional-answer/general', // Content registry key
|
|
3148
|
-
|
|
2853
|
+
prompt: '{{input}}',
|
|
2854
|
+
workingMemory: { input: 'What is the capital of France?' },
|
|
2855
|
+
identity: {
|
|
2856
|
+
sessionId: 'run-1',
|
|
2857
|
+
instance: { instanceId: 'agent-1', type: 'ai-reasoner' },
|
|
2858
|
+
aiRequestId,
|
|
2859
|
+
jobId: 'job-123',
|
|
2860
|
+
taskId: 'task-1',
|
|
2861
|
+
agentId: 'agent-1'
|
|
2862
|
+
},
|
|
3149
2863
|
primaryObjectType: {
|
|
3150
2864
|
type: 'professional-answer',
|
|
3151
2865
|
whenToUse: 'For professional Q&A responses'
|
|
3152
|
-
// schema is optional
|
|
3153
2866
|
},
|
|
3154
2867
|
modelConfig: {
|
|
3155
2868
|
model: 'gpt-4o',
|
|
@@ -3158,14 +2871,25 @@ const response = await gateway.invoke({
|
|
|
3158
2871
|
});
|
|
3159
2872
|
```
|
|
3160
2873
|
|
|
3161
|
-
**Using modelConfig for
|
|
2874
|
+
**Using modelConfig for model selection:**
|
|
3162
2875
|
```typescript
|
|
3163
|
-
|
|
3164
|
-
// Priority: modelConfig > config > gateway defaults
|
|
2876
|
+
const aiRequestId = 'req-2';
|
|
3165
2877
|
const response = await gateway.invoke({
|
|
3166
|
-
|
|
2878
|
+
aiRequestId,
|
|
3167
2879
|
agentId: 'agent-1',
|
|
2880
|
+
actionType: 'skill',
|
|
2881
|
+
actionRef: 'skills/sentiment',
|
|
3168
2882
|
instructions: 'Analyze the data',
|
|
2883
|
+
prompt: '{{input}}',
|
|
2884
|
+
workingMemory: { input: 'sample' },
|
|
2885
|
+
identity: {
|
|
2886
|
+
sessionId: 'run-1',
|
|
2887
|
+
instance: { instanceId: 'agent-1', type: 'ai-reasoner' },
|
|
2888
|
+
aiRequestId,
|
|
2889
|
+
jobId: 'job-123',
|
|
2890
|
+
taskId: 'task-1',
|
|
2891
|
+
agentId: 'agent-1'
|
|
2892
|
+
},
|
|
3169
2893
|
primaryObjectType: 'sentiment-analysis',
|
|
3170
2894
|
modelConfig: {
|
|
3171
2895
|
model: 'gpt-4-turbo',
|
|
@@ -3177,35 +2901,57 @@ const response = await gateway.invoke({
|
|
|
3177
2901
|
});
|
|
3178
2902
|
```
|
|
3179
2903
|
|
|
3180
|
-
**modelConfig
|
|
2904
|
+
**modelConfig overrides `config`:**
|
|
3181
2905
|
```typescript
|
|
3182
|
-
|
|
2906
|
+
const aiRequestId = 'req-3';
|
|
3183
2907
|
const response = await gateway.invoke({
|
|
3184
|
-
|
|
2908
|
+
aiRequestId,
|
|
3185
2909
|
agentId: 'agent-1',
|
|
2910
|
+
actionType: 'skill',
|
|
2911
|
+
actionRef: 'skills/classification',
|
|
3186
2912
|
instructions: 'Process request',
|
|
2913
|
+
prompt: '{{input}}',
|
|
2914
|
+
workingMemory: { input: 'sample' },
|
|
2915
|
+
identity: {
|
|
2916
|
+
sessionId: 'run-1',
|
|
2917
|
+
instance: { instanceId: 'agent-1', type: 'ai-reasoner' },
|
|
2918
|
+
aiRequestId,
|
|
2919
|
+
jobId: 'job-123',
|
|
2920
|
+
taskId: 'task-1',
|
|
2921
|
+
agentId: 'agent-1'
|
|
2922
|
+
},
|
|
3187
2923
|
primaryObjectType: 'classification',
|
|
3188
2924
|
config: {
|
|
3189
|
-
model: 'gpt-3.5-turbo',
|
|
3190
|
-
temperature: 0.5
|
|
2925
|
+
model: 'gpt-3.5-turbo',
|
|
2926
|
+
temperature: 0.5
|
|
3191
2927
|
},
|
|
3192
2928
|
modelConfig: {
|
|
3193
|
-
model: 'gpt-4-turbo',
|
|
3194
|
-
temperature: 0.9
|
|
2929
|
+
model: 'gpt-4-turbo',
|
|
2930
|
+
temperature: 0.9
|
|
3195
2931
|
}
|
|
3196
2932
|
});
|
|
3197
|
-
// Result: Uses gpt-4-turbo with temperature 0.9
|
|
3198
2933
|
```
|
|
3199
2934
|
|
|
3200
|
-
**Standard
|
|
2935
|
+
**Standard object type example (v3.0.6+):**
|
|
3201
2936
|
```typescript
|
|
3202
|
-
|
|
2937
|
+
const aiRequestId = 'req-4';
|
|
3203
2938
|
const response = await gateway.invoke({
|
|
3204
|
-
|
|
2939
|
+
aiRequestId,
|
|
3205
2940
|
agentId: 'agent-1',
|
|
2941
|
+
actionType: 'skill',
|
|
2942
|
+
actionRef: 'skills/sentiment-analysis',
|
|
3206
2943
|
instructions: 'Analyze the sentiment of this text',
|
|
3207
|
-
|
|
3208
|
-
|
|
2944
|
+
prompt: '{{input}}',
|
|
2945
|
+
workingMemory: { input: 'I love this product!' },
|
|
2946
|
+
identity: {
|
|
2947
|
+
sessionId: 'run-1',
|
|
2948
|
+
instance: { instanceId: 'agent-1', type: 'ai-reasoner' },
|
|
2949
|
+
aiRequestId,
|
|
2950
|
+
jobId: 'job-123',
|
|
2951
|
+
taskId: 'task-1',
|
|
2952
|
+
agentId: 'agent-1'
|
|
2953
|
+
},
|
|
2954
|
+
primaryObjectType: 'sentiment-analysis',
|
|
3209
2955
|
config: {
|
|
3210
2956
|
model: 'gpt-5-nano',
|
|
3211
2957
|
provider: 'openai'
|
|
@@ -3267,11 +3013,23 @@ The gateway uses **flex-md (Markdown-based format)** for all LLM communication,
|
|
|
3267
3013
|
**Example:**
|
|
3268
3014
|
|
|
3269
3015
|
```typescript
|
|
3016
|
+
const aiRequestId = 'flex-md-ex-1';
|
|
3270
3017
|
const response = await gateway.invoke({
|
|
3271
|
-
|
|
3018
|
+
aiRequestId,
|
|
3272
3019
|
agentId: 'agent-1',
|
|
3020
|
+
actionType: 'skill',
|
|
3021
|
+
actionRef: 'skills/sentiment',
|
|
3273
3022
|
instructions: 'Classify the sentiment of this text',
|
|
3274
|
-
|
|
3023
|
+
prompt: '{{input}}',
|
|
3024
|
+
workingMemory: { input: 'This product is amazing!' },
|
|
3025
|
+
identity: {
|
|
3026
|
+
sessionId: 's1',
|
|
3027
|
+
instance: { instanceId: 'agent-1', type: 'test' },
|
|
3028
|
+
aiRequestId,
|
|
3029
|
+
jobId: 'job-123',
|
|
3030
|
+
taskId: 'task-1',
|
|
3031
|
+
agentId: 'agent-1'
|
|
3032
|
+
},
|
|
3275
3033
|
primaryObjectType: {
|
|
3276
3034
|
type: 'classification',
|
|
3277
3035
|
schema: {
|
|
@@ -3436,14 +3194,26 @@ The `instructions-blocks.json` file supports nested objects that get automatical
|
|
|
3436
3194
|
- **Template Variables**: All `{{variable}}` placeholders are resolved using request context
|
|
3437
3195
|
- **Nested Access**: Dot notation (e.g., `output.complianceLevels.L2`) automatically traverses nested objects
|
|
3438
3196
|
|
|
3439
|
-
**
|
|
3197
|
+
**AIInvokeRequest with object types / schema (invoke):**
|
|
3440
3198
|
```typescript
|
|
3199
|
+
const aiRequestId = 'pa-1';
|
|
3441
3200
|
const response = await gateway.invoke({
|
|
3442
|
-
|
|
3201
|
+
aiRequestId,
|
|
3443
3202
|
agentId: 'agent-1',
|
|
3203
|
+
actionType: 'skill',
|
|
3204
|
+
actionRef: 'skills/professional-answer',
|
|
3444
3205
|
instructions: 'professional-answer/general',
|
|
3445
|
-
|
|
3206
|
+
prompt: '{{input}}',
|
|
3207
|
+
workingMemory: { input: 'User question here' },
|
|
3446
3208
|
inferenceType: 'professional-answer',
|
|
3209
|
+
identity: {
|
|
3210
|
+
sessionId: 's1',
|
|
3211
|
+
instance: { instanceId: 'agent-1', type: 'test' },
|
|
3212
|
+
aiRequestId,
|
|
3213
|
+
jobId: 'job-123',
|
|
3214
|
+
taskId: 'task-1',
|
|
3215
|
+
agentId: 'agent-1'
|
|
3216
|
+
},
|
|
3447
3217
|
objectTypes: [
|
|
3448
3218
|
{
|
|
3449
3219
|
type: 'professional-answer',
|
|
@@ -3460,44 +3230,50 @@ const response = await gateway.invoke({
|
|
|
3460
3230
|
}
|
|
3461
3231
|
}
|
|
3462
3232
|
],
|
|
3463
|
-
validateOutputSchema: true,
|
|
3464
3233
|
config: {
|
|
3465
3234
|
model: 'gpt-4o',
|
|
3466
3235
|
provider: 'openai'
|
|
3467
3236
|
}
|
|
3468
3237
|
});
|
|
3238
|
+
// Validate against schema in your app if needed — `validateOutputSchema` is not a request field.
|
|
3469
3239
|
```
|
|
3470
3240
|
|
|
3471
|
-
**Content
|
|
3241
|
+
**Content registry + AIInvokeRequest (recommended):**
|
|
3472
3242
|
|
|
3473
|
-
When using content-registry with
|
|
3243
|
+
When using content-registry with **`invoke()`**, the gateway typically:
|
|
3474
3244
|
1. Resolves instruction keys from content-registry
|
|
3475
|
-
2. Fetches instruction metadata (outputType
|
|
3476
|
-
3.
|
|
3477
|
-
4. Validates responses against schema if `validateOutputSchema: true`
|
|
3245
|
+
2. Fetches instruction metadata (**`outputType`**, **`outputSchema`**, **`parseOptions`** on **metadata**, not on the request body)
|
|
3246
|
+
3. May configure provider JSON modes via merged **`config`** (provider-specific)
|
|
3478
3247
|
|
|
3479
3248
|
```typescript
|
|
3480
|
-
|
|
3481
|
-
// Metadata can include: outputType, outputSchema, validationRules
|
|
3482
|
-
|
|
3249
|
+
const aiRequestId = 'pa-2';
|
|
3483
3250
|
const response = await gateway.invoke({
|
|
3484
|
-
|
|
3251
|
+
aiRequestId,
|
|
3485
3252
|
agentId: 'agent-1',
|
|
3486
|
-
|
|
3487
|
-
|
|
3253
|
+
actionType: 'skill',
|
|
3254
|
+
actionRef: 'skills/professional-answer',
|
|
3255
|
+
instructions: 'professional-answer/general',
|
|
3256
|
+
prompt: '{{input}}',
|
|
3257
|
+
workingMemory: { input: 'User question' },
|
|
3488
3258
|
inferenceType: 'professional-answer',
|
|
3259
|
+
identity: {
|
|
3260
|
+
sessionId: 's1',
|
|
3261
|
+
instance: { instanceId: 'agent-1', type: 'test' },
|
|
3262
|
+
aiRequestId,
|
|
3263
|
+
jobId: 'job-123',
|
|
3264
|
+
taskId: 'task-1',
|
|
3265
|
+
agentId: 'agent-1'
|
|
3266
|
+
},
|
|
3489
3267
|
objectTypes: [
|
|
3490
3268
|
{
|
|
3491
3269
|
type: 'professional-answer',
|
|
3492
3270
|
whenToUse: 'For professional Q&A responses',
|
|
3493
|
-
// Schema can come from instruction metadata or be provided here
|
|
3494
3271
|
schema: instructionMetadata?.outputSchema || {
|
|
3495
3272
|
type: 'object',
|
|
3496
3273
|
properties: { answer: { type: 'string' } }
|
|
3497
3274
|
}
|
|
3498
3275
|
}
|
|
3499
3276
|
],
|
|
3500
|
-
validateOutputSchema: true,
|
|
3501
3277
|
config: {
|
|
3502
3278
|
model: 'gpt-4o',
|
|
3503
3279
|
provider: 'openai'
|
|
@@ -3509,11 +3285,23 @@ const response = await gateway.invoke({
|
|
|
3509
3285
|
|
|
3510
3286
|
**1. JSON Output Mode (Default):**
|
|
3511
3287
|
```typescript
|
|
3288
|
+
const aiRequestId = 'out-json-1';
|
|
3512
3289
|
const response = await gateway.invoke({
|
|
3513
|
-
|
|
3290
|
+
aiRequestId,
|
|
3514
3291
|
agentId: 'agent-1',
|
|
3292
|
+
actionType: 'skill',
|
|
3293
|
+
actionRef: 'skills/qa',
|
|
3515
3294
|
instructions: 'Answer the question',
|
|
3516
|
-
prompt: '
|
|
3295
|
+
prompt: '{{input}}',
|
|
3296
|
+
workingMemory: { input: 'What is the capital of France?' },
|
|
3297
|
+
identity: {
|
|
3298
|
+
sessionId: 's1',
|
|
3299
|
+
instance: { instanceId: 'agent-1', type: 'test' },
|
|
3300
|
+
aiRequestId,
|
|
3301
|
+
jobId: 'job-123',
|
|
3302
|
+
taskId: 'task-1',
|
|
3303
|
+
agentId: 'agent-1'
|
|
3304
|
+
},
|
|
3517
3305
|
primaryObjectType: {
|
|
3518
3306
|
type: 'question-answer',
|
|
3519
3307
|
schema: {
|
|
@@ -3531,11 +3319,23 @@ const response = await gateway.invoke({
|
|
|
3531
3319
|
|
|
3532
3320
|
**2. Structured Text Output Mode:**
|
|
3533
3321
|
```typescript
|
|
3322
|
+
const aiRequestId = 'out-st-1';
|
|
3534
3323
|
const response = await gateway.invoke({
|
|
3535
|
-
|
|
3324
|
+
aiRequestId,
|
|
3536
3325
|
agentId: 'story-teller',
|
|
3326
|
+
actionType: 'skill',
|
|
3327
|
+
actionRef: 'skills/story',
|
|
3537
3328
|
instructions: 'Write a short story',
|
|
3538
|
-
prompt: '
|
|
3329
|
+
prompt: '{{input}}',
|
|
3330
|
+
workingMemory: { input: 'Create a story about a brave knight' },
|
|
3331
|
+
identity: {
|
|
3332
|
+
sessionId: 's1',
|
|
3333
|
+
instance: { instanceId: 'story-teller', type: 'test' },
|
|
3334
|
+
aiRequestId,
|
|
3335
|
+
jobId: 'job-123',
|
|
3336
|
+
taskId: 'task-1',
|
|
3337
|
+
agentId: 'story-teller'
|
|
3338
|
+
},
|
|
3539
3339
|
primaryObjectType: {
|
|
3540
3340
|
type: 'story',
|
|
3541
3341
|
whenToUse: 'For narrative content'
|
|
@@ -3554,11 +3354,23 @@ const response = await gateway.invoke({
|
|
|
3554
3354
|
|
|
3555
3355
|
**3. Two-Step Conversion Mode:**
|
|
3556
3356
|
```typescript
|
|
3357
|
+
const aiRequestId = 'out-2step-1';
|
|
3557
3358
|
const response = await gateway.invoke({
|
|
3558
|
-
|
|
3359
|
+
aiRequestId,
|
|
3559
3360
|
agentId: 'story-converter',
|
|
3361
|
+
actionType: 'skill',
|
|
3362
|
+
actionRef: 'skills/story-convert',
|
|
3560
3363
|
instructions: 'Write a detailed story',
|
|
3561
|
-
prompt: '
|
|
3364
|
+
prompt: '{{input}}',
|
|
3365
|
+
workingMemory: { input: 'Create a story about space exploration' },
|
|
3366
|
+
identity: {
|
|
3367
|
+
sessionId: 's1',
|
|
3368
|
+
instance: { instanceId: 'story-converter', type: 'test' },
|
|
3369
|
+
aiRequestId,
|
|
3370
|
+
jobId: 'job-123',
|
|
3371
|
+
taskId: 'task-1',
|
|
3372
|
+
agentId: 'story-converter'
|
|
3373
|
+
},
|
|
3562
3374
|
primaryObjectType: {
|
|
3563
3375
|
type: 'story',
|
|
3564
3376
|
schema: {
|
|
@@ -3591,11 +3403,11 @@ const response = await gateway.invoke({
|
|
|
3591
3403
|
- **Structured Text Spec + Structured Text Mode**: Instructions describe structure in free-form, expects structured text
|
|
3592
3404
|
- **Structured Text Spec + Two-Step Mode**: Instructions describe structure in free-form, gets structured text first, then converts to JSON
|
|
3593
3405
|
|
|
3594
|
-
**Troubleshooting**: If
|
|
3595
|
-
1.
|
|
3596
|
-
2.
|
|
3597
|
-
3. Use
|
|
3598
|
-
4. Enable
|
|
3406
|
+
**Troubleshooting**: If **`invoke()`** fails validation:
|
|
3407
|
+
1. Confirm **`actionType`**, **`actionRef`**, **`aiRequestId`**, and full **`identity`** (**`jobId`** / **`taskId`**) are set
|
|
3408
|
+
2. Remove top-level **`input`** — use **`prompt`** + **`workingMemory.input`**
|
|
3409
|
+
3. Use **`validateAIRequest()`** / **`assertValidAIRequest()`** before sending
|
|
3410
|
+
4. Enable **`AI_GATEWAY_DEBUG_REQUEST=true`** to inspect the request at the entry point
|
|
3599
3411
|
5. See [TROUBLESHOOTING.md](./TROUBLESHOOTING.md#airequest-validation-issues) for detailed solutions
|
|
3600
3412
|
|
|
3601
3413
|
#### Task Type ID (taskTypeId)
|
|
@@ -3631,14 +3443,26 @@ const taskTypeId = await gateway.generateTaskTypeId(question);
|
|
|
3631
3443
|
|
|
3632
3444
|
// All 15k requests will have the same taskTypeId
|
|
3633
3445
|
for (const record of records) {
|
|
3446
|
+
const aiRequestId = `sentiment-${record.id}`;
|
|
3634
3447
|
await gateway.invoke({
|
|
3635
|
-
|
|
3448
|
+
aiRequestId,
|
|
3636
3449
|
agentId: 'sentiment-analyzer',
|
|
3450
|
+
actionType: 'skill',
|
|
3451
|
+
actionRef: 'skills/sentiment-batch',
|
|
3637
3452
|
instructions: question,
|
|
3638
|
-
|
|
3639
|
-
|
|
3640
|
-
|
|
3641
|
-
|
|
3453
|
+
prompt: '{{input}}',
|
|
3454
|
+
workingMemory: { input: record.text },
|
|
3455
|
+
taskTypeId, // Same for all records
|
|
3456
|
+
identity: {
|
|
3457
|
+
sessionId: 'batch-1',
|
|
3458
|
+
instance: { instanceId: 'sentiment-analyzer', type: 'batch' },
|
|
3459
|
+
aiRequestId,
|
|
3460
|
+
jobId: `job-${record.id}`,
|
|
3461
|
+
taskId: `task-${record.id}`,
|
|
3462
|
+
agentId: 'sentiment-analyzer'
|
|
3463
|
+
},
|
|
3464
|
+
primaryObjectType: 'sentiment-analysis',
|
|
3465
|
+
config: { model: 'gpt-5-nano', provider: 'openai' }
|
|
3642
3466
|
});
|
|
3643
3467
|
}
|
|
3644
3468
|
```
|
|
@@ -3710,7 +3534,7 @@ interface EnhancedLLMResponse extends LLMResponse {
|
|
|
3710
3534
|
|
|
3711
3535
|
### Custom Activix instance
|
|
3712
3536
|
|
|
3713
|
-
Use the same **`collections`** names the gateway writes to (`ai-
|
|
3537
|
+
Use the same **`collections`** names the gateway writes to (`ai-actions`, `skill-executions`, `bad-requests`) and the same **`statusValues`** mapping as in section 2.
|
|
3714
3538
|
|
|
3715
3539
|
```typescript
|
|
3716
3540
|
import { Activix } from '@x12i/activix';
|
|
@@ -3725,7 +3549,7 @@ const statusValues = {
|
|
|
3725
3549
|
|
|
3726
3550
|
const customTracker = new Activix({
|
|
3727
3551
|
collections: [
|
|
3728
|
-
{ name: 'ai-
|
|
3552
|
+
{ name: 'ai-actions', statusValues },
|
|
3729
3553
|
{ name: 'skill-executions', statusValues },
|
|
3730
3554
|
{ name: 'bad-requests', statusValues }
|
|
3731
3555
|
]
|
|
@@ -3832,14 +3656,23 @@ const gateway = new AIGateway({
|
|
|
3832
3656
|
|
|
3833
3657
|
// In your agent execution
|
|
3834
3658
|
async function executeAgentTask(task: Task, jobId: string) {
|
|
3659
|
+
const aiRequestId = `agent-${task.id}-${Date.now()}`;
|
|
3835
3660
|
// Use invokeChat() for chat requests without structured output
|
|
3836
3661
|
const response = await gateway.invokeChat({
|
|
3837
|
-
|
|
3662
|
+
aiRequestId,
|
|
3838
3663
|
agentId: task.agentId,
|
|
3839
3664
|
instructions: task.instructions,
|
|
3840
|
-
|
|
3841
|
-
|
|
3842
|
-
taskTypeId: task.typeId
|
|
3665
|
+
prompt: '{{input}}',
|
|
3666
|
+
workingMemory: { input: task.input },
|
|
3667
|
+
taskTypeId: task.typeId, // Or use MD5 hash of question/instruction for consistent identification
|
|
3668
|
+
identity: {
|
|
3669
|
+
sessionId: jobId,
|
|
3670
|
+
instance: { instanceId: task.agentId, type: 'agent' },
|
|
3671
|
+
aiRequestId,
|
|
3672
|
+
jobId,
|
|
3673
|
+
taskId: task.id,
|
|
3674
|
+
agentId: task.agentId
|
|
3675
|
+
}
|
|
3843
3676
|
});
|
|
3844
3677
|
|
|
3845
3678
|
return {
|
|
@@ -3867,11 +3700,21 @@ if (model) {
|
|
|
3867
3700
|
defaultProvider: model.provider
|
|
3868
3701
|
});
|
|
3869
3702
|
|
|
3703
|
+
const aiRequestId = 'xmodels-1';
|
|
3870
3704
|
const response = await gateway.invokeChat({
|
|
3871
|
-
|
|
3705
|
+
aiRequestId,
|
|
3872
3706
|
agentId: 'agent-1',
|
|
3873
3707
|
instructions: 'You are a helpful assistant',
|
|
3874
|
-
|
|
3708
|
+
prompt: '{{input}}',
|
|
3709
|
+
workingMemory: { input: 'Hello!' },
|
|
3710
|
+
identity: {
|
|
3711
|
+
sessionId: 's1',
|
|
3712
|
+
instance: { instanceId: 'agent-1', type: 'test' },
|
|
3713
|
+
aiRequestId,
|
|
3714
|
+
jobId: 'job-123',
|
|
3715
|
+
taskId: 'task-1',
|
|
3716
|
+
agentId: 'agent-1'
|
|
3717
|
+
},
|
|
3875
3718
|
config: {
|
|
3876
3719
|
model: model.id
|
|
3877
3720
|
}
|
|
@@ -3886,11 +3729,21 @@ The gateway supports unified reasoning/thoughts configuration through the OpenRo
|
|
|
3886
3729
|
#### Request Configuration
|
|
3887
3730
|
|
|
3888
3731
|
```typescript
|
|
3732
|
+
const aiRequestId = 'reasoning-example';
|
|
3889
3733
|
const response = await gateway.invokeChat({
|
|
3890
|
-
|
|
3734
|
+
aiRequestId,
|
|
3891
3735
|
agentId: 'agent-1',
|
|
3892
3736
|
instructions: 'Solve this step-by-step',
|
|
3893
|
-
|
|
3737
|
+
prompt: '{{input}}',
|
|
3738
|
+
workingMemory: { input: 'What is 15 * 23?' },
|
|
3739
|
+
identity: {
|
|
3740
|
+
sessionId: 's1',
|
|
3741
|
+
instance: { instanceId: 'agent-1', type: 'test' },
|
|
3742
|
+
aiRequestId,
|
|
3743
|
+
jobId: 'job-reason',
|
|
3744
|
+
taskId: 'task-reason',
|
|
3745
|
+
agentId: 'agent-1'
|
|
3746
|
+
},
|
|
3894
3747
|
config: {
|
|
3895
3748
|
provider: 'openai',
|
|
3896
3749
|
model: 'gpt-5-nano',
|
|
@@ -3967,11 +3820,21 @@ interface EnhancedLLMResponse {
|
|
|
3967
3820
|
|
|
3968
3821
|
**Basic Reasoning Request:**
|
|
3969
3822
|
```typescript
|
|
3823
|
+
const aiRequestId = 'basic-reasoning';
|
|
3970
3824
|
const response = await gateway.invokeChat({
|
|
3971
|
-
|
|
3825
|
+
aiRequestId,
|
|
3972
3826
|
agentId: 'agent-1',
|
|
3973
3827
|
instructions: 'Explain your reasoning step by step',
|
|
3974
|
-
|
|
3828
|
+
prompt: '{{input}}',
|
|
3829
|
+
workingMemory: { input: 'Why is the sky blue?' },
|
|
3830
|
+
identity: {
|
|
3831
|
+
sessionId: 's1',
|
|
3832
|
+
instance: { instanceId: 'agent-1', type: 'test' },
|
|
3833
|
+
aiRequestId,
|
|
3834
|
+
jobId: 'job-reason',
|
|
3835
|
+
taskId: 'task-basic',
|
|
3836
|
+
agentId: 'agent-1'
|
|
3837
|
+
},
|
|
3975
3838
|
config: {
|
|
3976
3839
|
provider: 'openai',
|
|
3977
3840
|
model: 'gpt-5-nano',
|
|
@@ -3991,11 +3854,21 @@ console.log('Reasoning summary:', response.reasoning?.artifacts?.summary?.text);
|
|
|
3991
3854
|
**Encrypted Reasoning Continuity:**
|
|
3992
3855
|
```typescript
|
|
3993
3856
|
// First request with encrypted trace
|
|
3857
|
+
const aiRequestId1 = 'continuity-1';
|
|
3994
3858
|
const response1 = await gateway.invokeChat({
|
|
3995
|
-
|
|
3859
|
+
aiRequestId: aiRequestId1,
|
|
3996
3860
|
agentId: 'agent-1',
|
|
3997
3861
|
instructions: 'Solve this complex problem',
|
|
3998
|
-
|
|
3862
|
+
prompt: '{{input}}',
|
|
3863
|
+
workingMemory: { input: 'Calculate the trajectory of a satellite' },
|
|
3864
|
+
identity: {
|
|
3865
|
+
sessionId: 's1',
|
|
3866
|
+
instance: { instanceId: 'agent-1', type: 'test' },
|
|
3867
|
+
aiRequestId: aiRequestId1,
|
|
3868
|
+
jobId: 'job-cont',
|
|
3869
|
+
taskId: 'task-c1',
|
|
3870
|
+
agentId: 'agent-1'
|
|
3871
|
+
},
|
|
3999
3872
|
config: {
|
|
4000
3873
|
provider: 'openai',
|
|
4001
3874
|
model: 'o1-preview', // OpenAI o-series models support encrypted traces
|
|
@@ -4013,11 +3886,21 @@ if (encryptedArtifacts && encryptedArtifacts.length > 0) {
|
|
|
4013
3886
|
|
|
4014
3887
|
// Second request with continuity (encrypted artifacts would be passed back)
|
|
4015
3888
|
// Note: Continuity input format depends on provider-specific implementation
|
|
3889
|
+
const aiRequestId2 = 'continuity-2';
|
|
4016
3890
|
const response2 = await gateway.invokeChat({
|
|
4017
|
-
|
|
3891
|
+
aiRequestId: aiRequestId2,
|
|
4018
3892
|
agentId: 'agent-1',
|
|
4019
3893
|
instructions: 'Continue from previous reasoning',
|
|
4020
|
-
|
|
3894
|
+
prompt: '{{input}}',
|
|
3895
|
+
workingMemory: { input: 'Now apply this to Mars orbit' },
|
|
3896
|
+
identity: {
|
|
3897
|
+
sessionId: 's1',
|
|
3898
|
+
instance: { instanceId: 'agent-1', type: 'test' },
|
|
3899
|
+
aiRequestId: aiRequestId2,
|
|
3900
|
+
jobId: 'job-cont',
|
|
3901
|
+
taskId: 'task-c2',
|
|
3902
|
+
agentId: 'agent-1'
|
|
3903
|
+
},
|
|
4021
3904
|
config: {
|
|
4022
3905
|
provider: 'openai',
|
|
4023
3906
|
model: 'o1-preview',
|
|
@@ -4067,26 +3950,35 @@ import {
|
|
|
4067
3950
|
|
|
4068
3951
|
**See**: [TROUBLESHOOTING_TOOLBOX.md](./TROUBLESHOOTING_TOOLBOX.md) for complete API documentation.
|
|
4069
3952
|
|
|
4070
|
-
### Common Issue:
|
|
3953
|
+
### Common Issue: `invoke()` validation (`actionType`, `actionRef`, `identity`, `input`)
|
|
4071
3954
|
|
|
4072
|
-
**
|
|
3955
|
+
**Typical errors**: missing **`actionType`** / **`actionRef`**, missing **`identity.jobId`** / **`identity.taskId`**, or passing deprecated top-level **`input`** (use **`workingMemory.input`** with a **`prompt`** template).
|
|
4073
3956
|
|
|
4074
3957
|
**Quick Fix Checklist**:
|
|
4075
3958
|
|
|
4076
|
-
1. ✅ **
|
|
3959
|
+
1. ✅ **Structured calls must use `invoke()` with `AIInvokeRequest` fields**
|
|
4077
3960
|
```typescript
|
|
4078
|
-
|
|
3961
|
+
const aiRequestId = 'fix-1';
|
|
4079
3962
|
await gateway.invoke({
|
|
4080
|
-
|
|
3963
|
+
aiRequestId,
|
|
4081
3964
|
agentId: 'agent-1',
|
|
3965
|
+
actionType: 'skill',
|
|
3966
|
+
actionRef: 'skills/professional-answer',
|
|
4082
3967
|
instructions: 'professional-answer/general',
|
|
4083
|
-
|
|
4084
|
-
|
|
4085
|
-
|
|
4086
|
-
|
|
4087
|
-
|
|
4088
|
-
|
|
4089
|
-
|
|
3968
|
+
prompt: '{{input}}',
|
|
3969
|
+
workingMemory: { input: 'User question' },
|
|
3970
|
+
identity: {
|
|
3971
|
+
sessionId: 's1',
|
|
3972
|
+
instance: { instanceId: 'agent-1', type: 'test' },
|
|
3973
|
+
aiRequestId,
|
|
3974
|
+
jobId: 'job-123',
|
|
3975
|
+
taskId: 'task-1',
|
|
3976
|
+
agentId: 'agent-1'
|
|
3977
|
+
},
|
|
3978
|
+
primaryObjectType: {
|
|
3979
|
+
type: 'professional-answer',
|
|
3980
|
+
whenToUse: 'For professional Q&A responses'
|
|
3981
|
+
},
|
|
4090
3982
|
config: { model: 'gpt-4o', provider: 'openai' }
|
|
4091
3983
|
});
|
|
4092
3984
|
```
|
|
@@ -4142,14 +4034,21 @@ The gateway throws specific error types:
|
|
|
4142
4034
|
import { ProviderNotFoundError, FallbackExhaustedError } from '@x12i/ai-gateway';
|
|
4143
4035
|
|
|
4144
4036
|
try {
|
|
4037
|
+
const aiRequestId = 'chat-req-001';
|
|
4145
4038
|
const response = await gateway.invokeChat({
|
|
4146
|
-
aiRequestId
|
|
4147
|
-
sessionId: 'sess-1',
|
|
4148
|
-
instance: { instanceId: 'gw-1', type: 'gateway' },
|
|
4149
|
-
jobId: 'job-123',
|
|
4039
|
+
aiRequestId,
|
|
4150
4040
|
agentId: 'agent-1',
|
|
4151
4041
|
instructions: 'You are a helpful assistant',
|
|
4152
|
-
|
|
4042
|
+
prompt: '{{input}}',
|
|
4043
|
+
workingMemory: { input: 'Hello!' },
|
|
4044
|
+
identity: {
|
|
4045
|
+
sessionId: 'sess-1',
|
|
4046
|
+
instance: { instanceId: 'gw-1', type: 'gateway' },
|
|
4047
|
+
aiRequestId,
|
|
4048
|
+
jobId: 'job-123',
|
|
4049
|
+
taskId: 'task-1',
|
|
4050
|
+
agentId: 'agent-1'
|
|
4051
|
+
}
|
|
4153
4052
|
});
|
|
4154
4053
|
} catch (error) {
|
|
4155
4054
|
if (error instanceof FallbackExhaustedError) {
|
|
@@ -4168,7 +4067,7 @@ This package integrates with:
|
|
|
4168
4067
|
- **@x12i/ai-providers-router**: Core routing functionality
|
|
4169
4068
|
- **@xronoces/content-registry**: Instruction key resolution and content management (optional)
|
|
4170
4069
|
- **@x12i/x-models**: Usage tier tracking and model metadata
|
|
4171
|
-
- **@x12i/activix**
|
|
4070
|
+
- **@x12i/activix** v7 (`@x12i/xronox-store`): Activity logging and tracking (`^7.1.0` in this package)
|
|
4172
4071
|
- **@x12i/logxer**: Structured logging with correlation (`^4.2.1` in this package)
|
|
4173
4072
|
- **nx-config2**: Configuration management (via dependencies)
|
|
4174
4073
|
|