dyna-ai-gateway 0.0.1-beta.7 → 0.0.13-beta.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (3) hide show
  1. package/CHANGELOG.md +8 -1
  2. package/README.md +294 -19
  3. package/package.json +1 -1
package/CHANGELOG.md CHANGED
@@ -6,6 +6,13 @@ what breaks, the migration, and an AI prompt that performs the migration in your
6
6
 
7
7
  ## [Unreleased]
8
8
 
9
+ ## [0.0.13-beta.8] - 2026-11-09
10
+
11
+
12
+ ### Fixed
13
+
14
+ - Fix readme links
15
+
9
16
  ## [0.0.1-beta.7] - 2026-11-09
10
17
 
11
18
  ### Added
@@ -218,4 +225,4 @@ what breaks, the migration, and an AI prompt that performs the migration in your
218
225
  - The shipped `CHAT_MAIN` template puts its stable layers first: the base text, the output format and the profile addendum,
219
226
  then the per-call "Today is … locale … timezone" line, the call's instructions and context, the memory and the knowledge
220
227
  base. The same layers, one of them moved: a prompt cache can now match the prefix across calls. A tenant's own stored
221
- `CHAT_MAIN` version is not touched. See `docs/architecture/System prompt production layers.md`.
228
+ `CHAT_MAIN` version is not touched. See `docs/architecture/System prompt production layers.md`.
package/README.md CHANGED
@@ -1,31 +1,306 @@
1
+ ![dyna AI Gateway logo](https://raw.githubusercontent.com/aneldev/dyna-ai-gateway/main/assets/dyna-ai-gateway%20banner.png)
2
+
3
+ <p align="center">
4
+ <a href="https://aneldev.github.io/dyna-ai-gateway/"><strong>Documentation</strong></a>
5
+ &nbsp;·&nbsp;
6
+ <a href="https://aneldev.github.io/dyna-ai-gateway/docs/getting-started/README">Getting started</a>
7
+ &nbsp;·&nbsp;
8
+ <a href="https://aneldev.github.io/dyna-ai-gateway/docs/api/index/classes/AIGateway">API reference</a>
9
+ &nbsp;·&nbsp;
10
+ <a href="https://www.npmjs.com/package/dyna-ai-gateway">npm</a>
11
+ </p>
12
+
13
+ <p align="center">
14
+ <a href="https://www.npmjs.com/package/dyna-ai-gateway"><img alt="npm version" src="https://img.shields.io/npm/v/dyna-ai-gateway"></a>
15
+ <a href="https://aneldev.github.io/dyna-ai-gateway/"><img alt="docs" src="https://img.shields.io/badge/docs-aneldev.github.io-blue"></a>
16
+ <a href="https://github.com/aneldev/dyna-ai-gateway/blob/main/LICENSE"><img alt="license: MIT" src="https://img.shields.io/badge/license-MIT-green"></a>
17
+ </p>
18
+
19
+
1
20
  # dyna-ai-gateway
2
21
 
3
- Provider-agnostic AI gateway for Node.js (server-only): `ask` (grounded answers) and `run` (agent runs with tools, a human
4
- approval gate and an optional review of the answer), knowledge base (RAG), thread and user memory, usage control by the application, guardrails, traces and streaming, on top of the
5
- [Vercel AI SDK](https://ai-sdk.dev). Persistence is behind repository/adapter contracts; in-memory adapters are included.
6
22
 
7
- ```ts
8
- import {AIGateway} from "dyna-ai-gateway";
9
- import {AIGatewayVercelUi} from "dyna-ai-gateway/vercel-ui"; // the endpoints of a Vercel chat client (useChat)
23
+ A server-side TypeScript library that puts a language model inside a business application. The application calls
24
+ `ask()` for an answer or `run()` for an agent with tools; the gateway does the rest: prompt assembly, conversation
25
+ history, memory, knowledge base, tool policies, human approval, usage control, guardrails and tracing. Every provider is reached
26
+ through the [Vercel AI SDK](https://ai-sdk.dev); every store is behind an interface, with in-memory implementations
27
+ included.
28
+
29
+ ```
30
+ Application / Agent dyna-ai-gateway LLM providers
31
+ ┌──────────────────┐ ask() ┌───────────────────────────────────────┐ ┌─────────────┐
32
+ │ business logic │ run() │ prompt assembly · history · memory │ AI │ OpenAI │
33
+ │ HTTP, UI, jobs │ ────────▶ │ knowledge base · tools · policies │ SDK │ Anthropic │
34
+ │ │ ◀──────── │ human gate · usage · guardrails │ ─────▶ │ Google │
35
+ │ userContext │ events │ trace │ │ Mistral │
36
+ │ (authenticated) │ response └──────────────────┬────────────────────┘ │ Bedrock │
37
+ └──────────────────┘ │ repository / adapter └─────────────┘
38
+ ▼ interfaces
39
+ in-memory · MongoDB · PostgreSQL · S3
10
40
  ```
11
41
 
12
- - Documentation: https://aneldev.github.io/dyna-ai-gateway/ (getting started, API reference, architecture)
13
- - Tests: `src/tests/README.md` · Sandbox for developers (chat and debugger on a test's config): `src/dev-sandbox/README.md`
42
+ **No weirdos!**
43
+
44
+ > Nothing here is invented. It is the code every application around a model has to write anyway, written once and
45
+ > tested.
46
+
47
+ **Status: beta.** The API is considered stable and keeps improving; the documentation is being completed. Versions follow
48
+ [semver](https://semver.org); while the version is `0.x` a release may contain a breaking change, listed under *Breaking
49
+ changes* in the package's `CHANGELOG.md` with what breaks, the migration and an AI prompt that performs it. Every exported
50
+ symbol carries its JSDoc, so the editor shows the contract of each option.
14
51
 
15
- ## Requirements
52
+ | Quality | |
53
+ |---------------------|--------------------------------------------------------------------|
54
+ | Deterministic tests | 1000+, on a scripted model, no key needed |
55
+ | Real-provider tests | 30+, against OpenAI, Anthropic, Google, Mistral and Amazon Bedrock |
56
+ | Types | `tsc` over sources and tests; non-null assertions are a lint error |
16
57
 
17
- - Node.js 24 or newer, pnpm 10
18
- - Peer dependency: `zod` 4
58
+ ## Before you adopt it
19
59
 
20
- ## Development
60
+ | Question | Answer |
61
+ |-----------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
62
+ | Am I locked into one provider? | No. Five providers; the model is chosen per call through the profile, so a thread can switch model between two calls. |
63
+ | Where does my data live? | In your database. Every store is an interface (`IAIGatewayConfig.<store>.repository`); in-memory implementations are included. |
64
+ | How does the model reach my code? | `defineTool`: a function with a zod schema and a policy that allows, denies or holds each call on its parsed input. |
65
+ | Can a human step in? | Yes. A run pauses on a tool call or on the final answer; `resume()` takes approve, edit or reject. |
66
+ | How do I control the spend? | In your code. The gateway asks your usage adapter before every model call, with the tenant, user, profile, model, purpose and a token estimate, and reports what the call used after it. It holds no budget, plan or quota of its own. |
67
+ | Can I test without a real model? | Yes. The agent loop runs on a scripted model; 1,086 of the tests do exactly that. |
68
+ | Does it fit my auth and tenants? | The call carries the `userContext` your host authenticated; every store is keyed by its `tenantId`. |
69
+
70
+ Every part of the gateway is a class with one job, held behind an interface. The second constructor argument takes your
71
+ own instance in place of any of them: a knowledge-base retriever on your search engine, a trace that writes to your
72
+ observability stack, a scripted model in a test. The rest of the gateway does not notice the swap.
73
+
74
+ ```ts
75
+ const gateway = new AIGateway(config, {kbRetriever: new MyKbRetriever()});
76
+ ```
77
+
78
+ The seams: [Live adjustment and dependency injection](https://github.com/aneldev/dyna-ai-gateway/blob/main/docs/60-architecture/50-live-adjustment-and-dependency-injection.md).
79
+
80
+ ## Install
21
81
 
22
82
  ```bash
23
- pnpm install
24
- pnpm test-no-coverage # real-provider tests run only when their keys are in .env (see .env.sample)
25
- pnpm lint
26
- pnpm typecheck # tsc over sources and tests (tsconfig.tests.json)
27
- pnpm build # lint + typecheck + dist/index.js (CJS) + dist/index.mjs (ESM) + .d.ts; dist/vercel-ui.js / .mjs for the subpath
83
+ pnpm add dyna-ai-gateway zod
28
84
  ```
29
85
 
30
- Built from [dyna-ts-module-boilerplate](https://github.com/aneldev/dyna-ts-module-boilerplate); its guide is
31
- `README.boilerplate.md`.
86
+ Node.js 24 or newer. `zod` 4 is a peer dependency: the tools and the structured answers are typed with it.
87
+
88
+ ## Ask: an answer on the application's data
89
+
90
+ The application hands the gateway what the user asked and what the screen knows; the answer comes back as text or as a
91
+ typed object.
92
+
93
+ ```ts
94
+ import {z} from "zod";
95
+ import {AIGateway, EGatewayResponseKind} from "dyna-ai-gateway";
96
+
97
+ const gateway = new AIGateway(config); // the config: next section
98
+
99
+ const {response} = gateway.ask({
100
+ userContext: {tenantId: "acme", userId: "u-17", permissions: [], locale: "en-GB", timezone: "Europe/Oslo", now: new Date()},
101
+ profileId: "accounting",
102
+ prompt: "Which lines of this invoice are missing a VAT code?",
103
+ context: JSON.stringify(invoice), // what the user is looking at
104
+ outputSchema: z.object({lines: z.array(z.object({lineNo: z.number(), reason: z.string()}))}),
105
+ kbFilters: {libraryIds: ["vat-rules"], bookIds: []}, // the knowledge base this call may search
106
+ });
107
+
108
+ const result = await response;
109
+ if (result.kind === EGatewayResponseKind.ANSWER) {
110
+ result.structured; // {lines: [...]}, validated against the schema
111
+ result.citations; // the ids of the knowledge-base chunks the model was given
112
+ result.threadId; // pass it on the next call to continue the conversation
113
+ }
114
+ ```
115
+
116
+ A call that should stream reads `events` before awaiting `response`: text deltas, tool calls and results, the pause of a
117
+ human gate. The application owns the transport.
118
+
119
+ ### The configuration
120
+
121
+ One object: the stores, the providers and the switches. The in-memory stores ship with the package; a database adapter
122
+ implements the same interface.
123
+
124
+ ```ts
125
+ import {
126
+ IAIGatewayConfig,
127
+ IAIProfile,
128
+ EModelProvider,
129
+ InMemoryAIProfileRepository,
130
+ InMemoryBlobAdapter,
131
+ InMemoryModelRegistryRepository,
132
+ InMemoryPromptTemplateRepository,
133
+ InMemoryRunRepository,
134
+ InMemoryThreadRepository,
135
+ InMemoryTraceRepository,
136
+ InMemoryUsageAdapter,
137
+ InMemoryUserMemoryRepository,
138
+ InMemoryVectorIndexAdapter,
139
+ } from "dyna-ai-gateway";
140
+
141
+ const config: IAIGatewayConfig = {
142
+ llmClient: {providers: {[EModelProvider.OPENAI]: {apiKey: process.env.OPENAI_API_KEY ?? ""}}},
143
+ modelRegistry: {repository: new InMemoryModelRegistryRepository({models: [gpt5Mini, textEmbedding3Small]})},
144
+ aiProfileStore: {repository: new InMemoryAIProfileRepository({profiles: [accountingProfile]})},
145
+ promptTemplateStore: {repository: new InMemoryPromptTemplateRepository()},
146
+ threadStore: {repository: new InMemoryThreadRepository()},
147
+ fileStore: {adapter: new InMemoryBlobAdapter(), path: "ai-files"},
148
+ workingMemory: {enabled: true},
149
+ userMemoryStore: {repository: new InMemoryUserMemoryRepository(), enabled: true},
150
+ embedder: {embeddingModelId: "text-embedding-3-small", dimensions: 1536},
151
+ vectorStore: {adapter: new InMemoryVectorIndexAdapter({embeddingModelId: "text-embedding-3-small", dimensions: 1536})},
152
+ queryRewriter: {enabled: true},
153
+ runStore: {repository: new InMemoryRunRepository()},
154
+ humanGate: {userQuestions: {enabled: true}},
155
+ usageGuard: {adapter: new InMemoryUsageAdapter()},
156
+ contextAssembler: {promptCaching: true, sharePromptCacheAcrossTenants: false},
157
+ trace: {repository: new InMemoryTraceRepository()},
158
+ defaultCheapModelId: "gpt-5-mini",
159
+ };
160
+ ```
161
+
162
+ A **model** (`IModelInfo`) is registered with its provider, its capabilities and its price. A **profile** (`IAIProfile`) is
163
+ what a tenant's use case runs on: the model, the tools it may call, the approvals, a prompt addendum and the limits.
164
+ Each call names its profile, so one thread can switch model between two calls. Both are data in a store and can be changed
165
+ while the server runs.
166
+
167
+ ```ts
168
+ const accountingProfile: IAIProfile = {
169
+ id: "accounting",
170
+ tenantId: "acme",
171
+ name: "Accounting assistant",
172
+ modelId: "gpt-5-mini",
173
+ promptAddendum: "You assist the accounting team of a Norwegian company. Amounts are in NOK.",
174
+ allowedTools: ["get_invoice", "post_credit_note"],
175
+ approvals: [],
176
+ limits: {maxStepsPerRun: 6, maxOutputTokens: 2_000},
177
+ };
178
+ ```
179
+
180
+ ## Agent: tools, a policy per tool, a human in the loop
181
+
182
+ A tool is a function with a zod input schema and a policy. The policy decides, on the parsed input of each call, whether
183
+ the tool runs, is denied, or waits for a human. The application does not write a loop: `run()` is the loop.
184
+
185
+ ```ts
186
+ import {z} from "zod";
187
+ import {defineTool, EToolCallVerdict, EGatewayResponseKind, EResumeAction} from "dyna-ai-gateway";
188
+
189
+ const getInvoice = defineTool({
190
+ name: "get_invoice",
191
+ description: "Returns an invoice by its number.",
192
+ inputSchema: z.object({invoiceNo: z.string()}),
193
+ tags: ["billing"],
194
+ execute: async ({input, userContext}) => invoices.load(userContext.tenantId, input.invoiceNo),
195
+ });
196
+
197
+ const postCreditNote = defineTool({
198
+ name: "post_credit_note",
199
+ description: "Posts a credit note against an invoice.",
200
+ inputSchema: z.object({invoiceNo: z.string(), amount: z.number(), reason: z.string()}),
201
+ tags: ["billing"],
202
+ // Small amounts post at once; above the threshold the run pauses until a human decides
203
+ policy: ({input}) => ({verdict: input.amount > 5_000 ? EToolCallVerdict.NEEDS_APPROVAL : EToolCallVerdict.ALLOW}),
204
+ execute: async ({input, userContext}) => ledger.postCreditNote(userContext.tenantId, input),
205
+ });
206
+
207
+ const gateway = new AIGateway({...config, toolRegistry: {tools: [getInvoice, postCreditNote]}});
208
+
209
+ const paused = await gateway.run({
210
+ userContext,
211
+ profileId: "accounting",
212
+ prompt: "Credit invoice 2024-1187 in full, the goods came back damaged.",
213
+ kbFilters: {libraryIds: [], bookIds: []},
214
+ }).response;
215
+
216
+ // paused.kind === EGatewayResponseKind.PAUSED: the model read the invoice, then asked to post 12 400 NOK.
217
+ // The application shows the card to an approver; nothing has been posted. Later, from another request:
218
+ if (paused.kind === EGatewayResponseKind.PAUSED) {
219
+ const done = await gateway.resume({
220
+ userContext: approverContext,
221
+ runId: paused.envelopes[0].runId,
222
+ pauseId: paused.pauseId,
223
+ decisions: [{action: EResumeAction.APPROVE}], // or EDIT with a new input, or REJECT with a reason the model reads
224
+ }).response;
225
+ }
226
+ ```
227
+
228
+ What else a run gives: tools for one call only, a tool that ends the run, a tool offered only after another one ran, a
229
+ question the model asks the user as a form, a review of the final answer by a human before the user sees it, timeouts and
230
+ output caps per tool, a cap on steps and daily tokens per tenant. A paused run survives a restart of the server.
231
+
232
+ ## A chat screen in one module
233
+
234
+ `dyna-ai-gateway/vercel-ui` speaks the UI message protocol of the Vercel AI SDK's `useChat`: the approval cards, the
235
+ history of a thread and the resume of a paused run included.
236
+
237
+ ```ts
238
+ import {AIGatewayVercelUi} from "dyna-ai-gateway/vercel-ui";
239
+
240
+ const supportChat = new AIGatewayVercelUi({
241
+ mode: "run",
242
+ userContext: async ({request}) => sessionOf(request), // the application's authentication
243
+ call: {profileId: "support", kbFilters: {libraryIds: ["help-center"], bookIds: []}},
244
+ }, {gateway});
245
+
246
+ export const POST = (request: Request): Promise<Response> => supportChat.chat({request});
247
+ ```
248
+
249
+ The guide: [Vercel chat client](https://github.com/aneldev/dyna-ai-gateway/blob/main/docs/10-getting-started/50-vercel-chat-client.md).
250
+
251
+ ## What is inside
252
+
253
+ | | |
254
+ |-------------------|-------------------------------------------------------------------------------------------------------------------------------|
255
+ | Calls | `ask()`, `run()`, `resume()`; text or structured answer; streaming; `abortSignal` and `timeoutMs` on every call |
256
+ | Knowledge base | Markdown documents chunked, embedded and searched per tenant, library and book; query rewrite; citations |
257
+ | Memory | The thread's summary and facts, the user's facts across threads; both optional, both editable |
258
+ | Files | Images and PDFs sent to a model that reads them |
259
+ | Skills | Instructions the model loads only when a task needs them, on the profile or on the call; inlined in an `ask()` |
260
+ | Human in the loop | Approval of a tool call, the model's questions as a form, review of the answer, expiry of unanswered pauses |
261
+ | Guardrails | Built-in input and output rules, the application's own rules with evidence, a judge on the cheap model |
262
+ | Usage and trace | Your adapter asked before every model call and told what it used after it; one trace entry per model call with usage and cost |
263
+ | Operations | Graceful shutdown, recovery of the runs a crashed server left behind, deletion or anonymisation of a user's data |
264
+ | Providers | OpenAI, Anthropic, Google, Mistral, Amazon Bedrock |
265
+
266
+ ## Design
267
+
268
+ - **The application keeps control.** The gateway knows no HTTP route, no UI and no session: it receives a `userContext`
269
+ the host authenticated and returns events and a response.
270
+ - **Data is passed, services are injected.** A call loads its profile, memory and history once and hands them on as plain
271
+ data; only the stores and the provider client are injected. One class per job.
272
+ - **Nothing is hidden in the model.** Every decision a tool policy, a guardrail or the usage adapter makes is code the
273
+ application wrote, logged in the trace.
274
+ - **Persistence is a contract.** Every store is an interface; in-memory for tests and small deployments, MongoDB,
275
+ PostgreSQL and S3 adapters as separate packages.
276
+
277
+ The reasons behind each behaviour: [Decisions](https://github.com/aneldev/dyna-ai-gateway/blob/main/docs/60-architecture/90-decisions.md). The composition of the classes:
278
+ [Architecture and trade-offs](https://github.com/aneldev/dyna-ai-gateway/blob/main/docs/60-architecture/10-architecture-and-trade-offs.md).
279
+
280
+ ## Documentation
281
+
282
+ | | |
283
+ |---------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|
284
+ | [Docs site](https://aneldev.github.io/dyna-ai-gateway/) | these docs and the API reference from the JSDoc, with navigation |
285
+ | [Getting started](https://github.com/aneldev/dyna-ai-gateway/blob/main/docs/10-getting-started/00-README.md) | first steps, the examples, the [FAQ](https://github.com/aneldev/dyna-ai-gateway/blob/main/docs/10-getting-started/90-faq.md), the [Vercel chat client](https://github.com/aneldev/dyna-ai-gateway/blob/main/docs/10-getting-started/50-vercel-chat-client.md) |
286
+ | [Roadmap](https://github.com/aneldev/dyna-ai-gateway/blob/main/docs/70-roadmap.md) | what works today, what is planned, what is not |
287
+ | [Terminology](https://github.com/aneldev/dyna-ai-gateway/blob/main/docs/90-terminology.md) | the terms of the docs and where they come from |
288
+ | [Architecture](https://github.com/aneldev/dyna-ai-gateway/blob/main/docs/60-architecture/90-decisions.md) | the composition, the trade-offs, one chapter of decisions per class |
289
+ | [Practices](https://github.com/aneldev/dyna-ai-gateway/blob/main/docs/50-ai-best-practices/index.md) | how to apply the library well |
290
+ | [Workspace](https://github.com/aneldev/dyna-ai-gateway/blob/main/docs/85-workspace.md) | building and testing the packages of this repository |
291
+
292
+ ## Packages
293
+
294
+ | Package | What | Status |
295
+ |---------------------------------------------------------------------------|--------------------------------------------------------|---------|
296
+ | [`dyna-ai-gateway`](https://github.com/aneldev/dyna-ai-gateway/blob/main/packages/dyna-ai-gateway) | the gateway, in-memory stores included | beta |
297
+ | [`dyna-ai-gateway-mongodb`](https://github.com/aneldev/dyna-ai-gateway/blob/main/packages/dyna-ai-gateway-mongodb/README.md) | MongoDB repositories and the Atlas Vector Search index | planned |
298
+ | [`dyna-ai-gateway-postgres`](https://github.com/aneldev/dyna-ai-gateway/blob/main/packages/dyna-ai-gateway-postgres/README.md) | PostgreSQL repositories | planned |
299
+ | [`dyna-ai-gateway-s3`](https://github.com/aneldev/dyna-ai-gateway/blob/main/packages/dyna-ai-gateway-s3/README.md) | S3 blob store for the files | planned |
300
+
301
+ ## About
302
+
303
+ Built by [Anel](http://anel.co), a small Norwegian software company working in accounting, billing, payments and AI
304
+ integration in complex environments. The gateway is the piece those systems share.
305
+
306
+ MIT license.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "dyna-ai-gateway",
3
- "version": "0.0.1-beta.7",
3
+ "version": "0.0.13-beta.8",
4
4
  "description": "AI Gateway (beta): provider-agnostic LLM orchestration (ask/run, tools, human gate, KB/RAG, memory, usage control, traces) behind repository/adapter contracts",
5
5
  "homepage": "https://aneldev.github.io/dyna-ai-gateway/",
6
6
  "repository": {