llm-agent-runtime 0.1.5 → 0.1.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/USAGE.md +71 -14
- package/dist/create-runtime.d.ts.map +1 -1
- package/dist/create-runtime.js +10 -0
- package/dist/env-llm.d.ts +5 -5
- package/dist/env-llm.d.ts.map +1 -1
- package/dist/env-llm.js +25 -5
- package/dist/llm.d.ts.map +1 -1
- package/dist/llm.js +17 -0
- package/dist/services/flow-router.d.ts +1 -0
- package/dist/services/flow-router.d.ts.map +1 -1
- package/dist/services/flow-router.js +12 -0
- package/dist/types.d.ts +3 -1
- package/dist/types.d.ts.map +1 -1
- package/package.json +1 -1
package/USAGE.md
CHANGED
|
@@ -90,7 +90,7 @@ console.log(result.answer, result.intent, result.flow, result.toolOutputs);
|
|
|
90
90
|
|
|
91
91
|
| You configure | Library handles |
|
|
92
92
|
| --- | --- |
|
|
93
|
-
| `intents` (examples + `sticky`) | Hybrid LLM + embedding router |
|
|
93
|
+
| `intents` (examples + `sticky` + `regex`) | Hybrid regex + LLM + embedding router |
|
|
94
94
|
| `intentRegistry` (prompt + tools) | LangGraph agent + Redis/memory checkpointer |
|
|
95
95
|
| `toolOutputMap` | Parse tool calls into structured outputs |
|
|
96
96
|
| `completeOn` | Sticky-flow completion + Redis flow state |
|
|
@@ -124,6 +124,7 @@ intentRegistry: Record<string, { prompt: (ctx) => string; tools: (phoneKey) => T
|
|
|
124
124
|
| `defaultIntent` | first intent | Fallback intent |
|
|
125
125
|
| `buildSystemPrompt` | registry `prompt` | One builder for all intents |
|
|
126
126
|
| `resolvePhoneKey` | `phoneKey ?? threadId` | Key passed to `tools(phoneKey)` |
|
|
127
|
+
| `tracing` | auto-detect | LangSmith tracing config |
|
|
127
128
|
|
|
128
129
|
---
|
|
129
130
|
|
|
@@ -139,14 +140,28 @@ intents: [
|
|
|
139
140
|
{
|
|
140
141
|
name: "create_campaign",
|
|
141
142
|
examples: ["create a campaign"],
|
|
142
|
-
sticky: true,
|
|
143
|
+
sticky: true, // keep this flow across turns until completeOn fires
|
|
144
|
+
regex: /create.*(campaign|fundrais)/i, // fast-path: skip LLM if matched
|
|
143
145
|
},
|
|
144
146
|
]
|
|
145
147
|
```
|
|
146
148
|
|
|
147
|
-
- **examples** — embedding routing
|
|
148
|
-
- **sticky** — multi-step wizards
|
|
149
|
-
- **description** — LLM intent router
|
|
149
|
+
- **examples** — embedding routing
|
|
150
|
+
- **sticky** — multi-step wizards
|
|
151
|
+
- **description** — LLM intent router
|
|
152
|
+
- **regex** — fast-path: if message matches, skip LLM/embeddings entirely (zero tokens)
|
|
153
|
+
|
|
154
|
+
### Intent detection flow
|
|
155
|
+
|
|
156
|
+
```
|
|
157
|
+
1. explicitIntent passed? → Use it directly (score 1.0)
|
|
158
|
+
2. Wizard input + sticky? → Reuse cached intent (skip LLM)
|
|
159
|
+
3. Menu digit match? → Use mapped intent (score 1.0)
|
|
160
|
+
4. Regex matches? → Use matched intent (score 1.0, 0 tokens)
|
|
161
|
+
5. Embeddings + LLM (hybrid) → Existing flow
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
The regex fast-path is a token-saving optimization. When a user message matches an intent's regex pattern, the library returns that intent immediately without calling the LLM or embeddings. Define regexes for high-confidence, pattern-based intents (e.g. "generate an image of...") and leave regex undefined for ambiguous intents that need LLM classification.
|
|
150
165
|
|
|
151
166
|
---
|
|
152
167
|
|
|
@@ -271,34 +286,74 @@ await runtime.disconnect()
|
|
|
271
286
|
|
|
272
287
|
---
|
|
273
288
|
|
|
274
|
-
##
|
|
289
|
+
## LLM Providers
|
|
290
|
+
|
|
291
|
+
### Supported providers
|
|
292
|
+
|
|
293
|
+
| Provider | `LLM_PROVIDER` | API Key Env Var | Default Model |
|
|
294
|
+
| --- | --- | --- | --- |
|
|
295
|
+
| Ollama | `ollama` (default) | — | `qwen2.5:7b` |
|
|
296
|
+
| OpenAI | `openai` | `OPENAI_API_KEY` | `gpt-4o-mini` |
|
|
297
|
+
| OpenRouter | `openrouter` | `OPENROUTER_API_KEY` | `meta-llama/llama-4-scout` |
|
|
298
|
+
|
|
299
|
+
### Env vars
|
|
275
300
|
|
|
276
301
|
| Variable | Notes |
|
|
277
302
|
| --- | --- |
|
|
278
|
-
| `LLM_PROVIDER` | `ollama` (default) or `
|
|
303
|
+
| `LLM_PROVIDER` | `ollama` (default), `openai`, or `openrouter` |
|
|
279
304
|
| `EMBEDDINGS_PROVIDER` | defaults to `LLM_PROVIDER` |
|
|
280
305
|
| `OLLAMA_MODEL` | default `qwen2.5:7b` |
|
|
281
306
|
| `OLLAMA_BASE_URL` | e.g. `http://localhost:11434` |
|
|
282
|
-
| `INTENT_ROUTER_MODEL` | optional smaller router model |
|
|
283
307
|
| `OLLAMA_EMBEDDINGS_MODEL` | default `mxbai-embed-large:latest` |
|
|
308
|
+
| `INTENT_ROUTER_MODEL` | optional smaller router model (ollama) |
|
|
284
309
|
| `OPENAI_API_KEY` | required for openai |
|
|
285
310
|
| `OPENAI_MODEL` | default `gpt-4o-mini` |
|
|
286
311
|
| `OPENAI_EMBEDDINGS_MODEL` | default `text-embedding-3-small` |
|
|
312
|
+
| `OPENROUTER_API_KEY` | required for openrouter |
|
|
313
|
+
| `OPENROUTER_MODEL` | default `meta-llama/llama-4-scout` |
|
|
314
|
+
| `OPENROUTER_BASE_URL` | default `https://openrouter.ai/api/v1` |
|
|
315
|
+
| `OPENROUTER_INTENT_ROUTER_MODEL` | optional separate router model |
|
|
287
316
|
| `REDIS_URL` | your app usually passes this as `redisUrl` |
|
|
288
317
|
| `ANSWER_CACHE` | app convention: `"1"` to enable cache |
|
|
289
318
|
|
|
290
|
-
|
|
319
|
+
### OpenRouter example
|
|
320
|
+
|
|
321
|
+
```bash
|
|
322
|
+
LLM_PROVIDER=openrouter OPENROUTER_API_KEY=sk-or-... pnpm dev
|
|
323
|
+
```
|
|
324
|
+
|
|
325
|
+
Or pass explicitly:
|
|
291
326
|
|
|
292
327
|
```ts
|
|
293
|
-
|
|
328
|
+
createAgentRuntime({
|
|
329
|
+
llm: {
|
|
330
|
+
provider: "openrouter",
|
|
331
|
+
model: "meta-llama/llama-4-scout",
|
|
332
|
+
apiKey: "sk-or-...",
|
|
333
|
+
},
|
|
334
|
+
// ...
|
|
335
|
+
});
|
|
336
|
+
```
|
|
337
|
+
|
|
338
|
+
### LangSmith tracing
|
|
339
|
+
|
|
340
|
+
Tracing auto-enables when `LANGSMITH_API_KEY` is set in the environment. No code changes needed.
|
|
341
|
+
|
|
342
|
+
```bash
|
|
343
|
+
LANGSMITH_API_KEY=ls_... LLM_PROVIDER=openrouter OPENROUTER_API_KEY=sk-or-... pnpm dev
|
|
344
|
+
```
|
|
345
|
+
|
|
346
|
+
Or enable explicitly:
|
|
294
347
|
|
|
348
|
+
```ts
|
|
295
349
|
createAgentRuntime({
|
|
296
|
-
|
|
297
|
-
//
|
|
298
|
-
|
|
299
|
-
})
|
|
350
|
+
tracing: { enabled: true, projectName: "my-project" },
|
|
351
|
+
// ...
|
|
352
|
+
});
|
|
300
353
|
```
|
|
301
354
|
|
|
355
|
+
Tracing sets `LANGCHAIN_TRACING_V2`, `LANGCHAIN_API_KEY`, and `LANGCHAIN_PROJECT` env vars before model creation. The `langsmith` package must be installed (`pnpm add langsmith`).
|
|
356
|
+
|
|
302
357
|
---
|
|
303
358
|
|
|
304
359
|
## Minimal app shape (recommended)
|
|
@@ -332,11 +387,13 @@ const runtime = await createAgentRuntime({
|
|
|
332
387
|
name: "create_campaign",
|
|
333
388
|
examples: ["create a campaign"],
|
|
334
389
|
sticky: true,
|
|
390
|
+
regex: /create.*(campaign|fundrais)/i,
|
|
335
391
|
},
|
|
336
392
|
{
|
|
337
393
|
name: "buy_airtime_and_data",
|
|
338
394
|
examples: ["buy airtime", "vend airtime"],
|
|
339
395
|
sticky: true,
|
|
396
|
+
regex: /(buy|vend|purchase).*(airtime|data|credit)/i,
|
|
340
397
|
},
|
|
341
398
|
],
|
|
342
399
|
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"create-runtime.d.ts","sourceRoot":"","sources":["../src/create-runtime.ts"],"names":[],"mappings":"AAqBA,OAAO,KAAK,EACV,kBAAkB,EAClB,WAAW,EACX,YAAY,EAEb,MAAM,YAAY,CAAC;AAepB,MAAM,MAAM,YAAY,GAAG;IACzB,MAAM,EAAE,CAAC,KAAK,EAAE,WAAW,KAAK,OAAO,CAAC,YAAY,CAAC,CAAC;IACtD,OAAO,EAAE,MAAM,OAAO,CAAC,IAAI,CAAC,CAAC;IAC7B,UAAU,EAAE,MAAM,OAAO,CAAC,IAAI,CAAC,CAAC;IAChC,SAAS,EAAE,CAAC,QAAQ,EAAE,MAAM,KAAK,OAAO,CAAC,IAAI,CAAC,CAAC;IAC/C,gEAAgE;IAChE,WAAW,EAAE,CAAC,QAAQ,EAAE,MAAM,KAAK,OAAO,CAAC,IAAI,CAAC,CAAC;IACjD,eAAe,EAAE,CAAC,OAAO,EAAE,MAAM,KAAK,OAAO,CAAC,MAAM,GAAG,SAAS,CAAC,CAAC;IAClE,eAAe,EAAE,CAAC,OAAO,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,KAAK,OAAO,CAAC,IAAI,CAAC,CAAC;IACpE,4EAA4E;IAC5E,0BAA0B,EAAE,MAAM,IAAI,CAAC;CACxC,CAAC;AAEF;;;GAGG;AACH,wBAAsB,kBAAkB,CACtC,MAAM,EAAE,kBAAkB,GACzB,OAAO,CAAC,YAAY,CAAC,
|
|
1
|
+
{"version":3,"file":"create-runtime.d.ts","sourceRoot":"","sources":["../src/create-runtime.ts"],"names":[],"mappings":"AAqBA,OAAO,KAAK,EACV,kBAAkB,EAClB,WAAW,EACX,YAAY,EAEb,MAAM,YAAY,CAAC;AAepB,MAAM,MAAM,YAAY,GAAG;IACzB,MAAM,EAAE,CAAC,KAAK,EAAE,WAAW,KAAK,OAAO,CAAC,YAAY,CAAC,CAAC;IACtD,OAAO,EAAE,MAAM,OAAO,CAAC,IAAI,CAAC,CAAC;IAC7B,UAAU,EAAE,MAAM,OAAO,CAAC,IAAI,CAAC,CAAC;IAChC,SAAS,EAAE,CAAC,QAAQ,EAAE,MAAM,KAAK,OAAO,CAAC,IAAI,CAAC,CAAC;IAC/C,gEAAgE;IAChE,WAAW,EAAE,CAAC,QAAQ,EAAE,MAAM,KAAK,OAAO,CAAC,IAAI,CAAC,CAAC;IACjD,eAAe,EAAE,CAAC,OAAO,EAAE,MAAM,KAAK,OAAO,CAAC,MAAM,GAAG,SAAS,CAAC,CAAC;IAClE,eAAe,EAAE,CAAC,OAAO,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,KAAK,OAAO,CAAC,IAAI,CAAC,CAAC;IACpE,4EAA4E;IAC5E,0BAA0B,EAAE,MAAM,IAAI,CAAC;CACxC,CAAC;AAEF;;;GAGG;AACH,wBAAsB,kBAAkB,CACtC,MAAM,EAAE,kBAAkB,GACzB,OAAO,CAAC,YAAY,CAAC,CAoPvB"}
|
package/dist/create-runtime.js
CHANGED
|
@@ -28,6 +28,15 @@ const getTurnMessages = (allMessages, userMessage) => {
|
|
|
28
28
|
* Engine pipeline: resolveFlow → runAgent → getTurnMessages → extract → finalize
|
|
29
29
|
*/
|
|
30
30
|
export async function createAgentRuntime(config) {
|
|
31
|
+
// Wire LangSmith tracing: auto-detect from env or explicit config
|
|
32
|
+
const langsmithKey = process.env.LANGSMITH_API_KEY;
|
|
33
|
+
const tracingEnabled = config.tracing?.enabled ?? !!langsmithKey;
|
|
34
|
+
if (tracingEnabled) {
|
|
35
|
+
process.env.LANGCHAIN_TRACING_V2 = "true";
|
|
36
|
+
if (langsmithKey)
|
|
37
|
+
process.env.LANGCHAIN_API_KEY = langsmithKey;
|
|
38
|
+
process.env.LANGCHAIN_PROJECT = config.tracing?.projectName ?? "llm-agent-runtime";
|
|
39
|
+
}
|
|
31
40
|
const defaultIntent = config.defaultIntent ?? config.intents[0]?.name ?? "greetings";
|
|
32
41
|
const intentNames = new Set(config.intents.map((i) => i.name));
|
|
33
42
|
const aliases = config.intentAliases ?? {};
|
|
@@ -103,6 +112,7 @@ export async function createAgentRuntime(config) {
|
|
|
103
112
|
stickyFlows,
|
|
104
113
|
menuMap,
|
|
105
114
|
normalizeIntent,
|
|
115
|
+
intents: config.intents,
|
|
106
116
|
getEmbeddedIntents: intentEmbeddings.getEmbeddedIntents,
|
|
107
117
|
scoreIntentsForMessage: intentEmbeddings.scoreIntentsForMessage,
|
|
108
118
|
classifyIntentWithLlm: intentRouter.classifyIntentWithLlm,
|
package/dist/env-llm.d.ts
CHANGED
|
@@ -1,13 +1,13 @@
|
|
|
1
1
|
import type { LlmConfig } from "./types.js";
|
|
2
2
|
/**
|
|
3
|
-
* Build LlmConfig from process.env
|
|
3
|
+
* Build LlmConfig from process.env.
|
|
4
4
|
*
|
|
5
5
|
* Env knobs:
|
|
6
|
-
* - LLM_PROVIDER / EMBEDDINGS_PROVIDER — ollama | openai
|
|
7
|
-
* - OLLAMA_MODEL / OPENAI_MODEL
|
|
8
|
-
* - INTENT_ROUTER_MODEL / OPENAI_INTENT_ROUTER_MODEL
|
|
6
|
+
* - LLM_PROVIDER / EMBEDDINGS_PROVIDER — ollama | openai | openrouter
|
|
7
|
+
* - OLLAMA_MODEL / OPENAI_MODEL / OPENROUTER_MODEL
|
|
8
|
+
* - INTENT_ROUTER_MODEL / OPENAI_INTENT_ROUTER_MODEL / OPENROUTER_INTENT_ROUTER_MODEL
|
|
9
9
|
* - OLLAMA_EMBEDDINGS_MODEL / OPENAI_EMBEDDINGS_MODEL
|
|
10
|
-
* - OLLAMA_BASE_URL / OPENAI_API_KEY
|
|
10
|
+
* - OLLAMA_BASE_URL / OPENAI_API_KEY / OPENROUTER_API_KEY / OPENROUTER_BASE_URL
|
|
11
11
|
*/
|
|
12
12
|
export declare function llmConfigFromEnv(env?: NodeJS.ProcessEnv): LlmConfig;
|
|
13
13
|
//# sourceMappingURL=env-llm.d.ts.map
|
package/dist/env-llm.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"env-llm.d.ts","sourceRoot":"","sources":["../src/env-llm.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,SAAS,EAAe,MAAM,YAAY,CAAC;
|
|
1
|
+
{"version":3,"file":"env-llm.d.ts","sourceRoot":"","sources":["../src/env-llm.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,SAAS,EAAe,MAAM,YAAY,CAAC;AAqBzD;;;;;;;;;GASG;AACH,wBAAgB,gBAAgB,CAC9B,GAAG,GAAE,MAAM,CAAC,UAAwB,GACnC,SAAS,CAiDX"}
|
package/dist/env-llm.js
CHANGED
|
@@ -5,23 +5,43 @@ const asProvider = (value, fallback) => {
|
|
|
5
5
|
normalized === "chatgpt") {
|
|
6
6
|
return "openai";
|
|
7
7
|
}
|
|
8
|
+
if (normalized === "openrouter" ||
|
|
9
|
+
normalized === "or") {
|
|
10
|
+
return "openrouter";
|
|
11
|
+
}
|
|
8
12
|
if (normalized === "ollama")
|
|
9
13
|
return "ollama";
|
|
10
14
|
return fallback;
|
|
11
15
|
};
|
|
12
16
|
/**
|
|
13
|
-
* Build LlmConfig from process.env
|
|
17
|
+
* Build LlmConfig from process.env.
|
|
14
18
|
*
|
|
15
19
|
* Env knobs:
|
|
16
|
-
* - LLM_PROVIDER / EMBEDDINGS_PROVIDER — ollama | openai
|
|
17
|
-
* - OLLAMA_MODEL / OPENAI_MODEL
|
|
18
|
-
* - INTENT_ROUTER_MODEL / OPENAI_INTENT_ROUTER_MODEL
|
|
20
|
+
* - LLM_PROVIDER / EMBEDDINGS_PROVIDER — ollama | openai | openrouter
|
|
21
|
+
* - OLLAMA_MODEL / OPENAI_MODEL / OPENROUTER_MODEL
|
|
22
|
+
* - INTENT_ROUTER_MODEL / OPENAI_INTENT_ROUTER_MODEL / OPENROUTER_INTENT_ROUTER_MODEL
|
|
19
23
|
* - OLLAMA_EMBEDDINGS_MODEL / OPENAI_EMBEDDINGS_MODEL
|
|
20
|
-
* - OLLAMA_BASE_URL / OPENAI_API_KEY
|
|
24
|
+
* - OLLAMA_BASE_URL / OPENAI_API_KEY / OPENROUTER_API_KEY / OPENROUTER_BASE_URL
|
|
21
25
|
*/
|
|
22
26
|
export function llmConfigFromEnv(env = process.env) {
|
|
23
27
|
const provider = asProvider(env.LLM_PROVIDER, "ollama");
|
|
24
28
|
const embeddingsProvider = asProvider(env.EMBEDDINGS_PROVIDER ?? env.LLM_PROVIDER, provider);
|
|
29
|
+
const openrouterBaseUrl = env.OPENROUTER_BASE_URL ?? "https://openrouter.ai/api/v1";
|
|
30
|
+
if (provider === "openrouter") {
|
|
31
|
+
return {
|
|
32
|
+
provider,
|
|
33
|
+
embeddingsProvider: embeddingsProvider === "openrouter" ? "ollama" : embeddingsProvider,
|
|
34
|
+
model: env.OPENROUTER_MODEL ?? "meta-llama/llama-4-scout",
|
|
35
|
+
intentRouterModel: env.OPENROUTER_INTENT_ROUTER_MODEL ??
|
|
36
|
+
env.OPENROUTER_MODEL ??
|
|
37
|
+
"meta-llama/llama-4-scout",
|
|
38
|
+
embeddingsModel: embeddingsProvider === "openai"
|
|
39
|
+
? (env.OPENAI_EMBEDDINGS_MODEL ?? "text-embedding-3-small")
|
|
40
|
+
: (env.OLLAMA_EMBEDDINGS_MODEL ?? "mxbai-embed-large:latest"),
|
|
41
|
+
baseUrl: openrouterBaseUrl,
|
|
42
|
+
apiKey: env.OPENROUTER_API_KEY,
|
|
43
|
+
};
|
|
44
|
+
}
|
|
25
45
|
return {
|
|
26
46
|
provider,
|
|
27
47
|
embeddingsProvider,
|
package/dist/llm.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"llm.d.ts","sourceRoot":"","sources":["../src/llm.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,6CAA6C,CAAC;AACjF,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,4BAA4B,CAAC;AAG7D,OAAO,KAAK,EAAE,SAAS,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;
|
|
1
|
+
{"version":3,"file":"llm.d.ts","sourceRoot":"","sources":["../src/llm.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,6CAA6C,CAAC;AACjF,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,4BAA4B,CAAC;AAG7D,OAAO,KAAK,EAAE,SAAS,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AAoBzD,MAAM,MAAM,aAAa,GAAG;IAC1B,IAAI,EAAE,aAAa,CAAC;IACpB,YAAY,EAAE,aAAa,CAAC;IAC5B,UAAU,EAAE,UAAU,CAAC;IACvB,MAAM,EAAE;QACN,QAAQ,EAAE,WAAW,CAAC;QACtB,kBAAkB,EAAE,WAAW,CAAC;QAChC,KAAK,EAAE,MAAM,CAAC;QACd,iBAAiB,EAAE,MAAM,CAAC;QAC1B,eAAe,EAAE,MAAM,CAAC;KACzB,CAAC;CACH,CAAC;AAEF,wBAAgB,YAAY,CAAC,GAAG,EAAE,SAAS,GAAG,aAAa,CA8E1D"}
|
package/dist/llm.js
CHANGED
|
@@ -7,6 +7,10 @@ const normalizeProvider = (value) => {
|
|
|
7
7
|
normalized === "chatgpt") {
|
|
8
8
|
return "openai";
|
|
9
9
|
}
|
|
10
|
+
if (normalized === "openrouter" ||
|
|
11
|
+
normalized === "or") {
|
|
12
|
+
return "openrouter";
|
|
13
|
+
}
|
|
10
14
|
return "ollama";
|
|
11
15
|
};
|
|
12
16
|
export function createModels(llm) {
|
|
@@ -24,6 +28,19 @@ export function createModels(llm) {
|
|
|
24
28
|
apiKey: llm.apiKey,
|
|
25
29
|
});
|
|
26
30
|
}
|
|
31
|
+
if (provider === "openrouter") {
|
|
32
|
+
if (!llm.apiKey) {
|
|
33
|
+
throw new Error("llm.apiKey is required when provider is openrouter.");
|
|
34
|
+
}
|
|
35
|
+
return new ChatOpenAI({
|
|
36
|
+
model: modelName,
|
|
37
|
+
temperature,
|
|
38
|
+
apiKey: llm.apiKey,
|
|
39
|
+
configuration: {
|
|
40
|
+
baseURL: llm.baseUrl ?? "https://openrouter.ai/api/v1",
|
|
41
|
+
},
|
|
42
|
+
});
|
|
43
|
+
}
|
|
27
44
|
return new ChatOllama({
|
|
28
45
|
model: modelName,
|
|
29
46
|
temperature,
|
|
@@ -5,6 +5,7 @@ export declare function createFlowRouter(options: {
|
|
|
5
5
|
stickyFlows: Set<string>;
|
|
6
6
|
menuMap: Record<string, string>;
|
|
7
7
|
normalizeIntent: (intent: string) => string;
|
|
8
|
+
intents: IntentDefinition[];
|
|
8
9
|
getEmbeddedIntents: () => Promise<{
|
|
9
10
|
name: string;
|
|
10
11
|
embedding: number[];
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"flow-router.d.ts","sourceRoot":"","sources":["../../src/services/flow-router.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EACV,aAAa,EACb,oBAAoB,EACpB,iBAAiB,EACjB,kBAAkB,EAClB,cAAc,EACd,SAAS,EACT,gBAAgB,EAChB,eAAe,EAChB,MAAM,aAAa,CAAC;AACrB,OAAO,KAAK,EAAE,SAAS,EAAE,MAAM,iBAAiB,CAAC;AAOjD,eAAO,MAAM,aAAa,GAAI,SAAS,MAAM,KAAG,OAI/C,CAAC;AAEF,wBAAgB,gBAAgB,CAAC,OAAO,EAAE;IACxC,WAAW,EAAE,GAAG,CAAC,MAAM,CAAC,CAAC;IACzB,OAAO,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IAChC,eAAe,EAAE,CAAC,MAAM,EAAE,MAAM,KAAK,MAAM,CAAC;IAC5C,kBAAkB,EAAE,MAAM,OAAO,CAAC;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,SAAS,EAAE,MAAM,EAAE,CAAA;KAAE,EAAE,CAAC,CAAC;IAC3E,sBAAsB,EAAE,CACtB,OAAO,EAAE,MAAM,EACf,QAAQ,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,SAAS,EAAE,MAAM,EAAE,CAAA;KAAE,EAAE,KAC9C,OAAO,CAAC,oBAAoB,EAAE,CAAC,CAAC;IACrC,qBAAqB,EAAE,CAAC,KAAK,EAAE;QAC7B,OAAO,EAAE,MAAM,CAAC;QAChB,mBAAmB,EAAE,MAAM,CAAC;QAC5B,gBAAgB,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;KAClC,KAAK,OAAO,CAAC,eAAe,CAAC,CAAC;IAC/B,kBAAkB,EAAE,CAClB,YAAY,EAAE,oBAAoB,EAClC,SAAS,EAAE,eAAe,KACvB,oBAAoB,GAAG;QAAE,MAAM,EAAE,eAAe,CAAC,QAAQ,CAAC,CAAA;KAAE,CAAC;IAClE,SAAS,EAAE,SAAS,CAAC;IACrB,YAAY,CAAC,EAAE,CACb,KAAK,EAAE,iBAAiB,KACrB,kBAAkB,GAAG,OAAO,CAAC,kBAAkB,CAAC,CAAC;IACtD,UAAU,CAAC,EAAE;QACX,MAAM,CAAC,EAAE,MAAM,CAAC;QAChB,SAAS,CAAC,EAAE,MAAM,CAAC;QACnB,YAAY,CAAC,EAAE,MAAM,CAAC;KACvB,CAAC;CACH;sFAoBI;QACD,OAAO,EAAE,MAAM,CAAC;QAChB,cAAc,CAAC,EAAE,MAAM,CAAC;QACxB,eAAe,EAAE,aAAa,GAAG,IAAI,CAAC;QACtC,mBAAmB,EAAE,MAAM,CAAC;KAC7B,KAAG,OAAO,CAAC,cAAc,CAAC;+
|
|
1
|
+
{"version":3,"file":"flow-router.d.ts","sourceRoot":"","sources":["../../src/services/flow-router.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EACV,aAAa,EACb,oBAAoB,EACpB,iBAAiB,EACjB,kBAAkB,EAClB,cAAc,EACd,SAAS,EACT,gBAAgB,EAChB,eAAe,EAChB,MAAM,aAAa,CAAC;AACrB,OAAO,KAAK,EAAE,SAAS,EAAE,MAAM,iBAAiB,CAAC;AAOjD,eAAO,MAAM,aAAa,GAAI,SAAS,MAAM,KAAG,OAI/C,CAAC;AAEF,wBAAgB,gBAAgB,CAAC,OAAO,EAAE;IACxC,WAAW,EAAE,GAAG,CAAC,MAAM,CAAC,CAAC;IACzB,OAAO,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IAChC,eAAe,EAAE,CAAC,MAAM,EAAE,MAAM,KAAK,MAAM,CAAC;IAC5C,OAAO,EAAE,gBAAgB,EAAE,CAAC;IAC5B,kBAAkB,EAAE,MAAM,OAAO,CAAC;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,SAAS,EAAE,MAAM,EAAE,CAAA;KAAE,EAAE,CAAC,CAAC;IAC3E,sBAAsB,EAAE,CACtB,OAAO,EAAE,MAAM,EACf,QAAQ,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,SAAS,EAAE,MAAM,EAAE,CAAA;KAAE,EAAE,KAC9C,OAAO,CAAC,oBAAoB,EAAE,CAAC,CAAC;IACrC,qBAAqB,EAAE,CAAC,KAAK,EAAE;QAC7B,OAAO,EAAE,MAAM,CAAC;QAChB,mBAAmB,EAAE,MAAM,CAAC;QAC5B,gBAAgB,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;KAClC,KAAK,OAAO,CAAC,eAAe,CAAC,CAAC;IAC/B,kBAAkB,EAAE,CAClB,YAAY,EAAE,oBAAoB,EAClC,SAAS,EAAE,eAAe,KACvB,oBAAoB,GAAG;QAAE,MAAM,EAAE,eAAe,CAAC,QAAQ,CAAC,CAAA;KAAE,CAAC;IAClE,SAAS,EAAE,SAAS,CAAC;IACrB,YAAY,CAAC,EAAE,CACb,KAAK,EAAE,iBAAiB,KACrB,kBAAkB,GAAG,OAAO,CAAC,kBAAkB,CAAC,CAAC;IACtD,UAAU,CAAC,EAAE;QACX,MAAM,CAAC,EAAE,MAAM,CAAC;QAChB,SAAS,CAAC,EAAE,MAAM,CAAC;QACnB,YAAY,CAAC,EAAE,MAAM,CAAC;KACvB,CAAC;CACH;sFAoBI;QACD,OAAO,EAAE,MAAM,CAAC;QAChB,cAAc,CAAC,EAAE,MAAM,CAAC;QACxB,eAAe,EAAE,aAAa,GAAG,IAAI,CAAC;QACtC,mBAAmB,EAAE,MAAM,CAAC;KAC7B,KAAG,OAAO,CAAC,cAAc,CAAC;+BAyLlB,iBAAiB,KACvB,OAAO,CAAC,kBAAkB,CAAC;EAkB/B;AAED,wBAAgB,oBAAoB,CAAC,OAAO,EAAE,gBAAgB,EAAE,GAAG,GAAG,CAAC,MAAM,CAAC,CAE7E;AAED,YAAY,EAAE,SAAS,EAAE,cAAc,EAAE,CAAC"}
|
|
@@ -52,6 +52,18 @@ export function createFlowRouter(options) {
|
|
|
52
52
|
promptIntent: menuIntent,
|
|
53
53
|
};
|
|
54
54
|
}
|
|
55
|
+
// Regex fast-path: skip LLM/embeddings if a pattern matches
|
|
56
|
+
const intentsWithRegex = options.intents.filter((i) => i.regex);
|
|
57
|
+
for (const intent of intentsWithRegex) {
|
|
58
|
+
if (intent.regex.test(message)) {
|
|
59
|
+
const sticky = options.stickyFlows.has(intent.name);
|
|
60
|
+
return {
|
|
61
|
+
messageIntent: { intent: intent.name, score: 1 },
|
|
62
|
+
flow: { active: sticky, intent: sticky ? intent.name : null },
|
|
63
|
+
promptIntent: intent.name,
|
|
64
|
+
};
|
|
65
|
+
}
|
|
66
|
+
}
|
|
55
67
|
const embeddedIntents = await options.getEmbeddedIntents();
|
|
56
68
|
const [embeddingScores, llmResult] = await Promise.all([
|
|
57
69
|
options.scoreIntentsForMessage(message, embeddedIntents),
|
package/dist/types.d.ts
CHANGED
|
@@ -28,6 +28,8 @@ export type IntentDefinition = {
|
|
|
28
28
|
sticky?: boolean;
|
|
29
29
|
/** Short description for the LLM intent router. */
|
|
30
30
|
description?: string;
|
|
31
|
+
/** Fast-path regex: if message matches, skip LLM/embeddings and use this intent directly. */
|
|
32
|
+
regex?: RegExp;
|
|
31
33
|
};
|
|
32
34
|
/** Context passed into prompt builders (app-defined shape). */
|
|
33
35
|
export type PromptContext = {
|
|
@@ -65,7 +67,7 @@ export type FlowResolution = {
|
|
|
65
67
|
promptIntent: string;
|
|
66
68
|
llmResult?: LlmIntentResult;
|
|
67
69
|
};
|
|
68
|
-
export type LlmProvider = "ollama" | "openai";
|
|
70
|
+
export type LlmProvider = "ollama" | "openai" | "openrouter";
|
|
69
71
|
export type LlmConfig = {
|
|
70
72
|
provider: LlmProvider;
|
|
71
73
|
model: string;
|
package/dist/types.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"types.d.ts","sourceRoot":"","sources":["../src/types.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,6CAA6C,CAAC;AACjF,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,4BAA4B,CAAC;AAC7D,OAAO,KAAK,EAAE,qBAAqB,EAAE,MAAM,uBAAuB,CAAC;AACnE,OAAO,KAAK,EAAE,mBAAmB,EAAE,MAAM,iCAAiC,CAAC;AAE3E;;;;;;;;;GASG;AACH,MAAM,MAAM,aAAa,GAAG,MAAM,CAChC,MAAM,EACJ,MAAM,GACN;IACE,6CAA6C;IAC7C,EAAE,EAAE,MAAM,CAAC;IACX,qDAAqD;IACrD,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,6CAA6C;IAC7C,OAAO,CAAC,EAAE,OAAO,CAAC;CACnB,CACJ,CAAC;AAEF,2EAA2E;AAC3E,MAAM,MAAM,gBAAgB,GAAG;IAC7B,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,MAAM,EAAE,CAAC;IACnB,sEAAsE;IACtE,MAAM,CAAC,EAAE,OAAO,CAAC;IACjB,mDAAmD;IACnD,WAAW,CAAC,EAAE,MAAM,CAAC;
|
|
1
|
+
{"version":3,"file":"types.d.ts","sourceRoot":"","sources":["../src/types.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,6CAA6C,CAAC;AACjF,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,4BAA4B,CAAC;AAC7D,OAAO,KAAK,EAAE,qBAAqB,EAAE,MAAM,uBAAuB,CAAC;AACnE,OAAO,KAAK,EAAE,mBAAmB,EAAE,MAAM,iCAAiC,CAAC;AAE3E;;;;;;;;;GASG;AACH,MAAM,MAAM,aAAa,GAAG,MAAM,CAChC,MAAM,EACJ,MAAM,GACN;IACE,6CAA6C;IAC7C,EAAE,EAAE,MAAM,CAAC;IACX,qDAAqD;IACrD,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,6CAA6C;IAC7C,OAAO,CAAC,EAAE,OAAO,CAAC;CACnB,CACJ,CAAC;AAEF,2EAA2E;AAC3E,MAAM,MAAM,gBAAgB,GAAG;IAC7B,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,MAAM,EAAE,CAAC;IACnB,sEAAsE;IACtE,MAAM,CAAC,EAAE,OAAO,CAAC;IACjB,mDAAmD;IACnD,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,6FAA6F;IAC7F,KAAK,CAAC,EAAE,MAAM,CAAC;CAChB,CAAC;AAEF,+DAA+D;AAC/D,MAAM,MAAM,aAAa,GAAG;IAC1B,sBAAsB,EAAE,OAAO,CAAC;IAChC,MAAM,EAAE,MAAM,CAAC;IACf,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,kEAAkE;IAClE,OAAO,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;CACnC,CAAC;AAEF,MAAM,MAAM,aAAa,GAAG;IAC1B,MAAM,EAAE,CAAC,GAAG,EAAE,aAAa,KAAK,MAAM,CAAC;IACvC,KAAK,EAAE,CAAC,QAAQ,EAAE,MAAM,KAAK,qBAAqB,EAAE,CAAC;CACtD,CAAC;AAEF,MAAM,MAAM,cAAc,GAAG,MAAM,CAAC,MAAM,EAAE,aAAa,CAAC,CAAC;AAE3D,MAAM,MAAM,oBAAoB,GAAG;IACjC,MAAM,EAAE,MAAM,CAAC;IACf,KAAK,EAAE,MAAM,CAAC;CACf,CAAC;AAEF,MAAM,MAAM,eAAe,GAAG,oBAAoB,GAAG;IACnD,MAAM,EAAE,UAAU,GAAG,QAAQ,GAAG,QAAQ,CAAC;IACzC,MAAM,CAAC,EAAE,MAAM,CAAC;CACjB,CAAC;AAEF,MAAM,MAAM,SAAS,GAAG;IACtB,MAAM,EAAE,OAAO,CAAC;IAChB,MAAM,EAAE,MAAM,GAAG,IAAI,CAAC;CACvB,CAAC;AAEF,MAAM,MAAM,aAAa,GAAG;IAC1B,MAAM,EAAE,MAAM,CAAC;IACf,SAAS,EAAE,MAAM,CAAC;CACnB,CAAC;AAEF,MAAM,MAAM,cAAc,GAAG;IAC3B,aAAa,EAAE,oBAAoB,CAAC;IACpC,IAAI,EAAE,SAAS,CAAC;IAChB,YAAY,EAAE,MAAM,CAAC;IACrB,SAAS,CAAC,EAAE,eAAe,CAAC;CAC7B,CAAC;AAEF,MAAM,MAAM,WAAW,GAAG,QAAQ,GAAG,QAAQ,GAAG,YAAY,CAAC;AAE7D,MAAM,MAAM,SAAS,GAAG;IACtB,QAAQ,EAAE,WAAW,CAAC;IACtB,KAAK,EAAE,MAAM,CAAC;IACd,iBAAiB,CAAC,EAAE,MAAM,CAAC;IAC3B,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB,kBAAkB,CAAC,EAAE,WAAW,CAAC;IACjC,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,WAAW,CAAC,EAAE,MAAM,CAAC;CACtB,CAAC;AAEF,MAAM,MAAM,WAAW,GAAG;IACxB,uDAAuD;IACvD,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB,sDAAsD;IACtD,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,8CAA8C;IAC9C,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,KAAK,CAAC,EAAE,MAAM,CAAC;CAChB,CAAC;AAEF,MAAM,MAAM,aAAa,GAAG;IAC1B,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB,WAAW,CAAC,EAAE,MAAM,CAAC;CACtB,CAAC;AAEF,MAAM,MAAM,QAAQ,GAAG;IACrB,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,IAAI,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;CAChC,CAAC;AAEF;;;GAGG;AACH,MAAM,MAAM,aAAa,GAAG,CAC1B,IAAI,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,EAC7B,GAAG,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,KACzB,IAAI,CAAC;AAEV,mEAAmE;AACnE,MAAM,MAAM,gBAAgB,GAAG,MAAM,CAAC,MAAM,EAAE,aAAa,CAAC,CAAC;AAE7D,MAAM,MAAM,iBAAiB,GAAG;IAC9B,QAAQ,EAAE,MAAM,CAAC;IACjB,YAAY,EAAE,MAAM,CAAC;IACrB,IAAI,EAAE,SAAS,CAAC;IAChB,WAAW,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;CACtC,CAAC;AAEF,MAAM,MAAM,kBAAkB,GAAG;IAC/B,IAAI,EAAE,SAAS,CAAC;IAChB,aAAa,EAAE,OAAO,CAAC;IACvB,8EAA8E;IAC9E,MAAM,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;CAClC,CAAC;AAEF,MAAM,MAAM,WAAW,GAAG;IACxB,OAAO,EAAE,MAAM,CAAC;IAChB,QAAQ,EAAE,MAAM,CAAC;IACjB,kEAAkE;IAClE,OAAO,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IAClC,8CAA8C;IAC9C,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,QAAQ,CAAC,EAAE,MAAM,CAAC;CACnB,CAAC;AAEF,MAAM,MAAM,YAAY,GAAG;IACzB,QAAQ,EAAE,OAAO,EAAE,CAAC;IACpB,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,MAAM,EAAE,oBAAoB,CAAC;IAC7B,IAAI,EAAE,SAAS,CAAC;IAChB,WAAW,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IACrC,aAAa,EAAE,OAAO,CAAC;IACvB,MAAM,CAAC,EAAE;QAAE,MAAM,EAAE,eAAe,CAAC,QAAQ,CAAC,CAAC;QAAC,MAAM,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;IAChE,MAAM,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;CAClC,CAAC;AAEF,MAAM,MAAM,kBAAkB,GAAG;IAC/B,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB;;OAEG;IACH,GAAG,CAAC,EAAE,SAAS,CAAC;IAChB,oDAAoD;IACpD,OAAO,EAAE,gBAAgB,EAAE,CAAC;IAC5B,yCAAyC;IACzC,cAAc,EAAE,cAAc,CAAC;IAC/B;;;;OAIG;IACH,OAAO,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IACjC,mDAAmD;IACnD,aAAa,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IACvC,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,yCAAyC;IACzC,kBAAkB,CAAC,EAAE,MAAM,GAAG,CAAC,CAAC,OAAO,EAAE,gBAAgB,EAAE,KAAK,MAAM,CAAC,CAAC;IACxE,gFAAgF;IAChF,iBAAiB,CAAC,EAAE,CAAC,GAAG,EAAE,aAAa,KAAK,MAAM,CAAC;IACnD;;;OAGG;IACH,aAAa,CAAC,EAAE,aAAa,CAAC;IAC9B;;;OAGG;IACH,UAAU,CAAC,EAAE,MAAM,EAAE,CAAC;IACtB;;OAEG;IACH,cAAc,CAAC,EAAE,gBAAgB,CAAC;IAClC,sEAAsE;IACtE,kBAAkB,CAAC,EAAE,MAAM,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IACnD,iGAAiG;IACjG,yBAAyB,CAAC,EAAE,OAAO,CAAC;IACpC,qFAAqF;IACrF,kBAAkB,CAAC,EAAE,CAAC,QAAQ,EAAE,OAAO,EAAE,KAAK,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IACtE;;OAEG;IACH,YAAY,CAAC,EAAE,CAAC,KAAK,EAAE,iBAAiB,KAAK,kBAAkB,GAAG,OAAO,CAAC,kBAAkB,CAAC,CAAC;IAC9F,yEAAyE;IACzE,eAAe,CAAC,EAAE,CAAC,KAAK,EAAE;QACxB,QAAQ,EAAE,MAAM,CAAC;QACjB,QAAQ,CAAC,EAAE,MAAM,CAAC;QAClB,OAAO,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;KACnC,KAAK,MAAM,CAAC;IACb,KAAK,CAAC,EAAE,WAAW,CAAC;IACpB,OAAO,CAAC,EAAE,aAAa,CAAC;IACxB,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,gEAAgE;IAChE,UAAU,CAAC,EAAE;QACX,MAAM,CAAC,EAAE,MAAM,CAAC;QAChB,SAAS,CAAC,EAAE,MAAM,CAAC;QACnB,YAAY,CAAC,EAAE,MAAM,CAAC;QACtB,aAAa,CAAC,EAAE,MAAM,CAAC;QACvB,mBAAmB,CAAC,EAAE,MAAM,CAAC;KAC9B,CAAC;IACF,mEAAmE;IACnE,MAAM,CAAC,EAAE;QACP,IAAI,CAAC,EAAE,aAAa,CAAC;QACrB,YAAY,CAAC,EAAE,aAAa,CAAC;QAC7B,UAAU,CAAC,EAAE,UAAU,CAAC;KACzB,CAAC;IACF,mEAAmE;IACnE,YAAY,CAAC,EAAE,mBAAmB,CAAC;CACpC,CAAC"}
|
package/package.json
CHANGED