@molecule/api-ai-local 1.0.0 → 1.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +168 -0
- package/dist/provider.d.ts +15 -1
- package/dist/provider.d.ts.map +1 -1
- package/dist/provider.js +60 -10
- package/package.json +15 -9
package/README.md
ADDED
|
@@ -0,0 +1,168 @@
|
|
|
1
|
+
<!--
|
|
2
|
+
AUTO-GENERATED — DO NOT EDIT THIS FILE.
|
|
3
|
+
Generated by `mlcl sync-docs` from the package's src/index.ts JSDoc + mlcl/registry.json.
|
|
4
|
+
Edits here are overwritten on the next commit (molecule's pre-commit hook regenerates).
|
|
5
|
+
To change this document, edit the module-level JSDoc in src/index.ts.
|
|
6
|
+
Generated: 2026-08-04T01:47:33.798Z
|
|
7
|
+
-->
|
|
8
|
+
|
|
9
|
+
# @molecule/api-ai-local
|
|
10
|
+
|
|
11
|
+
> **Auto-generated, AI-first package reference** for the [molecule.dev](https://molecule.dev) ecosystem.
|
|
12
|
+
> It is written to be read by coding agents as much as by people, and is generated from this
|
|
13
|
+
> package's source — edit `src/index.ts` JSDoc, not this file.
|
|
14
|
+
|
|
15
|
+
Local (OpenAI-compatible) ai-local provider for molecule.dev.
|
|
16
|
+
|
|
17
|
+
Streams chat completions from any local inference server that speaks the
|
|
18
|
+
OpenAI `chat/completions` protocol (Ollama, LM Studio, llama.cpp, vLLM),
|
|
19
|
+
keyless by default.
|
|
20
|
+
|
|
21
|
+
## Type
|
|
22
|
+
|
|
23
|
+
`provider`
|
|
24
|
+
|
|
25
|
+
## Installation
|
|
26
|
+
|
|
27
|
+
```bash
|
|
28
|
+
npm install @molecule/api-ai-local @molecule/api-ai @molecule/api-bond @molecule/api-i18n @molecule/api-secrets
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
## API
|
|
32
|
+
|
|
33
|
+
### Interfaces
|
|
34
|
+
|
|
35
|
+
#### `LocalConfig`
|
|
36
|
+
|
|
37
|
+
Local (OpenAI-compatible) provider configuration.
|
|
38
|
+
|
|
39
|
+
```typescript
|
|
40
|
+
interface LocalConfig {
|
|
41
|
+
/** Called on each rate-limited/busy upstream response, before any retry sleep. */
|
|
42
|
+
onRateLimit?: AiRateLimitCallback
|
|
43
|
+
/**
|
|
44
|
+
* Base URL of the OpenAI-compatible endpoint, INCLUDING the version segment
|
|
45
|
+
* (e.g. `http://localhost:11434/v1`). Overrides `LOCAL_AI_BASE_URL` /
|
|
46
|
+
* `OLLAMA_BASE_URL`. Defaults to Ollama's `http://localhost:11434/v1`.
|
|
47
|
+
*/
|
|
48
|
+
baseUrl?: string
|
|
49
|
+
/**
|
|
50
|
+
* Optional API key. Most local servers ignore auth; when omitted (and no
|
|
51
|
+
* `LOCAL_AI_API_KEY` env var is set) no `Authorization` header is sent.
|
|
52
|
+
*/
|
|
53
|
+
apiKey?: string
|
|
54
|
+
/**
|
|
55
|
+
* Default model when a call doesn't specify one. Overrides `LOCAL_AI_MODEL`.
|
|
56
|
+
* Defaults to `llama3.1`.
|
|
57
|
+
*/
|
|
58
|
+
model?: string
|
|
59
|
+
}
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
### Classes
|
|
63
|
+
|
|
64
|
+
#### `LocalAIProvider`
|
|
65
|
+
|
|
66
|
+
Local (OpenAI-compatible) chat provider implementing the `AIProvider`
|
|
67
|
+
interface. Mirrors `@molecule/api-ai-openai` so the same handler code can
|
|
68
|
+
dispatch to a local endpoint.
|
|
69
|
+
|
|
70
|
+
### Functions
|
|
71
|
+
|
|
72
|
+
#### `createProvider(config)`
|
|
73
|
+
|
|
74
|
+
Create a local (OpenAI-compatible) AI provider instance.
|
|
75
|
+
|
|
76
|
+
Constructs WITHOUT requiring any secret — local endpoints run keyless.
|
|
77
|
+
|
|
78
|
+
```typescript
|
|
79
|
+
function createProvider(config?: LocalConfig): AIProvider
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
- `config` — Local provider configuration.
|
|
83
|
+
|
|
84
|
+
**Returns:** An `AIProvider` backed by an OpenAI-compatible local endpoint.
|
|
85
|
+
|
|
86
|
+
### Constants
|
|
87
|
+
|
|
88
|
+
#### `aiLocalSecretDefinitions`
|
|
89
|
+
|
|
90
|
+
Optional secret/config overrides recognised by the local AI bond.
|
|
91
|
+
|
|
92
|
+
```typescript
|
|
93
|
+
const aiLocalSecretDefinitions: SecretDefinition[]
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
#### `provider`
|
|
97
|
+
|
|
98
|
+
The provider implementation. Constructs keyless — no secret required.
|
|
99
|
+
|
|
100
|
+
```typescript
|
|
101
|
+
const provider: AIProvider
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
## Core Interface
|
|
105
|
+
|
|
106
|
+
Implements `@molecule/api-ai` interface.
|
|
107
|
+
|
|
108
|
+
## Bond Wiring
|
|
109
|
+
|
|
110
|
+
Setup function to register this provider with the bond system:
|
|
111
|
+
|
|
112
|
+
```typescript
|
|
113
|
+
import { bond } from '@molecule/api-bond'
|
|
114
|
+
import { provider } from '@molecule/api-ai-local'
|
|
115
|
+
|
|
116
|
+
export function setupAiLocal(): void {
|
|
117
|
+
bond('ai', 'local', provider)
|
|
118
|
+
}
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
## Injection Notes
|
|
122
|
+
|
|
123
|
+
### Requirements
|
|
124
|
+
|
|
125
|
+
Peer dependencies:
|
|
126
|
+
|
|
127
|
+
- `@molecule/api-ai` ^1.0.1
|
|
128
|
+
- `@molecule/api-bond` ^1.0.1
|
|
129
|
+
- `@molecule/api-i18n` ^1.0.1
|
|
130
|
+
- `@molecule/api-secrets` ^1.0.1
|
|
131
|
+
|
|
132
|
+
### Runtime Dependencies
|
|
133
|
+
|
|
134
|
+
- `@molecule/api-ai`
|
|
135
|
+
- `@molecule/api-bond`
|
|
136
|
+
- `@molecule/api-i18n`
|
|
137
|
+
- `@molecule/api-secrets`
|
|
138
|
+
|
|
139
|
+
**Error message disambiguation**: a plain 400 that ISN'T a context-length error (bad param,
|
|
140
|
+
malformed tool schema) gets its own non-retryable message distinct from the generic
|
|
141
|
+
"AI service error. Please try again." used for retryable failures.
|
|
142
|
+
|
|
143
|
+
## E2E Tests
|
|
144
|
+
|
|
145
|
+
Integration checklist — drive the real UI (live preview, no mocks), adapt
|
|
146
|
+
each item to this app's actual chat/AI screens, and check every box off one
|
|
147
|
+
by one. A box you can't check is an integration bug to fix — not a skip. The
|
|
148
|
+
sandbox HAS an AI provider bonded, so the flow runs live end-to-end; AI
|
|
149
|
+
output is NON-DETERMINISTIC, so assert on STRUCTURE/behavior, not exact text:
|
|
150
|
+
|
|
151
|
+
- [ ] A message sent through the real chat UI comes back as a RELEVANT AI
|
|
152
|
+
reply — not an echo of the prompt, a hardcoded stub, or an empty bubble.
|
|
153
|
+
Ask something with a checkable answer (e.g. "What is 2 + 2?") and confirm
|
|
154
|
+
the response actually contains it ("4"), proving a live model answered.
|
|
155
|
+
- [ ] If the app streams, tokens render INCREMENTALLY — text grows word by
|
|
156
|
+
word in the UI, not one final blob dumped after a long frozen spinner. (A
|
|
157
|
+
streamed `chat()` yields `text` chunks then a final `done`; a single late
|
|
158
|
+
blob means the reply was awaited whole and streaming is broken.)
|
|
159
|
+
- [ ] Multi-turn CONTEXT is preserved: a follow-up that refers back to the
|
|
160
|
+
previous turn (e.g. after "2 + 2", ask "now double that" -> understood as 8) works — proving the full `messages` history is sent, not just the last
|
|
161
|
+
line.
|
|
162
|
+
- [ ] A provider failure (bad/missing key, rate limit, timeout) surfaces as
|
|
163
|
+
a graceful in-UI error message, NOT a crash, blank screen, a spinner that
|
|
164
|
+
never resolves, or an unhandled 500. Force one and watch the UI recover.
|
|
165
|
+
- [ ] The provider key + the provider call are SERVER-side only: the key
|
|
166
|
+
never reaches the browser (check the network tab, the JS bundle, and page
|
|
167
|
+
globals), and no route proxies arbitrary prompts to the model without auth
|
|
168
|
+
- a token cap — an open AI endpoint is an unbounded bill and abuse vector.
|
package/dist/provider.d.ts
CHANGED
|
@@ -36,9 +36,23 @@ export declare class LocalAIProvider implements AIProvider {
|
|
|
36
36
|
/**
|
|
37
37
|
* Convert internal `ChatMessage` objects to the OpenAI message format.
|
|
38
38
|
* The system prompt becomes a leading `{role:'system'}` message.
|
|
39
|
+
*
|
|
40
|
+
* `tool_use`/`tool_result` blocks are serialized STRUCTURALLY (assistant
|
|
41
|
+
* `tool_calls` + standalone `{role:'tool'}` messages), as the OpenAI-
|
|
42
|
+
* compatible chat API requires. They used to flatten to placeholder text
|
|
43
|
+
* (`[tool_result for <id>]`), which silently dropped every tool result and
|
|
44
|
+
* made multi-turn agentic tool loops impossible — a local/BYO model looped on
|
|
45
|
+
* its first tool call and never finalized. See the same fix + regression test
|
|
46
|
+
* in `@molecule/api-ai-openai` (this bond is that provider with a configurable
|
|
47
|
+
* baseUrl and optional key). Single-shot text/vision requests never carry
|
|
48
|
+
* these block types and are unaffected.
|
|
39
49
|
*/
|
|
40
50
|
private formatMessages;
|
|
41
|
-
/**
|
|
51
|
+
/**
|
|
52
|
+
* Map a generic content-part `ContentBlock` (text/image/document/audio/video)
|
|
53
|
+
* to OpenAI's content-part shape. `tool_use`/`tool_result` are handled
|
|
54
|
+
* structurally in {@link formatMessages} and never reach here.
|
|
55
|
+
*/
|
|
42
56
|
private formatBlock;
|
|
43
57
|
/** Convert internal `AITool` definitions to OpenAI's `tools` array. */
|
|
44
58
|
private formatTools;
|
package/dist/provider.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"provider.d.ts","sourceRoot":"","sources":["../src/provider.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;GAWG;AAKH,OAAO,cAAc,CAAA;AAErB,OAAO,KAAK,EACV,UAAU,EAGV,SAAS,EAET,UAAU,EAEX,MAAM,kBAAkB,CAAA;AAGzB,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,YAAY,CAAA;AAqB7C;;;;GAIG;AACH,qBAAa,eAAgB,YAAW,UAAU;IAChD,QAAQ,CAAC,IAAI,WAAU;IACvB,OAAO,CAAC,MAAM,CAAQ;IACtB,OAAO,CAAC,YAAY,CAAQ;IAC5B,OAAO,CAAC,SAAS,CAAQ;IACzB,OAAO,CAAC,OAAO,CAAQ;IACvB,OAAO,CAAC,WAAW,CAAC,CAAqB;gBAE7B,MAAM,GAAE,WAAgB;IAepC;;;;;OAKG;IACI,IAAI,CAAC,MAAM,EAAE,UAAU,GAAG,aAAa,CAAC,SAAS,CAAC;IAoLzD
|
|
1
|
+
{"version":3,"file":"provider.d.ts","sourceRoot":"","sources":["../src/provider.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;GAWG;AAKH,OAAO,cAAc,CAAA;AAErB,OAAO,KAAK,EACV,UAAU,EAGV,SAAS,EAET,UAAU,EAEX,MAAM,kBAAkB,CAAA;AAGzB,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,YAAY,CAAA;AAqB7C;;;;GAIG;AACH,qBAAa,eAAgB,YAAW,UAAU;IAChD,QAAQ,CAAC,IAAI,WAAU;IACvB,OAAO,CAAC,MAAM,CAAQ;IACtB,OAAO,CAAC,YAAY,CAAQ;IAC5B,OAAO,CAAC,SAAS,CAAQ;IACzB,OAAO,CAAC,OAAO,CAAQ;IACvB,OAAO,CAAC,WAAW,CAAC,CAAqB;gBAE7B,MAAM,GAAE,WAAgB;IAepC;;;;;OAKG;IACI,IAAI,CAAC,MAAM,EAAE,UAAU,GAAG,aAAa,CAAC,SAAS,CAAC;IAoLzD;;;;;;;;;;;;;OAaG;IACH,OAAO,CAAC,cAAc;IA2DtB;;;;OAIG;IACH,OAAO,CAAC,WAAW;IA6BnB,uEAAuE;IACvE,OAAO,CAAC,WAAW;IAWnB;;;;;OAKG;IACH,OAAO,CAAE,yBAAyB;IA8ClC;;;;;OAKG;YACY,sBAAsB;IA2ErC;;;;;;OAMG;IACH,OAAO,CAAE,eAAe;IA+DxB;;;;;;;;;;OAUG;IACH,OAAO,CAAE,qBAAqB;CAqC/B;AAED;;;;;;;GAOG;AACH,wBAAgB,cAAc,CAAC,MAAM,CAAC,EAAE,WAAW,GAAG,UAAU,CAE/D"}
|
package/dist/provider.js
CHANGED
|
@@ -225,6 +225,16 @@ export class LocalAIProvider {
|
|
|
225
225
|
/**
|
|
226
226
|
* Convert internal `ChatMessage` objects to the OpenAI message format.
|
|
227
227
|
* The system prompt becomes a leading `{role:'system'}` message.
|
|
228
|
+
*
|
|
229
|
+
* `tool_use`/`tool_result` blocks are serialized STRUCTURALLY (assistant
|
|
230
|
+
* `tool_calls` + standalone `{role:'tool'}` messages), as the OpenAI-
|
|
231
|
+
* compatible chat API requires. They used to flatten to placeholder text
|
|
232
|
+
* (`[tool_result for <id>]`), which silently dropped every tool result and
|
|
233
|
+
* made multi-turn agentic tool loops impossible — a local/BYO model looped on
|
|
234
|
+
* its first tool call and never finalized. See the same fix + regression test
|
|
235
|
+
* in `@molecule/api-ai-openai` (this bond is that provider with a configurable
|
|
236
|
+
* baseUrl and optional key). Single-shot text/vision requests never carry
|
|
237
|
+
* these block types and are unaffected.
|
|
228
238
|
*/
|
|
229
239
|
formatMessages(messages, systemPrompt) {
|
|
230
240
|
const out = [];
|
|
@@ -235,14 +245,58 @@ export class LocalAIProvider {
|
|
|
235
245
|
out.push({ role: 'system', content: typeof m.content === 'string' ? m.content : '' });
|
|
236
246
|
continue;
|
|
237
247
|
}
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
248
|
+
if (typeof m.content === 'string') {
|
|
249
|
+
out.push({ role: m.role, content: m.content });
|
|
250
|
+
continue;
|
|
251
|
+
}
|
|
252
|
+
const parts = [];
|
|
253
|
+
const toolCalls = [];
|
|
254
|
+
const toolResults = [];
|
|
255
|
+
for (const block of m.content) {
|
|
256
|
+
switch (block.type) {
|
|
257
|
+
case 'tool_use':
|
|
258
|
+
toolCalls.push({
|
|
259
|
+
id: block.id,
|
|
260
|
+
type: 'function',
|
|
261
|
+
function: { name: block.name, arguments: JSON.stringify(block.input ?? {}) },
|
|
262
|
+
});
|
|
263
|
+
break;
|
|
264
|
+
case 'tool_result':
|
|
265
|
+
toolResults.push({
|
|
266
|
+
role: 'tool',
|
|
267
|
+
tool_call_id: block.tool_use_id,
|
|
268
|
+
content: typeof block.content === 'string' ? block.content : JSON.stringify(block.content),
|
|
269
|
+
});
|
|
270
|
+
break;
|
|
271
|
+
default:
|
|
272
|
+
parts.push(this.formatBlock(block));
|
|
273
|
+
}
|
|
274
|
+
}
|
|
275
|
+
if (toolResults.length > 0) {
|
|
276
|
+
for (const tr of toolResults)
|
|
277
|
+
out.push(tr);
|
|
278
|
+
}
|
|
279
|
+
else if (toolCalls.length > 0) {
|
|
280
|
+
out.push({
|
|
281
|
+
role: 'assistant',
|
|
282
|
+
content: parts.length > 0 ? parts : null,
|
|
283
|
+
tool_calls: toolCalls,
|
|
284
|
+
});
|
|
285
|
+
}
|
|
286
|
+
else {
|
|
287
|
+
out.push({
|
|
288
|
+
role: m.role,
|
|
289
|
+
content: parts.length === 1 && parts[0].type === 'text' ? parts[0].text : parts,
|
|
290
|
+
});
|
|
291
|
+
}
|
|
242
292
|
}
|
|
243
293
|
return out;
|
|
244
294
|
}
|
|
245
|
-
/**
|
|
295
|
+
/**
|
|
296
|
+
* Map a generic content-part `ContentBlock` (text/image/document/audio/video)
|
|
297
|
+
* to OpenAI's content-part shape. `tool_use`/`tool_result` are handled
|
|
298
|
+
* structurally in {@link formatMessages} and never reach here.
|
|
299
|
+
*/
|
|
246
300
|
formatBlock(block) {
|
|
247
301
|
switch (block.type) {
|
|
248
302
|
case 'text':
|
|
@@ -267,12 +321,8 @@ export class LocalAIProvider {
|
|
|
267
321
|
type: 'text',
|
|
268
322
|
text: `[Video attachment (${block.mediaType}) — not supported by this provider]`,
|
|
269
323
|
};
|
|
270
|
-
case 'tool_use':
|
|
271
|
-
return { type: 'text', text: `[tool_use ${block.name}]` };
|
|
272
|
-
case 'tool_result':
|
|
273
|
-
return { type: 'text', text: `[tool_result for ${block.tool_use_id}]` };
|
|
274
324
|
default:
|
|
275
|
-
return
|
|
325
|
+
return { type: 'text', text: '' };
|
|
276
326
|
}
|
|
277
327
|
}
|
|
278
328
|
/** Convert internal `AITool` definitions to OpenAI's `tools` array. */
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@molecule/api-ai-local",
|
|
3
|
-
"version": "1.0.
|
|
3
|
+
"version": "1.0.2",
|
|
4
4
|
"description": "AIProvider bond for any OpenAI-compatible local inference server (Ollama, LM Studio, llama.cpp, vLLM) — configurable base URL, keyless by default, streamed via raw fetch.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "dist/index.js",
|
|
@@ -17,7 +17,8 @@
|
|
|
17
17
|
}
|
|
18
18
|
},
|
|
19
19
|
"files": [
|
|
20
|
-
"dist"
|
|
20
|
+
"dist",
|
|
21
|
+
"README.md"
|
|
21
22
|
],
|
|
22
23
|
"keywords": [
|
|
23
24
|
"molecule",
|
|
@@ -25,19 +26,24 @@
|
|
|
25
26
|
"local"
|
|
26
27
|
],
|
|
27
28
|
"license": "Apache-2.0",
|
|
29
|
+
"repository": {
|
|
30
|
+
"type": "git",
|
|
31
|
+
"url": "https://github.com/molecule-dev/molecule.git",
|
|
32
|
+
"directory": "packages/api/bonds/ai/local"
|
|
33
|
+
},
|
|
28
34
|
"devDependencies": {
|
|
29
|
-
"@molecule/api-ai": "1.0.
|
|
30
|
-
"@molecule/api-bond": "1.0.
|
|
31
|
-
"@molecule/api-secrets": "1.0.
|
|
35
|
+
"@molecule/api-ai": "1.0.1",
|
|
36
|
+
"@molecule/api-bond": "1.0.1",
|
|
37
|
+
"@molecule/api-secrets": "1.0.1",
|
|
32
38
|
"@types/node": "26.1.2",
|
|
33
39
|
"typescript": "6.0.3",
|
|
34
40
|
"vitest": "4.1.10"
|
|
35
41
|
},
|
|
36
42
|
"peerDependencies": {
|
|
37
|
-
"@molecule/api-ai": "^1.0.
|
|
38
|
-
"@molecule/api-bond": "^1.0.
|
|
39
|
-
"@molecule/api-i18n": "^1.0.
|
|
40
|
-
"@molecule/api-secrets": "^1.0.
|
|
43
|
+
"@molecule/api-ai": "^1.0.1",
|
|
44
|
+
"@molecule/api-bond": "^1.0.1",
|
|
45
|
+
"@molecule/api-i18n": "^1.0.1",
|
|
46
|
+
"@molecule/api-secrets": "^1.0.1"
|
|
41
47
|
},
|
|
42
48
|
"peerDependenciesMeta": {
|
|
43
49
|
"@molecule/api-ai": {
|