@futdevpro/nts-dynamo 1.15.23 → 1.15.24
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md
CHANGED
|
@@ -290,6 +290,10 @@ Provides abstractions and implementations for AI operations:
|
|
|
290
290
|
- **Document AI**: Document chunking and preprocessing for AI models
|
|
291
291
|
- **Vector Search**: MongoDB Atlas vector database operations with automatic data vectorization
|
|
292
292
|
|
|
293
|
+
#### OpenAI-Compatible Providers
|
|
294
|
+
|
|
295
|
+
The OAI service-ek (`DyNTS_OAI_LLM_ServiceBase`, `DyNTS_OAI_LLMChat_ServiceBase`, `DyNTS_OAI_Embedding_ControlService`) bármely OpenAI-API-kompatibilis endpoint-tal működnek — lokál LLM (LM Studio, Ollama), self-hosted (vLLM, LocalAI) is — a `DyFM_OAI_ClientOptions.baseURL` mező segítségével. Részletek + provider-konfig sample-ök: [`__documentations/2026-05-17-oai-compatible-providers.md`](__documentations/2026-05-17-oai-compatible-providers.md).
|
|
296
|
+
|
|
293
297
|
### Assistant Module
|
|
294
298
|
|
|
295
299
|
Platform and AI provider agnostic solutions for creating intelligent assistants:
|
|
@@ -0,0 +1,229 @@
|
|
|
1
|
+
# 2026-05-17 — Using OpenAI-compatible providers with Dynamo-NTS
|
|
2
|
+
|
|
3
|
+
**Spec:** `__agent/feature-requests/FR-005-oai-compat-docs.md` (workspace root)
|
|
4
|
+
**Scope:** docs-only, LOW priority. Semmilyen TS-változás. NEM kell npm publish.
|
|
5
|
+
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## Mire jó
|
|
9
|
+
|
|
10
|
+
A Dynamo-NTS OAI service-ek — `DyNTS_OAI_LLM_ServiceBase`, `DyNTS_OAI_LLMChat_ServiceBase`, `DyNTS_OAI_Embedding_ControlService` — az `openai` npm SDK-t használják, és **out-of-the-box bármely OpenAI-API-kompatibilis endpoint-tal működnek** a `baseURL` override-on keresztül. Ez **MÁR MŰKÖDIK** (zéró kód-változás kell), csak eddig nem volt dokumentálva.
|
|
11
|
+
|
|
12
|
+
A `baseURL` mező a `@futdevpro/fsm-dynamo/ai/open-ai` package `DyFM_OAI_ClientOptions` interface-en él. Bővebben a fsm-dynamo-szintű részletekért: [`@futdevpro/fsm-dynamo/__documentations/2026-05-17-oai-compatible-providers-howto.md`](../../dynamo-fsm/__documentations/2026-05-17-oai-compatible-providers-howto.md).
|
|
13
|
+
|
|
14
|
+
Tipikus use-case-ek:
|
|
15
|
+
- **Lokál fejlesztés** OpenAI-credits nélkül (LM Studio, Ollama)
|
|
16
|
+
- **Self-hosted production** infrastruktúra (vLLM, LocalAI klaszter)
|
|
17
|
+
- **Cost control** — nyílt-súlyú modellek drága cloud API helyett
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## A pattern
|
|
22
|
+
|
|
23
|
+
```typescript
|
|
24
|
+
import { DyFM_OAI_Settings, DyFM_OAI_CallSettings } from '@futdevpro/fsm-dynamo/ai/open-ai';
|
|
25
|
+
import { DyNTS_OAI_LLMChat_ServiceBase } from '@futdevpro/nts-dynamo/ai/open-ai';
|
|
26
|
+
|
|
27
|
+
const settings: DyFM_OAI_Settings = new DyFM_OAI_Settings({
|
|
28
|
+
config: {
|
|
29
|
+
baseURL: 'http://<provider-host>:<port>/v1',
|
|
30
|
+
apiKey: '<placeholder-or-real>',
|
|
31
|
+
},
|
|
32
|
+
defaultSettings: new DyFM_OAI_CallSettings({
|
|
33
|
+
useModel: '<provider-specific-model-id>',
|
|
34
|
+
}),
|
|
35
|
+
});
|
|
36
|
+
|
|
37
|
+
class MyChat extends DyNTS_OAI_LLMChat_ServiceBase {}
|
|
38
|
+
const chat: MyChat = new MyChat(settings);
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
Csak `config.baseURL` + `config.apiKey` + `defaultSettings.useModel` kell. A többi (organization, project) lokál providereknél figyelmen kívül marad.
|
|
42
|
+
|
|
43
|
+
A `DyNTS_OAI_Embedding_ControlService` és más OAI alapú service-ek ugyanezt a `DyFM_OAI_Settings`-t fogadják — egy konfig, minden service.
|
|
44
|
+
|
|
45
|
+
---
|
|
46
|
+
|
|
47
|
+
## Provider-specifikus minták
|
|
48
|
+
|
|
49
|
+
### LM Studio (port 1234)
|
|
50
|
+
|
|
51
|
+
Desktop GUI (Mac/Windows/Linux), beépített OAI-compat szerverrel. Lépések:
|
|
52
|
+
1. Töltsd be a modellt a UI-ben (pl. embedding-hez `nomic-embed-text-v1.5`, chat-hez `llama-3.2-3b-instruct`)
|
|
53
|
+
2. **Local Server** tab → **Start Server**
|
|
54
|
+
|
|
55
|
+
```typescript
|
|
56
|
+
const lmStudio: DyFM_OAI_Settings = new DyFM_OAI_Settings({
|
|
57
|
+
config: {
|
|
58
|
+
baseURL: 'http://localhost:1234/v1',
|
|
59
|
+
apiKey: 'lm-studio', // bármilyen non-empty placeholder
|
|
60
|
+
},
|
|
61
|
+
defaultSettings: new DyFM_OAI_CallSettings({
|
|
62
|
+
useModel: 'nomic-embed-text-v1.5', // ahogy a Local Server tab megjeleníti
|
|
63
|
+
}),
|
|
64
|
+
});
|
|
65
|
+
|
|
66
|
+
// Embedding-hez
|
|
67
|
+
class Knowledge_DataService extends DyNTS_OAI_VectorDataService<Knowledge> {
|
|
68
|
+
constructor() { super(/* args */); }
|
|
69
|
+
}
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
### Ollama (port 11434, OAI-compat layer ≥ v0.1.14)
|
|
73
|
+
|
|
74
|
+
CLI-driven, multi-modell runner.
|
|
75
|
+
|
|
76
|
+
```bash
|
|
77
|
+
ollama pull llama3.2:3b
|
|
78
|
+
ollama pull nomic-embed-text
|
|
79
|
+
ollama serve
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
```typescript
|
|
83
|
+
const ollama: DyFM_OAI_Settings = new DyFM_OAI_Settings({
|
|
84
|
+
config: {
|
|
85
|
+
baseURL: 'http://localhost:11434/v1',
|
|
86
|
+
apiKey: 'ollama', // szintén placeholder
|
|
87
|
+
},
|
|
88
|
+
defaultSettings: new DyFM_OAI_CallSettings({
|
|
89
|
+
useModel: 'nomic-embed-text', // `ollama list` mutatja
|
|
90
|
+
}),
|
|
91
|
+
});
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
### vLLM (port 8000, production GPU)
|
|
95
|
+
|
|
96
|
+
Python-alapú batched + PagedAttention inference. OAI-compat REST endpoint beépítve.
|
|
97
|
+
|
|
98
|
+
```bash
|
|
99
|
+
pip install vllm
|
|
100
|
+
python -m vllm.entrypoints.openai.api_server \
|
|
101
|
+
--model meta-llama/Meta-Llama-3-8B-Instruct \
|
|
102
|
+
--port 8000 \
|
|
103
|
+
--api-key local-secret-token
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
```typescript
|
|
107
|
+
const vllm: DyFM_OAI_Settings = new DyFM_OAI_Settings({
|
|
108
|
+
config: {
|
|
109
|
+
baseURL: 'http://gpu-server:8000/v1',
|
|
110
|
+
apiKey: process.env.VLLM_API_KEY ?? 'local-secret-token',
|
|
111
|
+
},
|
|
112
|
+
defaultSettings: new DyFM_OAI_CallSettings({
|
|
113
|
+
useModel: 'meta-llama/Meta-Llama-3-8B-Instruct', // a `--model` érték
|
|
114
|
+
}),
|
|
115
|
+
});
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
### LocalAI (port 8080, REST multi-model)
|
|
119
|
+
|
|
120
|
+
Go-alapú, multi-backend (llama.cpp + whisper.cpp + stable-diffusion).
|
|
121
|
+
|
|
122
|
+
```bash
|
|
123
|
+
docker run -p 8080:8080 \
|
|
124
|
+
-v $PWD/models:/build/models \
|
|
125
|
+
localai/localai:latest
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
```typescript
|
|
129
|
+
const localai: DyFM_OAI_Settings = new DyFM_OAI_Settings({
|
|
130
|
+
config: {
|
|
131
|
+
baseURL: 'http://localai:8080/v1',
|
|
132
|
+
apiKey: 'sk-local', // placeholder
|
|
133
|
+
},
|
|
134
|
+
defaultSettings: new DyFM_OAI_CallSettings({
|
|
135
|
+
useModel: 'gpt-4', // LocalAI alias — `models.yaml`-ban mapped
|
|
136
|
+
}),
|
|
137
|
+
});
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
## Dev → cloud quick recipe
|
|
143
|
+
|
|
144
|
+
```typescript
|
|
145
|
+
const isDev: boolean = process.env.NODE_ENV === 'development';
|
|
146
|
+
|
|
147
|
+
export const aiSettings: DyFM_OAI_Settings = new DyFM_OAI_Settings({
|
|
148
|
+
config: isDev
|
|
149
|
+
? {
|
|
150
|
+
baseURL: 'http://localhost:11434/v1', // Ollama lokálban
|
|
151
|
+
apiKey: 'dev',
|
|
152
|
+
}
|
|
153
|
+
: {
|
|
154
|
+
apiKey: process.env.OPENAI_API_KEY,
|
|
155
|
+
organization: process.env.OPENAI_ORG_ID,
|
|
156
|
+
},
|
|
157
|
+
defaultSettings: new DyFM_OAI_CallSettings({
|
|
158
|
+
useModel: isDev ? 'llama3.2:3b' : 'gpt-4o',
|
|
159
|
+
}),
|
|
160
|
+
});
|
|
161
|
+
|
|
162
|
+
// Az összes Dynamo-NTS OAI service ugyanazt a settings-t fogyasztja:
|
|
163
|
+
class ProdChat extends DyNTS_OAI_LLMChat_ServiceBase {}
|
|
164
|
+
const chat: ProdChat = new ProdChat(aiSettings);
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
---
|
|
168
|
+
|
|
169
|
+
## Caveats — service-szintű
|
|
170
|
+
|
|
171
|
+
### Tool/function calling
|
|
172
|
+
A `DyNTS_OAI_LLMChat_ServiceBase` tool-calling support csak akkor működik a lokál provider-eknél, ha:
|
|
173
|
+
- **Ollama**: tool-aware modell (Llama-3.1+, Mistral újabb verziók)
|
|
174
|
+
- **vLLM**: v0.6+ + megfelelő chat-template
|
|
175
|
+
- **LM Studio**: model-függő, gyakran részleges
|
|
176
|
+
- **LocalAI**: function-call backend kell konfigolni
|
|
177
|
+
|
|
178
|
+
Production OpenAI-mintával írt service-ek **lokál providerre váltáskor** előbb tesztelni kell, hogy a tool-callok valóban triggerelnek-e.
|
|
179
|
+
|
|
180
|
+
### Streaming chat
|
|
181
|
+
A `DyNTS_OAI_LLMChat_ServiceBase` streaming (SSE) működik mindenhol, **de**:
|
|
182
|
+
- Ollama < v0.1.20 streaming-stabilitás kérdéses
|
|
183
|
+
- LocalAI streaming-rate alacsonyabb mint az emit-rate
|
|
184
|
+
- vLLM/LM Studio: rendszeresen frissül, érdemes a provider-changelogot követni
|
|
185
|
+
|
|
186
|
+
### Embedding-modellek
|
|
187
|
+
A `DyNTS_OAI_Embedding_ControlService` `useModel`-je a provider-specifikus dedikált embedding modell **pontos ID**-ja:
|
|
188
|
+
|
|
189
|
+
| Provider | Embedding model ID |
|
|
190
|
+
|---|---|
|
|
191
|
+
| OpenAI (cloud) | `text-embedding-3-small` / `text-embedding-3-large` |
|
|
192
|
+
| LM Studio | `nomic-embed-text-v1.5` / `BAAI/bge-large-en-v1.5` |
|
|
193
|
+
| Ollama | `nomic-embed-text` / `mxbai-embed-large` |
|
|
194
|
+
| vLLM | csak ha az indítási `--model` embedding-képes (pl. `intfloat/e5-mistral-7b-instruct`) |
|
|
195
|
+
| LocalAI | `text-embedding-ada-002` alias (vagy egyéb a `models.yaml`-ban) |
|
|
196
|
+
|
|
197
|
+
### Context window / max-tokens
|
|
198
|
+
A `DyFM_OAI_CallSettings.maxTokens` provider-szintű limit alá kell esnie, különben 400/422 jön vissza:
|
|
199
|
+
|
|
200
|
+
| Provider | Default context-window | Override |
|
|
201
|
+
|---|---|---|
|
|
202
|
+
| OpenAI gpt-4o | 128K | API tier |
|
|
203
|
+
| LM Studio | a betöltött modell native context | UI slider |
|
|
204
|
+
| Ollama | **2048 (legacy default!)** | `OLLAMA_NUM_CTX` env / Modelfile `PARAMETER num_ctx` |
|
|
205
|
+
| vLLM | a modell native context | `--max-model-len` flag |
|
|
206
|
+
| LocalAI | `models.yaml` `context_size` | per-model konfig |
|
|
207
|
+
|
|
208
|
+
### Error-handling
|
|
209
|
+
- OpenAI cloud: standard 4xx/5xx + structured error body
|
|
210
|
+
- Lokál providerek: 404 a modell-name-re (rossz ID), timeout első call-on (modell load), OOM vLLM-en
|
|
211
|
+
- Ollama-specifikus: ha a modell még nem `pull`-olt, csendben várakozik a pull-ig — látszólag hang
|
|
212
|
+
|
|
213
|
+
---
|
|
214
|
+
|
|
215
|
+
## Reference
|
|
216
|
+
|
|
217
|
+
| Fájl | Mit ad |
|
|
218
|
+
|---|---|
|
|
219
|
+
| `fsm-dynamo/.../oai-client-options.interface.ts` | `DyFM_OAI_ClientOptions.baseURL` — a kapcsoló mező |
|
|
220
|
+
| `fsm-dynamo/.../oai-settings.control-model.ts` | `DyFM_OAI_Settings` — settings wrapper |
|
|
221
|
+
| `dynamo-nts/.../oai-llm.service-base.ts` | `DyNTS_OAI_LLM_ServiceBase` — base LLM client (`openai` SDK instantiation) |
|
|
222
|
+
| `dynamo-nts/.../oai-llm-chat.service-base.ts` | `DyNTS_OAI_LLMChat_ServiceBase` — chat service |
|
|
223
|
+
| `dynamo-nts/.../oai-embedding.control-service.ts` | `DyNTS_OAI_Embedding_ControlService` — embedding service |
|
|
224
|
+
|
|
225
|
+
További (fsm-szintű) részletek, ugyanezzel a 4 provider-mintázattal: [`dynamo-fsm/__documentations/2026-05-17-oai-compatible-providers-howto.md`](../../dynamo-fsm/__documentations/2026-05-17-oai-compatible-providers-howto.md).
|
|
226
|
+
|
|
227
|
+
## Backward compatibility
|
|
228
|
+
|
|
229
|
+
Docs-only. `baseURL` mindig is létezett az OpenAI SDK-ban, a Dynamo-NTS csak átadja a config-ot (`oai-llm.service-base.ts:91-98`). Ez a doc csak tisztázza a használati mintázatot — semmilyen TS-változás nincs.
|
package/package.json
CHANGED