theroundtaible 0.3.10 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +52 -2
- package/dist/adapters/base.d.ts +6 -0
- package/dist/adapters/base.d.ts.map +1 -1
- package/dist/adapters/base.js +8 -0
- package/dist/adapters/base.js.map +1 -1
- package/dist/adapters/local-llm.d.ts +48 -0
- package/dist/adapters/local-llm.d.ts.map +1 -0
- package/dist/adapters/local-llm.js +218 -0
- package/dist/adapters/local-llm.js.map +1 -0
- package/dist/adapters/openai-api.js +1 -1
- package/dist/adapters/openai-api.js.map +1 -1
- package/dist/commands/init.d.ts.map +1 -1
- package/dist/commands/init.js +88 -55
- package/dist/commands/init.js.map +1 -1
- package/dist/consensus.d.ts.map +1 -1
- package/dist/consensus.js +16 -3
- package/dist/consensus.js.map +1 -1
- package/dist/orchestrator.d.ts.map +1 -1
- package/dist/orchestrator.js +13 -1
- package/dist/orchestrator.js.map +1 -1
- package/dist/types.d.ts +7 -1
- package/dist/types.d.ts.map +1 -1
- package/dist/utils/adapters.d.ts.map +1 -1
- package/dist/utils/adapters.js +17 -1
- package/dist/utils/adapters.js.map +1 -1
- package/dist/utils/context.js +2 -2
- package/dist/utils/context.js.map +1 -1
- package/dist/utils/local-detect.d.ts +12 -0
- package/dist/utils/local-detect.d.ts.map +1 -0
- package/dist/utils/local-detect.js +111 -0
- package/dist/utils/local-detect.js.map +1 -0
- package/package.json +1 -1
- package/templates/system-prompt.md +4 -0
package/README.md
CHANGED
|
@@ -66,11 +66,12 @@ roundtable --version
|
|
|
66
66
|
### Prerequisites
|
|
67
67
|
|
|
68
68
|
- Node.js 20+
|
|
69
|
-
- At least one AI
|
|
69
|
+
- At least one AI tool:
|
|
70
70
|
- [Claude Code](https://docs.anthropic.com/en/docs/claude-code) — `claude` command
|
|
71
71
|
- [Gemini CLI](https://github.com/google-gemini/gemini-cli) — `gemini` command
|
|
72
72
|
- [Codex CLI](https://github.com/openai/codex) — `codex` command
|
|
73
|
-
-
|
|
73
|
+
- OpenAI API key (`OPENAI_API_KEY` env var) as fallback
|
|
74
|
+
- [Ollama](https://ollama.com/) or [LM Studio](https://lmstudio.ai/) for local models
|
|
74
75
|
|
|
75
76
|
## Quick start
|
|
76
77
|
|
|
@@ -153,6 +154,7 @@ After `roundtable init`, your `.roundtable/config.json` controls everything:
|
|
|
153
154
|
| `gemini-cli` | `gemini -p "prompt"` | Gemini Advanced | `gemini-api` |
|
|
154
155
|
| `openai-cli` | `codex "prompt"` | ChatGPT Pro | `openai-api` |
|
|
155
156
|
| `openai-api` | OpenAI REST API | No (API key) | — |
|
|
157
|
+
| `local-llm` | OpenAI-compat or Ollama native | No (runs locally) | — |
|
|
156
158
|
|
|
157
159
|
## Code-Red mode
|
|
158
160
|
|
|
@@ -235,11 +237,59 @@ See [architecture-docs.md](architecture-docs.md) for the full technical architec
|
|
|
235
237
|
- [x] Chronicle (persistent decision memory)
|
|
236
238
|
- [x] Implementation Manifest (tracks what's been built)
|
|
237
239
|
- [x] Scoped Apply (knights declare files, apply enforces scope)
|
|
240
|
+
- [x] Local LLM support (Ollama, LM Studio)
|
|
238
241
|
- [ ] VS Code extension
|
|
239
242
|
- [ ] Web dashboard for session visualization
|
|
240
243
|
- [ ] More adapters (DeepSeek, Llama, Mistral)
|
|
241
244
|
- [ ] CI/CD integration (GitHub Actions)
|
|
242
245
|
|
|
246
|
+
## Local LLMs
|
|
247
|
+
|
|
248
|
+
TheRoundtAIble supports local models through [Ollama](https://ollama.com/) and [LM Studio](https://lmstudio.ai/). Run `roundtable init` and any running local server will be auto-detected.
|
|
249
|
+
|
|
250
|
+
### Which platform should I use?
|
|
251
|
+
|
|
252
|
+
**We recommend Ollama.** Here's why:
|
|
253
|
+
|
|
254
|
+
| | Ollama | LM Studio |
|
|
255
|
+
|---|---|---|
|
|
256
|
+
| Context window | Auto-detected, set programmatically | Manual setup required |
|
|
257
|
+
| Headless / CI | Yes (`ollama serve`) | No (GUI required) |
|
|
258
|
+
| Model switching | Automatic via API | Manual in GUI |
|
|
259
|
+
| Multi-model | Run multiple models concurrently | One model at a time |
|
|
260
|
+
| Setup | `ollama pull model-name` | Download through GUI |
|
|
261
|
+
|
|
262
|
+
Ollama gives TheRoundtAIble full programmatic control — context window size is detected automatically and allocated dynamically per prompt. LM Studio requires you to manually configure Context Length and Response Limit in the GUI.
|
|
263
|
+
|
|
264
|
+
### Hardware requirements
|
|
265
|
+
|
|
266
|
+
Local models run on your GPU. Bigger models = better discussion quality, but more VRAM:
|
|
267
|
+
|
|
268
|
+
| Model size | VRAM needed | Discussion quality |
|
|
269
|
+
|---|---|---|
|
|
270
|
+
| 7B parameters | ~4-6 GB | Basic responses, limited reasoning |
|
|
271
|
+
| 14B parameters | ~8-10 GB | Can participate, but may repeat itself |
|
|
272
|
+
| 30B+ parameters | ~16-24 GB | Meaningful contributions, can hold multi-round debates |
|
|
273
|
+
| 70B+ parameters | ~32-48 GB | Comparable to cloud models |
|
|
274
|
+
|
|
275
|
+
**Our honest take:** Models under 30B struggle with multi-round discussions. They tend to repeat themselves, ignore other knights' arguments, and miss the collaborative spirit (no roasting!). Cloud knights (Claude, Gemini, GPT) handle 100K+ tokens and produce richer debates. Local models shine when you want privacy or offline access — but for best results, mix them with at least one cloud knight.
|
|
276
|
+
|
|
277
|
+
### LM Studio setup
|
|
278
|
+
|
|
279
|
+
If you choose LM Studio, you **must** manually adjust these settings (Developer tab > Model Settings):
|
|
280
|
+
|
|
281
|
+
- **Context Length:** increase to at least 16384 (default 4096 is too small)
|
|
282
|
+
- **Response Limit:** uncheck the limit, or set to 4096+
|
|
283
|
+
|
|
284
|
+
Higher context = more VRAM and slower responses. Find the sweet spot for your GPU.
|
|
285
|
+
|
|
286
|
+
### How it works under the hood
|
|
287
|
+
|
|
288
|
+
- **Ollama:** Uses the native `/api/chat` endpoint with dynamic `num_ctx` — only allocates as much context as the prompt needs, saving GPU memory
|
|
289
|
+
- **LM Studio:** Uses the OpenAI-compatible `/v1/chat/completions` endpoint
|
|
290
|
+
- **Auto-detection:** `roundtable init` probes `localhost:11434` (Ollama) and `localhost:1234` (LM Studio), discovers loaded models, and filters out non-chat models (embeddings, TTS, etc.)
|
|
291
|
+
- **Context budgeting:** The orchestrator detects each local model's context window and adjusts the source code payload so it fits — cloud knights get the full context, local knights get a trimmed version
|
|
292
|
+
|
|
243
293
|
## Known Limitations
|
|
244
294
|
|
|
245
295
|
### `roundtable apply` — BETA, use at own risk
|
package/dist/adapters/base.d.ts
CHANGED
|
@@ -7,6 +7,12 @@ export declare abstract class BaseAdapter {
|
|
|
7
7
|
abstract readonly name: string;
|
|
8
8
|
abstract execute(prompt: string, timeoutMs?: number): Promise<string>;
|
|
9
9
|
abstract isAvailable(): Promise<boolean>;
|
|
10
|
+
/**
|
|
11
|
+
* Max chars for source context injection. Local adapters return a budget
|
|
12
|
+
* based on their detected context window. Cloud adapters return undefined
|
|
13
|
+
* (= use default 200KB).
|
|
14
|
+
*/
|
|
15
|
+
getMaxSourceChars(): number | undefined;
|
|
10
16
|
parseConsensus(response: string, round: number): ConsensusBlock | null;
|
|
11
17
|
}
|
|
12
18
|
//# sourceMappingURL=base.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"base.d.ts","sourceRoot":"","sources":["../../src/adapters/base.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,aAAa,CAAC;AAElD,OAAO,EAAE,YAAY,EAAE,aAAa,EAAE,MAAM,oBAAoB,CAAC;AACjE,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,oBAAoB,CAAC;AAG3D,OAAO,EAAE,YAAY,EAAE,aAAa,EAAE,CAAC;AACvC,YAAY,EAAE,gBAAgB,EAAE,CAAC;AAEjC,8BAAsB,WAAW;IAC/B,QAAQ,CAAC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IAE/B,QAAQ,CAAC,OAAO,CAAC,MAAM,EAAE,MAAM,EAAE,SAAS,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC;IAErE,QAAQ,CAAC,WAAW,IAAI,OAAO,CAAC,OAAO,CAAC;IAExC,cAAc,CAAC,QAAQ,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,GAAG,cAAc,GAAG,IAAI;CAGvE"}
|
|
1
|
+
{"version":3,"file":"base.d.ts","sourceRoot":"","sources":["../../src/adapters/base.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,aAAa,CAAC;AAElD,OAAO,EAAE,YAAY,EAAE,aAAa,EAAE,MAAM,oBAAoB,CAAC;AACjE,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,oBAAoB,CAAC;AAG3D,OAAO,EAAE,YAAY,EAAE,aAAa,EAAE,CAAC;AACvC,YAAY,EAAE,gBAAgB,EAAE,CAAC;AAEjC,8BAAsB,WAAW;IAC/B,QAAQ,CAAC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IAE/B,QAAQ,CAAC,OAAO,CAAC,MAAM,EAAE,MAAM,EAAE,SAAS,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC;IAErE,QAAQ,CAAC,WAAW,IAAI,OAAO,CAAC,OAAO,CAAC;IAExC;;;;OAIG;IACH,iBAAiB,IAAI,MAAM,GAAG,SAAS;IAIvC,cAAc,CAAC,QAAQ,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,GAAG,cAAc,GAAG,IAAI;CAGvE"}
|
package/dist/adapters/base.js
CHANGED
|
@@ -3,6 +3,14 @@ import { AdapterError, classifyError } from "../utils/errors.js";
|
|
|
3
3
|
// Re-export for backward compatibility (orchestrator imports from here)
|
|
4
4
|
export { AdapterError, classifyError };
|
|
5
5
|
export class BaseAdapter {
|
|
6
|
+
/**
|
|
7
|
+
* Max chars for source context injection. Local adapters return a budget
|
|
8
|
+
* based on their detected context window. Cloud adapters return undefined
|
|
9
|
+
* (= use default 200KB).
|
|
10
|
+
*/
|
|
11
|
+
getMaxSourceChars() {
|
|
12
|
+
return undefined;
|
|
13
|
+
}
|
|
6
14
|
parseConsensus(response, round) {
|
|
7
15
|
return parseConsensusFromResponse(response, this.name, round);
|
|
8
16
|
}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"base.js","sourceRoot":"","sources":["../../src/adapters/base.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,0BAA0B,EAAE,MAAM,iBAAiB,CAAC;AAC7D,OAAO,EAAE,YAAY,EAAE,aAAa,EAAE,MAAM,oBAAoB,CAAC;AAGjE,wEAAwE;AACxE,OAAO,EAAE,YAAY,EAAE,aAAa,EAAE,CAAC;AAGvC,MAAM,OAAgB,WAAW;IAO/B,cAAc,CAAC,QAAgB,EAAE,KAAa;QAC5C,OAAO,0BAA0B,CAAC,QAAQ,EAAE,IAAI,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC;IAChE,CAAC;CACF"}
|
|
1
|
+
{"version":3,"file":"base.js","sourceRoot":"","sources":["../../src/adapters/base.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,0BAA0B,EAAE,MAAM,iBAAiB,CAAC;AAC7D,OAAO,EAAE,YAAY,EAAE,aAAa,EAAE,MAAM,oBAAoB,CAAC;AAGjE,wEAAwE;AACxE,OAAO,EAAE,YAAY,EAAE,aAAa,EAAE,CAAC;AAGvC,MAAM,OAAgB,WAAW;IAO/B;;;;OAIG;IACH,iBAAiB;QACf,OAAO,SAAS,CAAC;IACnB,CAAC;IAED,cAAc,CAAC,QAAgB,EAAE,KAAa;QAC5C,OAAO,0BAA0B,CAAC,QAAQ,EAAE,IAAI,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC;IAChE,CAAC;CACF"}
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
import { BaseAdapter } from "./base.js";
|
|
2
|
+
type LocalSource = "Ollama" | "LM Studio" | undefined;
|
|
3
|
+
export declare class LocalLlmAdapter extends BaseAdapter {
|
|
4
|
+
readonly name: string;
|
|
5
|
+
private endpoint;
|
|
6
|
+
private model;
|
|
7
|
+
private source;
|
|
8
|
+
private defaultTimeout;
|
|
9
|
+
private detectedContextTokens;
|
|
10
|
+
constructor(endpoint: string, model: string, name: string, source?: LocalSource, timeoutMs?: number);
|
|
11
|
+
isAvailable(): Promise<boolean>;
|
|
12
|
+
/**
|
|
13
|
+
* Detect the model's context window size.
|
|
14
|
+
* - Ollama: POST /api/show → model_info["*.context_length"]
|
|
15
|
+
* - LM Studio: no reliable API detection (user sets it manually)
|
|
16
|
+
*/
|
|
17
|
+
detectContextWindow(): Promise<number | undefined>;
|
|
18
|
+
/**
|
|
19
|
+
* Max source chars budget based on detected or assumed context window.
|
|
20
|
+
* - Ollama: uses detected context from /api/show
|
|
21
|
+
* - LM Studio: assumes 16384 (common default, can't detect via API)
|
|
22
|
+
* - Unknown: returns undefined (use default 200KB)
|
|
23
|
+
* Reserves 4096 tokens for response + 3000 for system prompt/rounds overhead.
|
|
24
|
+
*/
|
|
25
|
+
getMaxSourceChars(): number | undefined;
|
|
26
|
+
execute(prompt: string, timeoutMs?: number): Promise<string>;
|
|
27
|
+
/**
|
|
28
|
+
* Ollama native API — uses /api/chat with dynamic num_ctx based on prompt size.
|
|
29
|
+
* Only allocates as much context as needed to save GPU memory.
|
|
30
|
+
*/
|
|
31
|
+
private executeOllama;
|
|
32
|
+
/**
|
|
33
|
+
* OpenAI-compatible API — used by LM Studio and unknown sources.
|
|
34
|
+
* Parses LM Studio-specific context window errors into actionable messages.
|
|
35
|
+
*/
|
|
36
|
+
private executeOpenAICompat;
|
|
37
|
+
/**
|
|
38
|
+
* Detect Ollama model context window via /api/show.
|
|
39
|
+
* Looks for context_length in model_info (e.g. "llama.context_length": 32768).
|
|
40
|
+
*/
|
|
41
|
+
private detectOllamaContext;
|
|
42
|
+
/**
|
|
43
|
+
* Detect LM Studio context window errors from response body.
|
|
44
|
+
*/
|
|
45
|
+
private isContextWindowError;
|
|
46
|
+
}
|
|
47
|
+
export {};
|
|
48
|
+
//# sourceMappingURL=local-llm.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"local-llm.d.ts","sourceRoot":"","sources":["../../src/adapters/local-llm.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,WAAW,EAAE,MAAM,WAAW,CAAC;AAGxC,KAAK,WAAW,GAAG,QAAQ,GAAG,WAAW,GAAG,SAAS,CAAC;AAEtD,qBAAa,eAAgB,SAAQ,WAAW;IAC9C,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IAEtB,OAAO,CAAC,QAAQ,CAAS;IACzB,OAAO,CAAC,KAAK,CAAS;IACtB,OAAO,CAAC,MAAM,CAAc;IAC5B,OAAO,CAAC,cAAc,CAAS;IAC/B,OAAO,CAAC,qBAAqB,CAAqB;gBAEtC,QAAQ,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,EAAE,MAAM,CAAC,EAAE,WAAW,EAAE,SAAS,GAAE,MAAgB;IAStG,WAAW,IAAI,OAAO,CAAC,OAAO,CAAC;IAcrC;;;;OAIG;IACG,mBAAmB,IAAI,OAAO,CAAC,MAAM,GAAG,SAAS,CAAC;IAQxD;;;;;;OAMG;IACM,iBAAiB,IAAI,MAAM,GAAG,SAAS;IAc1C,OAAO,CAAC,MAAM,EAAE,MAAM,EAAE,SAAS,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC;IAmBlE;;;OAGG;YACW,aAAa;IAmD3B;;;OAGG;YACW,mBAAmB;IAmDjC;;;OAGG;YACW,mBAAmB;IAgCjC;;OAEG;IACH,OAAO,CAAC,oBAAoB;CAS7B"}
|
|
@@ -0,0 +1,218 @@
|
|
|
1
|
+
import { BaseAdapter } from "./base.js";
|
|
2
|
+
import { classifyError } from "../utils/errors.js";
|
|
3
|
+
export class LocalLlmAdapter extends BaseAdapter {
|
|
4
|
+
name;
|
|
5
|
+
endpoint;
|
|
6
|
+
model;
|
|
7
|
+
source;
|
|
8
|
+
defaultTimeout;
|
|
9
|
+
detectedContextTokens;
|
|
10
|
+
constructor(endpoint, model, name, source, timeoutMs = 120_000) {
|
|
11
|
+
super();
|
|
12
|
+
this.endpoint = endpoint.replace(/\/+$/, ""); // strip trailing slash
|
|
13
|
+
this.model = model;
|
|
14
|
+
this.name = name;
|
|
15
|
+
this.source = source;
|
|
16
|
+
this.defaultTimeout = timeoutMs;
|
|
17
|
+
}
|
|
18
|
+
async isAvailable() {
|
|
19
|
+
try {
|
|
20
|
+
const controller = new AbortController();
|
|
21
|
+
const timer = setTimeout(() => controller.abort(), 3000);
|
|
22
|
+
const response = await fetch(`${this.endpoint}/v1/models`, {
|
|
23
|
+
signal: controller.signal,
|
|
24
|
+
});
|
|
25
|
+
clearTimeout(timer);
|
|
26
|
+
return response.ok;
|
|
27
|
+
}
|
|
28
|
+
catch {
|
|
29
|
+
return false;
|
|
30
|
+
}
|
|
31
|
+
}
|
|
32
|
+
/**
|
|
33
|
+
* Detect the model's context window size.
|
|
34
|
+
* - Ollama: POST /api/show → model_info["*.context_length"]
|
|
35
|
+
* - LM Studio: no reliable API detection (user sets it manually)
|
|
36
|
+
*/
|
|
37
|
+
async detectContextWindow() {
|
|
38
|
+
if (this.source === "Ollama") {
|
|
39
|
+
this.detectedContextTokens = await this.detectOllamaContext();
|
|
40
|
+
}
|
|
41
|
+
// LM Studio: no reliable API, user configures manually
|
|
42
|
+
return this.detectedContextTokens;
|
|
43
|
+
}
|
|
44
|
+
/**
|
|
45
|
+
* Max source chars budget based on detected or assumed context window.
|
|
46
|
+
* - Ollama: uses detected context from /api/show
|
|
47
|
+
* - LM Studio: assumes 16384 (common default, can't detect via API)
|
|
48
|
+
* - Unknown: returns undefined (use default 200KB)
|
|
49
|
+
* Reserves 4096 tokens for response + 3000 for system prompt/rounds overhead.
|
|
50
|
+
*/
|
|
51
|
+
getMaxSourceChars() {
|
|
52
|
+
// Use detected context, or conservative default for LM Studio
|
|
53
|
+
const contextTokens = this.detectedContextTokens
|
|
54
|
+
?? (this.source === "LM Studio" ? 16384 : undefined);
|
|
55
|
+
if (!contextTokens)
|
|
56
|
+
return undefined;
|
|
57
|
+
const responseReserve = 4096;
|
|
58
|
+
const overheadReserve = 3000; // system prompt, chronicle, discussion rounds
|
|
59
|
+
const availableTokens = Math.max(contextTokens - responseReserve - overheadReserve, 2000);
|
|
60
|
+
// ~4 chars per token (rough estimate)
|
|
61
|
+
return availableTokens * 4;
|
|
62
|
+
}
|
|
63
|
+
async execute(prompt, timeoutMs) {
|
|
64
|
+
try {
|
|
65
|
+
if (this.source === "Ollama") {
|
|
66
|
+
return await this.executeOllama(prompt, timeoutMs);
|
|
67
|
+
}
|
|
68
|
+
return await this.executeOpenAICompat(prompt, timeoutMs);
|
|
69
|
+
}
|
|
70
|
+
catch (error) {
|
|
71
|
+
// Retry once on transient "Model reloaded" error (LM Studio reloads model after settings change)
|
|
72
|
+
if (error instanceof Error && error.message.includes("Model reloaded")) {
|
|
73
|
+
await new Promise((r) => setTimeout(r, 3000));
|
|
74
|
+
if (this.source === "Ollama") {
|
|
75
|
+
return this.executeOllama(prompt, timeoutMs);
|
|
76
|
+
}
|
|
77
|
+
return this.executeOpenAICompat(prompt, timeoutMs);
|
|
78
|
+
}
|
|
79
|
+
throw error;
|
|
80
|
+
}
|
|
81
|
+
}
|
|
82
|
+
/**
|
|
83
|
+
* Ollama native API — uses /api/chat with dynamic num_ctx based on prompt size.
|
|
84
|
+
* Only allocates as much context as needed to save GPU memory.
|
|
85
|
+
*/
|
|
86
|
+
async executeOllama(prompt, timeoutMs) {
|
|
87
|
+
const timeout = timeoutMs ?? this.defaultTimeout;
|
|
88
|
+
const controller = new AbortController();
|
|
89
|
+
const timer = setTimeout(() => controller.abort(), timeout);
|
|
90
|
+
// Dynamic num_ctx: prompt tokens + response budget + safety margin
|
|
91
|
+
const estimatedPromptTokens = Math.ceil(prompt.length / 4);
|
|
92
|
+
const responsebudget = 4096;
|
|
93
|
+
const safetyMargin = 512;
|
|
94
|
+
let numCtx = estimatedPromptTokens + responsebudget + safetyMargin;
|
|
95
|
+
// Clamp to model's detected max (if known)
|
|
96
|
+
if (this.detectedContextTokens) {
|
|
97
|
+
numCtx = Math.min(numCtx, this.detectedContextTokens);
|
|
98
|
+
}
|
|
99
|
+
try {
|
|
100
|
+
const response = await fetch(`${this.endpoint}/api/chat`, {
|
|
101
|
+
method: "POST",
|
|
102
|
+
headers: { "Content-Type": "application/json" },
|
|
103
|
+
body: JSON.stringify({
|
|
104
|
+
model: this.model,
|
|
105
|
+
messages: [{ role: "user", content: prompt }],
|
|
106
|
+
stream: false,
|
|
107
|
+
options: { num_ctx: numCtx },
|
|
108
|
+
}),
|
|
109
|
+
signal: controller.signal,
|
|
110
|
+
});
|
|
111
|
+
if (!response.ok) {
|
|
112
|
+
const errorBody = await response.text();
|
|
113
|
+
throw new Error(`Ollama error (${response.status}): ${errorBody}`);
|
|
114
|
+
}
|
|
115
|
+
const data = (await response.json());
|
|
116
|
+
const content = data.message?.content;
|
|
117
|
+
if (!content) {
|
|
118
|
+
throw new Error("Ollama returned empty response");
|
|
119
|
+
}
|
|
120
|
+
return content;
|
|
121
|
+
}
|
|
122
|
+
catch (error) {
|
|
123
|
+
throw classifyError(error, this.name);
|
|
124
|
+
}
|
|
125
|
+
finally {
|
|
126
|
+
clearTimeout(timer);
|
|
127
|
+
}
|
|
128
|
+
}
|
|
129
|
+
/**
|
|
130
|
+
* OpenAI-compatible API — used by LM Studio and unknown sources.
|
|
131
|
+
* Parses LM Studio-specific context window errors into actionable messages.
|
|
132
|
+
*/
|
|
133
|
+
async executeOpenAICompat(prompt, timeoutMs) {
|
|
134
|
+
const timeout = timeoutMs ?? this.defaultTimeout;
|
|
135
|
+
const controller = new AbortController();
|
|
136
|
+
const timer = setTimeout(() => controller.abort(), timeout);
|
|
137
|
+
try {
|
|
138
|
+
// Don't send max_tokens — let the server auto-determine based on remaining
|
|
139
|
+
// context window. Sending max_tokens risks prompt + max_tokens > context_length
|
|
140
|
+
// which LM Studio rejects outright.
|
|
141
|
+
const response = await fetch(`${this.endpoint}/v1/chat/completions`, {
|
|
142
|
+
method: "POST",
|
|
143
|
+
headers: { "Content-Type": "application/json" },
|
|
144
|
+
body: JSON.stringify({
|
|
145
|
+
model: this.model,
|
|
146
|
+
messages: [{ role: "user", content: prompt }],
|
|
147
|
+
}),
|
|
148
|
+
signal: controller.signal,
|
|
149
|
+
});
|
|
150
|
+
if (!response.ok) {
|
|
151
|
+
const errorBody = await response.text();
|
|
152
|
+
// Detect LM Studio context window overflow
|
|
153
|
+
if (this.source === "LM Studio" && this.isContextWindowError(errorBody)) {
|
|
154
|
+
const estimatedTokens = Math.ceil(prompt.length / 4);
|
|
155
|
+
throw new Error(`LM Studio context window too small (prompt needs ~${estimatedTokens} tokens).\n` +
|
|
156
|
+
` Fix: In LM Studio → Developer → Model Settings → increase Context Length.\n` +
|
|
157
|
+
` Also uncheck the Response Limit, or set it higher.\n` +
|
|
158
|
+
` Note: higher context = more VRAM. Find the sweet spot for your GPU.`);
|
|
159
|
+
}
|
|
160
|
+
throw new Error(`Local LLM error (${response.status}): ${errorBody}`);
|
|
161
|
+
}
|
|
162
|
+
const data = (await response.json());
|
|
163
|
+
const content = data.choices?.[0]?.message?.content;
|
|
164
|
+
if (!content) {
|
|
165
|
+
throw new Error("Local LLM returned empty response");
|
|
166
|
+
}
|
|
167
|
+
return content;
|
|
168
|
+
}
|
|
169
|
+
catch (error) {
|
|
170
|
+
throw classifyError(error, this.name);
|
|
171
|
+
}
|
|
172
|
+
finally {
|
|
173
|
+
clearTimeout(timer);
|
|
174
|
+
}
|
|
175
|
+
}
|
|
176
|
+
/**
|
|
177
|
+
* Detect Ollama model context window via /api/show.
|
|
178
|
+
* Looks for context_length in model_info (e.g. "llama.context_length": 32768).
|
|
179
|
+
*/
|
|
180
|
+
async detectOllamaContext() {
|
|
181
|
+
try {
|
|
182
|
+
const controller = new AbortController();
|
|
183
|
+
const timer = setTimeout(() => controller.abort(), 5000);
|
|
184
|
+
const response = await fetch(`${this.endpoint}/api/show`, {
|
|
185
|
+
method: "POST",
|
|
186
|
+
headers: { "Content-Type": "application/json" },
|
|
187
|
+
body: JSON.stringify({ name: this.model }),
|
|
188
|
+
signal: controller.signal,
|
|
189
|
+
});
|
|
190
|
+
clearTimeout(timer);
|
|
191
|
+
if (!response.ok)
|
|
192
|
+
return undefined;
|
|
193
|
+
const data = (await response.json());
|
|
194
|
+
if (data.model_info) {
|
|
195
|
+
for (const [key, value] of Object.entries(data.model_info)) {
|
|
196
|
+
if (key.endsWith(".context_length") && typeof value === "number") {
|
|
197
|
+
return value;
|
|
198
|
+
}
|
|
199
|
+
}
|
|
200
|
+
}
|
|
201
|
+
return undefined;
|
|
202
|
+
}
|
|
203
|
+
catch {
|
|
204
|
+
return undefined;
|
|
205
|
+
}
|
|
206
|
+
}
|
|
207
|
+
/**
|
|
208
|
+
* Detect LM Studio context window errors from response body.
|
|
209
|
+
*/
|
|
210
|
+
isContextWindowError(errorBody) {
|
|
211
|
+
const lower = errorBody.toLowerCase();
|
|
212
|
+
return ((lower.includes("n_keep") && lower.includes("n_ctx")) ||
|
|
213
|
+
lower.includes("context length exceeded") ||
|
|
214
|
+
lower.includes("maximum context length") ||
|
|
215
|
+
lower.includes("too many tokens"));
|
|
216
|
+
}
|
|
217
|
+
}
|
|
218
|
+
//# sourceMappingURL=local-llm.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"local-llm.js","sourceRoot":"","sources":["../../src/adapters/local-llm.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,WAAW,EAAE,MAAM,WAAW,CAAC;AACxC,OAAO,EAAE,aAAa,EAAE,MAAM,oBAAoB,CAAC;AAInD,MAAM,OAAO,eAAgB,SAAQ,WAAW;IACrC,IAAI,CAAS;IAEd,QAAQ,CAAS;IACjB,KAAK,CAAS;IACd,MAAM,CAAc;IACpB,cAAc,CAAS;IACvB,qBAAqB,CAAqB;IAElD,YAAY,QAAgB,EAAE,KAAa,EAAE,IAAY,EAAE,MAAoB,EAAE,YAAoB,OAAO;QAC1G,KAAK,EAAE,CAAC;QACR,IAAI,CAAC,QAAQ,GAAG,QAAQ,CAAC,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC,CAAC,uBAAuB;QACrE,IAAI,CAAC,KAAK,GAAG,KAAK,CAAC;QACnB,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;QACjB,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC;QACrB,IAAI,CAAC,cAAc,GAAG,SAAS,CAAC;IAClC,CAAC;IAED,KAAK,CAAC,WAAW;QACf,IAAI,CAAC;YACH,MAAM,UAAU,GAAG,IAAI,eAAe,EAAE,CAAC;YACzC,MAAM,KAAK,GAAG,UAAU,CAAC,GAAG,EAAE,CAAC,UAAU,CAAC,KAAK,EAAE,EAAE,IAAI,CAAC,CAAC;YACzD,MAAM,QAAQ,GAAG,MAAM,KAAK,CAAC,GAAG,IAAI,CAAC,QAAQ,YAAY,EAAE;gBACzD,MAAM,EAAE,UAAU,CAAC,MAAM;aAC1B,CAAC,CAAC;YACH,YAAY,CAAC,KAAK,CAAC,CAAC;YACpB,OAAO,QAAQ,CAAC,EAAE,CAAC;QACrB,CAAC;QAAC,MAAM,CAAC;YACP,OAAO,KAAK,CAAC;QACf,CAAC;IACH,CAAC;IAED;;;;OAIG;IACH,KAAK,CAAC,mBAAmB;QACvB,IAAI,IAAI,CAAC,MAAM,KAAK,QAAQ,EAAE,CAAC;YAC7B,IAAI,CAAC,qBAAqB,GAAG,MAAM,IAAI,CAAC,mBAAmB,EAAE,CAAC;QAChE,CAAC;QACD,uDAAuD;QACvD,OAAO,IAAI,CAAC,qBAAqB,CAAC;IACpC,CAAC;IAED;;;;;;OAMG;IACM,iBAAiB;QACxB,8DAA8D;QAC9D,MAAM,aAAa,GAAG,IAAI,CAAC,qBAAqB;eAC3C,CAAC,IAAI,CAAC,MAAM,KAAK,WAAW,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC;QAEvD,IAAI,CAAC,aAAa;YAAE,OAAO,SAAS,CAAC;QAErC,MAAM,eAAe,GAAG,IAAI,CAAC;QAC7B,MAAM,eAAe,GAAG,IAAI,CAAC,CAAC,8CAA8C;QAC5E,MAAM,eAAe,GAAG,IAAI,CAAC,GAAG,CAAC,aAAa,GAAG,eAAe,GAAG,eAAe,EAAE,IAAI,CAAC,CAAC;QAC1F,sCAAsC;QACtC,OAAO,eAAe,GAAG,CAAC,CAAC;IAC7B,CAAC;IAED,KAAK,CAAC,OAAO,CAAC,MAAc,EAAE,SAAkB;QAC9C,IAAI,CAAC;YACH,IAAI,IAAI,CAAC,MAAM,KAAK,QAAQ,EAAE,CAAC;gBAC7B,OAAO,MAAM,IAAI,CAAC,aAAa,CAAC,MAAM,EAAE,SAAS,CAAC,CAAC;YACrD,CAAC;YACD,OAAO,MAAM,IAAI,CAAC,mBAAmB,CAAC,MAAM,EAAE,SAAS,CAAC,CAAC;QAC3D,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,iGAAiG;YACjG,IAAI,KAAK,YAAY,KAAK,IAAI,KAAK,CAAC,OAAO,CAAC,QAAQ,CAAC,gBAAgB,CAAC,EAAE,CAAC;gBACvE,MAAM,IAAI,OAAO,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,UAAU,CAAC,CAAC,EAAE,IAAI,CAAC,CAAC,CAAC;gBAC9C,IAAI,IAAI,CAAC,MAAM,KAAK,QAAQ,EAAE,CAAC;oBAC7B,OAAO,IAAI,CAAC,aAAa,CAAC,MAAM,EAAE,SAAS,CAAC,CAAC;gBAC/C,CAAC;gBACD,OAAO,IAAI,CAAC,mBAAmB,CAAC,MAAM,EAAE,SAAS,CAAC,CAAC;YACrD,CAAC;YACD,MAAM,KAAK,CAAC;QACd,CAAC;IACH,CAAC;IAED;;;OAGG;IACK,KAAK,CAAC,aAAa,CAAC,MAAc,EAAE,SAAkB;QAC5D,MAAM,OAAO,GAAG,SAAS,IAAI,IAAI,CAAC,cAAc,CAAC;QACjD,MAAM,UAAU,GAAG,IAAI,eAAe,EAAE,CAAC;QACzC,MAAM,KAAK,GAAG,UAAU,CAAC,GAAG,EAAE,CAAC,UAAU,CAAC,KAAK,EAAE,EAAE,OAAO,CAAC,CAAC;QAE5D,mEAAmE;QACnE,MAAM,qBAAqB,GAAG,IAAI,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;QAC3D,MAAM,cAAc,GAAG,IAAI,CAAC;QAC5B,MAAM,YAAY,GAAG,GAAG,CAAC;QACzB,IAAI,MAAM,GAAG,qBAAqB,GAAG,cAAc,GAAG,YAAY,CAAC;QAEnE,2CAA2C;QAC3C,IAAI,IAAI,CAAC,qBAAqB,EAAE,CAAC;YAC/B,MAAM,GAAG,IAAI,CAAC,GAAG,CAAC,MAAM,EAAE,IAAI,CAAC,qBAAqB,CAAC,CAAC;QACxD,CAAC;QAED,IAAI,CAAC;YACH,MAAM,QAAQ,GAAG,MAAM,KAAK,CAAC,GAAG,IAAI,CAAC,QAAQ,WAAW,EAAE;gBACxD,MAAM,EAAE,MAAM;gBACd,OAAO,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE;gBAC/C,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC;oBACnB,KAAK,EAAE,IAAI,CAAC,KAAK;oBACjB,QAAQ,EAAE,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,EAAE,CAAC;oBAC7C,MAAM,EAAE,KAAK;oBACb,OAAO,EAAE,EAAE,OAAO,EAAE,MAAM,EAAE;iBAC7B,CAAC;gBACF,MAAM,EAAE,UAAU,CAAC,MAAM;aAC1B,CAAC,CAAC;YAEH,IAAI,CAAC,QAAQ,CAAC,EAAE,EAAE,CAAC;gBACjB,MAAM,SAAS,GAAG,MAAM,QAAQ,CAAC,IAAI,EAAE,CAAC;gBACxC,MAAM,IAAI,KAAK,CAAC,iBAAiB,QAAQ,CAAC,MAAM,MAAM,SAAS,EAAE,CAAC,CAAC;YACrE,CAAC;YAED,MAAM,IAAI,GAAG,CAAC,MAAM,QAAQ,CAAC,IAAI,EAAE,CAElC,CAAC;YAEF,MAAM,OAAO,GAAG,IAAI,CAAC,OAAO,EAAE,OAAO,CAAC;YACtC,IAAI,CAAC,OAAO,EAAE,CAAC;gBACb,MAAM,IAAI,KAAK,CAAC,gCAAgC,CAAC,CAAC;YACpD,CAAC;YAED,OAAO,OAAO,CAAC;QACjB,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,MAAM,aAAa,CAAC,KAAK,EAAE,IAAI,CAAC,IAAI,CAAC,CAAC;QACxC,CAAC;gBAAS,CAAC;YACT,YAAY,CAAC,KAAK,CAAC,CAAC;QACtB,CAAC;IACH,CAAC;IAED;;;OAGG;IACK,KAAK,CAAC,mBAAmB,CAAC,MAAc,EAAE,SAAkB;QAClE,MAAM,OAAO,GAAG,SAAS,IAAI,IAAI,CAAC,cAAc,CAAC;QACjD,MAAM,UAAU,GAAG,IAAI,eAAe,EAAE,CAAC;QACzC,MAAM,KAAK,GAAG,UAAU,CAAC,GAAG,EAAE,CAAC,UAAU,CAAC,KAAK,EAAE,EAAE,OAAO,CAAC,CAAC;QAE5D,IAAI,CAAC;YACH,2EAA2E;YAC3E,gFAAgF;YAChF,oCAAoC;YACpC,MAAM,QAAQ,GAAG,MAAM,KAAK,CAAC,GAAG,IAAI,CAAC,QAAQ,sBAAsB,EAAE;gBACnE,MAAM,EAAE,MAAM;gBACd,OAAO,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE;gBAC/C,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC;oBACnB,KAAK,EAAE,IAAI,CAAC,KAAK;oBACjB,QAAQ,EAAE,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,EAAE,CAAC;iBAC9C,CAAC;gBACF,MAAM,EAAE,UAAU,CAAC,MAAM;aAC1B,CAAC,CAAC;YAEH,IAAI,CAAC,QAAQ,CAAC,EAAE,EAAE,CAAC;gBACjB,MAAM,SAAS,GAAG,MAAM,QAAQ,CAAC,IAAI,EAAE,CAAC;gBACxC,2CAA2C;gBAC3C,IAAI,IAAI,CAAC,MAAM,KAAK,WAAW,IAAI,IAAI,CAAC,oBAAoB,CAAC,SAAS,CAAC,EAAE,CAAC;oBACxE,MAAM,eAAe,GAAG,IAAI,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;oBACrD,MAAM,IAAI,KAAK,CACb,qDAAqD,eAAe,aAAa;wBACjF,+EAA+E;wBAC/E,wDAAwD;wBACxD,uEAAuE,CACxE,CAAC;gBACJ,CAAC;gBACD,MAAM,IAAI,KAAK,CAAC,oBAAoB,QAAQ,CAAC,MAAM,MAAM,SAAS,EAAE,CAAC,CAAC;YACxE,CAAC;YAED,MAAM,IAAI,GAAG,CAAC,MAAM,QAAQ,CAAC,IAAI,EAAE,CAElC,CAAC;YAEF,MAAM,OAAO,GAAG,IAAI,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,OAAO,CAAC;YACpD,IAAI,CAAC,OAAO,EAAE,CAAC;gBACb,MAAM,IAAI,KAAK,CAAC,mCAAmC,CAAC,CAAC;YACvD,CAAC;YAED,OAAO,OAAO,CAAC;QACjB,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,MAAM,aAAa,CAAC,KAAK,EAAE,IAAI,CAAC,IAAI,CAAC,CAAC;QACxC,CAAC;gBAAS,CAAC;YACT,YAAY,CAAC,KAAK,CAAC,CAAC;QACtB,CAAC;IACH,CAAC;IAED;;;OAGG;IACK,KAAK,CAAC,mBAAmB;QAC/B,IAAI,CAAC;YACH,MAAM,UAAU,GAAG,IAAI,eAAe,EAAE,CAAC;YACzC,MAAM,KAAK,GAAG,UAAU,CAAC,GAAG,EAAE,CAAC,UAAU,CAAC,KAAK,EAAE,EAAE,IAAI,CAAC,CAAC;YACzD,MAAM,QAAQ,GAAG,MAAM,KAAK,CAAC,GAAG,IAAI,CAAC,QAAQ,WAAW,EAAE;gBACxD,MAAM,EAAE,MAAM;gBACd,OAAO,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE;gBAC/C,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,EAAE,IAAI,EAAE,IAAI,CAAC,KAAK,EAAE,CAAC;gBAC1C,MAAM,EAAE,UAAU,CAAC,MAAM;aAC1B,CAAC,CAAC;YACH,YAAY,CAAC,KAAK,CAAC,CAAC;YAEpB,IAAI,CAAC,QAAQ,CAAC,EAAE;gBAAE,OAAO,SAAS,CAAC;YAEnC,MAAM,IAAI,GAAG,CAAC,MAAM,QAAQ,CAAC,IAAI,EAAE,CAElC,CAAC;YAEF,IAAI,IAAI,CAAC,UAAU,EAAE,CAAC;gBACpB,KAAK,MAAM,CAAC,GAAG,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,IAAI,CAAC,UAAU,CAAC,EAAE,CAAC;oBAC3D,IAAI,GAAG,CAAC,QAAQ,CAAC,iBAAiB,CAAC,IAAI,OAAO,KAAK,KAAK,QAAQ,EAAE,CAAC;wBACjE,OAAO,KAAK,CAAC;oBACf,CAAC;gBACH,CAAC;YACH,CAAC;YAED,OAAO,SAAS,CAAC;QACnB,CAAC;QAAC,MAAM,CAAC;YACP,OAAO,SAAS,CAAC;QACnB,CAAC;IACH,CAAC;IAED;;OAEG;IACK,oBAAoB,CAAC,SAAiB;QAC5C,MAAM,KAAK,GAAG,SAAS,CAAC,WAAW,EAAE,CAAC;QACtC,OAAO,CACL,CAAC,KAAK,CAAC,QAAQ,CAAC,QAAQ,CAAC,IAAI,KAAK,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC;YACrD,KAAK,CAAC,QAAQ,CAAC,yBAAyB,CAAC;YACzC,KAAK,CAAC,QAAQ,CAAC,wBAAwB,CAAC;YACxC,KAAK,CAAC,QAAQ,CAAC,iBAAiB,CAAC,CAClC,CAAC;IACJ,CAAC;CACF"}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"openai-api.js","sourceRoot":"","sources":["../../src/adapters/openai-api.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,WAAW,EAAE,MAAM,WAAW,CAAC;AACxC,OAAO,EAAE,aAAa,EAAE,MAAM,oBAAoB,CAAC;AACnD,OAAO,EAAE,MAAM,EAAE,MAAM,kBAAkB,CAAC;AAE1C,MAAM,OAAO,gBAAiB,SAAQ,WAAW;IACtC,IAAI,GAAG,KAAK,CAAC;IAEd,KAAK,CAAS;IACd,MAAM,CAAS;IACf,cAAc,CAAS;IAE/B,YAAY,QAAgB,SAAS,EAAE,SAAiB,gBAAgB,EAAE,YAAoB,OAAO;QACnG,KAAK,EAAE,CAAC;QACR,IAAI,CAAC,KAAK,GAAG,KAAK,CAAC;QACnB,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC;QACrB,IAAI,CAAC,cAAc,GAAG,SAAS,CAAC;IAClC,CAAC;IAED,KAAK,CAAC,WAAW;QACf,MAAM,GAAG,GAAG,MAAM,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACtC,OAAO,OAAO,GAAG,KAAK,QAAQ,IAAI,GAAG,CAAC,MAAM,GAAG,CAAC,CAAC;IACnD,CAAC;IAED,KAAK,CAAC,OAAO,CAAC,MAAc,EAAE,SAAkB;QAC9C,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACzC,IAAI,CAAC,MAAM,EAAE,CAAC;YACZ,MAAM,aAAa,CACjB,IAAI,KAAK,CAAC,+BAA+B,IAAI,CAAC,MAAM,4BAA4B,CAAC,EACjF,IAAI,CAAC,IAAI,CACV,CAAC;QACJ,CAAC;QAED,MAAM,OAAO,GAAG,SAAS,IAAI,IAAI,CAAC,cAAc,CAAC;QACjD,MAAM,UAAU,GAAG,IAAI,eAAe,EAAE,CAAC;QACzC,MAAM,KAAK,GAAG,UAAU,CAAC,GAAG,EAAE,CAAC,UAAU,CAAC,KAAK,EAAE,EAAE,OAAO,CAAC,CAAC;QAE5D,IAAI,CAAC;YACH,MAAM,QAAQ,GAAG,MAAM,KAAK,CAAC,4CAA4C,EAAE;gBACzE,MAAM,EAAE,MAAM;gBACd,OAAO,EAAE;oBACP,cAAc,EAAE,kBAAkB;oBAClC,aAAa,EAAE,UAAU,MAAM,EAAE;iBAClC;gBACD,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC;oBACnB,KAAK,EAAE,IAAI,CAAC,KAAK;oBACjB,QAAQ,EAAE,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,EAAE,CAAC;oBAC7C,
|
|
1
|
+
{"version":3,"file":"openai-api.js","sourceRoot":"","sources":["../../src/adapters/openai-api.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,WAAW,EAAE,MAAM,WAAW,CAAC;AACxC,OAAO,EAAE,aAAa,EAAE,MAAM,oBAAoB,CAAC;AACnD,OAAO,EAAE,MAAM,EAAE,MAAM,kBAAkB,CAAC;AAE1C,MAAM,OAAO,gBAAiB,SAAQ,WAAW;IACtC,IAAI,GAAG,KAAK,CAAC;IAEd,KAAK,CAAS;IACd,MAAM,CAAS;IACf,cAAc,CAAS;IAE/B,YAAY,QAAgB,SAAS,EAAE,SAAiB,gBAAgB,EAAE,YAAoB,OAAO;QACnG,KAAK,EAAE,CAAC;QACR,IAAI,CAAC,KAAK,GAAG,KAAK,CAAC;QACnB,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC;QACrB,IAAI,CAAC,cAAc,GAAG,SAAS,CAAC;IAClC,CAAC;IAED,KAAK,CAAC,WAAW;QACf,MAAM,GAAG,GAAG,MAAM,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACtC,OAAO,OAAO,GAAG,KAAK,QAAQ,IAAI,GAAG,CAAC,MAAM,GAAG,CAAC,CAAC;IACnD,CAAC;IAED,KAAK,CAAC,OAAO,CAAC,MAAc,EAAE,SAAkB;QAC9C,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACzC,IAAI,CAAC,MAAM,EAAE,CAAC;YACZ,MAAM,aAAa,CACjB,IAAI,KAAK,CAAC,+BAA+B,IAAI,CAAC,MAAM,4BAA4B,CAAC,EACjF,IAAI,CAAC,IAAI,CACV,CAAC;QACJ,CAAC;QAED,MAAM,OAAO,GAAG,SAAS,IAAI,IAAI,CAAC,cAAc,CAAC;QACjD,MAAM,UAAU,GAAG,IAAI,eAAe,EAAE,CAAC;QACzC,MAAM,KAAK,GAAG,UAAU,CAAC,GAAG,EAAE,CAAC,UAAU,CAAC,KAAK,EAAE,EAAE,OAAO,CAAC,CAAC;QAE5D,IAAI,CAAC;YACH,MAAM,QAAQ,GAAG,MAAM,KAAK,CAAC,4CAA4C,EAAE;gBACzE,MAAM,EAAE,MAAM;gBACd,OAAO,EAAE;oBACP,cAAc,EAAE,kBAAkB;oBAClC,aAAa,EAAE,UAAU,MAAM,EAAE;iBAClC;gBACD,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC;oBACnB,KAAK,EAAE,IAAI,CAAC,KAAK;oBACjB,QAAQ,EAAE,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,EAAE,CAAC;oBAC7C,qBAAqB,EAAE,KAAK;iBAC7B,CAAC;gBACF,MAAM,EAAE,UAAU,CAAC,MAAM;aAC1B,CAAC,CAAC;YAEH,IAAI,CAAC,QAAQ,CAAC,EAAE,EAAE,CAAC;gBACjB,MAAM,SAAS,GAAG,MAAM,QAAQ,CAAC,IAAI,EAAE,CAAC;gBACxC,MAAM,IAAI,KAAK,CAAC,qBAAqB,QAAQ,CAAC,MAAM,MAAM,SAAS,EAAE,CAAC,CAAC;YACzE,CAAC;YAED,MAAM,IAAI,GAAG,CAAC,MAAM,QAAQ,CAAC,IAAI,EAAE,CAElC,CAAC;YAEF,MAAM,OAAO,GAAG,IAAI,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,OAAO,CAAC;YACpD,IAAI,CAAC,OAAO,EAAE,CAAC;gBACb,MAAM,IAAI,KAAK,CAAC,oCAAoC,CAAC,CAAC;YACxD,CAAC;YAED,OAAO,OAAO,CAAC;QACjB,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,MAAM,aAAa,CAAC,KAAK,EAAE,IAAI,CAAC,IAAI,CAAC,CAAC;QACxC,CAAC;gBAAS,CAAC;YACT,YAAY,CAAC,KAAK,CAAC,CAAC;QACtB,CAAC;IACH,CAAC;CACF"}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"init.d.ts","sourceRoot":"","sources":["../../src/commands/init.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"init.d.ts","sourceRoot":"","sources":["../../src/commands/init.ts"],"names":[],"mappings":"AA4NA;;GAEG;AACH,wBAAsB,WAAW,CAAC,OAAO,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC,CAsNhE"}
|
package/dist/commands/init.js
CHANGED
|
@@ -6,6 +6,7 @@ import { execa } from "execa";
|
|
|
6
6
|
import chalk from "chalk";
|
|
7
7
|
import ora from "ora";
|
|
8
8
|
import { saveKey, getKey, getKeysPath } from "../utils/keys.js";
|
|
9
|
+
import { detectLocalModels } from "../utils/local-detect.js";
|
|
9
10
|
const rl = () => createInterface({ input: process.stdin, output: process.stdout });
|
|
10
11
|
/**
|
|
11
12
|
* Ask a yes/no question. Returns true for yes.
|
|
@@ -105,14 +106,61 @@ const DEFAULT_CAPABILITIES = {
|
|
|
105
106
|
/**
|
|
106
107
|
* Generate a config.json based on detected tools and user choices.
|
|
107
108
|
*/
|
|
108
|
-
function generateConfig(projectName, language, enabledKnights) {
|
|
109
|
-
|
|
109
|
+
function generateConfig(projectName, language, enabledKnights, localKnights = []) {
|
|
110
|
+
let priority = 1;
|
|
111
|
+
const knights = enabledKnights.map((k) => ({
|
|
110
112
|
name: k.name,
|
|
111
113
|
adapter: k.adapter,
|
|
112
114
|
capabilities: DEFAULT_CAPABILITIES[k.name] || ["general"],
|
|
113
|
-
priority:
|
|
115
|
+
priority: priority++,
|
|
114
116
|
...(k.fallback ? { fallback: k.fallback } : {}),
|
|
115
117
|
}));
|
|
118
|
+
// Append local knights
|
|
119
|
+
for (const lk of localKnights) {
|
|
120
|
+
knights.push({
|
|
121
|
+
name: lk.name,
|
|
122
|
+
adapter: lk.adapter,
|
|
123
|
+
capabilities: DEFAULT_CAPABILITIES[lk.name] || ["code", "logic"],
|
|
124
|
+
priority: priority++,
|
|
125
|
+
});
|
|
126
|
+
}
|
|
127
|
+
// Build adapter_config — start with cloud adapters
|
|
128
|
+
const adapter_config = {
|
|
129
|
+
"claude-cli": {
|
|
130
|
+
command: "claude",
|
|
131
|
+
args: ["-p", "{prompt}", "--print"],
|
|
132
|
+
},
|
|
133
|
+
"claude-api": {
|
|
134
|
+
model: "claude-sonnet-4-6",
|
|
135
|
+
env_key: "ANTHROPIC_API_KEY",
|
|
136
|
+
},
|
|
137
|
+
"gemini-cli": {
|
|
138
|
+
command: "gemini",
|
|
139
|
+
args: ["-p", "{prompt}"],
|
|
140
|
+
},
|
|
141
|
+
"gemini-api": {
|
|
142
|
+
model: "gemini-2.5-flash",
|
|
143
|
+
env_key: "GEMINI_API_KEY",
|
|
144
|
+
},
|
|
145
|
+
"openai-cli": {
|
|
146
|
+
command: "codex",
|
|
147
|
+
args: ["exec", "{prompt}"],
|
|
148
|
+
},
|
|
149
|
+
"openai-api": {
|
|
150
|
+
model: "gpt-5.2",
|
|
151
|
+
env_key: "OPENAI_API_KEY",
|
|
152
|
+
},
|
|
153
|
+
};
|
|
154
|
+
// Add local adapter configs
|
|
155
|
+
for (const lk of localKnights) {
|
|
156
|
+
const localCfg = {
|
|
157
|
+
endpoint: lk.endpoint,
|
|
158
|
+
model: lk.model,
|
|
159
|
+
name: lk.name,
|
|
160
|
+
source: lk.source,
|
|
161
|
+
};
|
|
162
|
+
adapter_config[lk.adapter] = localCfg;
|
|
163
|
+
}
|
|
116
164
|
return {
|
|
117
165
|
version: "1.0",
|
|
118
166
|
project: projectName,
|
|
@@ -127,32 +175,7 @@ function generateConfig(projectName, language, enabledKnights) {
|
|
|
127
175
|
ignore: [".git", "node_modules", "dist", "build", ".next"],
|
|
128
176
|
},
|
|
129
177
|
chronicle: ".roundtable/chronicle.md",
|
|
130
|
-
adapter_config
|
|
131
|
-
"claude-cli": {
|
|
132
|
-
command: "claude",
|
|
133
|
-
args: ["-p", "{prompt}", "--print"],
|
|
134
|
-
},
|
|
135
|
-
"claude-api": {
|
|
136
|
-
model: "claude-sonnet-4-6",
|
|
137
|
-
env_key: "ANTHROPIC_API_KEY",
|
|
138
|
-
},
|
|
139
|
-
"gemini-cli": {
|
|
140
|
-
command: "gemini",
|
|
141
|
-
args: ["-p", "{prompt}"],
|
|
142
|
-
},
|
|
143
|
-
"gemini-api": {
|
|
144
|
-
model: "gemini-2.5-flash",
|
|
145
|
-
env_key: "GEMINI_API_KEY",
|
|
146
|
-
},
|
|
147
|
-
"openai-cli": {
|
|
148
|
-
command: "codex",
|
|
149
|
-
args: ["exec", "{prompt}"],
|
|
150
|
-
},
|
|
151
|
-
"openai-api": {
|
|
152
|
-
model: "gpt-5.2",
|
|
153
|
-
env_key: "OPENAI_API_KEY",
|
|
154
|
-
},
|
|
155
|
-
},
|
|
178
|
+
adapter_config,
|
|
156
179
|
};
|
|
157
180
|
}
|
|
158
181
|
/**
|
|
@@ -178,10 +201,10 @@ export async function initCommand(version) {
|
|
|
178
201
|
const projectName = await askText(" Project name?", dirName);
|
|
179
202
|
// 2. Language
|
|
180
203
|
const language = await askText(" Discussion language?", "nl");
|
|
181
|
-
// 3. Detect tools
|
|
204
|
+
// 3. Detect all knights (CLI tools + local LLM servers)
|
|
182
205
|
console.log("");
|
|
183
206
|
const detectSpinner = ora(" Scouting for available knights...").start();
|
|
184
|
-
const tools = await detectTools();
|
|
207
|
+
const [tools, localModels] = await Promise.all([detectTools(), detectLocalModels()]);
|
|
185
208
|
detectSpinner.succeed(" Scouting complete");
|
|
186
209
|
for (const tool of tools) {
|
|
187
210
|
const icon = tool.available ? chalk.green(" +") : chalk.red(" -");
|
|
@@ -190,6 +213,9 @@ export async function initCommand(version) {
|
|
|
190
213
|
: chalk.dim("not found");
|
|
191
214
|
console.log(`${icon} ${chalk.bold(tool.name)} ${chalk.dim(`(${tool.command})`)} — ${status}`);
|
|
192
215
|
}
|
|
216
|
+
for (const model of localModels) {
|
|
217
|
+
console.log(`${chalk.green(" +")} ${chalk.bold(model.name)} ${chalk.dim(`(${model.source})`)} — ${chalk.green("local")}`);
|
|
218
|
+
}
|
|
193
219
|
// 4. Let user choose which knights to enable
|
|
194
220
|
console.log("");
|
|
195
221
|
const enabledKnights = [];
|
|
@@ -220,21 +246,13 @@ export async function initCommand(version) {
|
|
|
220
246
|
adapter: tool.adapter,
|
|
221
247
|
fallback: FALLBACKS[tool.adapter],
|
|
222
248
|
});
|
|
223
|
-
//
|
|
249
|
+
// Fallback API key — only ask once, remember forever
|
|
224
250
|
const fallbackAdapter = FALLBACKS[tool.adapter];
|
|
225
251
|
const fallbackEnvKey = fallbackAdapter ? API_ENV_KEYS[fallbackAdapter] : undefined;
|
|
226
252
|
if (fallbackEnvKey) {
|
|
227
253
|
const existingFallbackKey = await getKey(fallbackEnvKey);
|
|
228
254
|
if (existingFallbackKey) {
|
|
229
|
-
console.log(chalk.
|
|
230
|
-
const update = await confirm(` Replace existing ${tool.name} fallback API key?`, false);
|
|
231
|
-
if (update) {
|
|
232
|
-
const key = await askSecret(` Enter your new ${tool.name} API key:`);
|
|
233
|
-
if (key) {
|
|
234
|
-
await saveKey(fallbackEnvKey, key);
|
|
235
|
-
console.log(chalk.green(` ✓ ${tool.name} fallback API key updated`));
|
|
236
|
-
}
|
|
237
|
-
}
|
|
255
|
+
console.log(chalk.green(` ✓ ${tool.name} fallback API key already set`));
|
|
238
256
|
}
|
|
239
257
|
else {
|
|
240
258
|
const wantFallback = await confirm(` Set up a fallback API key for ${tool.name}? (used if CLI fails)`, false);
|
|
@@ -256,25 +274,15 @@ export async function initCommand(version) {
|
|
|
256
274
|
const apiAdapter = API_ADAPTERS[tool.adapter] || tool.adapter;
|
|
257
275
|
const envKey = API_ENV_KEYS[apiAdapter];
|
|
258
276
|
if (envKey) {
|
|
259
|
-
// Check if key already exists (env var or keystore)
|
|
260
277
|
const existingKey = await getKey(envKey);
|
|
261
278
|
if (existingKey) {
|
|
262
|
-
console.log(chalk.green(` ✓ ${tool.name} API key
|
|
263
|
-
const update = await confirm(` Replace existing ${tool.name} API key?`, false);
|
|
264
|
-
if (update) {
|
|
265
|
-
const key = await askSecret(` Enter your new ${tool.name} API key:`);
|
|
266
|
-
if (key) {
|
|
267
|
-
await saveKey(envKey, key);
|
|
268
|
-
console.log(chalk.green(` ✓ ${tool.name} API key updated in ${chalk.dim(getKeysPath())}`));
|
|
269
|
-
}
|
|
270
|
-
}
|
|
279
|
+
console.log(chalk.green(` ✓ ${tool.name} API key already set`));
|
|
271
280
|
}
|
|
272
281
|
else {
|
|
273
|
-
// Ask for the API key
|
|
274
282
|
const key = await askSecret(` Enter your ${tool.name} API key:`);
|
|
275
283
|
if (key) {
|
|
276
284
|
await saveKey(envKey, key);
|
|
277
|
-
console.log(chalk.green(` ✓ ${tool.name} API key saved
|
|
285
|
+
console.log(chalk.green(` ✓ ${tool.name} API key saved to ${chalk.dim(getKeysPath())}`));
|
|
278
286
|
}
|
|
279
287
|
else {
|
|
280
288
|
apiKeyReminders.push(` ${envKey} # ${tool.name}`);
|
|
@@ -288,13 +296,38 @@ export async function initCommand(version) {
|
|
|
288
296
|
}
|
|
289
297
|
}
|
|
290
298
|
}
|
|
291
|
-
|
|
299
|
+
// 4b. Seat local LLM models (already detected in step 3)
|
|
300
|
+
const enabledLocalKnights = [];
|
|
301
|
+
for (const model of localModels) {
|
|
302
|
+
const use = await confirm(` Seat ${model.name} at the table?`, true);
|
|
303
|
+
if (use) {
|
|
304
|
+
const slug = model.name.toLowerCase().replace(/[^a-z0-9]+/g, "-").replace(/(^-|-$)/g, "");
|
|
305
|
+
enabledLocalKnights.push({
|
|
306
|
+
name: model.name,
|
|
307
|
+
adapter: `local-llm-${slug}`,
|
|
308
|
+
endpoint: model.endpoint,
|
|
309
|
+
model: model.modelId,
|
|
310
|
+
source: model.source,
|
|
311
|
+
});
|
|
312
|
+
}
|
|
313
|
+
}
|
|
314
|
+
// Show LM Studio setup tip for each seated LM Studio model
|
|
315
|
+
const lmStudioKnights = enabledLocalKnights.filter((k) => k.source === "LM Studio");
|
|
316
|
+
if (lmStudioKnights.length > 0) {
|
|
317
|
+
console.log(chalk.yellow("\n ⚠ LM Studio setup required:"));
|
|
318
|
+
console.log(chalk.white(" The default Context Length (4096) is too small for roundtable prompts."));
|
|
319
|
+
console.log(chalk.white(" → Context Length: increase to at least 16384, or as high as your model/GPU allows"));
|
|
320
|
+
console.log(chalk.white(" → Response Limit: uncheck the limit, or set to 4096+"));
|
|
321
|
+
console.log(chalk.dim(" Find these in: Developer tab → Model Settings"));
|
|
322
|
+
console.log(chalk.dim(" Note: higher context = more VRAM + slower. Find the sweet spot for your setup."));
|
|
323
|
+
}
|
|
324
|
+
if (enabledKnights.length === 0 && enabledLocalKnights.length === 0) {
|
|
292
325
|
console.log(chalk.red("\n A roundtable with no knights is just a table."));
|
|
293
326
|
console.log(chalk.dim(" Re-run `roundtable init` and enable at least one knight."));
|
|
294
327
|
return;
|
|
295
328
|
}
|
|
296
329
|
// 5. Generate config
|
|
297
|
-
const config = generateConfig(projectName, language, enabledKnights);
|
|
330
|
+
const config = generateConfig(projectName, language, enabledKnights, enabledLocalKnights);
|
|
298
331
|
// 6. Create folder structure
|
|
299
332
|
console.log("");
|
|
300
333
|
const structureSpinner = ora(" Forging the roundtable...").start();
|