natureco-cli 5.64.2 → 5.64.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +30 -0
- package/README.md +14 -0
- package/package.json +1 -1
- package/src/commands/config.js +3 -0
- package/src/tools/computer_use_loop.js +122 -45
- package/src/tools/image_generation.js +44 -5
- package/src/tools/media_understanding.js +45 -5
- package/src/tools/video_generation.js +40 -4
- package/src/tools/workflow.js +4 -3
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,36 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to NatureCo CLI will be documented in this file.
|
|
4
4
|
|
|
5
|
+
## [5.64.4] - 2026-07-13 — Unified MiniMax media routing
|
|
6
|
+
|
|
7
|
+
### Added
|
|
8
|
+
- MiniMax chat configurations now automatically expose image understanding through `MiniMax-VL-01`, image generation through `image-01`, and video generation through `MiniMax-Hailuo-2.3` using the existing provider key.
|
|
9
|
+
- Added provider-native GUI vision transports for MiniMax VLM and Anthropic image blocks; OpenAI-compatible and Gemini vision paths remain supported.
|
|
10
|
+
- MiniMax video generation now submits an asynchronous task and polls its status until `Success`, `Fail`, or timeout.
|
|
11
|
+
|
|
12
|
+
### Fixed
|
|
13
|
+
- `media_understanding` no longer rejects MiniMax configurations or incorrectly claims Gemini support without an implementation.
|
|
14
|
+
- Image and video generation automatically select MiniMax when the active provider is MiniMax, while explicit provider overrides remain respected.
|
|
15
|
+
|
|
16
|
+
### Verification
|
|
17
|
+
- Live MiniMax VLM smoke test read a unique string from a synthetic PNG using the existing NatureCo provider key.
|
|
18
|
+
- Added HTTP contract and provider-routing regression tests for MiniMax VLM, Anthropic vision, MiniMax image generation, and MiniMax video generation selection.
|
|
19
|
+
|
|
20
|
+
## [5.64.3] - 2026-07-13 — Evidence-based GUI completion
|
|
21
|
+
|
|
22
|
+
### Fixed
|
|
23
|
+
- `computer_use_loop` no longer trusts the action model's `done` claim as proof of completion.
|
|
24
|
+
- GUI success now requires a real state-changing action, a changed screenshot hash, and a separate visual-verification pass with explicit evidence and at least 80% confidence.
|
|
25
|
+
- Failed GUI tools now display the error reason beside the persistent `✗` status line.
|
|
26
|
+
- Typed values are redacted from GUI step history.
|
|
27
|
+
- MiniMax M-series chat configurations automatically route screenshots to the Token Plan VLM endpoint using the existing provider key; no second API key is required.
|
|
28
|
+
|
|
29
|
+
### Configuration
|
|
30
|
+
- Added optional `guiVisionProviderUrl`, `guiVisionApiKey`, and `guiVisionModel` overrides; MiniMax users use the existing provider configuration by default.
|
|
31
|
+
|
|
32
|
+
### Tests
|
|
33
|
+
- Added regression coverage for no-action claims, unchanged screens, weak evidence, verified completion, automatic MiniMax VLM routing, and dedicated vision-provider overrides.
|
|
34
|
+
|
|
5
35
|
## [5.64.2] - 2026-07-13 — Visible and reliable macOS GUI automation
|
|
6
36
|
|
|
7
37
|
### Fixed
|
package/README.md
CHANGED
|
@@ -51,6 +51,8 @@ natureco code
|
|
|
51
51
|
|
|
52
52
|
| Version | Highlights |
|
|
53
53
|
|---------|-----------|
|
|
54
|
+
| **v5.64.4** | **Unified MiniMax media:** the existing MiniMax key now powers `MiniMax-VL-01` analysis, `image-01` generation, `MiniMax-Hailuo-2.3` video, and verified GUI vision—no second key required. |
|
|
55
|
+
| **v5.64.3** | **Evidence-based GUI completion:** desktop actions require a changed screen plus independent visual evidence before success. MiniMax screenshots use the Token Plan VLM with the existing provider key. |
|
|
54
56
|
| **v5.64.2** | **Reliable macOS GUI automation:** tool names stay visible in REPL transcripts, visual tasks use a verified screenshot loop, MiniMax GUI routing avoids duplicate paths, and failed/unverified actions no longer claim success. |
|
|
55
57
|
| **v5.64.1** | **Reliable follow-ups + lower token cost:** `natureco code` now preserves same-session context across workflow calls and caps repeated history by token budget (1,024 / 2,048 / 8,192). Provider labels are rendered correctly. |
|
|
56
58
|
| **v5.64.0** | **Unified secure agent foundation:** one execution gateway, hard-stop guardrails, schema-validated tools, rollback/checkpoints, sourced memory, resilient channel delivery, OS keychains and encrypted sync. |
|
|
@@ -133,6 +135,18 @@ NatureCo uses progressive disclosure and bounded context instead of sending ever
|
|
|
133
135
|
|
|
134
136
|
For a 32,000-character previous response, the Balanced profile bounds repeated history to about 2,048 tokens instead of roughly 8,000—a reduction of approximately 74% for that repeated context.
|
|
135
137
|
|
|
138
|
+
### 👁️ Verified GUI Automation
|
|
139
|
+
|
|
140
|
+
MiniMax Token Plan users need no extra key: NatureCo automatically sends screenshots to MiniMax VLM while keeping the M-series model for chat. A separate OpenAI-compatible vision provider remains an optional override:
|
|
141
|
+
|
|
142
|
+
```bash
|
|
143
|
+
natureco config set guiVisionProviderUrl https://api.openai.com/v1
|
|
144
|
+
natureco config set guiVisionApiKey "$OPENAI_API_KEY"
|
|
145
|
+
natureco config set guiVisionModel gpt-4.1-mini
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
NatureCo reports GUI success only after a state-changing action, a changed screen, and a separate visual verification with explicit evidence. Purchasing, booking, sending, or other consequential actions should still be reviewed at the final confirmation step.
|
|
149
|
+
|
|
136
150
|
### 📡 10 Messaging Channels
|
|
137
151
|
|
|
138
152
|
| Platform | Connect | Notes |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "natureco-cli",
|
|
3
|
-
"version": "5.64.
|
|
3
|
+
"version": "5.64.4",
|
|
4
4
|
"description": "Terminal-native AI agent CLI with bilingual TR/EN UI, multi-agent orchestration, persistent memory, secure tools and messaging integrations.",
|
|
5
5
|
"bin": {
|
|
6
6
|
"natureco": "bin/natureco.js"
|
package/src/commands/config.js
CHANGED
|
@@ -149,6 +149,9 @@ function config(args) {
|
|
|
149
149
|
apiKey: { type: 'string', description: 'API key for the provider', required: true },
|
|
150
150
|
providerUrl: { type: 'string', description: 'Base URL of the provider API', required: true },
|
|
151
151
|
providerModel: { type: 'string', description: 'Default model identifier', required: false, default: 'gpt-4' },
|
|
152
|
+
guiVisionProviderUrl: { type: 'string', description: 'OpenAI-compatible vision API base URL for GUI screenshot analysis', required: false },
|
|
153
|
+
guiVisionApiKey: { type: 'string', description: 'API key for the GUI vision provider', required: false },
|
|
154
|
+
guiVisionModel: { type: 'string', description: 'Vision-capable model used by computer_use_loop', required: false },
|
|
152
155
|
temperature: { type: 'number', description: 'Sampling temperature (0-2)', required: false, default: 0.7 },
|
|
153
156
|
maxTokens: { type: 'number', description: 'Maximum tokens per response', required: false, default: 2048 },
|
|
154
157
|
systemPrompt: { type: 'string', description: 'Custom system prompt', required: false },
|
|
@@ -9,12 +9,77 @@ const https = require('https');
|
|
|
9
9
|
const fs = require('fs');
|
|
10
10
|
const path = require('path');
|
|
11
11
|
const os = require('os');
|
|
12
|
-
const
|
|
12
|
+
const crypto = require('crypto');
|
|
13
|
+
const { buildChatEndpoint, isMiniMax, isAnthropic } = require('../utils/provider-detect');
|
|
13
14
|
|
|
14
15
|
function loadConfig() {
|
|
15
16
|
try { return JSON.parse(fs.readFileSync(path.join(os.homedir(), '.natureco', 'config.json'), 'utf8')); } catch { return {}; }
|
|
16
17
|
}
|
|
17
18
|
|
|
19
|
+
function resolveVisionConfig(cfg) {
|
|
20
|
+
const main = {
|
|
21
|
+
providerUrl: cfg.providerUrl,
|
|
22
|
+
apiKey: cfg.providerApiKey,
|
|
23
|
+
model: cfg.providerModel || 'default',
|
|
24
|
+
};
|
|
25
|
+
const explicit = cfg.guiVisionProviderUrl && cfg.guiVisionApiKey && cfg.guiVisionModel
|
|
26
|
+
? { providerUrl: cfg.guiVisionProviderUrl, apiKey: cfg.guiVisionApiKey, model: cfg.guiVisionModel }
|
|
27
|
+
: null;
|
|
28
|
+
if (explicit) return { success: true, ...explicit, dedicated: true };
|
|
29
|
+
if (isMiniMax(main.providerUrl) || /^MiniMax-M/i.test(main.model)) {
|
|
30
|
+
if (!main.providerUrl || !main.apiKey) return { success: false, error: 'Provider not configured' };
|
|
31
|
+
return { success: true, ...main, dedicated: false, transport: 'minimax-vlm' };
|
|
32
|
+
}
|
|
33
|
+
if (!main.providerUrl || !main.apiKey) return { success: false, error: 'Provider not configured' };
|
|
34
|
+
return { success: true, ...main, dedicated: false };
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
async function visionCall(providerUrl, apiKey, model, prompt, screenshot) {
|
|
38
|
+
if (isMiniMax(providerUrl) || /^MiniMax-M/i.test(model)) {
|
|
39
|
+
const base = providerUrl.replace(/\/+$/, '').replace(/\/v1$/, '');
|
|
40
|
+
const response = await fetch(base + '/v1/coding_plan/vlm', {
|
|
41
|
+
method: 'POST',
|
|
42
|
+
headers: { Authorization: 'Bearer ' + apiKey, 'Content-Type': 'application/json' },
|
|
43
|
+
body: JSON.stringify({ prompt, image_url: 'data:image/png;base64,' + screenshot.base64 }),
|
|
44
|
+
signal: AbortSignal.timeout(120000),
|
|
45
|
+
});
|
|
46
|
+
if (!response.ok) throw new Error('MiniMax VLM HTTP ' + response.status + ': ' + (await response.text()).slice(0, 300));
|
|
47
|
+
const data = await response.json();
|
|
48
|
+
if (data.base_resp?.status_code) throw new Error(data.base_resp.status_msg || 'MiniMax VLM request failed');
|
|
49
|
+
return String(data.content || '');
|
|
50
|
+
}
|
|
51
|
+
|
|
52
|
+
if (isAnthropic(providerUrl) || /^claude-/i.test(model)) {
|
|
53
|
+
const origin = new URL(providerUrl).origin;
|
|
54
|
+
const response = await fetch(origin + '/v1/messages', {
|
|
55
|
+
method: 'POST',
|
|
56
|
+
headers: { 'x-api-key': apiKey, 'anthropic-version': '2023-06-01', 'Content-Type': 'application/json' },
|
|
57
|
+
body: JSON.stringify({
|
|
58
|
+
model,
|
|
59
|
+
max_tokens: 500,
|
|
60
|
+
messages: [{ role: 'user', content: [
|
|
61
|
+
{ type: 'image', source: { type: 'base64', media_type: 'image/png', data: screenshot.base64 } },
|
|
62
|
+
{ type: 'text', text: prompt },
|
|
63
|
+
] }],
|
|
64
|
+
}),
|
|
65
|
+
signal: AbortSignal.timeout(120000),
|
|
66
|
+
});
|
|
67
|
+
if (!response.ok) throw new Error('Anthropic vision HTTP ' + response.status + ': ' + (await response.text()).slice(0, 300));
|
|
68
|
+
const data = await response.json();
|
|
69
|
+
return String(data.content?.find(item => item.type === 'text')?.text || '');
|
|
70
|
+
}
|
|
71
|
+
|
|
72
|
+
const messages = [{
|
|
73
|
+
role: 'user',
|
|
74
|
+
content: [
|
|
75
|
+
{ type: 'text', text: prompt },
|
|
76
|
+
{ type: 'image_url', image_url: { url: 'data:image/png;base64,' + screenshot.base64 } },
|
|
77
|
+
],
|
|
78
|
+
}];
|
|
79
|
+
const result = await apiCall(providerUrl, apiKey, { model, messages, stream: false, temperature: 0, max_tokens: 500 });
|
|
80
|
+
return String(result.choices?.[0]?.message?.content || '');
|
|
81
|
+
}
|
|
82
|
+
|
|
18
83
|
function apiCall(providerUrl, apiKey, body) {
|
|
19
84
|
return new Promise((resolve, reject) => {
|
|
20
85
|
const endpoint = buildChatEndpoint(providerUrl);
|
|
@@ -38,13 +103,41 @@ function apiCall(providerUrl, apiKey, body) {
|
|
|
38
103
|
});
|
|
39
104
|
}
|
|
40
105
|
|
|
41
|
-
function
|
|
106
|
+
function captureScreenshot() {
|
|
42
107
|
if (os.platform() !== 'darwin') throw new Error('computer_use_loop currently requires macOS');
|
|
43
108
|
const file = path.join(os.tmpdir(), 'ncloop_' + Date.now() + '.png');
|
|
44
109
|
require('child_process').execSync('screencapture -x "' + file + '"', { timeout: 5000 });
|
|
45
110
|
const buf = fs.readFileSync(file);
|
|
46
111
|
fs.unlinkSync(file);
|
|
47
|
-
return
|
|
112
|
+
return {
|
|
113
|
+
base64: buf.toString('base64'),
|
|
114
|
+
hash: crypto.createHash('sha256').update(buf).digest('hex'),
|
|
115
|
+
};
|
|
116
|
+
}
|
|
117
|
+
|
|
118
|
+
function evaluateCompletionEvidence({ mutationCount, initialHash, currentHash, verification }) {
|
|
119
|
+
if (mutationCount < 1) return { verified: false, error: 'No state-changing GUI action was executed' };
|
|
120
|
+
if (!initialHash || !currentHash || initialHash === currentHash) {
|
|
121
|
+
return { verified: false, error: 'The screen did not change after GUI actions' };
|
|
122
|
+
}
|
|
123
|
+
if (!verification || verification.verified !== true) {
|
|
124
|
+
return { verified: false, error: verification?.reason || 'Visual verifier could not confirm the goal' };
|
|
125
|
+
}
|
|
126
|
+
const confidence = Number(verification.confidence || 0);
|
|
127
|
+
if (confidence < 0.8 || !String(verification.evidence || '').trim()) {
|
|
128
|
+
return { verified: false, error: 'Visual verification evidence was insufficient' };
|
|
129
|
+
}
|
|
130
|
+
return { verified: true, evidence: String(verification.evidence).trim(), confidence };
|
|
131
|
+
}
|
|
132
|
+
|
|
133
|
+
async function verifyGoal(providerUrl, apiKey, model, goal, screenshot) {
|
|
134
|
+
const prompt = `You are a strict independent verifier. Inspect only the screenshot. Goal: ${goal}\nReturn JSON only: {"verified":boolean,"confidence":0..1,"evidence":"exact visible evidence","reason":"why not"}. Never infer success from the goal text. If the requested sent message, booking, purchase, or confirmation is not visibly present, verified must be false.`;
|
|
135
|
+
const reply = await visionCall(providerUrl, apiKey, model, prompt, screenshot);
|
|
136
|
+
try { return JSON.parse(reply); }
|
|
137
|
+
catch {
|
|
138
|
+
const match = reply.match(/\{[\s\S]*\}/);
|
|
139
|
+
return match ? JSON.parse(match[0]) : { verified: false, confidence: 0, reason: 'Verifier returned invalid JSON' };
|
|
140
|
+
}
|
|
48
141
|
}
|
|
49
142
|
|
|
50
143
|
function executeAction(action, params) {
|
|
@@ -110,17 +203,14 @@ Actions:
|
|
|
110
203
|
- wait: { "action": "wait" }
|
|
111
204
|
- done: { "action": "done", "reason": "gorev tamamlandi" }
|
|
112
205
|
|
|
206
|
+
"done" yalnız görev sonucunu mevcut ekranda açıkça görüyorsan kullanılabilir. Bir mesajı yazmak gönderildiği anlamına gelmez; gönderilen mesajın konuşmada görünmesi gerekir.
|
|
113
207
|
Sadece JSON yanit ver, baska metin ekleme. Her action'dan sonra otomatik screenshot alinir, sen sadece bir sonraki en mantikli adimi soyle.`;
|
|
114
208
|
|
|
115
209
|
async function loop(goal, maxSteps) {
|
|
116
210
|
const cfg = loadConfig();
|
|
117
|
-
const
|
|
118
|
-
|
|
119
|
-
const model =
|
|
120
|
-
|
|
121
|
-
if (!providerUrl || !apiKey) {
|
|
122
|
-
return { success: false, error: 'Provider not configured' };
|
|
123
|
-
}
|
|
211
|
+
const vision = resolveVisionConfig(cfg);
|
|
212
|
+
if (!vision.success) return vision;
|
|
213
|
+
const { providerUrl, apiKey, model } = vision;
|
|
124
214
|
|
|
125
215
|
if (os.platform() !== 'darwin') {
|
|
126
216
|
return { success: false, error: 'computer_use_loop currently requires macOS' };
|
|
@@ -128,42 +218,20 @@ async function loop(goal, maxSteps) {
|
|
|
128
218
|
|
|
129
219
|
const steps = [];
|
|
130
220
|
let completed = false;
|
|
221
|
+
let initialHash = null;
|
|
222
|
+
let mutationCount = 0;
|
|
223
|
+
let completionEvidence = null;
|
|
131
224
|
for (let i = 0; i < maxSteps; i++) {
|
|
132
225
|
try {
|
|
133
226
|
// 1. Screenshot
|
|
134
|
-
const
|
|
227
|
+
const screenshot = captureScreenshot();
|
|
228
|
+
if (!initialHash) initialHash = screenshot.hash;
|
|
135
229
|
steps.push({ step: i + 1, action: 'screenshot' });
|
|
136
230
|
|
|
137
231
|
// 2. LLM vision analysis
|
|
138
|
-
const messages = [
|
|
139
|
-
{ role: 'system', content: SYSTEM_PROMPT },
|
|
140
|
-
];
|
|
141
|
-
|
|
142
232
|
const history = steps.filter(s => s.action && s.action !== 'screenshot' && s.action !== 'done').slice(-5);
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
'Adim ' + h.step + ': ' + JSON.stringify(h)
|
|
146
|
-
).join('\n') });
|
|
147
|
-
}
|
|
148
|
-
|
|
149
|
-
messages.push({
|
|
150
|
-
role: 'user',
|
|
151
|
-
content: [
|
|
152
|
-
{ type: 'text', text: 'Gorev: ' + goal + '\n\nEkran goruntusunu analiz et. Siradaki action ne?' },
|
|
153
|
-
{ type: 'image_url', image_url: { url: 'data:image/png;base64,' + b64 } },
|
|
154
|
-
],
|
|
155
|
-
});
|
|
156
|
-
|
|
157
|
-
const body = {
|
|
158
|
-
model,
|
|
159
|
-
messages,
|
|
160
|
-
stream: false,
|
|
161
|
-
temperature: 0.2,
|
|
162
|
-
max_tokens: 500,
|
|
163
|
-
};
|
|
164
|
-
|
|
165
|
-
const result = await apiCall(providerUrl, apiKey, body);
|
|
166
|
-
const reply = result.choices?.[0]?.message?.content || '';
|
|
233
|
+
const historyText = history.length > 0 ? '\nOnceki adimlar:\n' + history.map(h => 'Adim ' + h.step + ': ' + JSON.stringify(h)).join('\n') : '';
|
|
234
|
+
const reply = await visionCall(providerUrl, apiKey, model, SYSTEM_PROMPT + '\n\nGorev: ' + goal + historyText + '\n\nEkran goruntusunu analiz et. Siradaki action ne?', screenshot);
|
|
167
235
|
let decision;
|
|
168
236
|
try {
|
|
169
237
|
decision = JSON.parse(reply);
|
|
@@ -176,13 +244,21 @@ async function loop(goal, maxSteps) {
|
|
|
176
244
|
|
|
177
245
|
// 3. Execute
|
|
178
246
|
if (decision.action === 'done') {
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
247
|
+
const verification = await verifyGoal(providerUrl, apiKey, model, goal, screenshot);
|
|
248
|
+
const evidence = evaluateCompletionEvidence({ mutationCount, initialHash, currentHash: screenshot.hash, verification });
|
|
249
|
+
if (evidence.verified) {
|
|
250
|
+
steps.push({ step: i + 1, action: 'verified', evidence: evidence.evidence, confidence: evidence.confidence });
|
|
251
|
+
completionEvidence = evidence;
|
|
252
|
+
completed = true;
|
|
253
|
+
break;
|
|
254
|
+
}
|
|
255
|
+
steps.push({ step: i + 1, action: 'verification_failed', error: evidence.error });
|
|
256
|
+
continue;
|
|
182
257
|
}
|
|
183
258
|
|
|
184
259
|
const execResult = executeAction(decision.action, decision);
|
|
185
|
-
|
|
260
|
+
const safeParams = decision.action === 'type' ? { action: 'type', text: '[redacted]' } : decision;
|
|
261
|
+
steps.push({ step: i + 1, action: decision.action, params: safeParams, result: execResult.success });
|
|
186
262
|
|
|
187
263
|
if (!execResult.success) {
|
|
188
264
|
// Retry once with wait
|
|
@@ -193,6 +269,7 @@ async function loop(goal, maxSteps) {
|
|
|
193
269
|
return { success: false, error: execResult.error, goal, totalSteps: steps.length, steps };
|
|
194
270
|
}
|
|
195
271
|
}
|
|
272
|
+
if (['click', 'type', 'keypress', 'scroll', 'drag'].includes(decision.action)) mutationCount++;
|
|
196
273
|
|
|
197
274
|
// Small delay between actions
|
|
198
275
|
require('child_process').execSync('sleep 0.5');
|
|
@@ -210,7 +287,7 @@ async function loop(goal, maxSteps) {
|
|
|
210
287
|
if (!completed) {
|
|
211
288
|
return { success: false, error: 'Goal was not verified', goal, totalSteps: steps.length, steps };
|
|
212
289
|
}
|
|
213
|
-
return { success: true, goal, totalSteps: steps.length, steps };
|
|
290
|
+
return { success: true, verified: true, evidence: completionEvidence.evidence, confidence: completionEvidence.confidence, goal, totalSteps: steps.length, steps };
|
|
214
291
|
}
|
|
215
292
|
|
|
216
293
|
const name = 'computer_use_loop';
|
|
@@ -228,4 +305,4 @@ async function execute(params) {
|
|
|
228
305
|
return await loop(params.goal, params.maxSteps || 30);
|
|
229
306
|
}
|
|
230
307
|
|
|
231
|
-
module.exports = { name, description, parameters, execute, executeAction };
|
|
308
|
+
module.exports = { name, description, parameters, execute, executeAction, evaluateCompletionEvidence, resolveVisionConfig, visionCall };
|
|
@@ -1,6 +1,31 @@
|
|
|
1
1
|
const { getConfig } = require('../utils/config');
|
|
2
|
+
const fs = require('fs');
|
|
3
|
+
const path = require('path');
|
|
2
4
|
|
|
3
5
|
const PROVIDERS = {
|
|
6
|
+
minimax: {
|
|
7
|
+
name: 'MiniMax image-01',
|
|
8
|
+
async generate({ prompt, n, apiKey, model, aspectRatio, baseUrl }) {
|
|
9
|
+
const response = await fetch(baseUrl + '/v1/image_generation', {
|
|
10
|
+
method: 'POST',
|
|
11
|
+
headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${apiKey}` },
|
|
12
|
+
body: JSON.stringify({ model: model || 'image-01', prompt, n: n || 1, response_format: 'base64', ...(aspectRatio ? { aspect_ratio: aspectRatio } : {}) }),
|
|
13
|
+
signal: AbortSignal.timeout(120000),
|
|
14
|
+
});
|
|
15
|
+
if (!response.ok) throw new Error(`MiniMax image error ${response.status}: ${(await response.text()).slice(0, 300)}`);
|
|
16
|
+
const data = await response.json();
|
|
17
|
+
if (data.base_resp?.status_code) throw new Error(data.base_resp.status_msg || `MiniMax image API error ${data.base_resp.status_code}`);
|
|
18
|
+
const images = data.data?.image_base64 || [];
|
|
19
|
+
if (!images.length) throw new Error('MiniMax image generation returned no images');
|
|
20
|
+
const outDir = path.join(process.cwd(), 'minimax-output');
|
|
21
|
+
fs.mkdirSync(outDir, { recursive: true });
|
|
22
|
+
return images.map((base64, index) => {
|
|
23
|
+
const file = path.join(outDir, `image-${Date.now()}-${index + 1}.png`);
|
|
24
|
+
fs.writeFileSync(file, Buffer.from(base64, 'base64'));
|
|
25
|
+
return { file, mime: 'image/png' };
|
|
26
|
+
});
|
|
27
|
+
}
|
|
28
|
+
},
|
|
4
29
|
// v4.8.4: Pollinations.ai — TAMAMEN ÜCRETSİZ, API key gerektirmez
|
|
5
30
|
// Default provider olarak ayarlandı (herkes kullanabilsin)
|
|
6
31
|
pollinations: {
|
|
@@ -56,6 +81,11 @@ const PROVIDERS = {
|
|
|
56
81
|
}
|
|
57
82
|
};
|
|
58
83
|
|
|
84
|
+
function selectImageProvider(config, params = {}) {
|
|
85
|
+
if (params.provider || config.imageProvider) return params.provider || config.imageProvider;
|
|
86
|
+
return /minimax\.io|minimaxi\.com|minimax\.cn/i.test(config.providerUrl || '') ? 'minimax' : undefined;
|
|
87
|
+
}
|
|
88
|
+
|
|
59
89
|
module.exports = {
|
|
60
90
|
name: 'image_generation',
|
|
61
91
|
description: 'Generate images using AI. Supports OpenAI DALL-E, FAL.ai, and Together AI providers.',
|
|
@@ -63,7 +93,8 @@ module.exports = {
|
|
|
63
93
|
type: 'object',
|
|
64
94
|
properties: {
|
|
65
95
|
prompt: { type: 'string', description: 'Text description of the image to generate' },
|
|
66
|
-
provider: { type: 'string', description: 'Image provider: openai, fal, together (
|
|
96
|
+
provider: { type: 'string', description: 'Image provider: minimax, openai, fal, together, pollinations (auto-detected)', enum: ['minimax', 'openai', 'fal', 'together', 'pollinations'] },
|
|
97
|
+
aspectRatio: { type: 'string', description: 'Aspect ratio such as 1:1, 16:9 or 9:16' },
|
|
67
98
|
model: { type: 'string', description: 'Model override (e.g. fast-sdxl for FAL, FLUX.1-schnell for Together)' },
|
|
68
99
|
size: { type: 'string', description: 'Image size for DALL-E: 1024x1024, 1792x1024, 1024x1792 (default: 1024x1024)' },
|
|
69
100
|
n: { type: 'number', description: 'Number of images to generate (default: 1, max: 4)' }
|
|
@@ -79,14 +110,17 @@ module.exports = {
|
|
|
79
110
|
const openaiKey = params.apiKey || config.openaiApiKey || process.env.OPENAI_API_KEY;
|
|
80
111
|
const falKey = params.apiKey || config.falApiKey || process.env.FAL_KEY;
|
|
81
112
|
const togetherKey = params.apiKey || config.togetherApiKey || process.env.TOGETHER_API_KEY;
|
|
113
|
+
const minimaxKey = params.apiKey || config.providerApiKey || process.env.MINIMAX_API_KEY;
|
|
114
|
+
const isMiniMax = /minimax\.io|minimaxi\.com|minimax\.cn/i.test(config.providerUrl || '');
|
|
82
115
|
|
|
83
116
|
// Explicit provider tercih edilmişse onu kullan
|
|
84
|
-
let provider = params
|
|
117
|
+
let provider = selectImageProvider(config, params) || (isMiniMax && minimaxKey ? 'minimax' : undefined);
|
|
85
118
|
let apiKey;
|
|
86
119
|
|
|
87
120
|
if (provider) {
|
|
88
121
|
// Explicit provider
|
|
89
|
-
if (provider === '
|
|
122
|
+
if (provider === 'minimax') apiKey = minimaxKey;
|
|
123
|
+
else if (provider === 'openai') apiKey = openaiKey;
|
|
90
124
|
else if (provider === 'fal') apiKey = falKey;
|
|
91
125
|
else if (provider === 'together') apiKey = togetherKey;
|
|
92
126
|
else if (provider === 'pollinations') apiKey = 'free'; // key gereksiz
|
|
@@ -114,6 +148,8 @@ module.exports = {
|
|
|
114
148
|
prompt: params.prompt,
|
|
115
149
|
apiKey,
|
|
116
150
|
model: params.model,
|
|
151
|
+
aspectRatio: params.aspectRatio,
|
|
152
|
+
baseUrl: (config.providerUrl || 'https://api.minimax.io').replace(/\/+$/, '').replace(/\/v1$/, ''),
|
|
117
153
|
size: params.size,
|
|
118
154
|
n: params.n
|
|
119
155
|
});
|
|
@@ -122,11 +158,14 @@ module.exports = {
|
|
|
122
158
|
success: true,
|
|
123
159
|
prompt: params.prompt,
|
|
124
160
|
provider: provider,
|
|
125
|
-
images: images.filter(i => i.url).map(i => ({ url: i.url, revisedPrompt: i.revised_prompt })),
|
|
126
|
-
count: images.filter(i => i.url).length
|
|
161
|
+
images: images.filter(i => i.url || i.file).map(i => ({ url: i.url, file: i.file, revisedPrompt: i.revised_prompt })),
|
|
162
|
+
count: images.filter(i => i.url || i.file).length
|
|
127
163
|
};
|
|
128
164
|
} catch (error) {
|
|
129
165
|
return { success: false, error: error.message };
|
|
130
166
|
}
|
|
131
167
|
}
|
|
132
168
|
};
|
|
169
|
+
|
|
170
|
+
module.exports._providers = PROVIDERS;
|
|
171
|
+
module.exports.selectImageProvider = selectImageProvider;
|
|
@@ -11,7 +11,7 @@ module.exports = {
|
|
|
11
11
|
imagePath: { type: 'string', description: 'Local image file path' },
|
|
12
12
|
imageUrl: { type: 'string', description: 'Remote image URL (if no local file)' },
|
|
13
13
|
prompt: { type: 'string', description: 'Analysis prompt (default: "Describe this image in detail")' },
|
|
14
|
-
provider: { type: 'string', description: 'Vision provider: openai, anthropic, groq (
|
|
14
|
+
provider: { type: 'string', description: 'Vision provider: minimax, openai, anthropic, gemini, groq (auto-detected)', enum: ['minimax', 'openai', 'anthropic', 'gemini', 'groq'] },
|
|
15
15
|
model: { type: 'string', description: 'Model override' }
|
|
16
16
|
}
|
|
17
17
|
},
|
|
@@ -20,11 +20,10 @@ module.exports = {
|
|
|
20
20
|
try {
|
|
21
21
|
const config = getConfig();
|
|
22
22
|
const prompt = params.prompt || 'Describe this image in detail';
|
|
23
|
-
let provider = params.provider || config.visionProvider || 'openai';
|
|
24
|
-
|
|
25
|
-
// v5.6.22: MiniMax, Ollama, local provider'lar vision desteklemiyor olabilir
|
|
26
23
|
const providerUrl = (config.providerUrl || '').toLowerCase();
|
|
27
|
-
|
|
24
|
+
let provider = params.provider || config.visionProvider || (providerUrl.includes('minimax') ? 'minimax' : providerUrl.includes('generativelanguage.googleapis.com') || providerUrl.includes('gemini') ? 'gemini' : 'openai');
|
|
25
|
+
|
|
26
|
+
if (!params.provider && (providerUrl.includes('ollama') || providerUrl.includes('localhost'))) {
|
|
28
27
|
return {
|
|
29
28
|
success: false,
|
|
30
29
|
error: `Görsel analiz bu provider (${providerUrl}) için desteklenmiyor. Lütfen OpenAI, Anthropic, Gemini veya Groq provider'ı kullanın.`,
|
|
@@ -58,6 +57,30 @@ module.exports = {
|
|
|
58
57
|
|
|
59
58
|
const dataUrl = params.imageUrl || `data:${mediaType};base64,${imageBase64}`;
|
|
60
59
|
|
|
60
|
+
if (provider === 'minimax') {
|
|
61
|
+
const apiKey = params.apiKey || config.providerApiKey || process.env.MINIMAX_CODE_PLAN_KEY || process.env.MINIMAX_API_KEY;
|
|
62
|
+
if (!apiKey) return { success: false, error: 'MiniMax API key gerekli' };
|
|
63
|
+
let imageDataUrl = dataUrl;
|
|
64
|
+
if (/^https?:\/\//i.test(imageDataUrl)) {
|
|
65
|
+
const downloaded = await fetch(imageDataUrl);
|
|
66
|
+
if (!downloaded.ok) throw new Error(`Görsel indirilemedi: HTTP ${downloaded.status}`);
|
|
67
|
+
const mime = (downloaded.headers.get('content-type') || 'image/jpeg').split(';')[0];
|
|
68
|
+
imageDataUrl = `data:${mime};base64,${Buffer.from(await downloaded.arrayBuffer()).toString('base64')}`;
|
|
69
|
+
}
|
|
70
|
+
const base = (config.providerUrl || 'https://api.minimax.io').replace(/\/+$/, '').replace(/\/v1$/, '');
|
|
71
|
+
const response = await fetch(base + '/v1/coding_plan/vlm', {
|
|
72
|
+
method: 'POST',
|
|
73
|
+
headers: { Authorization: `Bearer ${apiKey}`, 'Content-Type': 'application/json', 'MM-API-Source': 'NatureCo' },
|
|
74
|
+
body: JSON.stringify({ prompt, image_url: imageDataUrl }),
|
|
75
|
+
signal: AbortSignal.timeout(60000),
|
|
76
|
+
});
|
|
77
|
+
if (!response.ok) throw new Error(`MiniMax VLM error ${response.status}: ${(await response.text()).slice(0, 300)}`);
|
|
78
|
+
const data = await response.json();
|
|
79
|
+
if (data.base_resp?.status_code) throw new Error(data.base_resp.status_msg || `MiniMax VLM API error ${data.base_resp.status_code}`);
|
|
80
|
+
if (!data.content) throw new Error('MiniMax VLM boş yanıt döndürdü');
|
|
81
|
+
return { success: true, provider: 'minimax', model: 'MiniMax-VL-01', analysis: data.content };
|
|
82
|
+
}
|
|
83
|
+
|
|
61
84
|
if (provider === 'openai') {
|
|
62
85
|
const apiKey = params.apiKey || config.openaiApiKey || process.env.OPENAI_API_KEY;
|
|
63
86
|
if (!apiKey) return { success: false, error: 'OpenAI API key gerekli' };
|
|
@@ -83,6 +106,23 @@ module.exports = {
|
|
|
83
106
|
return { success: true, provider: 'openai', analysis: data.choices?.[0]?.message?.content || '' };
|
|
84
107
|
}
|
|
85
108
|
|
|
109
|
+
if (provider === 'gemini') {
|
|
110
|
+
const apiKey = params.apiKey || config.providerApiKey || process.env.GEMINI_API_KEY;
|
|
111
|
+
if (!apiKey) return { success: false, error: 'Gemini API key gerekli' };
|
|
112
|
+
const base = (config.providerUrl || 'https://generativelanguage.googleapis.com/v1beta').replace(/\/+$/, '');
|
|
113
|
+
const response = await fetch(base + '/openai/chat/completions', {
|
|
114
|
+
method: 'POST',
|
|
115
|
+
headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${apiKey}` },
|
|
116
|
+
body: JSON.stringify({ model: params.model || config.providerModel, messages: [{ role: 'user', content: [
|
|
117
|
+
{ type: 'text', text: prompt },
|
|
118
|
+
{ type: 'image_url', image_url: { url: dataUrl } },
|
|
119
|
+
] }], max_tokens: 1000 }),
|
|
120
|
+
});
|
|
121
|
+
if (!response.ok) throw new Error(`Gemini vision error ${response.status}`);
|
|
122
|
+
const data = await response.json();
|
|
123
|
+
return { success: true, provider: 'gemini', analysis: data.choices?.[0]?.message?.content || '' };
|
|
124
|
+
}
|
|
125
|
+
|
|
86
126
|
if (provider === 'anthropic') {
|
|
87
127
|
const apiKey = params.apiKey || config.anthropicApiKey || process.env.ANTHROPIC_API_KEY;
|
|
88
128
|
if (!apiKey) return { success: false, error: 'Anthropic API key gerekli' };
|
|
@@ -1,6 +1,31 @@
|
|
|
1
1
|
const { getConfig } = require('../utils/config');
|
|
2
2
|
|
|
3
3
|
const PROVIDERS = {
|
|
4
|
+
minimax: {
|
|
5
|
+
name: 'MiniMax Hailuo',
|
|
6
|
+
async generate({ prompt, apiKey, model, baseUrl }) {
|
|
7
|
+
const headers = { 'Content-Type': 'application/json', Authorization: `Bearer ${apiKey}` };
|
|
8
|
+
const submittedResponse = await fetch(baseUrl + '/v1/video_generation', {
|
|
9
|
+
method: 'POST', headers,
|
|
10
|
+
body: JSON.stringify({ model: model || 'MiniMax-Hailuo-2.3', prompt }),
|
|
11
|
+
signal: AbortSignal.timeout(120000),
|
|
12
|
+
});
|
|
13
|
+
if (!submittedResponse.ok) throw new Error(`MiniMax video error ${submittedResponse.status}: ${(await submittedResponse.text()).slice(0, 300)}`);
|
|
14
|
+
const submitted = await submittedResponse.json();
|
|
15
|
+
if (submitted.base_resp?.status_code) throw new Error(submitted.base_resp.status_msg || 'MiniMax video submission failed');
|
|
16
|
+
if (!submitted.task_id) throw new Error('MiniMax video response missing task_id');
|
|
17
|
+
for (let attempt = 0; attempt < 120; attempt++) {
|
|
18
|
+
await new Promise(resolve => setTimeout(resolve, 3000));
|
|
19
|
+
const statusResponse = await fetch(baseUrl + '/v1/query/video_generation?task_id=' + encodeURIComponent(submitted.task_id), { headers, signal: AbortSignal.timeout(30000) });
|
|
20
|
+
if (!statusResponse.ok) throw new Error(`MiniMax video status error ${statusResponse.status}`);
|
|
21
|
+
const status = await statusResponse.json();
|
|
22
|
+
if (status.base_resp?.status_code) throw new Error(status.base_resp.status_msg || 'MiniMax video generation failed');
|
|
23
|
+
if (status.status === 'Fail') throw new Error(status.base_resp?.status_msg || 'MiniMax video generation failed');
|
|
24
|
+
if (status.status === 'Success') return [{ url: status.video_url || null, fileId: status.file_id || null, taskId: submitted.task_id }];
|
|
25
|
+
}
|
|
26
|
+
throw new Error(`MiniMax video task ${submitted.task_id} timed out`);
|
|
27
|
+
}
|
|
28
|
+
},
|
|
4
29
|
runway: {
|
|
5
30
|
name: 'RunwayML',
|
|
6
31
|
async generate({ prompt, apiKey, model }) {
|
|
@@ -21,6 +46,11 @@ const PROVIDERS = {
|
|
|
21
46
|
}
|
|
22
47
|
};
|
|
23
48
|
|
|
49
|
+
function selectVideoProvider(config, params = {}) {
|
|
50
|
+
if (params.provider) return params.provider;
|
|
51
|
+
return /minimax\.io|minimaxi\.com|minimax\.cn/i.test(config.providerUrl || '') ? 'minimax' : 'runway';
|
|
52
|
+
}
|
|
53
|
+
|
|
24
54
|
module.exports = {
|
|
25
55
|
name: 'video_generation',
|
|
26
56
|
description: 'Generate videos using AI. Supports RunwayML provider.',
|
|
@@ -28,7 +58,7 @@ module.exports = {
|
|
|
28
58
|
type: 'object',
|
|
29
59
|
properties: {
|
|
30
60
|
prompt: { type: 'string', description: 'Text description of the video to generate' },
|
|
31
|
-
provider: { type: 'string', description: 'Video provider: runway (
|
|
61
|
+
provider: { type: 'string', description: 'Video provider: minimax or runway (auto-detected)', enum: ['minimax', 'runway'] },
|
|
32
62
|
model: { type: 'string', description: 'Model override (default: gen3a_turbo)' }
|
|
33
63
|
},
|
|
34
64
|
required: ['prompt']
|
|
@@ -37,14 +67,16 @@ module.exports = {
|
|
|
37
67
|
async execute(params) {
|
|
38
68
|
try {
|
|
39
69
|
const config = getConfig();
|
|
40
|
-
const provider = params
|
|
70
|
+
const provider = selectVideoProvider(config, params);
|
|
41
71
|
|
|
42
72
|
const providerConfig = PROVIDERS[provider];
|
|
43
73
|
if (!providerConfig) {
|
|
44
74
|
return { success: false, error: `Desteklenmeyen provider: ${provider}` };
|
|
45
75
|
}
|
|
46
76
|
|
|
47
|
-
const apiKey =
|
|
77
|
+
const apiKey = provider === 'minimax'
|
|
78
|
+
? (params.apiKey || config.providerApiKey || process.env.MINIMAX_API_KEY)
|
|
79
|
+
: (params.apiKey || config.runwayApiKey || process.env.RUNWAY_API_KEY);
|
|
48
80
|
if (!apiKey) {
|
|
49
81
|
return {
|
|
50
82
|
success: false,
|
|
@@ -55,7 +87,8 @@ module.exports = {
|
|
|
55
87
|
const videos = await providerConfig.generate({
|
|
56
88
|
prompt: params.prompt,
|
|
57
89
|
apiKey,
|
|
58
|
-
model: params.model
|
|
90
|
+
model: params.model,
|
|
91
|
+
baseUrl: (config.providerUrl || 'https://api.minimax.io').replace(/\/+$/, '').replace(/\/v1$/, '')
|
|
59
92
|
});
|
|
60
93
|
|
|
61
94
|
return {
|
|
@@ -70,3 +103,6 @@ module.exports = {
|
|
|
70
103
|
}
|
|
71
104
|
}
|
|
72
105
|
};
|
|
106
|
+
|
|
107
|
+
module.exports._providers = PROVIDERS;
|
|
108
|
+
module.exports.selectVideoProvider = selectVideoProvider;
|
package/src/tools/workflow.js
CHANGED
|
@@ -348,12 +348,13 @@ async function workflow(params) {
|
|
|
348
348
|
const label = TOOL_LABEL[ev.tool] || ev.tool;
|
|
349
349
|
const hint = briefHint(ev.args);
|
|
350
350
|
process.stdout.write('\n\x1b[2m 🔧 ' + label + (hint ? ' · ' + hint : '') + '\x1b[0m');
|
|
351
|
-
} else {
|
|
352
|
-
const rec = (ev.records || [])[0] || {};
|
|
351
|
+
} else {
|
|
352
|
+
const rec = (ev.records || [])[0] || {};
|
|
353
353
|
// Satırı burada bitir. Sonraki model çağrısının thinking göstergesi
|
|
354
354
|
// mevcut satırı \r + erase-line ile temizlediği için newline yoksa
|
|
355
355
|
// kullanıcı araç adını ve sonucunu hiç göremiyordu.
|
|
356
|
-
|
|
356
|
+
const error = rec.status === 'done' ? '' : String(rec.error || rec.result?.error || '').replace(/\s+/g, ' ').slice(0, 100);
|
|
357
|
+
process.stdout.write((rec.status === 'done' ? ' \x1b[32m✓\x1b[0m' : ' \x1b[31m✗\x1b[0m') + (error ? ' \x1b[31m' + error + '\x1b[0m' : '') + '\n');
|
|
357
358
|
}
|
|
358
359
|
} : null;
|
|
359
360
|
|