natureco-cli 5.64.1 → 5.64.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -2,6 +2,48 @@
2
2
 
3
3
  All notable changes to NatureCo CLI will be documented in this file.
4
4
 
5
+ ## [5.64.4] - 2026-07-13 — Unified MiniMax media routing
6
+
7
+ ### Added
8
+ - MiniMax chat configurations now automatically expose image understanding through `MiniMax-VL-01`, image generation through `image-01`, and video generation through `MiniMax-Hailuo-2.3` using the existing provider key.
9
+ - Added provider-native GUI vision transports for MiniMax VLM and Anthropic image blocks; OpenAI-compatible and Gemini vision paths remain supported.
10
+ - MiniMax video generation now submits an asynchronous task and polls its status until `Success`, `Fail`, or timeout.
11
+
12
+ ### Fixed
13
+ - `media_understanding` no longer rejects MiniMax configurations or incorrectly claims Gemini support without an implementation.
14
+ - Image and video generation automatically select MiniMax when the active provider is MiniMax, while explicit provider overrides remain respected.
15
+
16
+ ### Verification
17
+ - Live MiniMax VLM smoke test read a unique string from a synthetic PNG using the existing NatureCo provider key.
18
+ - Added HTTP contract and provider-routing regression tests for MiniMax VLM, Anthropic vision, MiniMax image generation, and MiniMax video generation selection.
19
+
20
+ ## [5.64.3] - 2026-07-13 — Evidence-based GUI completion
21
+
22
+ ### Fixed
23
+ - `computer_use_loop` no longer trusts the action model's `done` claim as proof of completion.
24
+ - GUI success now requires a real state-changing action, a changed screenshot hash, and a separate visual-verification pass with explicit evidence and at least 80% confidence.
25
+ - Failed GUI tools now display the error reason beside the persistent `✗` status line.
26
+ - Typed values are redacted from GUI step history.
27
+ - MiniMax M-series chat configurations automatically route screenshots to the Token Plan VLM endpoint using the existing provider key; no second API key is required.
28
+
29
+ ### Configuration
30
+ - Added optional `guiVisionProviderUrl`, `guiVisionApiKey`, and `guiVisionModel` overrides; MiniMax users use the existing provider configuration by default.
31
+
32
+ ### Tests
33
+ - Added regression coverage for no-action claims, unchanged screens, weak evidence, verified completion, automatic MiniMax VLM routing, and dedicated vision-provider overrides.
34
+
35
+ ## [5.64.2] - 2026-07-13 — Visible and reliable macOS GUI automation
36
+
37
+ ### Fixed
38
+ - REPL workflow tool activity no longer disappears when the next thinking indicator starts; every tool name and its success/failure marker remains visible in the transcript.
39
+ - Multi-step visual desktop tasks are routed to the screenshot-driven `computer_use_loop` instead of blind single click/type calls.
40
+ - The visual loop now uses the canonical provider endpoint builder, avoiding duplicated `/v1/v1` paths with MiniMax configurations.
41
+ - macOS special keys such as Enter, Tab and Escape now use correct AppleScript key codes.
42
+ - Screenshot capture and GUI-loop failures can no longer be reported as successful completion; missing files, failed actions and unverified max-step exits return explicit failures.
43
+
44
+ ### Tests
45
+ - Added regression coverage for GUI-loop action failures and MiniMax endpoint construction with and without a trailing `/v1`.
46
+
5
47
  ## [5.64.1] - 2026-07-13 — Reliable coding context and token-budgeted follow-ups
6
48
 
7
49
  ### Fixed
package/README.md CHANGED
@@ -51,6 +51,9 @@ natureco code
51
51
 
52
52
  | Version | Highlights |
53
53
  |---------|-----------|
54
+ | **v5.64.4** | **Unified MiniMax media:** the existing MiniMax key now powers `MiniMax-VL-01` analysis, `image-01` generation, `MiniMax-Hailuo-2.3` video, and verified GUI vision—no second key required. |
55
+ | **v5.64.3** | **Evidence-based GUI completion:** desktop actions require a changed screen plus independent visual evidence before success. MiniMax screenshots use the Token Plan VLM with the existing provider key. |
56
+ | **v5.64.2** | **Reliable macOS GUI automation:** tool names stay visible in REPL transcripts, visual tasks use a verified screenshot loop, MiniMax GUI routing avoids duplicate paths, and failed/unverified actions no longer claim success. |
54
57
  | **v5.64.1** | **Reliable follow-ups + lower token cost:** `natureco code` now preserves same-session context across workflow calls and caps repeated history by token budget (1,024 / 2,048 / 8,192). Provider labels are rendered correctly. |
55
58
  | **v5.64.0** | **Unified secure agent foundation:** one execution gateway, hard-stop guardrails, schema-validated tools, rollback/checkpoints, sourced memory, resilient channel delivery, OS keychains and encrypted sync. |
56
59
  | **v5.43.0** | **Security:** 9 vulnerabilities fixed in a 3-round audit (RCE chain, admin-rpc auth, cron persistence, channel access control). See [`SECURITY_AUDIT_SUMMARY.md`](SECURITY_AUDIT_SUMMARY.md). |
@@ -132,6 +135,18 @@ NatureCo uses progressive disclosure and bounded context instead of sending ever
132
135
 
133
136
  For a 32,000-character previous response, the Balanced profile bounds repeated history to about 2,048 tokens instead of roughly 8,000—a reduction of approximately 74% for that repeated context.
134
137
 
138
+ ### 👁️ Verified GUI Automation
139
+
140
+ MiniMax Token Plan users need no extra key: NatureCo automatically sends screenshots to MiniMax VLM while keeping the M-series model for chat. A separate OpenAI-compatible vision provider remains an optional override:
141
+
142
+ ```bash
143
+ natureco config set guiVisionProviderUrl https://api.openai.com/v1
144
+ natureco config set guiVisionApiKey "$OPENAI_API_KEY"
145
+ natureco config set guiVisionModel gpt-4.1-mini
146
+ ```
147
+
148
+ NatureCo reports GUI success only after a state-changing action, a changed screen, and a separate visual verification with explicit evidence. Purchasing, booking, sending, or other consequential actions should still be reviewed at the final confirmation step.
149
+
135
150
  ### 📡 10 Messaging Channels
136
151
 
137
152
  | Platform | Connect | Notes |
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "natureco-cli",
3
- "version": "5.64.1",
3
+ "version": "5.64.4",
4
4
  "description": "Terminal-native AI agent CLI with bilingual TR/EN UI, multi-agent orchestration, persistent memory, secure tools and messaging integrations.",
5
5
  "bin": {
6
6
  "natureco": "bin/natureco.js"
@@ -149,6 +149,9 @@ function config(args) {
149
149
  apiKey: { type: 'string', description: 'API key for the provider', required: true },
150
150
  providerUrl: { type: 'string', description: 'Base URL of the provider API', required: true },
151
151
  providerModel: { type: 'string', description: 'Default model identifier', required: false, default: 'gpt-4' },
152
+ guiVisionProviderUrl: { type: 'string', description: 'OpenAI-compatible vision API base URL for GUI screenshot analysis', required: false },
153
+ guiVisionApiKey: { type: 'string', description: 'API key for the GUI vision provider', required: false },
154
+ guiVisionModel: { type: 'string', description: 'Vision-capable model used by computer_use_loop', required: false },
152
155
  temperature: { type: 'number', description: 'Sampling temperature (0-2)', required: false, default: 0.7 },
153
156
  maxTokens: { type: 'number', description: 'Maximum tokens per response', required: false, default: 2048 },
154
157
  systemPrompt: { type: 'string', description: 'Custom system prompt', required: false },
@@ -77,7 +77,9 @@ async function computerUse(params) {
77
77
  const outputFile = file || path.join(os.tmpdir(), 'natureco_screen_' + Date.now() + '.png');
78
78
  try {
79
79
  if (PLATFORM === 'darwin') {
80
- spawnSync('screencapture', ['-x', outputFile], { timeout: 5000 });
80
+ const capture = spawnSync('screencapture', ['-x', outputFile], { timeout: 5000, encoding: 'utf8' });
81
+ if (capture.error) throw capture.error;
82
+ if (capture.status !== 0) throw new Error(capture.stderr || `screencapture exit ${capture.status}`);
81
83
  } else if (PLATFORM === 'win32') {
82
84
  spawnSync('powershell', ['-Command',
83
85
  'Add-Type -AssemblyName System.Windows.Forms; ' +
@@ -89,7 +91,13 @@ async function computerUse(params) {
89
91
  } else {
90
92
  spawnSync('import', ['-window', 'root', outputFile], { timeout: 5000 });
91
93
  }
92
- return { success: true, file: outputFile, platform: PLATFORM, note: 'Ekran goruntusu alindi' };
94
+ if (!fs.existsSync(outputFile)) throw new Error('Screenshot file was not created');
95
+ return {
96
+ success: true,
97
+ file: outputFile,
98
+ platform: PLATFORM,
99
+ note: 'Screenshot saved. This tool returns a file path, not visual analysis; use computer_use_loop for autonomous visual interaction.',
100
+ };
93
101
  } catch (e) {
94
102
  return { success: false, error: 'Screenshot hatasi: ' + e.message };
95
103
  }
@@ -151,9 +159,10 @@ async function computerUse(params) {
151
159
  }
152
160
 
153
161
  if (KEY_MAP_DARWIN[actualKey]) {
162
+ const keyCodes = { return: 36, tab: 48, escape: 53, up: 126, down: 125, left: 123, right: 124, delete: 51, forwarddelete: 117, home: 115, end: 119, 'page up': 116, 'page down': 121, space: 49 };
154
163
  const keyName = KEY_MAP_DARWIN[actualKey];
155
164
  const usingClause = mods.length > 0 ? ' using {' + mods.join(', ') + '}' : '';
156
- const r = osaScript('tell application "System Events" to keystroke ' + keyName + usingClause);
165
+ const r = osaScript('tell application "System Events" to key code ' + keyCodes[keyName] + usingClause);
157
166
  if (!r.success) return r;
158
167
  } else if (actualKey) {
159
168
  const usingClause = mods.length > 0 ? ' using {' + mods.join(', ') + '}' : '';
@@ -9,22 +9,80 @@ const https = require('https');
9
9
  const fs = require('fs');
10
10
  const path = require('path');
11
11
  const os = require('os');
12
+ const crypto = require('crypto');
13
+ const { buildChatEndpoint, isMiniMax, isAnthropic } = require('../utils/provider-detect');
12
14
 
13
15
  function loadConfig() {
14
16
  try { return JSON.parse(fs.readFileSync(path.join(os.homedir(), '.natureco', 'config.json'), 'utf8')); } catch { return {}; }
15
17
  }
16
18
 
17
- function isMiniMax(url) { return url && (url.includes('minimax.io') || url.includes('minimaxi.com') || url.includes('minimax.cn')); }
18
- function isGemini(url) { return url && (url.includes('generativelanguage.googleapis.com') || url.includes('gemini')); }
19
+ function resolveVisionConfig(cfg) {
20
+ const main = {
21
+ providerUrl: cfg.providerUrl,
22
+ apiKey: cfg.providerApiKey,
23
+ model: cfg.providerModel || 'default',
24
+ };
25
+ const explicit = cfg.guiVisionProviderUrl && cfg.guiVisionApiKey && cfg.guiVisionModel
26
+ ? { providerUrl: cfg.guiVisionProviderUrl, apiKey: cfg.guiVisionApiKey, model: cfg.guiVisionModel }
27
+ : null;
28
+ if (explicit) return { success: true, ...explicit, dedicated: true };
29
+ if (isMiniMax(main.providerUrl) || /^MiniMax-M/i.test(main.model)) {
30
+ if (!main.providerUrl || !main.apiKey) return { success: false, error: 'Provider not configured' };
31
+ return { success: true, ...main, dedicated: false, transport: 'minimax-vlm' };
32
+ }
33
+ if (!main.providerUrl || !main.apiKey) return { success: false, error: 'Provider not configured' };
34
+ return { success: true, ...main, dedicated: false };
35
+ }
36
+
37
+ async function visionCall(providerUrl, apiKey, model, prompt, screenshot) {
38
+ if (isMiniMax(providerUrl) || /^MiniMax-M/i.test(model)) {
39
+ const base = providerUrl.replace(/\/+$/, '').replace(/\/v1$/, '');
40
+ const response = await fetch(base + '/v1/coding_plan/vlm', {
41
+ method: 'POST',
42
+ headers: { Authorization: 'Bearer ' + apiKey, 'Content-Type': 'application/json' },
43
+ body: JSON.stringify({ prompt, image_url: 'data:image/png;base64,' + screenshot.base64 }),
44
+ signal: AbortSignal.timeout(120000),
45
+ });
46
+ if (!response.ok) throw new Error('MiniMax VLM HTTP ' + response.status + ': ' + (await response.text()).slice(0, 300));
47
+ const data = await response.json();
48
+ if (data.base_resp?.status_code) throw new Error(data.base_resp.status_msg || 'MiniMax VLM request failed');
49
+ return String(data.content || '');
50
+ }
51
+
52
+ if (isAnthropic(providerUrl) || /^claude-/i.test(model)) {
53
+ const origin = new URL(providerUrl).origin;
54
+ const response = await fetch(origin + '/v1/messages', {
55
+ method: 'POST',
56
+ headers: { 'x-api-key': apiKey, 'anthropic-version': '2023-06-01', 'Content-Type': 'application/json' },
57
+ body: JSON.stringify({
58
+ model,
59
+ max_tokens: 500,
60
+ messages: [{ role: 'user', content: [
61
+ { type: 'image', source: { type: 'base64', media_type: 'image/png', data: screenshot.base64 } },
62
+ { type: 'text', text: prompt },
63
+ ] }],
64
+ }),
65
+ signal: AbortSignal.timeout(120000),
66
+ });
67
+ if (!response.ok) throw new Error('Anthropic vision HTTP ' + response.status + ': ' + (await response.text()).slice(0, 300));
68
+ const data = await response.json();
69
+ return String(data.content?.find(item => item.type === 'text')?.text || '');
70
+ }
71
+
72
+ const messages = [{
73
+ role: 'user',
74
+ content: [
75
+ { type: 'text', text: prompt },
76
+ { type: 'image_url', image_url: { url: 'data:image/png;base64,' + screenshot.base64 } },
77
+ ],
78
+ }];
79
+ const result = await apiCall(providerUrl, apiKey, { model, messages, stream: false, temperature: 0, max_tokens: 500 });
80
+ return String(result.choices?.[0]?.message?.content || '');
81
+ }
19
82
 
20
83
  function apiCall(providerUrl, apiKey, body) {
21
84
  return new Promise((resolve, reject) => {
22
- const base = providerUrl.replace(/\/+$/, '');
23
- const endpoint = isMiniMax(base)
24
- ? base + '/v1/text/chatcompletion_v2'
25
- : isGemini(base)
26
- ? base + '/openai/chat/completions'
27
- : base + '/chat/completions';
85
+ const endpoint = buildChatEndpoint(providerUrl);
28
86
  const req = https.request(endpoint, {
29
87
  method: 'POST',
30
88
  headers: { 'Authorization': 'Bearer ' + apiKey, 'Content-Type': 'application/json' },
@@ -45,12 +103,41 @@ function apiCall(providerUrl, apiKey, body) {
45
103
  });
46
104
  }
47
105
 
48
- function screenshotBase64() {
106
+ function captureScreenshot() {
107
+ if (os.platform() !== 'darwin') throw new Error('computer_use_loop currently requires macOS');
49
108
  const file = path.join(os.tmpdir(), 'ncloop_' + Date.now() + '.png');
50
109
  require('child_process').execSync('screencapture -x "' + file + '"', { timeout: 5000 });
51
110
  const buf = fs.readFileSync(file);
52
111
  fs.unlinkSync(file);
53
- return buf.toString('base64');
112
+ return {
113
+ base64: buf.toString('base64'),
114
+ hash: crypto.createHash('sha256').update(buf).digest('hex'),
115
+ };
116
+ }
117
+
118
+ function evaluateCompletionEvidence({ mutationCount, initialHash, currentHash, verification }) {
119
+ if (mutationCount < 1) return { verified: false, error: 'No state-changing GUI action was executed' };
120
+ if (!initialHash || !currentHash || initialHash === currentHash) {
121
+ return { verified: false, error: 'The screen did not change after GUI actions' };
122
+ }
123
+ if (!verification || verification.verified !== true) {
124
+ return { verified: false, error: verification?.reason || 'Visual verifier could not confirm the goal' };
125
+ }
126
+ const confidence = Number(verification.confidence || 0);
127
+ if (confidence < 0.8 || !String(verification.evidence || '').trim()) {
128
+ return { verified: false, error: 'Visual verification evidence was insufficient' };
129
+ }
130
+ return { verified: true, evidence: String(verification.evidence).trim(), confidence };
131
+ }
132
+
133
+ async function verifyGoal(providerUrl, apiKey, model, goal, screenshot) {
134
+ const prompt = `You are a strict independent verifier. Inspect only the screenshot. Goal: ${goal}\nReturn JSON only: {"verified":boolean,"confidence":0..1,"evidence":"exact visible evidence","reason":"why not"}. Never infer success from the goal text. If the requested sent message, booking, purchase, or confirmation is not visibly present, verified must be false.`;
135
+ const reply = await visionCall(providerUrl, apiKey, model, prompt, screenshot);
136
+ try { return JSON.parse(reply); }
137
+ catch {
138
+ const match = reply.match(/\{[\s\S]*\}/);
139
+ return match ? JSON.parse(match[0]) : { verified: false, confidence: 0, reason: 'Verifier returned invalid JSON' };
140
+ }
54
141
  }
55
142
 
56
143
  function executeAction(action, params) {
@@ -73,7 +160,7 @@ function executeAction(action, params) {
73
160
  case 'type':
74
161
  return osaScript('tell application "System Events" to keystroke "' + ESC(params.text) + '"');
75
162
  case 'keypress': {
76
- const KEY_MAP = { enter: 'return', tab: 'tab', escape: 'escape', up: 'up', down: 'down', left: 'left', right: 'right', backspace: 'delete', space: 'space' };
163
+ const KEY_CODES = { enter: 36, return: 36, tab: 48, escape: 53, up: 126, down: 125, left: 123, right: 124, backspace: 51, delete: 117, space: 49 };
77
164
  const MOD_MAP = { cmd: 'command down', command: 'command down', option: 'option down', alt: 'option down', ctrl: 'control down', shift: 'shift down' };
78
165
  const parts = params.key.toLowerCase().split('+').map(p => p.trim());
79
166
  const mods = [];
@@ -82,9 +169,12 @@ function executeAction(action, params) {
82
169
  if (MOD_MAP[p]) mods.push(MOD_MAP[p]);
83
170
  else actual = p;
84
171
  }
85
- const k = KEY_MAP[actual] || actual;
86
172
  const using = mods.length > 0 ? ' using {' + mods.join(', ') + '}' : '';
87
- return osaScript('tell application "System Events" to keystroke ' + k + using);
173
+ if (Object.prototype.hasOwnProperty.call(KEY_CODES, actual)) {
174
+ return osaScript('tell application "System Events" to key code ' + KEY_CODES[actual] + using);
175
+ }
176
+ if (!actual) return { success: false, error: 'A key is required with the modifier' };
177
+ return osaScript('tell application "System Events" to keystroke "' + ESC(actual) + '"' + using);
88
178
  }
89
179
  case 'mouse_move':
90
180
  return osaScript('tell application "System Events" to set position of mouse to {' + params.x + ', ' + params.y + '}');
@@ -113,55 +203,35 @@ Actions:
113
203
  - wait: { "action": "wait" }
114
204
  - done: { "action": "done", "reason": "gorev tamamlandi" }
115
205
 
206
+ "done" yalnız görev sonucunu mevcut ekranda açıkça görüyorsan kullanılabilir. Bir mesajı yazmak gönderildiği anlamına gelmez; gönderilen mesajın konuşmada görünmesi gerekir.
116
207
  Sadece JSON yanit ver, baska metin ekleme. Her action'dan sonra otomatik screenshot alinir, sen sadece bir sonraki en mantikli adimi soyle.`;
117
208
 
118
209
  async function loop(goal, maxSteps) {
119
210
  const cfg = loadConfig();
120
- const providerUrl = cfg.providerUrl;
121
- const apiKey = cfg.providerApiKey;
122
- const model = cfg.providerModel || 'default';
211
+ const vision = resolveVisionConfig(cfg);
212
+ if (!vision.success) return vision;
213
+ const { providerUrl, apiKey, model } = vision;
123
214
 
124
- if (!providerUrl || !apiKey) {
125
- return JSON.stringify({ success: false, error: 'Provider not configured' });
215
+ if (os.platform() !== 'darwin') {
216
+ return { success: false, error: 'computer_use_loop currently requires macOS' };
126
217
  }
127
218
 
128
219
  const steps = [];
220
+ let completed = false;
221
+ let initialHash = null;
222
+ let mutationCount = 0;
223
+ let completionEvidence = null;
129
224
  for (let i = 0; i < maxSteps; i++) {
130
225
  try {
131
226
  // 1. Screenshot
132
- const b64 = screenshotBase64();
227
+ const screenshot = captureScreenshot();
228
+ if (!initialHash) initialHash = screenshot.hash;
133
229
  steps.push({ step: i + 1, action: 'screenshot' });
134
230
 
135
231
  // 2. LLM vision analysis
136
- const messages = [
137
- { role: 'system', content: SYSTEM_PROMPT },
138
- ];
139
-
140
232
  const history = steps.filter(s => s.action && s.action !== 'screenshot' && s.action !== 'done').slice(-5);
141
- if (history.length > 0) {
142
- messages.push({ role: 'user', content: 'Onceki adimlar:\n' + history.map(h =>
143
- 'Adim ' + h.step + ': ' + JSON.stringify(h)
144
- ).join('\n') });
145
- }
146
-
147
- messages.push({
148
- role: 'user',
149
- content: [
150
- { type: 'text', text: 'Gorev: ' + goal + '\n\nEkran goruntusunu analiz et. Siradaki action ne?' },
151
- { type: 'image_url', image_url: { url: 'data:image/png;base64,' + b64 } },
152
- ],
153
- });
154
-
155
- const body = {
156
- model,
157
- messages,
158
- stream: false,
159
- temperature: 0.2,
160
- max_tokens: 500,
161
- };
162
-
163
- const result = await apiCall(providerUrl, apiKey, body);
164
- const reply = result.choices?.[0]?.message?.content || '';
233
+ const historyText = history.length > 0 ? '\nOnceki adimlar:\n' + history.map(h => 'Adim ' + h.step + ': ' + JSON.stringify(h)).join('\n') : '';
234
+ const reply = await visionCall(providerUrl, apiKey, model, SYSTEM_PROMPT + '\n\nGorev: ' + goal + historyText + '\n\nEkran goruntusunu analiz et. Siradaki action ne?', screenshot);
165
235
  let decision;
166
236
  try {
167
237
  decision = JSON.parse(reply);
@@ -174,12 +244,21 @@ async function loop(goal, maxSteps) {
174
244
 
175
245
  // 3. Execute
176
246
  if (decision.action === 'done') {
177
- steps.push({ step: i + 1, action: 'done', reason: decision.reason || 'Goal achieved' });
178
- break;
247
+ const verification = await verifyGoal(providerUrl, apiKey, model, goal, screenshot);
248
+ const evidence = evaluateCompletionEvidence({ mutationCount, initialHash, currentHash: screenshot.hash, verification });
249
+ if (evidence.verified) {
250
+ steps.push({ step: i + 1, action: 'verified', evidence: evidence.evidence, confidence: evidence.confidence });
251
+ completionEvidence = evidence;
252
+ completed = true;
253
+ break;
254
+ }
255
+ steps.push({ step: i + 1, action: 'verification_failed', error: evidence.error });
256
+ continue;
179
257
  }
180
258
 
181
259
  const execResult = executeAction(decision.action, decision);
182
- steps.push({ step: i + 1, action: decision.action, params: decision, result: execResult.success });
260
+ const safeParams = decision.action === 'type' ? { action: 'type', text: '[redacted]' } : decision;
261
+ steps.push({ step: i + 1, action: decision.action, params: safeParams, result: execResult.success });
183
262
 
184
263
  if (!execResult.success) {
185
264
  // Retry once with wait
@@ -187,15 +266,17 @@ async function loop(goal, maxSteps) {
187
266
  const retryResult = executeAction(decision.action, decision);
188
267
  if (!retryResult.success) {
189
268
  steps.push({ step: i + 1, action: 'error', error: execResult.error });
269
+ return { success: false, error: execResult.error, goal, totalSteps: steps.length, steps };
190
270
  }
191
271
  }
272
+ if (['click', 'type', 'keypress', 'scroll', 'drag'].includes(decision.action)) mutationCount++;
192
273
 
193
274
  // Small delay between actions
194
275
  require('child_process').execSync('sleep 0.5');
195
276
 
196
277
  // Safety: if too many steps, break
197
278
  if (i >= maxSteps - 1) {
198
- steps.push({ step: i + 1, action: 'done', reason: 'Max steps reached' });
279
+ steps.push({ step: i + 1, action: 'error', error: 'Max steps reached before the goal was verified' });
199
280
  }
200
281
  } catch (e) {
201
282
  steps.push({ step: i + 1, action: 'error', error: e.message });
@@ -203,7 +284,10 @@ async function loop(goal, maxSteps) {
203
284
  }
204
285
  }
205
286
 
206
- return JSON.stringify({ success: true, goal, totalSteps: steps.length, steps });
287
+ if (!completed) {
288
+ return { success: false, error: 'Goal was not verified', goal, totalSteps: steps.length, steps };
289
+ }
290
+ return { success: true, verified: true, evidence: completionEvidence.evidence, confidence: completionEvidence.confidence, goal, totalSteps: steps.length, steps };
207
291
  }
208
292
 
209
293
  const name = 'computer_use_loop';
@@ -221,4 +305,4 @@ async function execute(params) {
221
305
  return await loop(params.goal, params.maxSteps || 30);
222
306
  }
223
307
 
224
- module.exports = { name, description, parameters, execute };
308
+ module.exports = { name, description, parameters, execute, executeAction, evaluateCompletionEvidence, resolveVisionConfig, visionCall };
@@ -1,6 +1,31 @@
1
1
  const { getConfig } = require('../utils/config');
2
+ const fs = require('fs');
3
+ const path = require('path');
2
4
 
3
5
  const PROVIDERS = {
6
+ minimax: {
7
+ name: 'MiniMax image-01',
8
+ async generate({ prompt, n, apiKey, model, aspectRatio, baseUrl }) {
9
+ const response = await fetch(baseUrl + '/v1/image_generation', {
10
+ method: 'POST',
11
+ headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${apiKey}` },
12
+ body: JSON.stringify({ model: model || 'image-01', prompt, n: n || 1, response_format: 'base64', ...(aspectRatio ? { aspect_ratio: aspectRatio } : {}) }),
13
+ signal: AbortSignal.timeout(120000),
14
+ });
15
+ if (!response.ok) throw new Error(`MiniMax image error ${response.status}: ${(await response.text()).slice(0, 300)}`);
16
+ const data = await response.json();
17
+ if (data.base_resp?.status_code) throw new Error(data.base_resp.status_msg || `MiniMax image API error ${data.base_resp.status_code}`);
18
+ const images = data.data?.image_base64 || [];
19
+ if (!images.length) throw new Error('MiniMax image generation returned no images');
20
+ const outDir = path.join(process.cwd(), 'minimax-output');
21
+ fs.mkdirSync(outDir, { recursive: true });
22
+ return images.map((base64, index) => {
23
+ const file = path.join(outDir, `image-${Date.now()}-${index + 1}.png`);
24
+ fs.writeFileSync(file, Buffer.from(base64, 'base64'));
25
+ return { file, mime: 'image/png' };
26
+ });
27
+ }
28
+ },
4
29
  // v4.8.4: Pollinations.ai — TAMAMEN ÜCRETSİZ, API key gerektirmez
5
30
  // Default provider olarak ayarlandı (herkes kullanabilsin)
6
31
  pollinations: {
@@ -56,6 +81,11 @@ const PROVIDERS = {
56
81
  }
57
82
  };
58
83
 
84
+ function selectImageProvider(config, params = {}) {
85
+ if (params.provider || config.imageProvider) return params.provider || config.imageProvider;
86
+ return /minimax\.io|minimaxi\.com|minimax\.cn/i.test(config.providerUrl || '') ? 'minimax' : undefined;
87
+ }
88
+
59
89
  module.exports = {
60
90
  name: 'image_generation',
61
91
  description: 'Generate images using AI. Supports OpenAI DALL-E, FAL.ai, and Together AI providers.',
@@ -63,7 +93,8 @@ module.exports = {
63
93
  type: 'object',
64
94
  properties: {
65
95
  prompt: { type: 'string', description: 'Text description of the image to generate' },
66
- provider: { type: 'string', description: 'Image provider: openai, fal, together (default: openai)', enum: ['openai', 'fal', 'together'] },
96
+ provider: { type: 'string', description: 'Image provider: minimax, openai, fal, together, pollinations (auto-detected)', enum: ['minimax', 'openai', 'fal', 'together', 'pollinations'] },
97
+ aspectRatio: { type: 'string', description: 'Aspect ratio such as 1:1, 16:9 or 9:16' },
67
98
  model: { type: 'string', description: 'Model override (e.g. fast-sdxl for FAL, FLUX.1-schnell for Together)' },
68
99
  size: { type: 'string', description: 'Image size for DALL-E: 1024x1024, 1792x1024, 1024x1792 (default: 1024x1024)' },
69
100
  n: { type: 'number', description: 'Number of images to generate (default: 1, max: 4)' }
@@ -79,14 +110,17 @@ module.exports = {
79
110
  const openaiKey = params.apiKey || config.openaiApiKey || process.env.OPENAI_API_KEY;
80
111
  const falKey = params.apiKey || config.falApiKey || process.env.FAL_KEY;
81
112
  const togetherKey = params.apiKey || config.togetherApiKey || process.env.TOGETHER_API_KEY;
113
+ const minimaxKey = params.apiKey || config.providerApiKey || process.env.MINIMAX_API_KEY;
114
+ const isMiniMax = /minimax\.io|minimaxi\.com|minimax\.cn/i.test(config.providerUrl || '');
82
115
 
83
116
  // Explicit provider tercih edilmişse onu kullan
84
- let provider = params.provider || config.imageProvider;
117
+ let provider = selectImageProvider(config, params) || (isMiniMax && minimaxKey ? 'minimax' : undefined);
85
118
  let apiKey;
86
119
 
87
120
  if (provider) {
88
121
  // Explicit provider
89
- if (provider === 'openai') apiKey = openaiKey;
122
+ if (provider === 'minimax') apiKey = minimaxKey;
123
+ else if (provider === 'openai') apiKey = openaiKey;
90
124
  else if (provider === 'fal') apiKey = falKey;
91
125
  else if (provider === 'together') apiKey = togetherKey;
92
126
  else if (provider === 'pollinations') apiKey = 'free'; // key gereksiz
@@ -114,6 +148,8 @@ module.exports = {
114
148
  prompt: params.prompt,
115
149
  apiKey,
116
150
  model: params.model,
151
+ aspectRatio: params.aspectRatio,
152
+ baseUrl: (config.providerUrl || 'https://api.minimax.io').replace(/\/+$/, '').replace(/\/v1$/, ''),
117
153
  size: params.size,
118
154
  n: params.n
119
155
  });
@@ -122,11 +158,14 @@ module.exports = {
122
158
  success: true,
123
159
  prompt: params.prompt,
124
160
  provider: provider,
125
- images: images.filter(i => i.url).map(i => ({ url: i.url, revisedPrompt: i.revised_prompt })),
126
- count: images.filter(i => i.url).length
161
+ images: images.filter(i => i.url || i.file).map(i => ({ url: i.url, file: i.file, revisedPrompt: i.revised_prompt })),
162
+ count: images.filter(i => i.url || i.file).length
127
163
  };
128
164
  } catch (error) {
129
165
  return { success: false, error: error.message };
130
166
  }
131
167
  }
132
168
  };
169
+
170
+ module.exports._providers = PROVIDERS;
171
+ module.exports.selectImageProvider = selectImageProvider;
@@ -11,7 +11,7 @@ module.exports = {
11
11
  imagePath: { type: 'string', description: 'Local image file path' },
12
12
  imageUrl: { type: 'string', description: 'Remote image URL (if no local file)' },
13
13
  prompt: { type: 'string', description: 'Analysis prompt (default: "Describe this image in detail")' },
14
- provider: { type: 'string', description: 'Vision provider: openai, anthropic, groq (default: openai)', enum: ['openai', 'anthropic', 'groq'] },
14
+ provider: { type: 'string', description: 'Vision provider: minimax, openai, anthropic, gemini, groq (auto-detected)', enum: ['minimax', 'openai', 'anthropic', 'gemini', 'groq'] },
15
15
  model: { type: 'string', description: 'Model override' }
16
16
  }
17
17
  },
@@ -20,11 +20,10 @@ module.exports = {
20
20
  try {
21
21
  const config = getConfig();
22
22
  const prompt = params.prompt || 'Describe this image in detail';
23
- let provider = params.provider || config.visionProvider || 'openai';
24
-
25
- // v5.6.22: MiniMax, Ollama, local provider'lar vision desteklemiyor olabilir
26
23
  const providerUrl = (config.providerUrl || '').toLowerCase();
27
- if (!params.provider && (providerUrl.includes('minimax') || providerUrl.includes('ollama') || providerUrl.includes('localhost'))) {
24
+ let provider = params.provider || config.visionProvider || (providerUrl.includes('minimax') ? 'minimax' : providerUrl.includes('generativelanguage.googleapis.com') || providerUrl.includes('gemini') ? 'gemini' : 'openai');
25
+
26
+ if (!params.provider && (providerUrl.includes('ollama') || providerUrl.includes('localhost'))) {
28
27
  return {
29
28
  success: false,
30
29
  error: `Görsel analiz bu provider (${providerUrl}) için desteklenmiyor. Lütfen OpenAI, Anthropic, Gemini veya Groq provider'ı kullanın.`,
@@ -58,6 +57,30 @@ module.exports = {
58
57
 
59
58
  const dataUrl = params.imageUrl || `data:${mediaType};base64,${imageBase64}`;
60
59
 
60
+ if (provider === 'minimax') {
61
+ const apiKey = params.apiKey || config.providerApiKey || process.env.MINIMAX_CODE_PLAN_KEY || process.env.MINIMAX_API_KEY;
62
+ if (!apiKey) return { success: false, error: 'MiniMax API key gerekli' };
63
+ let imageDataUrl = dataUrl;
64
+ if (/^https?:\/\//i.test(imageDataUrl)) {
65
+ const downloaded = await fetch(imageDataUrl);
66
+ if (!downloaded.ok) throw new Error(`Görsel indirilemedi: HTTP ${downloaded.status}`);
67
+ const mime = (downloaded.headers.get('content-type') || 'image/jpeg').split(';')[0];
68
+ imageDataUrl = `data:${mime};base64,${Buffer.from(await downloaded.arrayBuffer()).toString('base64')}`;
69
+ }
70
+ const base = (config.providerUrl || 'https://api.minimax.io').replace(/\/+$/, '').replace(/\/v1$/, '');
71
+ const response = await fetch(base + '/v1/coding_plan/vlm', {
72
+ method: 'POST',
73
+ headers: { Authorization: `Bearer ${apiKey}`, 'Content-Type': 'application/json', 'MM-API-Source': 'NatureCo' },
74
+ body: JSON.stringify({ prompt, image_url: imageDataUrl }),
75
+ signal: AbortSignal.timeout(60000),
76
+ });
77
+ if (!response.ok) throw new Error(`MiniMax VLM error ${response.status}: ${(await response.text()).slice(0, 300)}`);
78
+ const data = await response.json();
79
+ if (data.base_resp?.status_code) throw new Error(data.base_resp.status_msg || `MiniMax VLM API error ${data.base_resp.status_code}`);
80
+ if (!data.content) throw new Error('MiniMax VLM boş yanıt döndürdü');
81
+ return { success: true, provider: 'minimax', model: 'MiniMax-VL-01', analysis: data.content };
82
+ }
83
+
61
84
  if (provider === 'openai') {
62
85
  const apiKey = params.apiKey || config.openaiApiKey || process.env.OPENAI_API_KEY;
63
86
  if (!apiKey) return { success: false, error: 'OpenAI API key gerekli' };
@@ -83,6 +106,23 @@ module.exports = {
83
106
  return { success: true, provider: 'openai', analysis: data.choices?.[0]?.message?.content || '' };
84
107
  }
85
108
 
109
+ if (provider === 'gemini') {
110
+ const apiKey = params.apiKey || config.providerApiKey || process.env.GEMINI_API_KEY;
111
+ if (!apiKey) return { success: false, error: 'Gemini API key gerekli' };
112
+ const base = (config.providerUrl || 'https://generativelanguage.googleapis.com/v1beta').replace(/\/+$/, '');
113
+ const response = await fetch(base + '/openai/chat/completions', {
114
+ method: 'POST',
115
+ headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${apiKey}` },
116
+ body: JSON.stringify({ model: params.model || config.providerModel, messages: [{ role: 'user', content: [
117
+ { type: 'text', text: prompt },
118
+ { type: 'image_url', image_url: { url: dataUrl } },
119
+ ] }], max_tokens: 1000 }),
120
+ });
121
+ if (!response.ok) throw new Error(`Gemini vision error ${response.status}`);
122
+ const data = await response.json();
123
+ return { success: true, provider: 'gemini', analysis: data.choices?.[0]?.message?.content || '' };
124
+ }
125
+
86
126
  if (provider === 'anthropic') {
87
127
  const apiKey = params.apiKey || config.anthropicApiKey || process.env.ANTHROPIC_API_KEY;
88
128
  if (!apiKey) return { success: false, error: 'Anthropic API key gerekli' };
@@ -1,6 +1,31 @@
1
1
  const { getConfig } = require('../utils/config');
2
2
 
3
3
  const PROVIDERS = {
4
+ minimax: {
5
+ name: 'MiniMax Hailuo',
6
+ async generate({ prompt, apiKey, model, baseUrl }) {
7
+ const headers = { 'Content-Type': 'application/json', Authorization: `Bearer ${apiKey}` };
8
+ const submittedResponse = await fetch(baseUrl + '/v1/video_generation', {
9
+ method: 'POST', headers,
10
+ body: JSON.stringify({ model: model || 'MiniMax-Hailuo-2.3', prompt }),
11
+ signal: AbortSignal.timeout(120000),
12
+ });
13
+ if (!submittedResponse.ok) throw new Error(`MiniMax video error ${submittedResponse.status}: ${(await submittedResponse.text()).slice(0, 300)}`);
14
+ const submitted = await submittedResponse.json();
15
+ if (submitted.base_resp?.status_code) throw new Error(submitted.base_resp.status_msg || 'MiniMax video submission failed');
16
+ if (!submitted.task_id) throw new Error('MiniMax video response missing task_id');
17
+ for (let attempt = 0; attempt < 120; attempt++) {
18
+ await new Promise(resolve => setTimeout(resolve, 3000));
19
+ const statusResponse = await fetch(baseUrl + '/v1/query/video_generation?task_id=' + encodeURIComponent(submitted.task_id), { headers, signal: AbortSignal.timeout(30000) });
20
+ if (!statusResponse.ok) throw new Error(`MiniMax video status error ${statusResponse.status}`);
21
+ const status = await statusResponse.json();
22
+ if (status.base_resp?.status_code) throw new Error(status.base_resp.status_msg || 'MiniMax video generation failed');
23
+ if (status.status === 'Fail') throw new Error(status.base_resp?.status_msg || 'MiniMax video generation failed');
24
+ if (status.status === 'Success') return [{ url: status.video_url || null, fileId: status.file_id || null, taskId: submitted.task_id }];
25
+ }
26
+ throw new Error(`MiniMax video task ${submitted.task_id} timed out`);
27
+ }
28
+ },
4
29
  runway: {
5
30
  name: 'RunwayML',
6
31
  async generate({ prompt, apiKey, model }) {
@@ -21,6 +46,11 @@ const PROVIDERS = {
21
46
  }
22
47
  };
23
48
 
49
+ function selectVideoProvider(config, params = {}) {
50
+ if (params.provider) return params.provider;
51
+ return /minimax\.io|minimaxi\.com|minimax\.cn/i.test(config.providerUrl || '') ? 'minimax' : 'runway';
52
+ }
53
+
24
54
  module.exports = {
25
55
  name: 'video_generation',
26
56
  description: 'Generate videos using AI. Supports RunwayML provider.',
@@ -28,7 +58,7 @@ module.exports = {
28
58
  type: 'object',
29
59
  properties: {
30
60
  prompt: { type: 'string', description: 'Text description of the video to generate' },
31
- provider: { type: 'string', description: 'Video provider: runway (default: runway)', enum: ['runway'] },
61
+ provider: { type: 'string', description: 'Video provider: minimax or runway (auto-detected)', enum: ['minimax', 'runway'] },
32
62
  model: { type: 'string', description: 'Model override (default: gen3a_turbo)' }
33
63
  },
34
64
  required: ['prompt']
@@ -37,14 +67,16 @@ module.exports = {
37
67
  async execute(params) {
38
68
  try {
39
69
  const config = getConfig();
40
- const provider = params.provider || 'runway';
70
+ const provider = selectVideoProvider(config, params);
41
71
 
42
72
  const providerConfig = PROVIDERS[provider];
43
73
  if (!providerConfig) {
44
74
  return { success: false, error: `Desteklenmeyen provider: ${provider}` };
45
75
  }
46
76
 
47
- const apiKey = params.apiKey || config.runwayApiKey || process.env.RUNWAY_API_KEY;
77
+ const apiKey = provider === 'minimax'
78
+ ? (params.apiKey || config.providerApiKey || process.env.MINIMAX_API_KEY)
79
+ : (params.apiKey || config.runwayApiKey || process.env.RUNWAY_API_KEY);
48
80
  if (!apiKey) {
49
81
  return {
50
82
  success: false,
@@ -55,7 +87,8 @@ module.exports = {
55
87
  const videos = await providerConfig.generate({
56
88
  prompt: params.prompt,
57
89
  apiKey,
58
- model: params.model
90
+ model: params.model,
91
+ baseUrl: (config.providerUrl || 'https://api.minimax.io').replace(/\/+$/, '').replace(/\/v1$/, '')
59
92
  });
60
93
 
61
94
  return {
@@ -70,3 +103,6 @@ module.exports = {
70
103
  }
71
104
  }
72
105
  };
106
+
107
+ module.exports._providers = PROVIDERS;
108
+ module.exports.selectVideoProvider = selectVideoProvider;
@@ -300,7 +300,7 @@ async function workflow(params) {
300
300
  '- Dosya degistirmeden ONCE read_file ile oku, edit_file ile hedefli degistir. Yerini bilmiyorsan file_search/grep_search ile kesfet.',
301
301
  '- Kod yazinca/degistirince bash ile calistir/test et, hata varsa duzelt. Coklu dosya = her biri icin AYRI write_file. Hep TAM yol; "masaustu" = ' + desktop + '.',
302
302
  '- Arac sonuclari <tool_results> icinde doner; is bitince arac cagirmadan tek cumlelik ozet yaz. Basit sohbette arac cagirma, dogrudan yanitla.',
303
- execFull ? '- Kullanici uygulama ac / tarayici kontrol / muzik cal / ekran / GUI istediyse yukaridaki computer-use araclarini KULLAN — "yapamam/engellendi" DEME, dogrudan ilgili araci cagir.' : '',
303
+ execFull ? '- Kullanici uygulama ac / tarayici kontrol / muzik cal / ekran / GUI istediyse yukaridaki computer-use araclarini KULLAN — "yapamam/engellendi" DEME, dogrudan ilgili araci cagir. Gorsel geri bildirim gerektiren cok adimli GUI gorevlerinde tekil screenshot/click yerine computer_use_loop kullan; basariyi arac dogrulamadan tamamlandi deme.' : '',
304
304
  fullToolsBlock,
305
305
  treeDigest ? ('\n\nBILDIGIN KALICI HAFIZA (bu kullaniciya ait, onceki oturumlardan hatirladiklarin; kullaniciya ozel bir sey sorulursa ONCE BUNU KULLAN — dosya arama, uydurma):\n' + treeDigest) : '',
306
306
  treeIndex ? ('\n\nHafiza agaci yapisi (yukarida olmayan detay icin memory_tree(action:read/search) ile ilgili kok/dali oku):\n' + treeIndex) : '',
@@ -337,7 +337,7 @@ async function workflow(params) {
337
337
  }
338
338
 
339
339
  // Araç aktivitesi gösterimi (TTY streaming): her araç icin "🔧 label · hint ✓/✗"
340
- const TOOL_LABEL = { write_file: 'dosya yaz', read_file: 'oku', edit_file: 'düzenle', bash: 'komut', file_search: 'ara', list_dir: 'listele', skill_view: 'skill', browser: 'tarayıcı', browser_use: 'tarayıcı', mac_app_open: 'uygulama aç', mac_app_quit: 'uygulama kapat', computer_use: 'GUI', social_open: 'medya aç', macos_screenshot: 'ekran görüntüsü' };
340
+ const TOOL_LABEL = { write_file: 'dosya yaz', read_file: 'oku', edit_file: 'düzenle', bash: 'komut', file_search: 'ara', list_dir: 'listele', skill_view: 'skill', browser: 'tarayıcı', browser_use: 'tarayıcı', mac_app_open: 'uygulama aç', mac_app_quit: 'uygulama kapat', computer_use: 'GUI', computer_use_loop: 'GUI görsel döngü', social_open: 'medya aç', macos_screenshot: 'ekran görüntüsü' };
341
341
  function briefHint(args) {
342
342
  if (!args || typeof args !== 'object') return '';
343
343
  const v = args.appName || args.query || args.name || args.url || args.command || args.pattern || args.path || args.action;
@@ -348,9 +348,13 @@ async function workflow(params) {
348
348
  const label = TOOL_LABEL[ev.tool] || ev.tool;
349
349
  const hint = briefHint(ev.args);
350
350
  process.stdout.write('\n\x1b[2m 🔧 ' + label + (hint ? ' · ' + hint : '') + '\x1b[0m');
351
- } else {
352
- const rec = (ev.records || [])[0] || {};
353
- process.stdout.write(rec.status === 'done' ? ' \x1b[32m✓\x1b[0m' : ' \x1b[31m✗\x1b[0m');
351
+ } else {
352
+ const rec = (ev.records || [])[0] || {};
353
+ // Satırı burada bitir. Sonraki model çağrısının thinking göstergesi
354
+ // mevcut satırı \r + erase-line ile temizlediği için newline yoksa
355
+ // kullanıcı araç adını ve sonucunu hiç göremiyordu.
356
+ const error = rec.status === 'done' ? '' : String(rec.error || rec.result?.error || '').replace(/\s+/g, ' ').slice(0, 100);
357
+ process.stdout.write((rec.status === 'done' ? ' \x1b[32m✓\x1b[0m' : ' \x1b[31m✗\x1b[0m') + (error ? ' \x1b[31m' + error + '\x1b[0m' : '') + '\n');
354
358
  }
355
359
  } : null;
356
360