imagegen-smarto 0.1.2 → 0.1.4

package/README.md

@@ -1,19 +1,11 @@
 # imagegen-smarto
 
-Codex skill for sending natural-language image generation and editing requests
-through a custom SmartO OpenAI-compatible VPS relay.
+Codex skill for natural-language image generation and editing through a custom
+SmartO OpenAI-compatible VPS relay.
 
-The skill does not call Codex's built-in `image_gen` tool. It places a private
-routing marker in the Responses instructions; the relay adds its
-`image_generation` tool for requests that load this skill, merging it with
-existing function, shell, file, or other tools. Ordinary non-image requests
-remain unchanged.
-
-When the SmartO profile is active, this skill can also replace only the image
-provider inside workflows that normally use `$imagegen`, including the
-official `hatch-pet` workflow. The workflow's prompts, references, QA, and
-file handling remain unchanged; only image generation is routed through
-SmartO.
+It implicitly handles requests to create, draw, generate, modify, perform
+image-to-image, or perform text-to-image generation. Requests to view,
+analyze, describe, or recognize an image remain ordinary requests.
 
 ## Install with npm
 
@@ -26,7 +18,13 @@ npm install --global imagegen-smarto
 The npm postinstall step copies the skill to `CODEX_HOME/skills/imagegen-smarto`
 or, when `CODEX_HOME` is not set, `~/.codex/skills/imagegen-smarto`.
 
-To reinstall or update the skill without reinstalling the npm package:
+To update to the latest published package:
+
+```bash
+npm install --global imagegen-smarto
+```
+
+To recopy the skill from the already installed npm package:
 
 ```bash
 imagegen-smarto install
@@ -83,10 +81,35 @@ python3 ~/.codex/skills/.system/skill-installer/scripts/install-skill-from-githu
   --method git
 ```
 
-After installation, use a profile that enables `imagegen-smarto` and disables
-the official `imagegen` skill. The skill is configured for implicit invocation,
-so an image request can be written in natural language without first typing
-`$imagegen-smarto`.
+After installation, use the profile that points requests to SmartO. The skill
+is configured for implicit invocation, so an image request can be written in
+natural language without first typing `$imagegen-smarto`.
+
+## How image generation works
+
+When the skill is triggered, Codex runs the installed `imagegen-smarto`
+command. The command reads the active provider URL, model, and Codex
+credential, sends a streaming Responses request with the SmartO marker, parses
+the returned `image_generation_call`, and saves the PNG locally. It prints an
+absolute `IMAGE_MARKDOWN=...` line so Codex can display the result inline.
+
+You can also test the execution path directly:
+
+```bash
+imagegen-smarto generate --prompt "一只戴红色围巾的小猫，儿童绘本风格"
+```
+
+For editing, add one or more reference images:
+
+```bash
+imagegen-smarto generate \
+  --prompt "把背景改成海边日落，保留主体" \
+  --image /absolute/path/to/source.png
+```
+
+The command always uses `stream=true` internally because the relay requires
+streaming for image generation. The skill does not need to describe or modify
+the relay request body.
 
 ## Local profile switch
 
@@ -97,5 +120,4 @@ codex --profile smarto
 codex --profile official
 ```
 
-`smarto` selects the custom relay and enables this skill. `official` selects
-the built-in OpenAI provider and enables Codex's original `imagegen` skill.
+`smarto` selects the custom relay and enables this skill.

package/bin/cli.js

@@ -5,6 +5,7 @@
 const fs = require('node:fs')
 const os = require('node:os')
 const path = require('node:path')
+const { generateImage } = require('./generate')
 
 const packageRoot = path.resolve(__dirname, '..')
 const bundledSkillPath = path.join(packageRoot, 'imagegen-smarto')
@@ -16,9 +17,12 @@ Usage:
   imagegen-smarto install [--codex-home <path>]
   imagegen-smarto uninstall [--codex-home <path>]
   imagegen-smarto path [--codex-home <path>]
+  imagegen-smarto generate --prompt "your prompt" [options]
   imagegen-smarto --help
 
-The default Codex home is CODEX_HOME or ~/.codex.`)
+The default Codex home is CODEX_HOME or ~/.codex.
+
+Run imagegen-smarto generate --help for image generation options.`)
 }
 
 function parseArgs(argv) {
@@ -80,8 +84,14 @@ function uninstallSkill(codexHome) {
   return target
 }
 
-function main() {
-  const { command, codexHome: explicitCodexHome, quiet } = parseArgs(process.argv.slice(2))
+async function main() {
+  const argv = process.argv.slice(2)
+  if (argv[0] === 'generate') {
+    await generateImage(argv.slice(1))
+    return
+  }
+
+  const { command, codexHome: explicitCodexHome, quiet } = parseArgs(argv)
 
   if (command === 'help') {
     printHelp()
@@ -113,7 +123,10 @@ function main() {
 }
 
 try {
-  main()
+  main().catch((error) => {
+    console.error(`imagegen-smarto: ${error.message}`)
+    process.exitCode = 1
+  })
 } catch (error) {
   console.error(`imagegen-smarto: ${error.message}`)
   process.exitCode = 1

package/bin/generate.js

@@ -0,0 +1,391 @@
+#!/usr/bin/env node
+
+'use strict'
+
+const fs = require('node:fs')
+const os = require('node:os')
+const path = require('node:path')
+
+const IMAGEGEN_MARKER = '__CODEX_VPS_IMAGEGEN__'
+const DEFAULT_TIMEOUT_MS = 10 * 60 * 1000
+
+function printGenerateHelp() {
+  console.log(`imagegen-smarto generate - generate or edit an image through the active relay
+
+Usage:
+  imagegen-smarto generate --prompt "your prompt" [options]
+  imagegen-smarto generate "your prompt" [options]
+
+Options:
+  --prompt <text>       Prompt for generation or editing
+  --image <path>        Reference image; may be repeated for image editing
+  --output <path>       Output PNG path (default: a temporary file)
+  --model <model>       Override the model from Codex config
+  --base-url <url>      Override the active provider base URL
+  --timeout <seconds>   Request timeout (default: 600)
+  --help                Show this help`)
+}
+
+function parseGenerateArgs(argv) {
+  const options = {
+    images: [],
+    prompt: null,
+    output: null,
+    model: null,
+    baseUrl: null,
+    timeoutMs: DEFAULT_TIMEOUT_MS,
+  }
+  const positional = []
+
+  for (let index = 0; index < argv.length; index += 1) {
+    const arg = argv[index]
+    if (arg === '--help' || arg === '-h') {
+      options.help = true
+      continue
+    }
+
+    if (arg === '--prompt') {
+      options.prompt = argv[++index]
+      if (!options.prompt) throw new Error('--prompt requires text')
+      continue
+    }
+
+    if (arg === '--image' || arg === '--input-image') {
+      const image = argv[++index]
+      if (!image) throw new Error(`${arg} requires a path`)
+      options.images.push(image)
+      continue
+    }
+
+    if (arg === '--output') {
+      options.output = argv[++index]
+      if (!options.output) throw new Error('--output requires a path')
+      continue
+    }
+
+    if (arg === '--model') {
+      options.model = argv[++index]
+      if (!options.model) throw new Error('--model requires a model name')
+      continue
+    }
+
+    if (arg === '--base-url') {
+      options.baseUrl = argv[++index]
+      if (!options.baseUrl) throw new Error('--base-url requires a URL')
+      continue
+    }
+
+    if (arg === '--timeout') {
+      const seconds = Number(argv[++index])
+      if (!Number.isFinite(seconds) || seconds <= 0) {
+        throw new Error('--timeout must be a positive number of seconds')
+      }
+      options.timeoutMs = seconds * 1000
+      continue
+    }
+
+    if (arg.startsWith('-')) {
+      throw new Error(`unknown option: ${arg}`)
+    }
+    positional.push(arg)
+  }
+
+  if (!options.prompt && positional.length > 0) {
+    options.prompt = positional.join(' ')
+  }
+
+  return options
+}
+
+function resolveCodexHome() {
+  const configuredPath = process.env.CODEX_HOME
+  if (configuredPath && configuredPath.trim()) {
+    return path.resolve(configuredPath)
+  }
+  return path.join(os.homedir(), '.codex')
+}
+
+function readCodexConfig(codexHome) {
+  const configPath = path.join(codexHome, 'config.toml')
+  try {
+    return fs.readFileSync(configPath, 'utf8')
+  } catch (error) {
+    if (error.code === 'ENOENT') return ''
+    throw new Error(`cannot read ${configPath}: ${error.message}`)
+  }
+}
+
+function tomlStringValue(text, key, section) {
+  let sectionText = text
+
+  if (section) {
+    const sectionHeader = `[model_providers.${section}]`
+    const sectionStart = text.indexOf(sectionHeader)
+    if (sectionStart >= 0) {
+      const contentStart = text.indexOf('\n', sectionStart)
+      const nextSection = text.slice(contentStart + 1).search(/^\s*\[/m)
+      sectionText = nextSection >= 0
+        ? text.slice(contentStart + 1, contentStart + 1 + nextSection)
+        : text.slice(contentStart + 1)
+    } else {
+      sectionText = ''
+    }
+  }
+
+  const match = sectionText.match(new RegExp(`^\\s*${key}\\s*=\\s*["']([^"']+)["']`, 'm'))
+  return match ? match[1] : null
+}
+
+function readActiveProviderConfig(codexHome) {
+  const config = readCodexConfig(codexHome)
+  const provider = tomlStringValue(config, 'model_provider') || 'openai'
+  const baseUrl = process.env.IMAGEGEN_SMARTO_BASE_URL
+    || process.env.OPENAI_BASE_URL
+    || tomlStringValue(config, 'base_url', provider)
+    || tomlStringValue(config, 'base_url')
+  const model = process.env.IMAGEGEN_SMARTO_MODEL
+    || tomlStringValue(config, 'model')
+
+  if (!baseUrl) {
+    throw new Error(
+      `no active provider base_url found in ${path.join(codexHome, 'config.toml')}; `
+      + 'set IMAGEGEN_SMARTO_BASE_URL or use a Codex profile with a base_url',
+    )
+  }
+  if (!model) {
+    throw new Error(
+      `no active model found in ${path.join(codexHome, 'config.toml')}; `
+      + 'set IMAGEGEN_SMARTO_MODEL or pass --model',
+    )
+  }
+
+  return { baseUrl, model }
+}
+
+function readApiKey(codexHome) {
+  if (process.env.IMAGEGEN_SMARTO_API_KEY) return process.env.IMAGEGEN_SMARTO_API_KEY
+  if (process.env.OPENAI_API_KEY) return process.env.OPENAI_API_KEY
+
+  const authPath = path.join(codexHome, 'auth.json')
+  try {
+    const auth = JSON.parse(fs.readFileSync(authPath, 'utf8'))
+    const candidates = [
+      auth.OPENAI_API_KEY,
+      auth.openai_api_key,
+      auth.api_key,
+      auth.access_token,
+      auth.tokens?.access_token,
+    ]
+    const key = candidates.find((value) => typeof value === 'string' && value.trim())
+    if (key) return key
+  } catch (error) {
+    if (error.code !== 'ENOENT') {
+      throw new Error(`cannot read ${authPath}: ${error.message}`)
+    }
+  }
+
+  throw new Error(
+    `no API credential found; set OPENAI_API_KEY or log in so ${authPath} contains OPENAI_API_KEY`,
+  )
+}
+
+function normalizeResponsesUrl(baseUrl) {
+  const normalized = baseUrl.replace(/\/+$/, '')
+  return normalized.endsWith('/responses') ? normalized : `${normalized}/responses`
+}
+
+function imageMimeType(filePath) {
+  const extension = path.extname(filePath).toLowerCase()
+  if (extension === '.jpg' || extension === '.jpeg') return 'image/jpeg'
+  if (extension === '.webp') return 'image/webp'
+  if (extension === '.gif') return 'image/gif'
+  return 'image/png'
+}
+
+function imageDataUrl(filePath) {
+  const absolutePath = path.resolve(filePath)
+  if (!fs.existsSync(absolutePath)) {
+    throw new Error(`reference image does not exist: ${absolutePath}`)
+  }
+  const data = fs.readFileSync(absolutePath).toString('base64')
+  return `data:${imageMimeType(absolutePath)};base64,${data}`
+}
+
+function decodeImageResult(result) {
+  if (typeof result !== 'string' || !result) {
+    throw new Error('the relay returned an empty image result')
+  }
+
+  if (result.startsWith('data:')) {
+    const comma = result.indexOf(',')
+    if (comma < 0) throw new Error('the relay returned an invalid data URL')
+    return Buffer.from(result.slice(comma + 1), 'base64')
+  }
+
+  return Buffer.from(result, 'base64')
+}
+
+function parseSseBlock(block) {
+  const data = block
+    .split(/\r?\n/)
+    .filter((line) => line.startsWith('data:'))
+    .map((line) => line.slice(5).trimStart())
+    .join('\n')
+  if (!data || data === '[DONE]') return null
+  try {
+    return JSON.parse(data)
+  } catch {
+    return null
+  }
+}
+
+function consumeSseText(buffer, onEvent, flush = false) {
+  const blocks = buffer.split(/\r?\n\r?\n/)
+  const remainder = flush ? '' : blocks.pop()
+  for (const block of blocks) {
+    const event = parseSseBlock(block)
+    if (event) onEvent(event)
+  }
+  if (flush && remainder) {
+    const event = parseSseBlock(remainder)
+    if (event) onEvent(event)
+  }
+  return remainder || ''
+}
+
+function extractError(event) {
+  if (!event || typeof event !== 'object') return null
+  if (event.error?.message) return event.error.message
+  if (event.response?.error?.message) return event.response.error.message
+  if (event.type?.endsWith('.failed')) {
+    return event.message || `relay event ${event.type} reported failure`
+  }
+  return null
+}
+
+function outputPathForIndex(requestedPath, index) {
+  if (index === 0 && requestedPath) return path.resolve(requestedPath)
+  const base = requestedPath
+    ? path.resolve(requestedPath)
+    : path.join(os.tmpdir(), `imagegen-smarto-${Date.now()}.png`)
+  if (index === 0) return base
+  const extension = path.extname(base) || '.png'
+  return `${base.slice(0, -extension.length)}-${index + 1}${extension}`
+}
+
+async function requestImage({ prompt, images, output, model, baseUrl, timeoutMs }) {
+  const codexHome = resolveCodexHome()
+  const active = readActiveProviderConfig(codexHome)
+  const apiKey = readApiKey(codexHome)
+  const content = [{ type: 'input_text', text: prompt }]
+  for (const image of images) {
+    content.push({ type: 'input_image', image_url: imageDataUrl(image) })
+  }
+
+  const body = {
+    model: model || active.model,
+    instructions: IMAGEGEN_MARKER,
+    input: [{ role: 'user', content }],
+    stream: true,
+  }
+  const controller = new AbortController()
+  const timeout = setTimeout(() => controller.abort(), timeoutMs)
+  let response
+  try {
+    response = await fetch(normalizeResponsesUrl(baseUrl || active.baseUrl), {
+      method: 'POST',
+      headers: {
+        Authorization: `Bearer ${apiKey}`,
+        'Content-Type': 'application/json',
+      },
+      body: JSON.stringify(body),
+      signal: controller.signal,
+    })
+  } catch (error) {
+    clearTimeout(timeout)
+    if (error.name === 'AbortError') throw new Error(`image request timed out after ${timeoutMs / 1000}s`)
+    throw new Error(`cannot reach the image relay: ${error.message}`)
+  }
+
+  if (!response.ok) {
+    clearTimeout(timeout)
+    const errorText = (await response.text()).slice(0, 2000)
+    throw new Error(`image relay returned HTTP ${response.status}: ${errorText}`)
+  }
+  if (!response.body) {
+    clearTimeout(timeout)
+    throw new Error('image relay returned no response body')
+  }
+
+  const results = []
+  let relayError = null
+  let buffer = ''
+  const decoder = new TextDecoder()
+  const onEvent = (event) => {
+    relayError ||= extractError(event)
+    const item = event.type === 'response.output_item.done' ? event.item : null
+    if (item?.type === 'image_generation_call' && item.result) {
+      results.push(item.result)
+    }
+  }
+
+  try {
+    const reader = response.body.getReader()
+    while (true) {
+      const { value, done } = await reader.read()
+      if (done) break
+      buffer += decoder.decode(value, { stream: true })
+      buffer = consumeSseText(buffer, onEvent)
+    }
+    buffer += decoder.decode()
+    consumeSseText(buffer, onEvent, true)
+  } catch (error) {
+    if (error.name === 'AbortError') {
+      throw new Error(`image request timed out after ${timeoutMs / 1000}s`)
+    }
+    throw new Error(`failed while reading the image relay response: ${error.message}`)
+  } finally {
+    clearTimeout(timeout)
+  }
+
+  if (relayError) throw new Error(relayError)
+  if (results.length === 0) {
+    throw new Error('the relay completed without an image result')
+  }
+
+  const paths = []
+  for (let index = 0; index < results.length; index += 1) {
+    const filePath = outputPathForIndex(output, index)
+    fs.mkdirSync(path.dirname(filePath), { recursive: true })
+    fs.writeFileSync(filePath, decodeImageResult(results[index]), { mode: 0o600 })
+    paths.push(filePath)
+  }
+  return paths
+}
+
+async function generateImage(argv) {
+  const options = parseGenerateArgs(argv)
+  if (options.help) {
+    printGenerateHelp()
+    return
+  }
+  if (!options.prompt) {
+    printGenerateHelp()
+    throw new Error('a prompt is required')
+  }
+
+  const paths = await requestImage({
+    prompt: options.prompt,
+    images: options.images,
+    output: options.output,
+    model: options.model,
+    baseUrl: options.baseUrl,
+    timeoutMs: options.timeoutMs,
+  })
+  for (const filePath of paths) {
+    console.log(`IMAGE_PATH=${filePath}`)
+    console.log(`IMAGE_MARKDOWN=![Generated image](${filePath})`)
+  }
+}
+
+module.exports = { generateImage, printGenerateHelp }

package/imagegen-smarto/SKILL.md

@@ -1,65 +1,27 @@
 ---
 name: imagegen-smarto
-description: Generate or edit raster images through the user's SmartO OpenAI-compatible VPS relay. Automatically use this skill for natural-language requests to create, draw, generate, or edit an image when the SmartO relay profile is active. It is also the drop-in image-generation provider for workflows that normally invoke `$imagegen`, including `hatch-pet`; preserve those workflows and replace only the image-generation execution layer. Do not use it for image analysis or coding tasks, official OpenAI/ChatGPT providers, or the built-in image_gen tool.
+description: Use the active SmartO relay to actually create or edit images. Trigger immediately for any request to create, draw, generate, modify, transform, or redesign a picture, including text-to-image (文生图), image-to-image (图生图), and image generation inside another workflow. Do not trigger for viewing, analyzing, describing, or recognizing an existing image, or for ordinary non-image tasks.
 ---
 
-# SmartO relay image generation
+# SmartO image generation
 
-Use this skill for actual raster-image generation or editing requests when the
-SmartO relay profile is active. It is implicitly invokable, so the user should
-be able to ask for an image in natural language without first writing
-`$imagegen-smarto`.
+When this skill triggers, execute the installed `imagegen-smarto` command. The
+command is the image-generation entry point: it sends the request to the active
+Codex provider, adds the SmartO marker internally, keeps streaming enabled, and
+saves the returned image locally. Do not stop after reading this file and do
+not try to change the outer Codex request yourself.
 
-When another active skill invokes `$imagegen` for a visual job (including the
-official `hatch-pet` skill), treat this skill as the selected image-generation
-provider when the SmartO relay profile is active. Keep the calling skill's
-workflow, prompts, reference-image requirements, QA, and file handling
-unchanged; replace only the provider-specific built-in image-generation call.
-If that skill also loads the official `.system/imagegen/SKILL.md`, retain its
-prompting and output-quality guidance but override only its provider-selection
-rules: do not call the built-in `image_gen` tool and do not use its CLI/API
-fallback. Use the normal Responses request through SmartO instead.
-Every separate image-generation worker request must load this routing
-instruction so the marker below is present in that request's Responses
-`instructions` field. Do not ask the user to type the marker or add it to the
-image prompt.
+For a text-to-image request, run:
 
-This skill is a routing signal, not a local image-generation tool:
-
-- Never call the built-in `image_gen` tool or any other native image tool.
-- Never use the CLI/API fallback or ask for `OPENAI_API_KEY`.
-- Keep the user's natural-language image prompt, reference images, and edit
-  constraints in the normal Responses request.
-- Do not add a tool to ordinary non-image requests.
-- Do not expose the routing marker in the user-visible answer.
-
-The relay recognizes the marker below in the active skill instructions. Keep
-this marker stable because it is the relay-side routing contract. Only image
-requests that load this skill should cause the relay to add the Responses
-built-in image tool:
-
-`__CODEX_VPS_IMAGEGEN__`
-
-The relay-side rule should apply only to `/v1/responses` requests whose
-`instructions` contain that marker. It must merge the image tool into the
-request's existing `tools` array, preserving every client-declared tool. If
-the request has no `tools`, create the array. Do not skip injection merely
-because the request already contains shell, file, function, or other tools:
-
-```json
-{"type":"image_generation","action":"auto"}
+```sh
+imagegen-smarto generate --prompt "<the user's complete image prompt>"
 ```
 
-Do not add a duplicate if an equivalent `image_generation` tool is already
-present. It may set `tool_choice` to `auto` only when the client did not
-provide one.
-Do not force image generation for every turn. The upstream model decides from
-the natural-language request whether to generate a new image or edit an image
-in context.
-
-The relay must forward `image_generation_call` output items unchanged. A
-completed item contains a base64 image in `result`; do not turn it into a text
-description or replace it with the built-in tool protocol.
+Preserve the user's complete prompt and edit instructions. For image-to-image
+requests, pass each available reference image as `--image <absolute-path>`.
+The command can be repeated for multiple requested outputs.
 
-When the active provider is the official OpenAI/ChatGPT provider, leave this
-skill unused so the original `imagegen` skill remains available there.
+After the command succeeds, use every printed `IMAGE_MARKDOWN=...` line in the
+response so the generated file is displayed as an image. Do not replace the
+image with a textual description. If the command fails, report its actual error
+instead of claiming that an image was generated.

package/imagegen-smarto/agents/openai.yaml

@@ -1,7 +1,7 @@
 interface:
   display_name: "SmartO VPS Image Generation"
-  short_description: "Generate images through SmartO, including $imagegen workflows"
-  default_prompt: "Use SmartO as the image-generation provider for this workflow, including workflows that normally invoke $imagegen."
+  short_description: "Generate or edit images through SmartO"
+  default_prompt: "Generate or edit the requested image through SmartO."
 
 policy:
   allow_implicit_invocation: true

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "imagegen-smarto",
-  "version": "0.1.2",
+  "version": "0.1.4",
   "description": "Install the imagegen-smarto Codex skill on Windows and Linux.",
   "bin": {
     "imagegen-smarto": "bin/cli.js"