imagegen-smarto 0.1.2 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +46 -20
- package/bin/cli.js +17 -4
- package/bin/generate.js +391 -0
- package/imagegen-smarto/SKILL.md +30 -53
- package/imagegen-smarto/agents/openai.yaml +2 -2
- package/imagegen-smarto/references/prompting.md +99 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,19 +1,11 @@
|
|
|
1
1
|
# imagegen-smarto
|
|
2
2
|
|
|
3
|
-
Codex skill for
|
|
4
|
-
|
|
3
|
+
Codex skill for natural-language image generation and editing through a custom
|
|
4
|
+
SmartO OpenAI-compatible VPS relay.
|
|
5
5
|
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
existing function, shell, file, or other tools. Ordinary non-image requests
|
|
10
|
-
remain unchanged.
|
|
11
|
-
|
|
12
|
-
When the SmartO profile is active, this skill can also replace only the image
|
|
13
|
-
provider inside workflows that normally use `$imagegen`, including the
|
|
14
|
-
official `hatch-pet` workflow. The workflow's prompts, references, QA, and
|
|
15
|
-
file handling remain unchanged; only image generation is routed through
|
|
16
|
-
SmartO.
|
|
6
|
+
It implicitly handles requests to create, draw, generate, modify, perform
|
|
7
|
+
image-to-image, or perform text-to-image generation. Requests to view,
|
|
8
|
+
analyze, describe, or recognize an image remain ordinary requests.
|
|
17
9
|
|
|
18
10
|
## Install with npm
|
|
19
11
|
|
|
@@ -26,7 +18,13 @@ npm install --global imagegen-smarto
|
|
|
26
18
|
The npm postinstall step copies the skill to `CODEX_HOME/skills/imagegen-smarto`
|
|
27
19
|
or, when `CODEX_HOME` is not set, `~/.codex/skills/imagegen-smarto`.
|
|
28
20
|
|
|
29
|
-
To
|
|
21
|
+
To update to the latest published package:
|
|
22
|
+
|
|
23
|
+
```bash
|
|
24
|
+
npm install --global imagegen-smarto
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
To recopy the skill from the already installed npm package:
|
|
30
28
|
|
|
31
29
|
```bash
|
|
32
30
|
imagegen-smarto install
|
|
@@ -83,10 +81,39 @@ python3 ~/.codex/skills/.system/skill-installer/scripts/install-skill-from-githu
|
|
|
83
81
|
--method git
|
|
84
82
|
```
|
|
85
83
|
|
|
86
|
-
After installation, use
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
84
|
+
After installation, use the profile that points requests to SmartO. The skill
|
|
85
|
+
is configured for implicit invocation, so an image request can be written in
|
|
86
|
+
natural language without first typing `$imagegen-smarto`.
|
|
87
|
+
|
|
88
|
+
## Image generation workflow
|
|
89
|
+
|
|
90
|
+
When the skill is triggered, Codex first turns the request into the same kind
|
|
91
|
+
of structured, production-oriented prompt used by the official image skill.
|
|
92
|
+
Detailed prompts are preserved and normalized; generic prompts receive only
|
|
93
|
+
useful composition or presentation detail. Edits explicitly lock the parts
|
|
94
|
+
that must remain unchanged.
|
|
95
|
+
|
|
96
|
+
Codex then runs the installed `imagegen-smarto` command. The command uses the
|
|
97
|
+
active SmartO provider and credential, saves the returned PNG locally, and
|
|
98
|
+
prints an absolute `IMAGE_MARKDOWN=...` line so Codex can display it inline.
|
|
99
|
+
|
|
100
|
+
You can also test the execution path directly:
|
|
101
|
+
|
|
102
|
+
```bash
|
|
103
|
+
imagegen-smarto generate --prompt "一只戴红色围巾的小猫,儿童绘本风格"
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
For editing, add one or more reference images:
|
|
107
|
+
|
|
108
|
+
```bash
|
|
109
|
+
imagegen-smarto generate \
|
|
110
|
+
--prompt "把背景改成海边日落,保留主体" \
|
|
111
|
+
--image /absolute/path/to/source.png
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
The command handles its relay protocol internally; the skill only prepares the
|
|
115
|
+
final image prompt, supplies reference-image paths, and consumes the returned
|
|
116
|
+
image result.
|
|
90
117
|
|
|
91
118
|
## Local profile switch
|
|
92
119
|
|
|
@@ -97,5 +124,4 @@ codex --profile smarto
|
|
|
97
124
|
codex --profile official
|
|
98
125
|
```
|
|
99
126
|
|
|
100
|
-
`smarto` selects the custom relay and enables this skill.
|
|
101
|
-
the built-in OpenAI provider and enables Codex's original `imagegen` skill.
|
|
127
|
+
`smarto` selects the custom relay and enables this skill.
|
package/bin/cli.js
CHANGED
|
@@ -5,6 +5,7 @@
|
|
|
5
5
|
const fs = require('node:fs')
|
|
6
6
|
const os = require('node:os')
|
|
7
7
|
const path = require('node:path')
|
|
8
|
+
const { generateImage } = require('./generate')
|
|
8
9
|
|
|
9
10
|
const packageRoot = path.resolve(__dirname, '..')
|
|
10
11
|
const bundledSkillPath = path.join(packageRoot, 'imagegen-smarto')
|
|
@@ -16,9 +17,12 @@ Usage:
|
|
|
16
17
|
imagegen-smarto install [--codex-home <path>]
|
|
17
18
|
imagegen-smarto uninstall [--codex-home <path>]
|
|
18
19
|
imagegen-smarto path [--codex-home <path>]
|
|
20
|
+
imagegen-smarto generate --prompt "your prompt" [options]
|
|
19
21
|
imagegen-smarto --help
|
|
20
22
|
|
|
21
|
-
The default Codex home is CODEX_HOME or ~/.codex
|
|
23
|
+
The default Codex home is CODEX_HOME or ~/.codex.
|
|
24
|
+
|
|
25
|
+
Run imagegen-smarto generate --help for image generation options.`)
|
|
22
26
|
}
|
|
23
27
|
|
|
24
28
|
function parseArgs(argv) {
|
|
@@ -80,8 +84,14 @@ function uninstallSkill(codexHome) {
|
|
|
80
84
|
return target
|
|
81
85
|
}
|
|
82
86
|
|
|
83
|
-
function main() {
|
|
84
|
-
const
|
|
87
|
+
async function main() {
|
|
88
|
+
const argv = process.argv.slice(2)
|
|
89
|
+
if (argv[0] === 'generate') {
|
|
90
|
+
await generateImage(argv.slice(1))
|
|
91
|
+
return
|
|
92
|
+
}
|
|
93
|
+
|
|
94
|
+
const { command, codexHome: explicitCodexHome, quiet } = parseArgs(argv)
|
|
85
95
|
|
|
86
96
|
if (command === 'help') {
|
|
87
97
|
printHelp()
|
|
@@ -113,7 +123,10 @@ function main() {
|
|
|
113
123
|
}
|
|
114
124
|
|
|
115
125
|
try {
|
|
116
|
-
main()
|
|
126
|
+
main().catch((error) => {
|
|
127
|
+
console.error(`imagegen-smarto: ${error.message}`)
|
|
128
|
+
process.exitCode = 1
|
|
129
|
+
})
|
|
117
130
|
} catch (error) {
|
|
118
131
|
console.error(`imagegen-smarto: ${error.message}`)
|
|
119
132
|
process.exitCode = 1
|
package/bin/generate.js
ADDED
|
@@ -0,0 +1,391 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
|
|
3
|
+
'use strict'
|
|
4
|
+
|
|
5
|
+
const fs = require('node:fs')
|
|
6
|
+
const os = require('node:os')
|
|
7
|
+
const path = require('node:path')
|
|
8
|
+
|
|
9
|
+
const IMAGEGEN_MARKER = '__CODEX_VPS_IMAGEGEN__'
|
|
10
|
+
const DEFAULT_TIMEOUT_MS = 10 * 60 * 1000
|
|
11
|
+
|
|
12
|
+
function printGenerateHelp() {
|
|
13
|
+
console.log(`imagegen-smarto generate - generate or edit an image through the active relay
|
|
14
|
+
|
|
15
|
+
Usage:
|
|
16
|
+
imagegen-smarto generate --prompt "your prompt" [options]
|
|
17
|
+
imagegen-smarto generate "your prompt" [options]
|
|
18
|
+
|
|
19
|
+
Options:
|
|
20
|
+
--prompt <text> Prompt for generation or editing
|
|
21
|
+
--image <path> Reference image; may be repeated for image editing
|
|
22
|
+
--output <path> Output PNG path (default: a temporary file)
|
|
23
|
+
--model <model> Override the model from Codex config
|
|
24
|
+
--base-url <url> Override the active provider base URL
|
|
25
|
+
--timeout <seconds> Request timeout (default: 600)
|
|
26
|
+
--help Show this help`)
|
|
27
|
+
}
|
|
28
|
+
|
|
29
|
+
function parseGenerateArgs(argv) {
|
|
30
|
+
const options = {
|
|
31
|
+
images: [],
|
|
32
|
+
prompt: null,
|
|
33
|
+
output: null,
|
|
34
|
+
model: null,
|
|
35
|
+
baseUrl: null,
|
|
36
|
+
timeoutMs: DEFAULT_TIMEOUT_MS,
|
|
37
|
+
}
|
|
38
|
+
const positional = []
|
|
39
|
+
|
|
40
|
+
for (let index = 0; index < argv.length; index += 1) {
|
|
41
|
+
const arg = argv[index]
|
|
42
|
+
if (arg === '--help' || arg === '-h') {
|
|
43
|
+
options.help = true
|
|
44
|
+
continue
|
|
45
|
+
}
|
|
46
|
+
|
|
47
|
+
if (arg === '--prompt') {
|
|
48
|
+
options.prompt = argv[++index]
|
|
49
|
+
if (!options.prompt) throw new Error('--prompt requires text')
|
|
50
|
+
continue
|
|
51
|
+
}
|
|
52
|
+
|
|
53
|
+
if (arg === '--image' || arg === '--input-image') {
|
|
54
|
+
const image = argv[++index]
|
|
55
|
+
if (!image) throw new Error(`${arg} requires a path`)
|
|
56
|
+
options.images.push(image)
|
|
57
|
+
continue
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
if (arg === '--output') {
|
|
61
|
+
options.output = argv[++index]
|
|
62
|
+
if (!options.output) throw new Error('--output requires a path')
|
|
63
|
+
continue
|
|
64
|
+
}
|
|
65
|
+
|
|
66
|
+
if (arg === '--model') {
|
|
67
|
+
options.model = argv[++index]
|
|
68
|
+
if (!options.model) throw new Error('--model requires a model name')
|
|
69
|
+
continue
|
|
70
|
+
}
|
|
71
|
+
|
|
72
|
+
if (arg === '--base-url') {
|
|
73
|
+
options.baseUrl = argv[++index]
|
|
74
|
+
if (!options.baseUrl) throw new Error('--base-url requires a URL')
|
|
75
|
+
continue
|
|
76
|
+
}
|
|
77
|
+
|
|
78
|
+
if (arg === '--timeout') {
|
|
79
|
+
const seconds = Number(argv[++index])
|
|
80
|
+
if (!Number.isFinite(seconds) || seconds <= 0) {
|
|
81
|
+
throw new Error('--timeout must be a positive number of seconds')
|
|
82
|
+
}
|
|
83
|
+
options.timeoutMs = seconds * 1000
|
|
84
|
+
continue
|
|
85
|
+
}
|
|
86
|
+
|
|
87
|
+
if (arg.startsWith('-')) {
|
|
88
|
+
throw new Error(`unknown option: ${arg}`)
|
|
89
|
+
}
|
|
90
|
+
positional.push(arg)
|
|
91
|
+
}
|
|
92
|
+
|
|
93
|
+
if (!options.prompt && positional.length > 0) {
|
|
94
|
+
options.prompt = positional.join(' ')
|
|
95
|
+
}
|
|
96
|
+
|
|
97
|
+
return options
|
|
98
|
+
}
|
|
99
|
+
|
|
100
|
+
function resolveCodexHome() {
|
|
101
|
+
const configuredPath = process.env.CODEX_HOME
|
|
102
|
+
if (configuredPath && configuredPath.trim()) {
|
|
103
|
+
return path.resolve(configuredPath)
|
|
104
|
+
}
|
|
105
|
+
return path.join(os.homedir(), '.codex')
|
|
106
|
+
}
|
|
107
|
+
|
|
108
|
+
function readCodexConfig(codexHome) {
|
|
109
|
+
const configPath = path.join(codexHome, 'config.toml')
|
|
110
|
+
try {
|
|
111
|
+
return fs.readFileSync(configPath, 'utf8')
|
|
112
|
+
} catch (error) {
|
|
113
|
+
if (error.code === 'ENOENT') return ''
|
|
114
|
+
throw new Error(`cannot read ${configPath}: ${error.message}`)
|
|
115
|
+
}
|
|
116
|
+
}
|
|
117
|
+
|
|
118
|
+
function tomlStringValue(text, key, section) {
|
|
119
|
+
let sectionText = text
|
|
120
|
+
|
|
121
|
+
if (section) {
|
|
122
|
+
const sectionHeader = `[model_providers.${section}]`
|
|
123
|
+
const sectionStart = text.indexOf(sectionHeader)
|
|
124
|
+
if (sectionStart >= 0) {
|
|
125
|
+
const contentStart = text.indexOf('\n', sectionStart)
|
|
126
|
+
const nextSection = text.slice(contentStart + 1).search(/^\s*\[/m)
|
|
127
|
+
sectionText = nextSection >= 0
|
|
128
|
+
? text.slice(contentStart + 1, contentStart + 1 + nextSection)
|
|
129
|
+
: text.slice(contentStart + 1)
|
|
130
|
+
} else {
|
|
131
|
+
sectionText = ''
|
|
132
|
+
}
|
|
133
|
+
}
|
|
134
|
+
|
|
135
|
+
const match = sectionText.match(new RegExp(`^\\s*${key}\\s*=\\s*["']([^"']+)["']`, 'm'))
|
|
136
|
+
return match ? match[1] : null
|
|
137
|
+
}
|
|
138
|
+
|
|
139
|
+
function readActiveProviderConfig(codexHome) {
|
|
140
|
+
const config = readCodexConfig(codexHome)
|
|
141
|
+
const provider = tomlStringValue(config, 'model_provider') || 'openai'
|
|
142
|
+
const baseUrl = process.env.IMAGEGEN_SMARTO_BASE_URL
|
|
143
|
+
|| process.env.OPENAI_BASE_URL
|
|
144
|
+
|| tomlStringValue(config, 'base_url', provider)
|
|
145
|
+
|| tomlStringValue(config, 'base_url')
|
|
146
|
+
const model = process.env.IMAGEGEN_SMARTO_MODEL
|
|
147
|
+
|| tomlStringValue(config, 'model')
|
|
148
|
+
|
|
149
|
+
if (!baseUrl) {
|
|
150
|
+
throw new Error(
|
|
151
|
+
`no active provider base_url found in ${path.join(codexHome, 'config.toml')}; `
|
|
152
|
+
+ 'set IMAGEGEN_SMARTO_BASE_URL or use a Codex profile with a base_url',
|
|
153
|
+
)
|
|
154
|
+
}
|
|
155
|
+
if (!model) {
|
|
156
|
+
throw new Error(
|
|
157
|
+
`no active model found in ${path.join(codexHome, 'config.toml')}; `
|
|
158
|
+
+ 'set IMAGEGEN_SMARTO_MODEL or pass --model',
|
|
159
|
+
)
|
|
160
|
+
}
|
|
161
|
+
|
|
162
|
+
return { baseUrl, model }
|
|
163
|
+
}
|
|
164
|
+
|
|
165
|
+
function readApiKey(codexHome) {
|
|
166
|
+
if (process.env.IMAGEGEN_SMARTO_API_KEY) return process.env.IMAGEGEN_SMARTO_API_KEY
|
|
167
|
+
if (process.env.OPENAI_API_KEY) return process.env.OPENAI_API_KEY
|
|
168
|
+
|
|
169
|
+
const authPath = path.join(codexHome, 'auth.json')
|
|
170
|
+
try {
|
|
171
|
+
const auth = JSON.parse(fs.readFileSync(authPath, 'utf8'))
|
|
172
|
+
const candidates = [
|
|
173
|
+
auth.OPENAI_API_KEY,
|
|
174
|
+
auth.openai_api_key,
|
|
175
|
+
auth.api_key,
|
|
176
|
+
auth.access_token,
|
|
177
|
+
auth.tokens?.access_token,
|
|
178
|
+
]
|
|
179
|
+
const key = candidates.find((value) => typeof value === 'string' && value.trim())
|
|
180
|
+
if (key) return key
|
|
181
|
+
} catch (error) {
|
|
182
|
+
if (error.code !== 'ENOENT') {
|
|
183
|
+
throw new Error(`cannot read ${authPath}: ${error.message}`)
|
|
184
|
+
}
|
|
185
|
+
}
|
|
186
|
+
|
|
187
|
+
throw new Error(
|
|
188
|
+
`no API credential found; set OPENAI_API_KEY or log in so ${authPath} contains OPENAI_API_KEY`,
|
|
189
|
+
)
|
|
190
|
+
}
|
|
191
|
+
|
|
192
|
+
function normalizeResponsesUrl(baseUrl) {
|
|
193
|
+
const normalized = baseUrl.replace(/\/+$/, '')
|
|
194
|
+
return normalized.endsWith('/responses') ? normalized : `${normalized}/responses`
|
|
195
|
+
}
|
|
196
|
+
|
|
197
|
+
function imageMimeType(filePath) {
|
|
198
|
+
const extension = path.extname(filePath).toLowerCase()
|
|
199
|
+
if (extension === '.jpg' || extension === '.jpeg') return 'image/jpeg'
|
|
200
|
+
if (extension === '.webp') return 'image/webp'
|
|
201
|
+
if (extension === '.gif') return 'image/gif'
|
|
202
|
+
return 'image/png'
|
|
203
|
+
}
|
|
204
|
+
|
|
205
|
+
function imageDataUrl(filePath) {
|
|
206
|
+
const absolutePath = path.resolve(filePath)
|
|
207
|
+
if (!fs.existsSync(absolutePath)) {
|
|
208
|
+
throw new Error(`reference image does not exist: ${absolutePath}`)
|
|
209
|
+
}
|
|
210
|
+
const data = fs.readFileSync(absolutePath).toString('base64')
|
|
211
|
+
return `data:${imageMimeType(absolutePath)};base64,${data}`
|
|
212
|
+
}
|
|
213
|
+
|
|
214
|
+
function decodeImageResult(result) {
|
|
215
|
+
if (typeof result !== 'string' || !result) {
|
|
216
|
+
throw new Error('the relay returned an empty image result')
|
|
217
|
+
}
|
|
218
|
+
|
|
219
|
+
if (result.startsWith('data:')) {
|
|
220
|
+
const comma = result.indexOf(',')
|
|
221
|
+
if (comma < 0) throw new Error('the relay returned an invalid data URL')
|
|
222
|
+
return Buffer.from(result.slice(comma + 1), 'base64')
|
|
223
|
+
}
|
|
224
|
+
|
|
225
|
+
return Buffer.from(result, 'base64')
|
|
226
|
+
}
|
|
227
|
+
|
|
228
|
+
function parseSseBlock(block) {
|
|
229
|
+
const data = block
|
|
230
|
+
.split(/\r?\n/)
|
|
231
|
+
.filter((line) => line.startsWith('data:'))
|
|
232
|
+
.map((line) => line.slice(5).trimStart())
|
|
233
|
+
.join('\n')
|
|
234
|
+
if (!data || data === '[DONE]') return null
|
|
235
|
+
try {
|
|
236
|
+
return JSON.parse(data)
|
|
237
|
+
} catch {
|
|
238
|
+
return null
|
|
239
|
+
}
|
|
240
|
+
}
|
|
241
|
+
|
|
242
|
+
function consumeSseText(buffer, onEvent, flush = false) {
|
|
243
|
+
const blocks = buffer.split(/\r?\n\r?\n/)
|
|
244
|
+
const remainder = flush ? '' : blocks.pop()
|
|
245
|
+
for (const block of blocks) {
|
|
246
|
+
const event = parseSseBlock(block)
|
|
247
|
+
if (event) onEvent(event)
|
|
248
|
+
}
|
|
249
|
+
if (flush && remainder) {
|
|
250
|
+
const event = parseSseBlock(remainder)
|
|
251
|
+
if (event) onEvent(event)
|
|
252
|
+
}
|
|
253
|
+
return remainder || ''
|
|
254
|
+
}
|
|
255
|
+
|
|
256
|
+
function extractError(event) {
|
|
257
|
+
if (!event || typeof event !== 'object') return null
|
|
258
|
+
if (event.error?.message) return event.error.message
|
|
259
|
+
if (event.response?.error?.message) return event.response.error.message
|
|
260
|
+
if (event.type?.endsWith('.failed')) {
|
|
261
|
+
return event.message || `relay event ${event.type} reported failure`
|
|
262
|
+
}
|
|
263
|
+
return null
|
|
264
|
+
}
|
|
265
|
+
|
|
266
|
+
function outputPathForIndex(requestedPath, index) {
|
|
267
|
+
if (index === 0 && requestedPath) return path.resolve(requestedPath)
|
|
268
|
+
const base = requestedPath
|
|
269
|
+
? path.resolve(requestedPath)
|
|
270
|
+
: path.join(os.tmpdir(), `imagegen-smarto-${Date.now()}.png`)
|
|
271
|
+
if (index === 0) return base
|
|
272
|
+
const extension = path.extname(base) || '.png'
|
|
273
|
+
return `${base.slice(0, -extension.length)}-${index + 1}${extension}`
|
|
274
|
+
}
|
|
275
|
+
|
|
276
|
+
async function requestImage({ prompt, images, output, model, baseUrl, timeoutMs }) {
|
|
277
|
+
const codexHome = resolveCodexHome()
|
|
278
|
+
const active = readActiveProviderConfig(codexHome)
|
|
279
|
+
const apiKey = readApiKey(codexHome)
|
|
280
|
+
const content = [{ type: 'input_text', text: prompt }]
|
|
281
|
+
for (const image of images) {
|
|
282
|
+
content.push({ type: 'input_image', image_url: imageDataUrl(image) })
|
|
283
|
+
}
|
|
284
|
+
|
|
285
|
+
const body = {
|
|
286
|
+
model: model || active.model,
|
|
287
|
+
instructions: IMAGEGEN_MARKER,
|
|
288
|
+
input: [{ role: 'user', content }],
|
|
289
|
+
stream: true,
|
|
290
|
+
}
|
|
291
|
+
const controller = new AbortController()
|
|
292
|
+
const timeout = setTimeout(() => controller.abort(), timeoutMs)
|
|
293
|
+
let response
|
|
294
|
+
try {
|
|
295
|
+
response = await fetch(normalizeResponsesUrl(baseUrl || active.baseUrl), {
|
|
296
|
+
method: 'POST',
|
|
297
|
+
headers: {
|
|
298
|
+
Authorization: `Bearer ${apiKey}`,
|
|
299
|
+
'Content-Type': 'application/json',
|
|
300
|
+
},
|
|
301
|
+
body: JSON.stringify(body),
|
|
302
|
+
signal: controller.signal,
|
|
303
|
+
})
|
|
304
|
+
} catch (error) {
|
|
305
|
+
clearTimeout(timeout)
|
|
306
|
+
if (error.name === 'AbortError') throw new Error(`image request timed out after ${timeoutMs / 1000}s`)
|
|
307
|
+
throw new Error(`cannot reach the image relay: ${error.message}`)
|
|
308
|
+
}
|
|
309
|
+
|
|
310
|
+
if (!response.ok) {
|
|
311
|
+
clearTimeout(timeout)
|
|
312
|
+
const errorText = (await response.text()).slice(0, 2000)
|
|
313
|
+
throw new Error(`image relay returned HTTP ${response.status}: ${errorText}`)
|
|
314
|
+
}
|
|
315
|
+
if (!response.body) {
|
|
316
|
+
clearTimeout(timeout)
|
|
317
|
+
throw new Error('image relay returned no response body')
|
|
318
|
+
}
|
|
319
|
+
|
|
320
|
+
const results = []
|
|
321
|
+
let relayError = null
|
|
322
|
+
let buffer = ''
|
|
323
|
+
const decoder = new TextDecoder()
|
|
324
|
+
const onEvent = (event) => {
|
|
325
|
+
relayError ||= extractError(event)
|
|
326
|
+
const item = event.type === 'response.output_item.done' ? event.item : null
|
|
327
|
+
if (item?.type === 'image_generation_call' && item.result) {
|
|
328
|
+
results.push(item.result)
|
|
329
|
+
}
|
|
330
|
+
}
|
|
331
|
+
|
|
332
|
+
try {
|
|
333
|
+
const reader = response.body.getReader()
|
|
334
|
+
while (true) {
|
|
335
|
+
const { value, done } = await reader.read()
|
|
336
|
+
if (done) break
|
|
337
|
+
buffer += decoder.decode(value, { stream: true })
|
|
338
|
+
buffer = consumeSseText(buffer, onEvent)
|
|
339
|
+
}
|
|
340
|
+
buffer += decoder.decode()
|
|
341
|
+
consumeSseText(buffer, onEvent, true)
|
|
342
|
+
} catch (error) {
|
|
343
|
+
if (error.name === 'AbortError') {
|
|
344
|
+
throw new Error(`image request timed out after ${timeoutMs / 1000}s`)
|
|
345
|
+
}
|
|
346
|
+
throw new Error(`failed while reading the image relay response: ${error.message}`)
|
|
347
|
+
} finally {
|
|
348
|
+
clearTimeout(timeout)
|
|
349
|
+
}
|
|
350
|
+
|
|
351
|
+
if (relayError) throw new Error(relayError)
|
|
352
|
+
if (results.length === 0) {
|
|
353
|
+
throw new Error('the relay completed without an image result')
|
|
354
|
+
}
|
|
355
|
+
|
|
356
|
+
const paths = []
|
|
357
|
+
for (let index = 0; index < results.length; index += 1) {
|
|
358
|
+
const filePath = outputPathForIndex(output, index)
|
|
359
|
+
fs.mkdirSync(path.dirname(filePath), { recursive: true })
|
|
360
|
+
fs.writeFileSync(filePath, decodeImageResult(results[index]), { mode: 0o600 })
|
|
361
|
+
paths.push(filePath)
|
|
362
|
+
}
|
|
363
|
+
return paths
|
|
364
|
+
}
|
|
365
|
+
|
|
366
|
+
async function generateImage(argv) {
|
|
367
|
+
const options = parseGenerateArgs(argv)
|
|
368
|
+
if (options.help) {
|
|
369
|
+
printGenerateHelp()
|
|
370
|
+
return
|
|
371
|
+
}
|
|
372
|
+
if (!options.prompt) {
|
|
373
|
+
printGenerateHelp()
|
|
374
|
+
throw new Error('a prompt is required')
|
|
375
|
+
}
|
|
376
|
+
|
|
377
|
+
const paths = await requestImage({
|
|
378
|
+
prompt: options.prompt,
|
|
379
|
+
images: options.images,
|
|
380
|
+
output: options.output,
|
|
381
|
+
model: options.model,
|
|
382
|
+
baseUrl: options.baseUrl,
|
|
383
|
+
timeoutMs: options.timeoutMs,
|
|
384
|
+
})
|
|
385
|
+
for (const filePath of paths) {
|
|
386
|
+
console.log(`IMAGE_PATH=${filePath}`)
|
|
387
|
+
console.log(`IMAGE_MARKDOWN=`)
|
|
388
|
+
}
|
|
389
|
+
}
|
|
390
|
+
|
|
391
|
+
module.exports = { generateImage, printGenerateHelp }
|
package/imagegen-smarto/SKILL.md
CHANGED
|
@@ -1,65 +1,42 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: imagegen-smarto
|
|
3
|
-
description:
|
|
3
|
+
description: Use the active SmartO relay to create or edit raster images. Trigger immediately for requests to create, draw, generate, modify, transform, or redesign a picture, including text-to-image (文生图), image-to-image (图生图), and image generation inside another workflow. Do not trigger for viewing, analyzing, describing, or recognizing an existing image, ordinary non-image tasks, or output better produced as SVG or code-native graphics.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
-
# SmartO
|
|
6
|
+
# SmartO image generation
|
|
7
7
|
|
|
8
|
-
Use
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
`$imagegen-smarto`.
|
|
8
|
+
Use `imagegen-smarto generate` to produce the requested image. The command is
|
|
9
|
+
the image-generation tool for this skill; execute it instead of stopping after
|
|
10
|
+
reading these instructions.
|
|
12
11
|
|
|
13
|
-
|
|
14
|
-
official `hatch-pet` skill), treat this skill as the selected image-generation
|
|
15
|
-
provider when the SmartO relay profile is active. Keep the calling skill's
|
|
16
|
-
workflow, prompts, reference-image requirements, QA, and file handling
|
|
17
|
-
unchanged; replace only the provider-specific built-in image-generation call.
|
|
18
|
-
If that skill also loads the official `.system/imagegen/SKILL.md`, retain its
|
|
19
|
-
prompting and output-quality guidance but override only its provider-selection
|
|
20
|
-
rules: do not call the built-in `image_gen` tool and do not use its CLI/API
|
|
21
|
-
fallback. Use the normal Responses request through SmartO instead.
|
|
22
|
-
Every separate image-generation worker request must load this routing
|
|
23
|
-
instruction so the marker below is present in that request's Responses
|
|
24
|
-
`instructions` field. Do not ask the user to type the marker or add it to the
|
|
25
|
-
image prompt.
|
|
12
|
+
## Workflow
|
|
26
13
|
|
|
27
|
-
|
|
14
|
+
1. Decide whether this is generation or an edit. Treat supplied images as edit
|
|
15
|
+
inputs only when the user asks to change or combine them; otherwise label
|
|
16
|
+
their reference role in the prompt.
|
|
17
|
+
2. Rewrite the user's request into a strong image prompt using
|
|
18
|
+
[references/prompting.md](references/prompting.md). Preserve all explicit
|
|
19
|
+
requirements. Normalize detailed prompts; augment generic prompts only when
|
|
20
|
+
the addition materially improves the result.
|
|
21
|
+
3. Generate one requested asset or variant per command:
|
|
28
22
|
|
|
29
|
-
|
|
30
|
-
-
|
|
31
|
-
|
|
32
|
-
constraints in the normal Responses request.
|
|
33
|
-
- Do not add a tool to ordinary non-image requests.
|
|
34
|
-
- Do not expose the routing marker in the user-visible answer.
|
|
23
|
+
```sh
|
|
24
|
+
imagegen-smarto generate --prompt "<final structured prompt>"
|
|
25
|
+
```
|
|
35
26
|
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
requests that load this skill should cause the relay to add the Responses
|
|
39
|
-
built-in image tool:
|
|
27
|
+
4. For an edit, add each available source or reference image by absolute path.
|
|
28
|
+
Up to five images may be supplied:
|
|
40
29
|
|
|
41
|
-
|
|
30
|
+
```sh
|
|
31
|
+
imagegen-smarto generate --prompt "<final structured edit prompt>" \
|
|
32
|
+
--image /absolute/path/to/input.png
|
|
33
|
+
```
|
|
42
34
|
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
35
|
+
5. If the user named an output location, pass `--output <path>`. Do not
|
|
36
|
+
overwrite an existing asset unless replacement was explicitly requested.
|
|
37
|
+
6. On success, include every printed `IMAGE_MARKDOWN=...` value in the answer
|
|
38
|
+
so Codex displays the generated file. Report the saved path. For a failed
|
|
39
|
+
command, report the actual error and do not claim that an image exists.
|
|
48
40
|
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
```
|
|
52
|
-
|
|
53
|
-
Do not add a duplicate if an equivalent `image_generation` tool is already
|
|
54
|
-
present. It may set `tool_choice` to `auto` only when the client did not
|
|
55
|
-
provide one.
|
|
56
|
-
Do not force image generation for every turn. The upstream model decides from
|
|
57
|
-
the natural-language request whether to generate a new image or edit an image
|
|
58
|
-
in context.
|
|
59
|
-
|
|
60
|
-
The relay must forward `image_generation_call` output items unchanged. A
|
|
61
|
-
completed item contains a base64 image in `result`; do not turn it into a text
|
|
62
|
-
description or replace it with the built-in tool protocol.
|
|
63
|
-
|
|
64
|
-
When the active provider is the official OpenAI/ChatGPT provider, leave this
|
|
65
|
-
skill unused so the original `imagegen` skill remains available there.
|
|
41
|
+
For edits, state invariants explicitly in the prompt: `change only X; keep Y
|
|
42
|
+
unchanged`. When iterating, make one targeted change and repeat the invariants.
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
interface:
|
|
2
2
|
display_name: "SmartO VPS Image Generation"
|
|
3
|
-
short_description: "Generate images through SmartO
|
|
4
|
-
default_prompt: "
|
|
3
|
+
short_description: "Generate or edit images through SmartO"
|
|
4
|
+
default_prompt: "Generate or edit the requested image through SmartO."
|
|
5
5
|
|
|
6
6
|
policy:
|
|
7
7
|
allow_implicit_invocation: true
|
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
# Image prompting
|
|
2
|
+
|
|
3
|
+
Shape the user's request into a concise production-oriented specification.
|
|
4
|
+
Use only the lines that improve the request; the schema is scaffolding, not a
|
|
5
|
+
form that must always be filled.
|
|
6
|
+
|
|
7
|
+
## Specificity
|
|
8
|
+
|
|
9
|
+
- If the prompt is already detailed, preserve it and only normalize its order
|
|
10
|
+
and wording.
|
|
11
|
+
- If the prompt is generic, add tasteful composition, framing, intended-use,
|
|
12
|
+
polish, or scene detail when it materially improves the output.
|
|
13
|
+
- Do not invent extra characters, props, brands, slogans, palettes, or story
|
|
14
|
+
beats. Do not choose arbitrary left/right placement without layout context.
|
|
15
|
+
- Keep exact user constraints and requested text unchanged.
|
|
16
|
+
|
|
17
|
+
## Prompt structure
|
|
18
|
+
|
|
19
|
+
For complex requests, use short labeled lines in this order:
|
|
20
|
+
|
|
21
|
+
```text
|
|
22
|
+
Use case: <taxonomy slug>
|
|
23
|
+
Asset type: <where the image will be used> (optional)
|
|
24
|
+
Primary request: <the user's main request>
|
|
25
|
+
Input images: <Image 1: role; Image 2: role> (optional)
|
|
26
|
+
Scene/backdrop: <environment>
|
|
27
|
+
Subject: <main subject and important details>
|
|
28
|
+
Style/medium: <photo, illustration, 3D, etc.>
|
|
29
|
+
Composition/framing: <viewpoint, crop, placement, negative space>
|
|
30
|
+
Lighting/mood: <lighting and atmosphere>
|
|
31
|
+
Color palette: <requested or implied palette>
|
|
32
|
+
Materials/textures: <important surface detail>
|
|
33
|
+
Text (verbatim): "<exact text>"
|
|
34
|
+
Constraints: <must preserve and must include>
|
|
35
|
+
Avoid: <negative constraints>
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Simple requests can remain short. Add only the fields needed to clarify the
|
|
39
|
+
result.
|
|
40
|
+
|
|
41
|
+
## Use-case taxonomy
|
|
42
|
+
|
|
43
|
+
Generation:
|
|
44
|
+
|
|
45
|
+
- `photorealistic-natural`: candid or editorial scenes with natural lighting
|
|
46
|
+
and real texture.
|
|
47
|
+
- `product-mockup`: product, packaging, catalog, or merchandise imagery.
|
|
48
|
+
- `ui-mockup`: practical app or web interface imagery at a stated fidelity.
|
|
49
|
+
- `infographic-diagram`: structured diagrams with explicit layout and labels.
|
|
50
|
+
- `scientific-educational`: accurate teaching visuals for a named audience.
|
|
51
|
+
- `ads-marketing`: campaign imagery with audience, positioning, and exact copy.
|
|
52
|
+
- `productivity-visual`: slides, charts, workflows, and business visuals.
|
|
53
|
+
- `logo-brand`: simple, scalable mark exploration with a strong silhouette.
|
|
54
|
+
- `illustration-story`: comics, children's art, and narrative scenes.
|
|
55
|
+
- `stylized-concept`: style-driven concept art or rendered scenes.
|
|
56
|
+
- `historical-scene`: period-specific scenes requiring factual accuracy.
|
|
57
|
+
|
|
58
|
+
Editing:
|
|
59
|
+
|
|
60
|
+
- `text-localization`: replace only in-image text and preserve layout.
|
|
61
|
+
- `identity-preserve`: preserve face, body, pose, hair, and expression.
|
|
62
|
+
- `precise-object-edit`: replace or remove a named element only.
|
|
63
|
+
- `lighting-weather`: change environmental conditions while preserving content.
|
|
64
|
+
- `background-extraction`: create a clean transparent-background cutout.
|
|
65
|
+
- `style-transfer`: apply reference style without introducing extra elements.
|
|
66
|
+
- `compositing`: combine indexed inputs with matched scale, light, perspective.
|
|
67
|
+
- `sketch-to-render`: preserve layout, proportions, and perspective.
|
|
68
|
+
|
|
69
|
+
## Composition and realism
|
|
70
|
+
|
|
71
|
+
- Specify framing and viewpoint only when useful: close-up, full body, wide,
|
|
72
|
+
eye-level, low-angle, top-down.
|
|
73
|
+
- Call out negative space when the image must leave room for UI or copy.
|
|
74
|
+
- For people, clarify body framing, gaze, pose, and object interactions when
|
|
75
|
+
they matter.
|
|
76
|
+
- For photorealism, explicitly request photorealism and concrete natural detail
|
|
77
|
+
such as skin texture, fabric wear, material grain, and imperfect surfaces.
|
|
78
|
+
|
|
79
|
+
## Edits and references
|
|
80
|
+
|
|
81
|
+
- Label every supplied image by index and role: edit target, style reference,
|
|
82
|
+
composition reference, or compositing input.
|
|
83
|
+
- For edits, use `change only X; keep Y unchanged` and list all invariants.
|
|
84
|
+
- For compositing, state what moves from each indexed image and require matched
|
|
85
|
+
lighting, perspective, and scale.
|
|
86
|
+
- Repeat invariants on every edit iteration to reduce drift.
|
|
87
|
+
|
|
88
|
+
## Text in images
|
|
89
|
+
|
|
90
|
+
- Quote exact text, require verbatim rendering, and specify typography and
|
|
91
|
+
placement when those details matter.
|
|
92
|
+
- Spell uncommon words letter by letter when accuracy is critical.
|
|
93
|
+
- Require no extra characters or text beyond the supplied copy.
|
|
94
|
+
|
|
95
|
+
## Iteration
|
|
96
|
+
|
|
97
|
+
Start with a clean base prompt. Inspect the result against subject, style,
|
|
98
|
+
composition, exact text, invariants, and avoid items. Refine with one targeted
|
|
99
|
+
change at a time instead of rewriting unrelated parts of the prompt.
|