@ishlabs/cli 0.8.3 → 0.8.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +6 -0
- package/dist/auth.d.ts +1 -0
- package/dist/auth.js +12 -3
- package/dist/commands/ask.js +59 -16
- package/dist/commands/iteration.js +45 -11
- package/dist/commands/profile.js +65 -12
- package/dist/commands/study-run.js +49 -0
- package/dist/commands/study-tester.js +5 -2
- package/dist/commands/study.js +71 -16
- package/dist/connect.js +7 -7
- package/dist/index.js +119 -2
- package/dist/lib/api-client.js +29 -7
- package/dist/lib/command-helpers.d.ts +14 -0
- package/dist/lib/command-helpers.js +40 -0
- package/dist/lib/docs.js +430 -13
- package/dist/lib/output.js +437 -63
- package/dist/lib/skill-content.js +102 -9
- package/dist/lib/types.d.ts +3 -1
- package/dist/upgrade.js +3 -3
- package/package.json +1 -1
|
@@ -96,6 +96,10 @@ use **ask** for quick reactions to text/image variants.
|
|
|
96
96
|
## High-frequency commands
|
|
97
97
|
|
|
98
98
|
\`\`\`bash
|
|
99
|
+
# First command on a cold start — confirms login + active context:
|
|
100
|
+
ish status # or: ish whoami
|
|
101
|
+
# → user, active workspace/study/ask, token validity, API url
|
|
102
|
+
|
|
99
103
|
# Auth & active selection (saved to ~/.ish/config.json)
|
|
100
104
|
ish login
|
|
101
105
|
ish workspace use w-6ec
|
|
@@ -142,9 +146,47 @@ See \`references/workflows.md\` in this skill for end-to-end transcripts:
|
|
|
142
146
|
stdout is piped**, so an agent rarely needs \`--json\` explicitly.
|
|
143
147
|
- \`--fields a,b,c\` strips JSON output to the listed fields (saves
|
|
144
148
|
tokens). \`--verbose\` adds full UUIDs and timestamps.
|
|
149
|
+
- **Stdout is data only.** All progress, status, and "Open in browser"
|
|
150
|
+
hints go to stderr; \`--json | jq -e .\` parses cleanly without
|
|
151
|
+
defensive piping.
|
|
152
|
+
- **List responses are a six-key envelope:** \`{items, total, returned,
|
|
153
|
+
limit, offset, has_more}\`. Use \`has_more\` to detect truncation;
|
|
154
|
+
don't count items yourself.
|
|
155
|
+
- **Use \`runtime_status\`, not \`status\`, on study responses.** Values:
|
|
156
|
+
\`draft | running | completed | completed_with_errors | cancelled\`.
|
|
157
|
+
Derived from iteration testers' actual state — never reports
|
|
158
|
+
\`failed\` while completed runs exist. The CLI also surfaces
|
|
159
|
+
\`status_inferred\` + a stderr warning when raw \`status\` and the
|
|
160
|
+
testers disagree.
|
|
161
|
+
- **\`study generate --json\` returns \`modality_rationale\`** (one
|
|
162
|
+
short sentence). Inspect it before adding iterations; if the LLM
|
|
163
|
+
picked the wrong modality, override via
|
|
164
|
+
\`ish study update <id> --modality text\`.
|
|
165
|
+
- **Failed testers expose \`error_message\`.** \`study tester --json\`
|
|
166
|
+
and \`study results --json\` (in \`testers[]\`) include
|
|
167
|
+
\`error_message: "<reason>"\` for any tester with \`status: failed\`.
|
|
168
|
+
Don't drill into logs — read the field. \`study results\` also
|
|
169
|
+
includes a top-level \`failed_count\` alongside \`completed_count\`.
|
|
170
|
+
- **\`ask add-questions\` is additive by default.** Appending a
|
|
171
|
+
follow-up question to a completed round preserves prior comments,
|
|
172
|
+
picks, and ratings; only the new question is dispatched. Pass
|
|
173
|
+
\`--redispatch-all\` for the legacy reset-and-rerun behavior.
|
|
174
|
+
- **\`ask results --json\` adds \`cross_round_summary\` for 2+ rounds.**
|
|
175
|
+
Top-level field with per-round picks/winner snapshots and
|
|
176
|
+
\`picks_delta\` (R1 → last). Don't diff two \`ask results\` calls by
|
|
177
|
+
hand.
|
|
178
|
+
- **\`--workspace\` is accepted on every workspace-scoped subcommand.**
|
|
179
|
+
Pass it reflexively on any \`ask\`/\`study\`/\`iteration\`/\`profile\`/
|
|
180
|
+
\`source\` subcommand — when workspace is inferable from an ID alias
|
|
181
|
+
it's silently ignored.
|
|
145
182
|
- Exit codes carry meaning: 0 success, 2 usage/validation,
|
|
146
183
|
3 auth, 4 not-found, 5 transient. See
|
|
147
184
|
\`ish docs get-page reference/json-mode\`.
|
|
185
|
+
- **Tier limits surface as \`error_code: "usage_limit_reached"\`**
|
|
186
|
+
(HTTP 403, exit 1, non-retryable). The error body includes
|
|
187
|
+
\`tier\`, \`limit\`, \`current\`, \`max\`, \`upgrade_url\`. Do not
|
|
188
|
+
retry — branch on the code and surface the upgrade link. See
|
|
189
|
+
\`ish docs get-page reference/billing-limits\`.
|
|
148
190
|
- Aliases (\`s-…\`, \`a-…\`, \`tp-…\`, \`i-…\`, \`t-…\`, \`tps-…\`, \`w-…\`)
|
|
149
191
|
are accepted anywhere a UUID is. See
|
|
150
192
|
\`ish docs get-page reference/aliases\`.
|
|
@@ -153,22 +195,42 @@ See \`references/workflows.md\` in this skill for end-to-end transcripts:
|
|
|
153
195
|
|
|
154
196
|
1. **Don't paste flags from memory.** The CLI evolves; flags change.
|
|
155
197
|
Run \`ish <command> --help\` to confirm before constructing a command.
|
|
156
|
-
2. **Don't
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
198
|
+
2. **Don't pipe \`--json\` through \`python\`/\`jq\` to reshape output** —
|
|
199
|
+
the CLI already has the affordances:
|
|
200
|
+
- Inspect a few specific entities? \`ish profile get tp-1b9 tp-fc1
|
|
201
|
+
tp-2fc\` (also works for \`study get\`, \`iteration get\`, \`ask
|
|
202
|
+
get\`). Returns a \`{items:[...], total:N}\` envelope.
|
|
203
|
+
- Want only certain fields? \`--fields alias,name,country,occupation\`.
|
|
204
|
+
- Need counts of a nested array? \`ask get\` / \`ask create --wait\`
|
|
205
|
+
already include \`testers_count\`, \`responses_total\`,
|
|
206
|
+
\`responses_complete\` (per-round and aggregate). Don't recount.
|
|
207
|
+
- Want machine-readable A/B verdicts? \`ask results --json\` already
|
|
208
|
+
ships \`aggregates: { picks, ratings, winner }\` per round.
|
|
209
|
+
3. **Don't run \`ish study run\` against an empty study.** \`ish study
|
|
210
|
+
create\` and \`ish study generate\` no longer auto-create iteration
|
|
211
|
+
A — the first explicit \`ish iteration create\` becomes A. Running
|
|
212
|
+
\`study run\` on a study with zero iterations exits 2; create one
|
|
213
|
+
first via \`ish iteration create --url …\` / \`--content-url …\` /
|
|
214
|
+
\`--content-text …\`. Or pass \`--content-text\` / \`--url\` directly
|
|
215
|
+
on \`study create\` for a one-shot study + iteration A.
|
|
216
|
+
4. **Don't pass \`--profile\` together with demographic filters** — they
|
|
161
217
|
are mutually exclusive. Either explicit IDs or
|
|
162
218
|
\`--country\`/\`--gender\`/\`--min-age\`/\`--max-age\` + \`--sample\`.
|
|
163
|
-
|
|
219
|
+
5. **Don't change audience between rounds of an ask.** It's fixed at
|
|
164
220
|
ask creation. Use \`ish ask add-testers\` to *extend* it; you can't
|
|
165
221
|
replace it.
|
|
166
|
-
|
|
222
|
+
6. **Don't try to put credentials in the URL** for gated study URLs.
|
|
167
223
|
Configure them once on the workspace via
|
|
168
224
|
\`ish workspace site-access …\` (basic-auth, cookie, login).
|
|
169
225
|
See \`ish docs get-page concepts/site-access\`.
|
|
170
|
-
|
|
226
|
+
7. **Don't commit \`~/.ish/config.json\`** — it stores tokens and active
|
|
171
227
|
workspace/study/ask selections. It lives in \`$HOME\`, not the repo.
|
|
228
|
+
8. **Don't retry \`usage_limit_reached\` errors.** Tier caps
|
|
229
|
+
(\`maxProducts\`, \`maxStudiesPerProduct\`, \`maxIterationsPerStudy\`,
|
|
230
|
+
\`maxCustomTesterProfiles\`) are enforced server-side. The error body
|
|
231
|
+
carries \`tier\`, \`limit\`, \`current\`, \`max\`, \`upgrade_url\` — show
|
|
232
|
+
the upgrade link or delete an existing resource to free headroom.
|
|
233
|
+
See \`ish docs get-page reference/billing-limits\` for the table.
|
|
172
234
|
|
|
173
235
|
## Authentication
|
|
174
236
|
|
|
@@ -238,9 +300,17 @@ ish ask run --new --name "hero shots" \\
|
|
|
238
300
|
--variant image:./hero-b.png::label=B \\
|
|
239
301
|
--sample 30 --wants-pick --wait
|
|
240
302
|
|
|
241
|
-
|
|
303
|
+
# Read the verdict directly — no comment-parsing required:
|
|
304
|
+
ish ask results --json | jq '.rounds[0].aggregates'
|
|
305
|
+
# → { "picks": { "A": 22, "B": 8 },
|
|
306
|
+
# "winner": { "letter": "A", "count": 22, "tied": false } }
|
|
242
307
|
\`\`\`
|
|
243
308
|
|
|
309
|
+
For \`--wants-pick\` / \`--wants-ratings\` rounds, \`ask results --json\`
|
|
310
|
+
adds an \`aggregates\` field per round with \`picks\`, \`ratings\` (mean
|
|
311
|
+
+ n per variant), and a \`winner\`. See \`ish docs get-page
|
|
312
|
+
reference/json-mode\` for the full shape.
|
|
313
|
+
|
|
244
314
|
Add a follow-up round with no audience change:
|
|
245
315
|
|
|
246
316
|
\`\`\`bash
|
|
@@ -348,6 +418,27 @@ ish iteration create --url "$URL"
|
|
|
348
418
|
the full Ask payload.
|
|
349
419
|
- For \`profile generate --json\`, \`simulation_config\` is trimmed by
|
|
350
420
|
default (~9× smaller). Pass \`--include-simulation-config\` to include it.
|
|
421
|
+
- On \`error_code: "usage_limit_reached"\` (HTTP 403), don't retry —
|
|
422
|
+
read \`tier\`, \`limit\`, \`current\`, \`max\`, and \`upgrade_url\` from
|
|
423
|
+
the JSON body to construct a recovery message. \`profile generate\` /
|
|
424
|
+
\`study generate\` refuse the entire batch when the post-generation
|
|
425
|
+
count would exceed the cap; re-issue with a smaller \`--count\`.
|
|
426
|
+
|
|
427
|
+
## Common reshaping → use the CLI, not jq/python
|
|
428
|
+
|
|
429
|
+
| You want to… | Don't | Do |
|
|
430
|
+
|-------------------------------------------|----------------------------------------|--------------------------------------------------------------------|
|
|
431
|
+
| Look up 2-3 specific profiles | \`profile list --json \\| jq '.items[] \\| select(...)'\` | \`ish profile get tp-1b9 tp-fc1 tp-2fc\` |
|
|
432
|
+
| Show only some fields | \`--json \\| jq '{alias, name, country}'\` | \`--fields alias,name,country\` |
|
|
433
|
+
| Count testers on an ask | \`--json \\| jq '.testers \\| length'\` | \`ish ask get a-… --fields alias,testers_count\` |
|
|
434
|
+
| Count responses on a round | \`--json \\| jq '.rounds[0].responses \\| length'\` | \`ish ask get a-… --fields alias,rounds,responses_complete,responses_total\` |
|
|
435
|
+
| Pick the A/B winner | \`--json \\| jq '.rounds[0].responses…'\` | \`ish ask results a-… --json\` then read \`.rounds[].aggregates.winner\` |
|
|
436
|
+
| List of testers from \`study run\` | \`--json \\| jq '.testers[].id'\` | \`--json\` already has \`tester_ids[]\` and \`tester_aliases[]\` |
|
|
437
|
+
|
|
438
|
+
The bias here is intentional: \`ish\` ships shapes designed for agent
|
|
439
|
+
consumption. If you find yourself reaching for \`jq\` or \`python\` to
|
|
440
|
+
reshape ish output, run \`ish docs search <keyword>\` first — there is
|
|
441
|
+
almost always a flag that does the work for you.
|
|
351
442
|
`;
|
|
352
443
|
const COMMANDS_MD = `# ish commands — quick index
|
|
353
444
|
|
|
@@ -376,6 +467,8 @@ ish <command> --help
|
|
|
376
467
|
| | Cursor / Cline / Roo project | |
|
|
377
468
|
| \`login\` | Browser-based auth | — |
|
|
378
469
|
| \`logout\` | Clear saved credentials | — |
|
|
470
|
+
| \`status\` | Show active session (user, workspace, | concepts/active-context |
|
|
471
|
+
| | study, ask, token validity) — alias \`whoami\` | |
|
|
379
472
|
| \`connect\` | Cloudflare tunnel exposing localhost | — |
|
|
380
473
|
| \`upgrade\` | Self-update | — |
|
|
381
474
|
|
package/dist/lib/types.d.ts
CHANGED
|
@@ -97,6 +97,7 @@ export interface StudyCreateInput {
|
|
|
97
97
|
assignments?: Assignment[];
|
|
98
98
|
interview_questions?: InterviewQuestion[];
|
|
99
99
|
start_frame_id?: string;
|
|
100
|
+
iteration?: IterationCreateInput;
|
|
100
101
|
}
|
|
101
102
|
export interface StudyUpdateInput {
|
|
102
103
|
name?: string;
|
|
@@ -124,7 +125,7 @@ export interface Iteration {
|
|
|
124
125
|
updated_at: string;
|
|
125
126
|
}
|
|
126
127
|
export interface IterationCreateInput {
|
|
127
|
-
name
|
|
128
|
+
name?: string;
|
|
128
129
|
description?: string;
|
|
129
130
|
details?: Record<string, unknown>;
|
|
130
131
|
}
|
|
@@ -290,6 +291,7 @@ export interface AddTestersInput {
|
|
|
290
291
|
}
|
|
291
292
|
export interface AddRoundQuestionsInput {
|
|
292
293
|
questions: InterviewQuestion[];
|
|
294
|
+
redispatch_all?: boolean;
|
|
293
295
|
}
|
|
294
296
|
export interface AskAudienceTester {
|
|
295
297
|
id: string;
|
package/dist/upgrade.js
CHANGED
|
@@ -40,10 +40,10 @@ export async function upgrade(currentVersion, targetVersion) {
|
|
|
40
40
|
}
|
|
41
41
|
const latest = targetVersion || (await getLatestVersion());
|
|
42
42
|
if (latest === currentVersion) {
|
|
43
|
-
console.
|
|
43
|
+
console.error(`Already up to date (v${currentVersion}).`);
|
|
44
44
|
return;
|
|
45
45
|
}
|
|
46
|
-
console.
|
|
46
|
+
console.error(`Updating ish v${currentVersion} → v${latest}...`);
|
|
47
47
|
const target = getPlatformTarget();
|
|
48
48
|
const ext = process.platform === "win32" ? ".exe" : "";
|
|
49
49
|
const assetName = `ish-${target}${ext}`;
|
|
@@ -100,5 +100,5 @@ export async function upgrade(currentVersion, targetVersion) {
|
|
|
100
100
|
chmodSync(tmpPath, 0o755);
|
|
101
101
|
renameSync(tmpPath, execPath);
|
|
102
102
|
}
|
|
103
|
-
console.
|
|
103
|
+
console.error(`Updated to v${latest}.`);
|
|
104
104
|
}
|