@ishlabs/cli 0.8.1 → 0.8.3

package/README.md

@@ -1,6 +1,6 @@
 # ish
 
-CLI tool to expose your localhost to [Ish](https://ishlabs.io) for simulation testing.
+CLI tool for [Ish](https://ishlabs.io) — run studies, send asks, and expose your localhost for AI tester sessions.
 
 ## Install
 
@@ -29,41 +29,343 @@ brew tap ishlabs/tap
 brew install ish
 ```
 
-## Usage
+## Authenticate
 
 ```bash
-ish connect <port>
+ish login              # browser-based auth, stores tokens in ~/.ish/config.json
+ish logout             # clear saved credentials
 ```
 
-### Options
+The CLI resolves your auth token in this order:
+1. `--token` CLI flag
+2. `ISH_TOKEN` env var
+3. Saved token from `ish login`
 
-| Flag | Description |
-|------|-------------|
-| `-t, --token <token>` | Auth token (or set `ISH_TOKEN` env var, or enter interactively) |
-| `--api-url <url>` | Backend API URL (default: `https://api.ishlabs.io` or `ISH_API_URL` env var) |
-| `--version` | Show version |
+## Concepts
 
-### Token Configuration
+Two top-level research primitives, both consume reusable tester profiles:
 
-The CLI resolves your auth token in this order:
+```
+Workspace (= product, top-level container)
+│
+├── Tester Profiles        ← reusable audience personas
+│     └── Audience Sources (transcripts / audio / images that seed generation)
+│
+├── Study  ─────────────── "structured research artifact"
+│     ├── modality (interactive | text | video | audio | image | document)
+│     ├── content-type (for non-interactive studies: email | social_post | ad | etc.)
+│     ├── assignments (tasks the tester does)
+│     ├── questionnaire (questions testers answer — text, slider, likert, choice)
+│     └── Iterations       ← the unit of execution within a study
+│           └── Testers    ← instance of a Profile inside this Iteration
+│                 └── Interactions / results
+│
+└── Ask  ──────────────── "lightweight reaction artifact"
+      ├── Audience (testers, fixed at creation, max 5 rounds per ask)
+      └── Rounds           ← the unit of execution within an ask
+            └── Responses (per-tester reactions to a variant)
+```
+
+**The two primary run verbs:**
+
+| | **`ish study run`** | **`ish ask run`** |
+|---|---|---|
+| Default | Run the latest iteration on the active study | Append a round to the active ask |
+| Fresh setup | Create the iteration first: `ish iteration create …` | `--new` (creates ask + round 1 in one shot) |
+| Specific target | `--iteration <id>` | Pass the ask id as positional arg |
+| Audience | `--profile <ids>`, `--sample <N>`, `--all`, or demographic filters (`--country`, `--gender`, `--min-age`, `--max-age`, `--search`, `--visibility`) — else reuse iteration's testers | `--profile <ids>`, `--sample <N>`, `--all-simulatable`, or demographic filters (with `--new`) |
+
+## Commands
+
+### Auth & infra
+
+```bash
+ish login                          # browser auth
+ish logout
+ish connect <port>                 # Cloudflare tunnel exposing localhost
+ish upgrade                        # self-update (single-binary installs only)
+ish upgrade --release 0.8.1        # pin a specific release
+```
+
+`ish upgrade` only works on standalone-binary installs (curl/Homebrew). On npm-installed CLIs (`npm install -g @ishlabs/cli`) it refuses with a pointer to `npm install -g @ishlabs/cli@latest` — overwriting `node` would break every other Node tool.
+
+### Workspaces, studies, iterations, profiles, configs (CRUD groups)
+
+```bash
+ish workspace  list | create | get | update | delete | use
+ish workspace  site-access status | basic-auth | cookie | login | affirm-public | clear
+ish study      list | create | generate | get | results | update | delete | use
+ish iteration  list | create | get | update | delete
+ish profile    list | create | generate | get | update | delete
+ish source     upload | get
+ish config     list | create | get | schema | update | delete
+```
+
+Testers live as a nested group on a study (low-level — usually created via `study run`):
+
+```bash
+ish study tester <id>                              # show tester details and results
+ish study tester create | batch-create | delete    # low-level escape hatches
+```
+
+`use` saves an active workspace/study/ask to `~/.ish/config.json` so you don't repeat `--workspace`/`--study`/`--ask` on every command. Aliases like `w-6ec`, `s-b2c`, `a-…`, `i-…`, `t-…` work anywhere an ID is expected.
+
+### Define a study — `ish study create`
+
+A study locks in the persistent shape: modality, optional content-type taxonomy, the tasks testers do (`--assignment`), and the questionnaire they answer (`--question` or `--questionnaire`). The actual URL or file lives on an iteration (next step).
+
+```bash
+# Interactive study, one assignment + a single-question questionnaire:
+ish study create --name "Onboarding UX" --modality interactive \
+    --assignment "Sign up:Complete the signup flow" \
+    --question "How easy was it?"
+
+# Multiple assignments + a richer questionnaire from a file:
+ish study create --name "Checkout" --modality interactive \
+    --assignment "Browse:Find a product you like" \
+    --assignment "Buy:Add to cart and checkout" \
+    --questionnaire ./questionnaire.json
+
+# Bulk assignments from a file (text/email study):
+ish study create --name "Newsletter" --modality text --content-type email \
+    --assignments-file ./assignments.json
+```
+
+`--question` adds a simple text question to the questionnaire (repeatable, defaults to type=text, timing=after). For richer types (slider, likert, single/multiple-choice, number) and custom timing, pass a JSON manifest via `--questionnaire <file.json>`. The two flags are mutually exclusive — pick one.
+
+### Configure a run — `ish iteration create`
+
+Iterations carry the URL (interactive) or content (media) for a run. Create one before `ish study run`. Local files passed to `--content-url`, `--image-urls`, etc. are uploaded automatically; `@filepath` reads text from a file.
+
+```bash
+# Interactive — URL:
+ish iteration create --study s-b2c --url https://example.com
+
+# Interactive on mobile:
+ish iteration create --url https://example.com --screen-format mobile_portrait
+
+# Text/email (inline or @file):
+ish iteration create --content-text @./email.html --title "Newsletter"
+
+# Video (URL or local file):
+ish iteration create --content-url ./video.mp4
+
+# Image set:
+ish iteration create --image-urls "./a.png,./b.png"
+
+# Document (PDF):
+ish iteration create --content-url ./report.pdf
+
+# Video ad with copy text:
+ish iteration create --content-url ./ad.mp4 --copy-text "Buy now — 50% off!"
+
+# Social post with caption:
+ish iteration create --image-urls ./post.png \
+    --copy-text @./caption.txt --social-platform instagram
+```
+
+### Configure site access — `ish workspace site-access`
+
+For studies that target a gated URL (HTTP basic auth, a session-cookie wall like Vercel preview protection, or a login form), set credentials on the workspace once. Testers reuse them whenever a study points at a matching origin. Credentials are encrypted at rest; the CLI never reads them back, only checks whether each method is configured.
+
+```bash
+# Show what's configured:
+ish workspace site-access status
+
+# HTTP basic auth (e.g. staging gate):
+ish workspace site-access basic-auth --username alice --password hunter2
+
+# Session cookie (Vercel preview, Lovable, etc.):
+ish workspace site-access cookie --name session --value abc123
+
+# Login form — credentials the tester types into the page:
+ish workspace site-access login --username demo --password demo
+
+# Mark the site as public (silences the "credentials needed?" check):
+ish workspace site-access affirm-public
+
+# Clear one method, or everything:
+ish workspace site-access clear cookie
+ish workspace site-access clear all
+```
+
+`--origin` defaults to the workspace `base_url` (set via `ish workspace update --base-url`). Pass `-` for `--password`/`--value` to read from stdin and keep secrets out of shell history:
+
+```bash
+printf %s "$STAGING_PW" | ish workspace site-access basic-auth --username alice --password -
+```
 
-1. `--token` CLI argument
-2. `ISH_TOKEN` environment variable
-3. Saved token from `ish login` (stored in `~/.ish/config.json`)
+### Run a study — `ish study run`
 
-## Examples
+Picks the latest iteration on the study (or `--iteration <id>`), creates testers (explicit `--profile`, demographic-filtered sample, or reusing the iteration's existing testers), and dispatches simulations. If the study has no iterations, run `ish iteration create` first.
 
 ```bash
-# Expose port 3000
-ish connect 3000
+# Run the latest iteration, reusing its testers (after `study use`):
+ish study run -y
 
-# With explicit token
-ish connect 3000 --token YOUR_TOKEN
+# Run the latest iteration with an explicit audience:
+ish study run --profile tp-795,tp-af2
 
-# Using environment variable
+# Sample 3 Swedish profiles aged 35–50 from the workspace pool:
+ish study run --country SE --min-age 35 --max-age 50 --sample 3
+
+# Run with every female profile in the workspace:
+ish study run --gender female --all
+
+# Run a specific iteration:
+ish study run --iteration i-d4e
+
+# Override the simulation config (e.g. for a media study):
+ish study run --config c-c3c
+
+# Block until all simulations finish (or timeout):
+ish study run --wait
+ish study run --wait --timeout 600
+
+# Local browser (no remote Browserbase):
+ish study run --local --headed --slow-mo 500
+```
+
+**Audience flags** (shared with `ish ask`): `--profile <ids>` for explicit IDs, or any of `--country`, `--gender`, `--min-age`, `--max-age`, `--search`, `--visibility` paired with `--sample <N>` or `--all` to seed from the workspace pool. Mutually exclusive with `--profile`.
+
+**Other study verbs (low-level):**
+
+```bash
+ish study poll --study <id>                                      # one-shot progress
+ish study poll <tester_id>                                       # single tester status
+ish study wait --study <id>                                      # block until all done
+ish study wait --iteration <id>                                  # block on one iteration
+ish study wait <tester_id> --timeout 600                         # block on one tester
+ish study cancel <tester_id>                                     # cancel a running sim
+```
+
+`<tester_id>` is a tester alias (`t-...`) or UUID. Get them from `ish study run --json`'s `tester_aliases[]` / `tester_ids[]` arrays:
+
+```bash
+ish study run --study s-b2c -y --json | jq -r '.tester_aliases[]'   # → t-072, t-1ed, ...
+```
+
+### Run an ask — `ish ask run`
+
+Smart wrapper around the ask flow. With `--new`, creates a fresh ask + round 1 (audience from `--profile`, `--sample`, `--all-simulatable`, or any demographic filter — `--country`, `--gender`, `--min-age`, `--max-age`, `--search`, `--visibility`). Without `--new`, appends a new round to the active or specified ask.
+
+```bash
+# Append a round to the active ask:
+ish ask run --prompt "And now which?" \
+    --variant text:"X" --variant text:"Y" --wait
+
+# Append to a specific ask:
+ish ask run a-6ec --prompt "Round 2" \
+    --variant text:"A" --variant text:"B"
+
+# Create a fresh ask with round 1, sample 30 profiles, wait for results:
+ish ask run --new --name "tagline AB" \
+    --prompt "Which sounds better?" \
+    --variant text:"Short and punchy." \
+    --variant text:"A longer, descriptive line." \
+    --sample 30 --wants-pick --wait
+
+# Demographic-filtered sample (Swedish profiles aged 35–50):
+ish ask run --new --name "SE 35-50" \
+    --prompt "Which sounds better?" \
+    --variant text:"A" --variant text:"B" \
+    --country SE --min-age 35 --max-age 50 --sample 10 --wants-pick
+
+# Image comparison from local files (auto-uploaded), explicit profiles:
+ish ask run --new --name "hero shots" \
+    --prompt "Which feels premium?" \
+    --variant image:./hero-a.png::label=A \
+    --variant image:./hero-b.png::label=B \
+    --profile tp-d4e,tp-a17 --wants-ratings
+
+# Text from a markdown file + JSON questionnaire:
+ish ask run --new --name "newsletter" \
+    --prompt "Would you read this?" \
+    --variant text:@./body.md \
+    --questions ./questions.json --sample 30
+```
+
+`--questions` takes a JSON array shaped `[{"question": "...", "type": "open_ended"|"slider"|"choice"|"likert"}]`. The server requires the key `question` (not `text`). Minimal example:
+
+```json
+[
+  { "question": "What stood out?", "type": "open_ended" },
+  { "question": "Rate it 1-5", "type": "slider" }
+]
+```
+
+**Audience flags** (`--profile`, `--sample`, `--all-simulatable`, `--country`, `--gender`, `--min-age`, `--max-age`, `--search`, `--visibility`, `--name`, `--description`, `--workspace`) only apply with `--new` — the audience is fixed at ask creation.
+
+**Other ask verbs:**
+
+```bash
+ish ask list [--archived]
+ish ask get [id] [--round <n>]
+ish ask results [id] [--round <n>]
+ish ask wait [id] [--round <n>] [--timeout <s>]
+ish ask add-round [id] --prompt … --variant …       # explicit form of `run`
+ish ask add-questions [id] --round <n> --questions ./qs.json
+ish ask add-testers [id] --profile …                # extend audience for one round
+ish ask create …                                    # explicit form of `run --new`
+ish ask update | archive | unarchive | delete | use
+```
+
+### Generate tester profiles
+
+`ish profile generate` runs the same audience-generation flow used in the web UI: an LLM reads your description and any uploaded sources (transcripts, customer records, audio interviews, screenshots) and returns either a single profile or a full audience.
+
+```bash
+# 5 profiles from a written brief:
+ish profile generate \
+  --description "Tech-savvy millennials in the US who use mobile banking" \
+  --count 5
+
+# One profile from a transcript — the file is uploaded automatically:
+ish profile generate --source ./interviews/sarah.txt --count 1
+
+# Audio call with a written brief, diarized:
+ish profile generate \
+  --description "Voices behind support tickets" \
+  --source ./call.mp3 --diarize --count 3
+```
+
+For explicit control over uploads — e.g. reusing the same source across multiple `generate` runs — upload first and pass the returned alias:
+
+```bash
+ish source upload ./call.mp3 --diarize
+# → tps-3a4 (status: processed)
+
+ish profile generate --source tps-3a4 --propose-count
+# → { proposed_count: 4, rationale: "..." }
+
+ish profile generate --source tps-3a4 --count 4
+```
+
+### Expose localhost
+
+For interactive studies that need to reach a service running on your machine:
+
+```bash
+ish connect 3000                       # Cloudflare tunnel to localhost:3000
 ISH_TOKEN=YOUR_TOKEN ish connect 8080
 ```
 
+`connect` is a long-running command — keep it open while testers run. The Cloudflare tunnel URL prints prominently after "Connected"; pass `--json` for one-line machine-readable output (`{"status":"connected","tunnel_url":"...","local_port":3000,"registered":true}`) suitable for scripts.
+
+## Global flags
+
+| Flag | Description |
+|------|-------------|
+| `-t, --token <token>` | Auth token (or set `ISH_TOKEN` env var) |
+| `--api-url <url>` | Backend API URL (default `https://api.ishlabs.io` or `ISH_API_URL`) |
+| `--json` | Output JSON (auto-enabled when piped) |
+| `--fields <a,b,c>` | Comma-separated fields to include in JSON output |
+| `--verbose` | Include full UUIDs and timestamps in JSON output |
+| `-q, --quiet` | Suppress progress messages on stderr |
+| `-V, --version` | Show CLI version |
+
+All commands print machine-readable JSON when stdout is piped or `--json` is passed, so AI agents can chain them together without parsing tables.
+
 ## License
 
-Copyright (c) 2025 Ish Labs. All rights reserved. See [LICENSE](LICENSE).
+Copyright (c) 2026 Ish Labs. All rights reserved. See [LICENSE](LICENSE).

package/dist/auth.d.ts

@@ -5,13 +5,29 @@
 export declare function getAppUrl(): string;
 export declare function getSupabaseUrl(): string;
 export declare function getSupabaseAnonKey(): string;
+/**
+ * Resolve the Supabase project (URL + anon key) that issued the given JWT.
+ * Falls back to env vars / production defaults when the token is unparsable
+ * or its issuer isn't in our known list.
+ */
+export declare function resolveSupabaseProjectFromToken(accessToken: string | undefined): {
+    url: string;
+    anonKey: string;
+};
 export declare function decodeJwtExp(token: string): number;
 export declare function isTokenExpired(token: string, bufferSeconds?: number): boolean;
 export declare function login(appUrl?: string): Promise<{
     accessToken: string;
     refreshToken: string;
 }>;
-export declare function refreshTokens(refreshToken: string, supabaseUrl?: string, anonKey?: string): Promise<{
+export declare function refreshTokens(refreshToken: string, options?: {
+    /** The (possibly expired) access token. Used to pick the correct Supabase project. */
+    accessToken?: string;
+    /** Force a specific Supabase project URL (e.g. for tests). */
+    supabaseUrl?: string;
+    /** Force a specific anon/publishable key. */
+    anonKey?: string;
+}): Promise<{
     accessToken: string;
     refreshToken: string;
 }>;

package/dist/auth.js

@@ -7,16 +7,68 @@ import { execFile } from "node:child_process";
 const POLL_INTERVAL = 2_000;
 const LOGIN_TIMEOUT = 5 * 60 * 1000; // 5 minutes (matches server-side token TTL)
 const DEFAULT_APP_URL = "https://app.ishlabs.io";
-const DEFAULT_SUPABASE_URL = "https://hngymyxdyamokpbeakps.supabase.co";
-const DEFAULT_SUPABASE_ANON_KEY = "sb_publishable_JlS-HfwNyDqLNbrfbrkUlw_PSdZJdo2";
+// Known Supabase projects, keyed by hostname. The CLI may be talking to either
+// production or development depending on how the user logged in — the access
+// token's `iss` claim tells us which project minted the refresh token, and we
+// must send refresh requests back to that same project (with its matching
+// publishable/anon key) or Supabase will reject them.
+const SUPABASE_PROJECTS = {
+    // Production
+    "muqvgnqyubmqnfnqwxuk.supabase.co": {
+        url: "https://muqvgnqyubmqnfnqwxuk.supabase.co",
+        anonKey: "sb_publishable_pxXwY9EaWFwkR7h728NWvQ_NFqGfh8K",
+    },
+    // Development
+    "hngymyxdyamokpbeakps.supabase.co": {
+        url: "https://hngymyxdyamokpbeakps.supabase.co",
+        anonKey: "sb_publishable_JlS-HfwNyDqLNbrfbrkUlw_PSdZJdo2",
+    },
+};
+const DEFAULT_SUPABASE_PROJECT = SUPABASE_PROJECTS["muqvgnqyubmqnfnqwxuk.supabase.co"];
 export function getAppUrl() {
     return process.env.ISH_APP_URL ?? DEFAULT_APP_URL;
 }
 export function getSupabaseUrl() {
-    return process.env.ISH_SUPABASE_URL ?? DEFAULT_SUPABASE_URL;
+    return process.env.ISH_SUPABASE_URL ?? DEFAULT_SUPABASE_PROJECT.url;
 }
 export function getSupabaseAnonKey() {
-    return process.env.ISH_SUPABASE_ANON_KEY ?? DEFAULT_SUPABASE_ANON_KEY;
+    return process.env.ISH_SUPABASE_ANON_KEY ?? DEFAULT_SUPABASE_PROJECT.anonKey;
+}
+/**
+ * Resolve the Supabase project (URL + anon key) that issued the given JWT.
+ * Falls back to env vars / production defaults when the token is unparsable
+ * or its issuer isn't in our known list.
+ */
+export function resolveSupabaseProjectFromToken(accessToken) {
+    const envUrl = process.env.ISH_SUPABASE_URL;
+    const envKey = process.env.ISH_SUPABASE_ANON_KEY;
+    if (envUrl && envKey)
+        return { url: envUrl, anonKey: envKey };
+    if (accessToken) {
+        try {
+            const payload = JSON.parse(Buffer.from(accessToken.split(".")[1], "base64url").toString());
+            const iss = payload.iss;
+            if (iss) {
+                const host = new URL(iss).host;
+                const project = SUPABASE_PROJECTS[host];
+                if (project)
+                    return project;
+                // Unknown but well-formed issuer — trust it for the URL, use env or
+                // default key for apikey (best-effort; will likely fail without env override).
+                return {
+                    url: `https://${host}`,
+                    anonKey: envKey ?? DEFAULT_SUPABASE_PROJECT.anonKey,
+                };
+            }
+        }
+        catch {
+            // fall through
+        }
+    }
+    return {
+        url: envUrl ?? DEFAULT_SUPABASE_PROJECT.url,
+        anonKey: envKey ?? DEFAULT_SUPABASE_PROJECT.anonKey,
+    };
 }
 // --- Browser open ---
 function openBrowser(url) {
@@ -78,13 +130,14 @@ export async function login(appUrl) {
     throw new Error("Login timed out. Please try again.");
 }
 // --- Token refresh ---
-export async function refreshTokens(refreshToken, supabaseUrl, anonKey) {
-    const url = supabaseUrl ?? getSupabaseUrl();
-    const key = anonKey ?? getSupabaseAnonKey();
-    const resp = await fetch(`${url}/auth/v1/token?grant_type=refresh_token`, {
+export async function refreshTokens(refreshToken, options) {
+    const project = options?.supabaseUrl && options?.anonKey
+        ? { url: options.supabaseUrl, anonKey: options.anonKey }
+        : resolveSupabaseProjectFromToken(options?.accessToken);
+    const resp = await fetch(`${project.url}/auth/v1/token?grant_type=refresh_token`, {
         method: "POST",
         headers: {
-            apikey: key,
+            apikey: project.anonKey,
             "Content-Type": "application/json",
         },
         body: JSON.stringify({ refresh_token: refreshToken }),

package/dist/commands/ask.d.ts

@@ -0,0 +1,5 @@
+/**
+ * ish ask — Create and run asks (multi-round surveys with variants).
+ */
+import type { Command } from "commander";
+export declare function registerAskCommands(program: Command): void;