@toast-ninja/toast-cli 0.1.4 → 0.1.6

package/README.md

@@ -36,6 +36,15 @@ scope:
 toast auth --org toast-ninja
 ```
 
+You can authenticate multiple orgs on the same machine. The CLI stores tokens by
+org and picks the token that matches `--repo`:
+
+```bash
+toast auth --org toast-ninja
+toast auth --org another-org
+toast review-ci next --repo another-org/backend --pr 1177 --head "$GITHUB_SHA"
+```
+
 If browser auth is not available, use GitHub device auth explicitly:
 
 ```bash
@@ -65,9 +74,13 @@ toast review-ci wait \
   --timeout 20m
 ```
 
-`wait` exits `0` only when Toast returns `acknowledged` for the exact head SHA. It exits `1` for actionable non-green states and `2` for auth, network, argument, or timeout errors.
+`wait` exits `0` only when Toast returns `ready` for the exact head SHA. It exits `1` for actionable non-green states and `2` for auth, network, argument, or timeout errors.
 It waits on Toast's server-side Review CI events through bounded requests, so a
-local polling interval is not needed.
+local polling interval is not needed. If a wait stays pending, the CLI prints
+the current `waiting_for` reason to stderr; when a bounded wait wakes from a
+webhook or timeout, Toast reconciles exact GitHub PR state when needed.
+Reconciled responses include concrete GitHub check names in
+`github_checks` / `waiting_for.github_checks` when CI is pending or failing.
 
 CI can also provide a token directly:
 
@@ -89,12 +102,17 @@ toast review-ci next \
   --head "$GITHUB_SHA"
 ```
 
-Install agent guidance for Codex-compatible skill directories:
+Install agent guidance from the project checkout:
 
 ```bash
-toast review-ci instructions --install-codex
+toast review-ci instructions --install
 ```
 
+This installs the Codex skill and writes a bounded Toast Review CI section to
+`CLAUDE.md` and `AGENTS.md`. The `--install-agent` flag is an alias for
+`--install`; the older `--install-codex` flag still installs only the Codex
+skill.
+
 Or print the guidance:
 
 ```bash
@@ -103,12 +121,22 @@ toast review-ci instructions
 
 ## AI Agent Instructions
 
-Before using `gh` to rediscover review context, run:
+Use Review CI as the source of truth for PR readiness. Before using `gh` to
+rediscover review context, run:
+
+```bash
+toast review-ci next --repo ORG/REPO --pr NUMBER --head HEAD_SHA --json
+```
+
+Keep iterating until the response status is `ready`. For `needs_fix`, fix every
+`blocking_items` entry, reply with what changed, resolve threads when requested,
+push, refresh the exact head SHA, then run:
 
 ```bash
-toast review-ci next --repo ORG/REPO --pr NUMBER --head HEAD_SHA
+toast review-ci wait --repo ORG/REPO --pr NUMBER --head HEAD_SHA --json
 ```
 
-Fix every unresolved thread/comment listed by URL. Reply with what changed,
-resolve threads when requested, push the fix, then run `toast review-ci wait`
-for the exact head SHA before reporting completion.
+If `wait` returns `ready`, report completion. If it returns `needs_fix`, use the
+returned `blocking_items` and continue the fix loop instead of claiming the PR
+is done. For `pending` or `degraded:reconciliation_incomplete`, run `wait` for
+the same head SHA and use `waiting_for` to see what Toast is waiting on.

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "@toast-ninja/toast-cli",
-	"version": "0.1.4",
+	"version": "0.1.6",
 	"description": "Toast CLI for agent CI workflows",
 	"bin": {
 		"toast": "bin/toast.js"

package/skills/review-ci/SKILL.md

@@ -5,21 +5,42 @@ description: Use when addressing GitHub pull request review feedback in a repo t
 
 # Toast Review CI
 
-Use Toast Review CI before rediscovering review context with `gh`.
+Use Toast Review CI as the source of truth for PR review readiness before
+rediscovering review context with `gh`.
 
-1. Run:
+Goal: keep iterating until Review CI returns `status: "ready"` for the exact
+current head SHA. Only report completion after Review CI returns `ready`.
+
+## Review Loop
+
+1. Get the current PR head SHA, then run:
 
 ```bash
-toast review-ci next --repo ORG/REPO --pr NUMBER --head HEAD_SHA
+toast review-ci next --repo ORG/REPO --pr NUMBER --head HEAD_SHA --json
 ```
 
-2. Fix every `blocking_items` entry. Prefer the URL and body from Toast over a fresh GitHub search.
-3. Reply with what changed when a reply is requested. Include the relevant commit link if available.
-4. Resolve review threads when `suggested_reply_requirements.resolve_thread_required` is true.
-5. Push the fix, then wait on the exact new head:
+2. Follow the returned status:
+   - `ready`: the PR is ready. Report completion.
+   - `needs_fix`: fix every `blocking_items` entry. Prefer the URL and body
+     from Toast over a fresh GitHub search.
+   - `pending` or `degraded:reconciliation_incomplete`: run
+     `toast review-ci wait` for the same head SHA, using `waiting_for` to see
+     what Toast is waiting on, then continue from step 6.
+   - `head_changed`: refresh the head SHA and restart the loop.
+   - `draft`, `github_blocked`, `policy_blocked:unknown_actor`, or
+     `degraded:timeout`: fix the blocker when possible; otherwise report the
+     blocker instead of claiming the PR is ready.
+3. When fixing `blocking_items`, reply with what changed when a reply is
+   requested. Include the relevant commit link if available.
+4. Resolve review threads when
+   `suggested_reply_requirements.resolve_thread_required` is true.
+5. Push the fix, refresh the exact new head SHA, then wait:
 
 ```bash
-toast review-ci wait --repo ORG/REPO --pr NUMBER --head HEAD_SHA
+toast review-ci wait --repo ORG/REPO --pr NUMBER --head HEAD_SHA --json
 ```
 
-Only report completion after Review CI returns `acknowledged`.
+6. If `wait` returns `ready`, report completion. If it returns `needs_fix`, use
+   the returned `blocking_items` and continue the fix loop. If it returns
+   `head_changed` or another non-ready status, continue from the matching step
+   above.

package/src/cli.js

@@ -13,11 +13,14 @@ const { requestJson } = require('./http')
 const DEFAULT_API_URL = 'https://api.toast.ninja'
 const DEFAULT_WAIT_REQUEST_SECONDS = 25
 const WAIT_REQUEST_GRACE_MS = 5000
+const WAIT_DIAGNOSTIC_INTERVAL_MS = 60 * 1000
+const WAIT_STALLED_AFTER_MS = 5 * 60 * 1000
 const WAIT_STATUSES = new Set([
 	'pending',
 	'degraded:reconciliation_incomplete',
 	'head_changed',
 ])
+const READY_STATUSES = new Set(['ready', 'acknowledged'])
 const REVIEW_CI_SKILL_PATH = path.join(
 	__dirname,
 	'..',
@@ -25,6 +28,8 @@ const REVIEW_CI_SKILL_PATH = path.join(
 	'review-ci',
 	'SKILL.md',
 )
+const REVIEW_CI_INSTRUCTIONS_START = '<!-- toast-review-ci:start -->'
+const REVIEW_CI_INSTRUCTIONS_END = '<!-- toast-review-ci:end -->'
 const REVIEW_CI_COMMANDS = new Set(['review-ci'])
 const AUTH_COMMANDS = new Set(['auth', 'login'])
 
@@ -172,8 +177,44 @@ const normalizeApiUrl = apiUrl =>
 const getApiUrl = ({ options, env, config }) =>
 	normalizeApiUrl(options['api-url'] || env.TOAST_API_URL || config.apiUrl)
 
-const getToken = ({ options, env, config }) =>
-	options.token || env.TOAST_TOKEN || config.token
+const getScopedTokenEntry = ({ config, repo }) => {
+	if (!repo) return null
+
+	const repoEntry = config.tokensByRepo?.[repo.fullName]
+	if (repoEntry?.token) return repoEntry
+
+	const orgEntry = config.tokensByOrg?.[repo.owner]
+	if (orgEntry?.token) return orgEntry
+
+	return null
+}
+
+const legacyConfigMatchesRepo = ({ config, repo }) => {
+	if (!config.token) return false
+	if (!config.org && !config.repo) return false
+
+	if (config.repo) {
+		try {
+			return parseRepo(config.repo).fullName === repo.fullName
+		} catch (e) {
+			return false
+		}
+	}
+
+	return config.org === repo.owner
+}
+
+const getToken = ({ options, env, config, repo }) => {
+	if (options.token) return options.token
+	if (env.TOAST_TOKEN) return env.TOAST_TOKEN
+
+	const scopedEntry = getScopedTokenEntry({ config, repo })
+	if (scopedEntry) return scopedEntry.token
+
+	if (legacyConfigMatchesRepo({ config, repo })) return config.token
+
+	return null
+}
 
 const toMillis = value => {
 	if (value instanceof Date) return value.getTime()
@@ -195,6 +236,16 @@ const parseDurationMs = value => {
 	return amount
 }
 
+const formatDurationMs = ms => {
+	const seconds = Math.max(0, Math.round(Number(ms) / 1000))
+	if (seconds < 60) return `${seconds}s`
+
+	const minutes = Math.floor(seconds / 60)
+	const remainingSeconds = seconds % 60
+
+	return remainingSeconds ? `${minutes}m${remainingSeconds}s` : `${minutes}m`
+}
+
 const parseBrowserCallbackAuth = ({ requestUrl, expectedState }) => {
 	const callbackUrl = new URL(requestUrl, 'http://127.0.0.1')
 	const state = callbackUrl.searchParams.get('state')
@@ -334,10 +385,12 @@ const requestReviewState = async ({
 	const pullNumber = requireOption(options, 'pr')
 	const head = requireOption(options, 'head')
 	const apiUrl = getApiUrl({ options, env, config })
-	const token = getToken({ options, env, config })
+	const token = getToken({ options, env, config, repo })
 
 	if (!token)
-		throw new Error('Missing Toast token. Run `toast auth` or set TOAST_TOKEN.')
+		throw new Error(
+			`Missing Toast token for ${repo.owner}. Run \`toast auth --org ${repo.owner}\` or set TOAST_TOKEN.`,
+		)
 
 	const requestOptions = {
 		method: 'GET',
@@ -396,6 +449,75 @@ const writeReviewState = ({ state, stdout, asJson, includeBlockingItems }) => {
 	})
 }
 
+const toList = values => (values || []).filter(Boolean).join(', ')
+
+const getWaitReasonParts = state => {
+	const waitingFor = state.waiting_for || {}
+	const reasons = []
+
+	if (waitingFor.pending_actors?.length) {
+		reasons.push(`open review request for ${toList(waitingFor.pending_actors)}`)
+	}
+	if (waitingFor.blocking_actors?.length) {
+		reasons.push(
+			`missing current-head review signal from ${toList(
+				waitingFor.blocking_actors,
+			)}`,
+		)
+	}
+	if (waitingFor.github_mergeability) {
+		reasons.push(`GitHub mergeability is ${waitingFor.github_mergeability}`)
+	}
+	if (waitingFor.github_checks?.length) {
+		reasons.push(`GitHub checks: ${toList(waitingFor.github_checks)}`)
+	}
+	if (state.status === 'head_changed') {
+		reasons.push(
+			`Toast has head ${state.head || '<unknown>'}; requested ${
+				state.requested_head || '<unknown>'
+			}`,
+		)
+	}
+	if (state.status === 'degraded:reconciliation_incomplete') {
+		reasons.push('exact GitHub reconciliation is incomplete; retrying')
+	}
+
+	return reasons
+}
+
+const getWaitStateKey = state =>
+	JSON.stringify({
+		status: state.status,
+		head: state.head,
+		next_action: state.next_action,
+		counts: state.counts,
+		waiting_for: state.waiting_for,
+		github_mergeability: state.github_mergeability,
+	})
+
+const writeWaitDiagnostic = ({ state, stderr, elapsedMs, unchangedMs }) => {
+	const reasons = getWaitReasonParts(state)
+	const reasonText = reasons.length
+		? reasons.join('; ')
+		: 'Toast did not return waiting_for details; run `toast review-ci next --json` to inspect the current state'
+
+	stderr.write(
+		`Review CI still ${state.status} after ${formatDurationMs(
+			elapsedMs,
+		)}: ${reasonText}.\n`,
+	)
+
+	if (unchangedMs >= WAIT_STALLED_AFTER_MS) {
+		stderr.write(
+			`Review CI state has not changed for ${formatDurationMs(
+				unchangedMs,
+			)}; inspect GitHub review requests/checks, re-request the stuck reviewer, or report this blocker.\n`,
+		)
+	}
+}
+
+const isReadyState = state => READY_STATUSES.has(state.status)
+
 const handleStatus = async context => {
 	const config = context.readConfig()
 	const { options } = parseArgs(getReviewStateArgs(context))
@@ -403,7 +525,7 @@ const handleStatus = async context => {
 
 	writeReviewState({ state, stdout: context.stdout, asJson: options.json })
 
-	return state.status === 'acknowledged' ? 0 : 1
+	return isReadyState(state) ? 0 : 1
 }
 
 const handleNext = async context => {
@@ -423,7 +545,7 @@ const handleNext = async context => {
 		includeBlockingItems: true,
 	})
 
-	return state.status === 'acknowledged' ? 0 : 1
+	return isReadyState(state) ? 0 : 1
 }
 
 const handleWait = async context => {
@@ -432,6 +554,9 @@ const handleWait = async context => {
 	const timeoutMs = parseDurationMs(options.timeout || '20m')
 	if (options.interval) parseDurationMs(options.interval)
 	const startedAt = toMillis(context.now())
+	let lastStateKey = null
+	let lastStateChangedAt = startedAt
+	let lastDiagnosticAt = null
 
 	while (true) {
 		const elapsedMs = toMillis(context.now()) - startedAt
@@ -453,7 +578,7 @@ const handleWait = async context => {
 			timeoutMs: waitTimeoutSeconds * 1000 + WAIT_REQUEST_GRACE_MS,
 		})
 
-		if (state.status === 'acknowledged') {
+		if (isReadyState(state)) {
 			writeReviewState({ state, stdout: context.stdout, asJson: options.json })
 			return 0
 		}
@@ -462,34 +587,106 @@ const handleWait = async context => {
 			writeReviewState({ state, stdout: context.stdout, asJson: options.json })
 			return 1
 		}
+
+		const observedAt = toMillis(context.now())
+		const stateKey = getWaitStateKey(state)
+		if (stateKey !== lastStateKey) {
+			lastStateKey = stateKey
+			lastStateChangedAt = observedAt
+		}
+		if (
+			lastDiagnosticAt === null ||
+			observedAt - lastDiagnosticAt >= WAIT_DIAGNOSTIC_INTERVAL_MS
+		) {
+			writeWaitDiagnostic({
+				state,
+				stderr: context.stderr,
+				elapsedMs: observedAt - startedAt,
+				unchangedMs: observedAt - lastStateChangedAt,
+			})
+			lastDiagnosticAt = observedAt
+		}
 	}
 
-	context.stderr.write('Timed out waiting for Review CI acknowledgement\n')
+	context.stderr.write('Timed out waiting for Review CI readiness\n')
 	return 2
 }
 
 const completeLogin = ({ context, auth, apiUrl }) => {
 	const existingConfig = { ...context.readConfig() }
-	const org = auth.org || (auth.repo ? parseRepo(auth.repo).owner : null)
+	const repo = auth.repo ? parseRepo(auth.repo).fullName : null
+	const org = auth.org || (repo ? parseRepo(repo).owner : null)
 
 	if (!org) throw new Error('Login response was missing org')
-	delete existingConfig.repo
 
 	const expiresAt = new Date(
 		toMillis(context.now()) + auth.expires_in * 1000,
 	).toISOString()
+	const tokensByOrg = { ...(existingConfig.tokensByOrg || {}) }
+	const tokensByRepo = { ...(existingConfig.tokensByRepo || {}) }
+
+	if (existingConfig.token) {
+		const legacyEntry = {
+			token: existingConfig.token,
+			...(existingConfig.githubLogin
+				? { githubLogin: existingConfig.githubLogin }
+				: {}),
+			...(existingConfig.expiresAt ? { expiresAt: existingConfig.expiresAt } : {}),
+		}
+
+		if (existingConfig.repo) {
+			try {
+				const legacyRepo = parseRepo(existingConfig.repo)
+				tokensByRepo[legacyRepo.fullName] = {
+					...legacyEntry,
+					org: legacyRepo.owner,
+					repo: legacyRepo.fullName,
+				}
+			} catch (e) {
+				// Ignore malformed legacy scope and keep the new login usable.
+			}
+		} else if (existingConfig.org) {
+			tokensByOrg[existingConfig.org] = {
+				...legacyEntry,
+				org: existingConfig.org,
+			}
+		} else {
+			// An unscoped legacy token has no safe owner to migrate.
+		}
+	}
+
+	const entry = {
+		token: auth.token,
+		githubLogin: auth.github_login,
+		expiresAt,
+		org,
+		...(repo ? { repo } : {}),
+	}
+	if (repo) {
+		tokensByRepo[repo] = entry
+	} else {
+		tokensByOrg[org] = entry
+	}
 
-	context.writeConfig({
+	const nextConfig = {
 		...existingConfig,
 		apiUrl,
 		org,
+		...(repo ? { repo } : {}),
 		token: auth.token,
 		githubLogin: auth.github_login,
 		expiresAt,
-	})
-	context.stdout.write(`Authenticated ${auth.github_login} for ${org}\n`)
+		tokensByOrg,
+		tokensByRepo,
+	}
+	if (!repo) delete nextConfig.repo
+	if (!Object.keys(tokensByOrg).length) delete nextConfig.tokensByOrg
+	if (!Object.keys(tokensByRepo).length) delete nextConfig.tokensByRepo
+
+	context.writeConfig(nextConfig)
+	context.stdout.write(`Authenticated ${auth.github_login} for ${repo || org}\n`)
 	context.stdout.write(
-		'Agent setup: run `toast review-ci instructions --install-codex` to install Review CI guidance.\n',
+		'Agent setup: run `toast review-ci instructions --install` to install Review CI guidance.\n',
 	)
 }
 
@@ -646,6 +843,9 @@ const handleAuth = async context => {
 
 const readReviewCiSkill = () => fs.readFileSync(REVIEW_CI_SKILL_PATH, 'utf8')
 
+const stripSkillFrontmatter = markdown =>
+	markdown.replace(/^---\r?\n[\s\S]*?\r?\n---(?:\r?\n)*/, '')
+
 const getCodexSkillPath = env =>
 	path.join(
 		env.CODEX_HOME || path.join(os.homedir(), '.codex'),
@@ -659,12 +859,118 @@ const installCodexSkill = ({ env, stdout }) => {
 
 	fs.mkdirSync(path.dirname(skillPath), { recursive: true, mode: 0o700 })
 	fs.writeFileSync(skillPath, readReviewCiSkill(), { mode: 0o600 })
-	stdout.write(`Installed Review CI agent instructions to ${skillPath}\n`)
+	stdout.write(`Installed Review CI Codex skill to ${skillPath}\n`)
+}
+
+const buildMarkdownInstructionsSection = () =>
+	`${REVIEW_CI_INSTRUCTIONS_START}\n${stripSkillFrontmatter(
+		readReviewCiSkill(),
+	).trim()}\n${REVIEW_CI_INSTRUCTIONS_END}\n`
+
+const getMalformedMarkerError = fileName =>
+	`${fileName} has a Toast Review CI start marker without an end marker; repair the marked section and rerun.`
+
+const removeMarkedInstructionFragments = ({ content, fileName }) => {
+	let next = content
+
+	while (next.includes(REVIEW_CI_INSTRUCTIONS_START)) {
+		const start = next.indexOf(REVIEW_CI_INSTRUCTIONS_START)
+		const end = next.indexOf(REVIEW_CI_INSTRUCTIONS_END, start)
+
+		if (end === -1) {
+			throw new Error(getMalformedMarkerError(fileName))
+		}
+
+		next = `${next.slice(0, start)}${next.slice(
+			end + REVIEW_CI_INSTRUCTIONS_END.length,
+		)}`
+	}
+
+	return next.split(REVIEW_CI_INSTRUCTIONS_END).join('')
+}
+
+const upsertMarkedSection = ({ content, fileName, section }) => {
+	const start = content.indexOf(REVIEW_CI_INSTRUCTIONS_START)
+	const end = content.indexOf(REVIEW_CI_INSTRUCTIONS_END, start)
+
+	if (start !== -1) {
+		if (end === -1) {
+			throw new Error(getMalformedMarkerError(fileName))
+		}
+
+		const cleanedBefore = removeMarkedInstructionFragments({
+			content: content.slice(0, start),
+			fileName,
+		})
+		const before = cleanedBefore.trim()
+			? `${cleanedBefore.replace(/\s*$/, '')}\n\n`
+			: ''
+		const after = removeMarkedInstructionFragments({
+			content: content.slice(end + REVIEW_CI_INSTRUCTIONS_END.length),
+			fileName,
+		}).replace(/^\s*/, '')
+
+		return `${before}${section}${after ? `\n${after}` : ''}`
+	}
+
+	const cleaned = removeMarkedInstructionFragments({ content, fileName })
+	const prefix = cleaned.trim() ? `${cleaned.replace(/\s*$/, '')}\n\n` : ''
+
+	return `${prefix}${section}`
+}
+
+const getMarkdownInstructionsUpdate = ({ cwd, fileName }) => {
+	const filePath = path.join(cwd, fileName)
+	const exists = fs.existsSync(filePath)
+	const existing = exists ? fs.readFileSync(filePath, 'utf8') : ''
+	const next = upsertMarkedSection({
+		content: existing,
+		fileName,
+		section: buildMarkdownInstructionsSection(),
+	})
+
+	return {
+		exists,
+		fileName,
+		filePath,
+		next,
+	}
+}
+
+const writeMarkdownInstructionsUpdate = ({ update, stdout }) => {
+	if (update.exists) {
+		fs.writeFileSync(update.filePath, update.next)
+	} else {
+		fs.writeFileSync(update.filePath, update.next, { mode: 0o644 })
+	}
+	stdout.write(
+		`Installed Review CI ${update.fileName} instructions to ${update.filePath}\n`,
+	)
+}
+
+const installAgentInstructions = ({ cwd, env, stdout }) => {
+	const markdownUpdates = ['CLAUDE.md', 'AGENTS.md'].map(fileName =>
+		getMarkdownInstructionsUpdate({ cwd, fileName }),
+	)
+
+	markdownUpdates.forEach(update =>
+		writeMarkdownInstructionsUpdate({ update, stdout }),
+	)
+	installCodexSkill({ env, stdout })
 }
 
 const handleInstructions = async context => {
 	const { options } = parseArgs(context.commandArgs || [])
 
+	if (options.install || options['install-agent']) {
+		installAgentInstructions({
+			cwd: context.cwd,
+			env: context.env,
+			stdout: context.stdout,
+		})
+		return 0
+	}
+
 	if (options['install-codex']) {
 		installCodexSkill({ env: context.env, stdout: context.stdout })
 		return 0
@@ -681,7 +987,7 @@ const printHelp = stdout => {
   toast review-ci status --repo ORG/REPO --pr NUMBER --head SHA
   toast review-ci next --repo ORG/REPO --pr NUMBER --head SHA
   toast review-ci wait --repo ORG/REPO --pr NUMBER --head SHA
-  toast review-ci instructions [--install-codex]
+  toast review-ci instructions [--install|--install-agent|--install-codex]
 `)
 }
 
@@ -707,6 +1013,7 @@ const main = async ({
 	createBrowserCallbackServer = createBrowserCallbackServerDefault,
 	openBrowser = openBrowserDefault,
 	getGitRemoteUrl = getGitRemoteUrlDefault,
+	cwd = process.cwd(),
 	stdout = process.stdout,
 	stderr = process.stderr,
 }) => {
@@ -721,6 +1028,7 @@ const main = async ({
 		createBrowserCallbackServer,
 		openBrowser,
 		getGitRemoteUrl,
+		cwd,
 		stdout,
 		stderr,
 	}