@geoqiao/pi-ask 1.3.0 → 1.3.1

package/CHANGELOG.md

@@ -1,3 +1,9 @@
+## 1.3.1
+
+### Patch Changes
+
+- 1e56b27: Ask only when context review leaves a critical requirement, outcome-changing preference, or high-impact authorization gap unresolved, or when the user explicitly requests interactive questions. Align tool prompts, the bundled skill, and docs so alternatives, ordinary comparisons, and already authorized work do not automatically trigger interviews. Preserve safety and payload/TUI/RPC constraints; add static prompt regression checks and document model-evaluation scenarios without claiming verified model behavior.
+
 ## 1.3.0
 
 ### Minor Changes

package/README.md

@@ -194,7 +194,11 @@ Accepted notation follows pi-tui key ids. Common aliases are normalized, for exa
 
 After installation, the extension registers the `ask_user` tool plus `/ask-settings`, `/answer`, `/answer:again`, and `/ask:replay` commands.
 
-Agents can auto-discover and call `ask_user` when they need clarification instead of guessing. They can mark any number of grounded preferences with `recommended: true` and use option descriptions for reasons. In interactive sessions, it opens a terminal UI flow for structured answers, supports native pi-style `@` file references while typing answers or notes, and returns normalized answers back to the agent. Ask settings are available both from `?` in the ask flow and from the `/ask-settings` command. Behaviour and notification settings are binary `on`/`off` toggles that save immediately when the config file is writable; save failures revert the toggle and show a manual-edit message. The settings overlay includes a guarded double-press reset-to-defaults action; keymaps, notification channels, and extraction settings are changed by editing the shown config file path.
+Agents first read available context and call `ask_user` only for unresolved critical requirements, outcome-changing preferences, or consequential/hard-to-reverse actions beyond existing authorization. Explicitly requested interviews and requirements gathering also use the tool.
+
+Clear small changes, settled choices, authorized reversible steps, and routine implementation details proceed without reconfirmation. Multiple options or an architecture/naming/research label alone do not trigger questions; clear comparison/research requests get analysis first. Questions stay on current blockers, and settled decisions reopen only for materially new information. Delegated autonomy does not waive safety boundaries, and cancellation or missing/unclear answers never supplies high-risk approval. This is prompt guidance, not a runtime permission check; see the [policy and verification limits](docs/contract.md#skill-alignment-advisory) and [skill examples](skills/ask-user/SKILL.md#examples-and-behavioral-evaluation-cases).
+
+Agents can mark any number of grounded preferences with `recommended: true` and use option descriptions for reasons. In interactive sessions, it opens a terminal UI flow for structured answers, supports native pi-style `@` file references while typing answers or notes, and returns normalized answers back to the agent. Ask settings are available both from `?` in the ask flow and from the `/ask-settings` command. Behaviour and notification settings are binary `on`/`off` toggles that save immediately when the config file is writable; save failures revert the toggle and show a manual-edit message. The settings overlay includes a guarded double-press reset-to-defaults action; keymaps, notification channels, and extraction settings are changed by editing the shown config file path.
 
 ### Pi RPC fallback
 

package/docs/README.md

@@ -8,7 +8,7 @@ This folder keeps only the documentation needed to understand and maintain the e
 - `contract.md` — external behavior, tool payload/result details, and UX guarantees
 - `remote-events.md` — local inter-extension event contract, bridge examples, and smoke-test steps
 - `architecture.md` — module boundaries and invariants
-- `../skills/ask-user/SKILL.md` — auto-bundled agent-side decision-gate guidance; enabled by default, but can be disabled via `pi config`; inspired by https://github.com/edlsh/pi-ask-user
+- `../skills/ask-user/SKILL.md` — context-first clarification policy and behavioral evaluation cases (not model test results); enabled by default, but can be disabled via `pi config`; inspired by https://github.com/edlsh/pi-ask-user
 
 ## Reading order
 

package/docs/contract.md

@@ -1,6 +1,6 @@
 # Ask tool contract
 
-`ask_user` is a pi-native clarification tool for cases where implementation depends on user preference or missing requirements.
+`ask_user` is a pi-native clarification tool for material requirement, preference, or high-impact authorization gaps left after context review, and for explicitly requested interviews.
 
 This document defines the stable external behavior. It does not explain internal helper-by-helper implementation.
 
@@ -196,8 +196,8 @@ This document defines the stable external behavior. It does not explain internal
 - option-targeted elaboration items include the specific noted option plus whether it is currently selected
 - question-targeted elaboration items include whether the question already has a committed answer
 - `elaboration.instruction` tells the agent to answer the clarification directly first, then re-ask only the affected questions if a choice is still needed
-- after clarification, agents should prefer another structured follow-up over plain-text multiple choice when a decision is still unresolved
-- once prior answers narrow the branch, agents should bundle the next 2-3 related unresolved questions into one follow-up ask when possible, instead of using a long sequence of single-question asks
+- after clarification, agents should use a structured follow-up rather than plain-text multiple choice only when a material decision still blocks progress
+- follow-ups should preserve settled answers and ask only current blockers; related blockers may be bundled when they do not depend on each other's answers, without filling a question quota
 - `elaboration` is only present when `mode === "elaborate"`
 - elaborate `content` text and transcript rendering describe each note directly using the full question prompt and option label, and include the current committed answer text when available, instead of a generic elaboration banner
 - when the user selects `Elaborate` without adding notes, elaborate `content` text and transcript rendering still include the committed answer text so the agent can elaborate on that answer directly
@@ -332,9 +332,15 @@ The non-interactive fallback message includes normalized pending questions and o
 
 ## Skill alignment (advisory)
 
-The auto-bundled skill profile at `skills/ask-user/SKILL.md` defines agent-side decision-gate guidance for when to call `ask_user`. It is enabled by default when the package is installed, but can be disabled via `pi config`.
+The tool description, `promptSnippet`, `promptGuidelines`, and auto-bundled [`ask-user` skill](../skills/ask-user/SKILL.md) share the same context-first policy. The skill is enabled by default when installed and can be disabled via `pi config`; the tool guidance does not depend on loading it.
 
-It is advisory only. If there is any conflict, contract + tests win.
+Read available code, docs, conversation, prior answers, and existing authorization before asking. Use `ask_user` only for a critical requirement still missing/conflicting, an unresolved preference that materially changes the outcome, or missing authorization for a consequential or hard-to-reverse action beyond the approved scope. Explicit requests for interviews, requirements gathering, or interactive questions also use `ask_user`; a requested written questionnaire remains a prose artifact.
+
+Do not ask about matters already resolved by context or reconfirm settled choices/authorization. Proceed with reversible steps and ordinary implementation details within scope, stating useful assumptions. Multiple viable options and labels such as architecture, naming, or research alone do not justify asking. Complete clear comparison/research requests first rather than automatically starting an interview. Ask only current blockers (or the current requested interview topic), and reopen settled decisions only for materially new information.
+
+Delegated autonomy does not waive safety boundaries. Cancellation, missing answers, or ambiguous responses are not high-risk approval; leave unauthorized actions blocked. Neither `cancelled: false` nor advisory `required` metadata establishes approval.
+
+This policy is advisory, not runtime authorization enforcement; contract + tests take precedence for tool behavior. `tests/ask-tool.test.ts` checks the actual registered prompt fields, retained payload/RPC guidance, and removal of old blanket triggers. These are static string checks, not proof of model behavior. The skill records behavioral evaluation cases, including small edits, approved plans, missing requirements, unauthorized high-risk actions, explicit interviews, and ordinary comparisons. Actual evaluation requires model sessions and inspection of context reads, questions, and subsequent actions, with and without the skill; record false positives/negatives separately from static test results.
 
 ## Source of truth
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@geoqiao/pi-ask",
-  "version": "1.3.0",
+  "version": "1.3.1",
   "description": "Pi package that adds an interactive ask_user clarification tool.",
   "license": "MIT",
   "type": "module",

package/skills/ask-user/SKILL.md

@@ -1,112 +1,70 @@
 ---
 name: ask-user
-description: "Use ask_user as a decision, research, and requirements gate before ambiguous or high-stakes choices."
+description: "Use ask_user when context review leaves a critical requirement, outcome-changing preference, or high-impact authorization gap unresolved, or the user explicitly requests an interview, requirements gathering, or interactive questions. Not for multiple options or comparison/research requests alone."
 metadata:
-  short-description: Decision, research, and requirements gate
+  short-description: Clarify material gaps after reading context
 ---
 
-# Ask User decision/research gate
+# Ask User: clarify material gaps
 
-Use this skill to force explicit user alignment before consequential decisions, preference-sensitive planning, research scoping, or requirements gathering.
+Read available context before asking. Use `ask_user` to resolve a gap that materially changes the result, not to transfer routine decisions back to the user.
 
-This skill is for decision control, research scoping, requirements gathering, and preference-sensitive planning, not general chat.
+## When to ask
 
-## Trigger
+Read relevant code, docs, conversation, prior answers, and the user's existing requirements and authorization. Resolve factual uncertainty through available evidence rather than asking the user to repeat it. Do not invent preferences or permission from a code convention.
 
-Classify the next step as one of:
+Use `ask_user` when that review still leaves:
 
-- `high_stakes`
-- `ambiguous`
-- `both`
-- `clear`
+- a critical requirement missing or conflicting, so proceeding would produce materially different results;
+- an important preference unresolved that materially changes the outcome and cannot be inferred from the user's requirements;
+- missing authorization for a consequential or hard-to-reverse next action beyond the approved scope.
 
-Use `ask_user` when the next step is ambiguous, preference-sensitive, or high-stakes and the user has not already made the decision explicitly.
+Also use `ask_user` when the user explicitly requests an interview, requirements gathering, or interactive questions. Ask about the current topic and use existing answers rather than restarting discovery. If the user requests a written questionnaire or checklist instead, provide that artifact in prose.
 
-Use `ask_user` for any domain where user input changes the plan, recommendation, research direction, output format, criteria, constraints, or next action.
+## When to proceed
 
-Also use `ask_user` when the user asks to gather requirements, interview them, ask questions, scope research, plan work, compare options, or answer a set of open product/design/architecture/research questions. Do not respond with a plain-text questionnaire unless the user explicitly asks for a checklist or written questionnaire.
+- Do not ask about facts or requirements already established in code, docs, or the conversation.
+- Do not reconfirm settled choices or existing authorization. Carry out reversible steps and routine implementation details within the authorized scope; state useful assumptions without turning them into approval requests.
+- Multiple viable options do not by themselves justify a question. Labels such as architecture, schema, naming, UX, planning, or research do not establish a material gap or a need for new permission.
+- Complete clear comparison/research requests first. Analyze evidence and trade-offs, give conditional conclusions where appropriate, and do not automatically turn a comparison into an interview. Ask only if a remaining material gap actually blocks the requested analysis or next action.
+- Treat "your call" as delegation within the stated scope, not as missing preference information. Do not let delegated autonomy waive safety boundaries or expand authorization to unrelated high-impact actions.
 
-### Treat as `high_stakes` when the next step changes:
+## Keep questions focused
 
-- architecture, schema, API contract, deployment, or security posture
-- production-facing behavior in a costly-to-undo way
-- large refactors, migrations, or destructive edits
-- legal, financial, medical, career, hiring, vendor, purchasing, travel, or other costly-to-reverse decisions
-- public-facing claims, sensitive communications, or consequential recommendations
+Explain the blocking gap and its consequence briefly, then ask one concrete decision per question using `ask_user`, not a plain-text multiple-choice detour. Ask only current blockers; bundle related blockers only if they can be answered independently. In a requested interview, keep each batch on the current topic and follow the user's requested pacing.
 
-### Treat as `ambiguous` when:
+Use the answer explicitly and preserve resolved decisions. After an elaboration or note, answer the clarification first; use a structured follow-up only if a material decision still blocks progress. Respect `continuation.preservedAnswers` and revisit only affected unresolved questions. Reopen a settled decision only when materially new information changes its assumptions, scope, or consequences, and explain what changed.
 
-- requirements, goals, constraints, evaluation criteria, or success criteria are missing/conflicting
-- multiple valid options exist and the trade-off is preference-sensitive
-- research scope, audience, budget, timeline, risk tolerance, or output format is unclear
-- you would otherwise make a material assumption
+Cancellation, missing answers, or ambiguous responses are not approval for high-risk actions. If authorization remains missing, leave that action blocked and explain the boundary; do not automatically repeat the same question or silently choose a risky default. Continue only independent work still covered by existing authorization. `cancelled: false` and `required: true` do not prove an answer or approval exists: required is advisory, and RPC dismissal can skip a question without cancelling the flow.
 
-## Handshake (required)
+## Payload and presentation
 
-1. Gather evidence first from code/docs/tools.
-2. Summarize neutral context (current state, constraints, trade-offs, recommendation).
-3. Ask one focused `ask_user` decision question, or bundle 2-5 closely related questions when the user is explicitly in requirement-gathering/interview mode.
-4. Restate the user decision and proceed explicitly with it.
-5. Re-open only for materially new ambiguity.
-
-## Question spew prevention
-
-Before sending any assistant response that contains 2+ substantive questions for the user, stop and decide whether those questions should be interactive.
-
-Use `ask_user` instead of prose when:
-
-- the questions are meant to collect requirements, goals, constraints, preferences, scope, priorities, criteria, or missing context
-- answers will materially change the next artifact, recommendation, research direction, plan, implementation, architecture, schema, UX, stack choice, or decision criteria
-- the user previously corrected you with phrases like "ask those questions", "ask interactively", or "use ask_user"
-
-Plain-text questions are acceptable only when:
-
-- the user asked for a written checklist/list of open questions
-- the questions are rhetorical or purely explanatory
-- there is exactly one small factual clarification and an interactive flow would be heavier than needed
-
-If there are too many questions, group them into the smallest coherent `ask_user` batches and ask the highest-impact batch first.
-
-## Question budget and escalation
-
-- Max 1 `ask_user` call per decision boundary in normal cases.
-- Max 2 calls for the same boundary if first answer is unclear/cancelled.
-- Never re-ask the same trade-off without new evidence.
-
-Attempt 2 (only if needed) must be narrower and include:
-
-- `Proceed with recommended option`
-- `Choose another option`
-- `Stop for now`
-
-After attempt 2:
-
-- for `high_stakes` or `both`: stop as blocked until explicit decision
-- for `ambiguous` only: if user delegates ("your call"), proceed with the most reversible default and state assumptions
+- Include a stable question `id`, non-empty `prompt`, and a non-empty machine-readable `value` and visible `label` for every option. Question ids must be unique within a call; option values must be unique within a question.
+- Keep labels short and options distinct and outcome-oriented. Do not add filler options. Use `description` for meaningful trade-offs.
+- Mark grounded preferences with `recommended: true` and explain the reason in `description`; recommendations are presentation-only and never preselected.
+- Choose `single` for one expected answer, `multi` for multiple possible selections, and `preview` for richer comparison detail. Every declared preview option must include non-empty `preview` text; descriptions alone do not suffice.
+- TUI provides tabbed questions, native single/multi selection, a preview pane, and an internal `Type your own` fallback for every question type, including preview.
+- RPC presents questions sequentially with one real option or `Type something…` per question. Use typed input for multiple choices; previews and descriptions flatten into option text. Do not promise same-screen forms, native checkbox cards, a custom preview pane, notes, or a review tab in RPC.
 
-## ask_user payload quality
+## Examples and behavioral evaluation cases
 
-- Ask one concrete decision at a time.
-- Provide clear, distinct options. Do not add filler options.
-- Choose question type from semantics: `single` means one answer is expected, `multi` means multiple answers could reasonably be selected, and `preview` means options need preview-pane detail with non-empty preview text.
-- Avoid defaulting mechanically; infer from whether options are mutually exclusive, can coexist, or need preview-pane detail.
-- Keep option labels short and outcome-oriented.
-- Include trade-off descriptions when non-obvious.
-- Include a stable question `id`, non-empty `prompt`, and a non-empty `value` and visible `label` for every option.
-- Mark grounded preferences with `recommended: true` and explain the reason in `description`; recommendations are presentation-only and never preselected.
-- For research/planning, ask about goals, constraints, evaluation criteria, audience, budget, timeline, risk tolerance, and desired output only when they materially affect the result.
-- Prefer non-`preview` questions when a free-form answer may be useful, since those include an internal `Type your own` option.
+These are expected behaviors for model evaluation, not evidence that a model follows the policy. Static prompt tests check registered wording and tool constraints only.
 
-## Guardrails
+| Context / request | Expected behavior |
+| --- | --- |
+| "Fix this typo and update its test" with the target and expected text supplied | Read the relevant files and make the small change; no interview about naming, style, or alternatives. |
+| "Use the existing helper; implement the approved plan" | Follow the helper and plan; do not re-ask which approach to use. |
+| "Add data expiry" with no retention period in code, docs, or prior requirements | Ask for the retention requirement before implementing deletion semantics. |
+| "Draft the customer announcement" but the audience and disclosure scope remain unresolved and would materially change the message | Ask only for the missing consequential content preferences, not every possible tone choice. |
+| "Prepare a migration plan" followed by a proposed production migration or irreversible deletion | Ask for explicit authorization before execution; permission to plan is not permission to execute. |
+| "Interview me to gather requirements" | Use `ask_user` for the current interview topic, respecting the requested pacing and prior answers. |
+| "Compare SQLite and PostgreSQL for a small local app" | Analyze the comparison first, using evidence and conditional trade-offs; do not automatically ask the user to choose a database or define a full product brief. |
+| "Choose the local implementation details yourself" with several reversible approaches | Use established patterns within scope and state useful assumptions; do not ask merely because alternatives exist. |
+| A high-risk authorization question is cancelled, skipped, or answered vaguely | Do not execute the action or infer approval from completion metadata; explain the unresolved boundary. |
+| A chosen migration was approved for staging, but new evidence shows the target is production | Reopen only the changed target/authorization decision; do not restart the whole interview. |
 
-- Do not ask before reading available context.
-- Do not use for trivial formatting/style micro-decisions.
-- Do not continue implementation after unclear high-stakes answers.
+For actual model evaluation, run these cases in fresh sessions with the local extension alone and with the bundled skill loaded. Inspect context reads, whether and what the model asks, and its next action; include prior-answer and cancellation turns. Record model/version, loaded instructions, observed behavior, and false positives/negatives. Never execute real high-risk actions for these checks.
 
 ## Conflict rule
 
-If this skill conflicts with implementation behavior or tests, the project contract wins:
-
-1. `docs/contract.md`
-2. `tests/*.test.ts`
-3. this skill
+This guidance is advisory, not a runtime authorization mechanism. For tool behavior, [`docs/contract.md`](../../docs/contract.md) and the package tests take precedence over this skill.

package/src/ask-tool-helpers.ts

@@ -13,21 +13,18 @@ import type {
 } from "./types.ts";
 
 export const ASK_TOOL_DESCRIPTION =
-	"Interactive clarification tool for cases where the next step depends on user preferences, missing requirements, or choosing between multiple valid directions. Ask a short structured interview, collect normalized answers, and continue using those answers explicitly instead of guessing. TUI mode supports single-select, multi-select, and preview-pane questions; RPC mode presents questions sequentially, offers one portable choice per question plus a typed-answer fallback, and flattens preview details into option text. Always include a stable `id` and non-empty `prompt` for every question, plus a machine-readable `value` and visible `label` for every option. Use `preview` only when every option includes `preview` text; descriptions alone are not enough.";
+	"Interactive clarification after reading context: ask only for unresolved critical requirements, outcome-changing preferences, or consequential/hard-to-reverse actions beyond existing authorization; also for explicitly requested interviews, requirements gathering, or interactive questions. Multiple options alone do not justify asking. TUI supports single-select, multi-select, and preview-pane questions; RPC asks sequentially with one choice or typed input, flattening previews into option text. Include a stable `id` and non-empty `prompt` for each question, plus a non-empty machine-readable `value` and visible `label` for each option. Use `preview` only when every option has non-empty `preview` text; descriptions alone do not suffice.";
 
 export const ASK_TOOL_PROMPT_GUIDELINES = [
-	"Use `ask_user` before making preference-sensitive decisions about scope, tone, UX, naming, architecture, docs, or implementation direction.",
-	"When multiple valid directions exist, call `ask_user` with 1-3 concise questions instead of committing to one path on your own.",
-	"When calling `ask_user`, prefer one focused decision per question. Use short labels. Provide clear, distinct options. Do not add filler options.",
-	"When calling `ask_user`, always include a stable `id` and non-empty `prompt` for every question.",
-	"When calling `ask_user`, always include a non-empty machine-readable `value` and visible `label` for every option.",
-	"When calling `ask_user`, mark grounded preferences with `recommended: true` and use the option `description` to state the reason.",
-	"When calling `ask_user`, choose question `type` from the question semantics: `single` means one answer is expected, `multi` means multiple answers could reasonably be selected, and `preview` means options need preview-pane detail.",
-	'When calling `ask_user`, use `type: "preview"` only when every option includes non-empty `preview` text. Option descriptions do not satisfy this requirement.',
-	"After an `ask_user` elaboration or follow-up note, prefer another structured `ask_user` follow-up if a choice is still needed instead of switching to plain-text multiple choice in chat.",
-	"When prior `ask_user` answers narrow the branch, bundle the next 2-3 related unresolved decisions into one follow-up `ask_user` call when possible.",
-	"Use one-at-a-time `ask_user` follow-up calls only when the next question materially depends on the previous answer.",
-	"Do not promise same-screen forms, native checkbox cards, or a custom preview pane when `ask_user` is rendered through RPC; the portable fallback asks questions sequentially, and users type multiple choices through `Type something…` when one scalar selection is insufficient.",
+	"Before `ask_user`, read available context: code, docs, conversation, and prior answers. Ask only if a critical requirement or outcome-changing preference remains unresolved, or a consequential or hard-to-reverse action exceeds existing authorization.",
+	"Use `ask_user` for explicitly requested interviews, requirements gathering, or interactive questions. Multiple options or architecture/naming/research labels alone do not justify asking; analyze clear comparison/research requests first.",
+	"Do not use `ask_user` to reconfirm settled choices or authorization. Proceed with authorized reversible steps and routine implementation details; state useful assumptions.",
+	"With `ask_user`, delegated autonomy does not waive safety boundaries. Cancellation, missing answers, or ambiguity are not high-risk approval; keep unauthorized high-risk actions blocked.",
+	"In `ask_user`, ask only current blockers (or the requested interview topic), one decision per question; bundle independent related questions. Answer elaboration notes first; re-ask only remaining blockers. Reopen settled decisions only for materially new information.",
+	"For `ask_user`, include a stable `id` and non-empty `prompt` for each question, and a non-empty machine-readable `value` and visible `label` for each option. Keep labels short and options distinct; no filler.",
+	"For `ask_user`, mark grounded preferences with `recommended: true` and explain the reason in `description`; recommendations are not preselected.",
+	"For `ask_user`, use `single` for one answer, `multi` for multiple possible selections, and `preview` only when every option has non-empty `preview` text; descriptions alone do not suffice.",
+	"For `ask_user` in RPC, questions are sequential with one choice or `Type something…` (including typed multiple choices); previews flatten into option text. Do not promise same-screen forms, native checkbox cards, or a custom preview pane.",
 ] as const;
 
 interface ValidateParamsOptions {

package/src/ask-tool.ts

@@ -30,7 +30,7 @@ export function registerAskTool(
 		label: "Ask User",
 		description: ASK_TOOL_DESCRIPTION,
 		promptSnippet:
-			"Clarify ambiguous or preference-sensitive decisions with a short interactive interview before proceeding",
+			"Ask only for material requirement/preference or high-impact authorization gaps left after context review, or explicitly requested interviews",
 		promptGuidelines: [...ASK_TOOL_PROMPT_GUIDELINES],
 		parameters: AskParamsSchema,
 		prepareArguments: (args) => prepareAskParams(args) as AskParams,