pi-critique-model 1.0.0 → 1.1.0

package/README.md

@@ -1,28 +1,36 @@
 <div align="center">
 
-![Critique banner](docs/banner.png)
+![Critique banner](https://raw.githubusercontent.com/noguerol/critique/main/docs/banner.jpeg)
 
 </div>
 
-# Critique — Adversarial Code Review for pi
+# Critique — Adversarial Review for pi
 
-**Critique reviews the last work step with a separate model and feeds the review back to the working model as advisory feedback.** Two thinking machines face off — the working model produced a work step, and an independent reviewer picks it apart for correctness, robustness, maintainability and efficiency. The verdict comes back as structured Markdown: `APPROVED`, `APPROVED_WITH_SUGGESTIONS`, or `CHANGES_RECOMMENDED` — and the working model stays the final judge. The feedback is **non-mandatory**: it applies, partially applies, or rejects each point as it sees fit.
+**Critique questions the last work step with a separate model and can challenge user instructions before the model starts.** It is not limited to code: it can review implementation work, writing, plans, research, data analysis, ops, or any other project work. Optionally, Critique also runs a fast pre-flight check on sufficiently rich user prompts and shows a short `Critique` widget when the instruction deserves pushback. The feedback is **non-mandatory**: the user and the working model remain the final judges.
+
+Additionally, the **Questions** feature detects ambiguous user input and offers clarifying suggestions before the model acts on it.
 
 ---
 
 ## Features
 
-- **Independent reviewer** — a separate model judges the work step from the serialized context, with no tools of its own (can't modify the codebase, can't see your secrets)
+- **Independent reviewer** — a separate model questions the work step from serialized context, with no tools of its own (can't modify files or see hidden state)
 - **Auto-detects the work step** — splits the session branch into *episodes* at user-message boundaries; the last episode containing tool calls or assistant output is the work step
 - **Token-budgeted context** — the user prompt, assistant messages, tool calls and tool results are truncated to a self-contained block so the reviewer sees what it needs without overflow
-- **Multi-step review** — review the last `N` episodes in one shot (`/critique 3`) to catch interactions across steps
-- **Focus note** — `/critique check the error handling` biases the review without limiting it; the reviewer still scans for everything
-- **Structured output** — fixed `Verdict / Issues / Suggestions / Summary` schema so the feedback is always actionable and machine-parseable
-- **Independent model picker** — `/critique config` lets you choose the reviewer; the default is a *different* model than the working one, ensuring a genuinely independent perspective
+- **Multi-step review** — review the last `N` episodes in one shot (`/critique 3`) to catch cross-step issues
+- **Focus note** — `/critique check the assumptions` biases the review without limiting it; the reviewer still scans for everything
+- **Structured output** — fixed `Verdict / Issues / Suggestions / Summary` schema for actionable, parseable feedback
+- **Settings menu** — `/critique config` opens an editable menu so you can change one setting at a time without rerunning a full wizard
 - **Advisory injection** — on by default; toggle off (or use `/critique view`) to keep the review as read-only
 - **Cancelable loader** — in TUI mode, the review runs behind a loader that you can abort with `Esc`
 - **No provider surprises** — empty reviews are flagged, provider errors are surfaced as errors instead of silently producing nothing
-- **Persistent config** — `~/.pi/agent/critique.json` stores the model choice and auto-inject toggle across all projects
+- **Automatic prompt critique** — optional pre-flight challenge for user instructions before the model starts; trivial inputs (`ok`, tiny commands, quick corrections) are ignored
+- **Three prompt-critique levels** — `Inconsistencies only` (low sensitivity), `Critical` (moderate), or `Corrosive` (high)
+- **Prompt-critique model source** — use either the active working model or the configured critique model
+- **Interactive Critique widget** — ultra-short one-sentence advice in the user's interaction language with `Accept`, `Discard`, or `Reply`; auto-discards after 30 seconds
+- **Persistent config** — `~/.pi/agent/critique.json` stores model choice, auto-inject, and automatic prompt-critique settings across all projects
+- **Questions feature** — optional clarifying widget that detects ambiguous user input and offers three options: two suggested interpretations and a free-text answer; auto-discards after 30 seconds
+- **Questions frequency** — three sensitivity levels: `Essential only` (minimal), `Normal` (moderate), or `Many questions` (high sensitivity)
 
 ## Install
 
@@ -52,38 +60,38 @@ pi remove npm:pi-critique-model
 
 > **Security:** pi packages run with full system access — extensions execute arbitrary code. Install only packages you trust and review the source.
 
-**Requirements:** a working pi installation with at least two models configured (one is the *working* model, the other becomes the *reviewer*). Models can be from the same provider as long as they have different IDs.
+**Requirements:** a working pi installation with at least one configured model. For independent reviews, configure a second model as reviewer.
 
 ## Quick Start
 
 ```
-/critique config        # (optional) pick a reviewer model — defaults to "different from the working one"
+/critique config        # (optional) open the settings menu
 ...                     # let the main model do some work
 /critique               # review the last work step and feed the feedback back
 ```
 
-The main model then sees the review appended to its next turn and decides what to apply. With auto-inject off (or `/critique view`), the review is only displayed to you — useful when you're just exploring whether to apply changes.
+The main model then sees the review appended to its next turn and decides what to apply. With auto-inject off (or `/critique view`), the review is only displayed to you.
 
 To focus the review:
 
 ```
-/critique check the error handling on the retry logic
+/critique check the assumptions behind the plan
 /critique 3                             # review the last 3 work steps
-/critique 2 look at the test coverage   # combine count + focus
+/critique 2 look at the evidence gaps   # combine count + focus
 ```
 
 ## Commands
 
 | Command | Description |
 |---------|-------------|
-| `/critique` | Review the last work step and inject the feedback back into the working model |
+| `/critique` | Review/question the last work step and inject feedback into the working model |
 | `/critique <focus>` | Review the last work step with an additional focus note |
 | `/critique N` | Review the last `N` work steps (max 5) |
 | `/critique N <focus>` | Combine count and focus |
 | `/critique view` | Show the review only, without injecting it |
 | `/critique view <focus>` | View-only, with a focus note |
 | `/critique view N` | View-only, last `N` steps |
-| `/critique config` | Pick the critique model from pi's native active models and toggle auto-inject |
+| `/critique config` | Open the editable settings menu for model, auto-inject, prompt critique, and questions |
 
 **Argument parsing:**
 
@@ -95,7 +103,7 @@ To focus the review:
 
 ### 1. Extract the work step
 
-The session branch is split into *episodes* at user-message boundaries. The last episode containing tool calls or assistant output is the work step: the user request that triggered it, every tool call (with arguments), and every tool result (diffs, command output, errors). Content is truncated to a token budget so the reviewer sees a focused, self-contained context:
+The session branch is split into *episodes* at user-message boundaries. The last episode containing tool calls or assistant output is the work step: the user request that triggered it, every tool call (with arguments), and every tool result (diffs, command output, data, errors). Content is truncated to a token budget so the reviewer sees focused, self-contained context:
 
 | Field | Max chars |
 |-------|-----------|
@@ -109,11 +117,11 @@ Truncated content is marked with `… [truncated]` so the reviewer can tell what
 
 ### 2. Ask the reviewer
 
-The critique model is called directly through `ctx.modelRegistry.complete()` with **no tools** — it only judges. The reviewer is told:
+The critique model is called directly through `ctx.modelRegistry.complete()` with **no tools** — it only judges/questions. The reviewer is told:
 
 - The work step inside `<work-step>` tags
 - An optional `<focus-note>` if you passed one
-- "Do not invent issues: if the work is correct, say so and keep suggestions minimal"
+- "Do not invent issues: if the work is sound, say so and keep suggestions minimal"
 
 It replies in a fixed Markdown structure:
 
@@ -131,29 +139,65 @@ APPROVED | APPROVED_WITH_SUGGESTIONS | CHANGES_RECOMMENDED
 2-4 sentence overall assessment.
 ```
 
-The reviewer is also told explicitly to base its judgment *only* on the provided work step, so a reviewer on a smaller/cheaper model still gives useful feedback.
+The reviewer is told to base its judgment *only* on the provided work step, across any domain: code, writing, planning, research, data, ops, etc.
 
 ### 3. Inject the feedback
 
 The review is sent back to the working model as a follow-up user message:
 
 ```
-[Critique — advisory review of your last work step]
+[Critique — advisory]
+
+Reviewer: `provider/model`. Advice only: apply useful points; briefly reject bad ones.
 
-A separate reviewer model (`provider/model`) reviewed the work you just
-performed. This feedback is **advisory, not mandatory**: you are the final
-judge. Apply only the points that genuinely improve the work, and if you
-disagree with any of them, briefly explain why and continue.
+---
 
---- Review ---
 <the review>
 ```
 
 The main model then has the freedom to apply, partially apply, or reject each point. If auto-inject is off, the review is only shown to you.
 
+### 4. Optional automatic prompt critique
+
+When enabled in `/critique config`, Critique listens to user input before prompt-template expansion and before the agent starts. A local heuristic skips acknowledgements, slash commands, tiny corrections, and short commands. For richer instructions, a tool-free model call decides whether there is anything worth challenging.
+
+If critique is useful, pi shows an ultra-short `Critique` widget in the user's interaction language. The model always returns both an issue and a proposed fix:
+
+- `Accept` includes the proposed fix as extra guidance for the model.
+- `Discard` sends the original prompt unchanged.
+- `Reply` lets the user answer the critique/fix before the model sees the prompt.
+- No interaction within 30 seconds auto-discards the advice and sends the original prompt unchanged.
+
+Automatic prompt critique can use either the active working model or the configured critique model. It does not require a separate model.
+
+### 5. Optional Questions feature
+
+When enabled in `/critique config`, Critique analyzes user input for ambiguity — unclear pronouns, vague directives, or scope-unclear requests — and shows a `Questions` widget before the model receives the prompt.
+
+The widget presents three options in the user's interaction language:
+
+- **A** — a suggested interpretation (e.g., "Focus on the primary task")
+- **B** — an alternative interpretation (e.g., "Address all aspects comprehensively")
+- **C** — the user types their own answer in a free-text editor
+- **Dismiss** — send the original prompt unchanged (auto-dismisses after 30 seconds)
+
+The selected answer is prepended to the user's prompt as a clarification, so the working model receives a more precise instruction.
+
+#### Questions Frequency
+
+The sensitivity of the Questions feature is controlled by the **Questions frequency** setting:
+
+| Level | Description |
+|-------|-------------|
+| **Essential only** | Only ask when the input is clearly ambiguous (long, multi-sentence, or contains unclear pronouns/references) |
+| **Normal** | Moderate sensitivity; ask for most inputs that could benefit from clarification |
+| **Many questions** | High sensitivity; ask frequently, even for shorter or mildly ambiguous inputs |
+
 ## Model Selection
 
-`/critique config` opens the critic model picker. It only offers models with configured auth, **only** from pi's native model registry — the same list you see in `/model`. The picker shows at most ten entries at a time and scrolls past that.
+`/critique config` opens an editable settings menu. Select a setting to change only that value, toggle booleans directly, or choose `Done`/`Esc` to close. Changes are persisted as soon as each setting is edited.
+
+The `Critique model` entry opens the critic model picker. It only offers models with configured auth, **only** from pi's native model registry — the same list you see in `/model`. The picker shows at most ten entries at a time and scrolls past that.
 
 **Auto** (the default) prefers a *different* model than the working one, so the review is genuinely independent. If no second model is available, it falls back to the working model and warns you when it runs.
 
@@ -168,14 +212,24 @@ The config is persisted as JSON at `~/.pi/agent/critique.json`:
 ```json
 {
   "model": "anthropic/claude-sonnet-4",
-  "autoInject": true
+  "autoInject": true,
+  "autoPromptCritique": false,
+  "autoPromptCritiqueLevel": "inconsistencies",
+  "autoPromptCritiqueModel": "working",
+  "questions": false,
+  "questionsFrequency": "normal"
 }
 ```
 
 - **`model`** — canonical `provider/modelId` of the reviewer. Empty string = Auto (different from working model).
 - **`autoInject`** — when `true`, the review is injected back into the working model. When `false`, the review is only displayed.
+- **`autoPromptCritique`** — when `true`, sufficiently rich user instructions are challenged before the model starts.
+- **`autoPromptCritiqueLevel`** — `inconsistencies` only flags real misunderstanding risks; `critical` is moderate; `corrosive` is highly sensitive and skips only clearly logical/complete prompts.
+- **`autoPromptCritiqueModel`** — `working` uses the active model; `critique` uses the configured critique model.
+- **`questions`** — when `true`, ambiguous user input triggers a clarifying Questions widget before the model receives the prompt.
+- **`questionsFrequency`** — sensitivity of the Questions feature: `essential` (minimal), `normal` (moderate), or `verbose` (high; "many questions").
 
-The picker offers any model with configured auth that's available in pi's registry; the config persists per-machine (in `getAgentDir()`), shared across all projects.
+The settings menu offers any model with configured auth that's available in pi's registry; the config persists per-machine (in `getAgentDir()`), shared across all projects.
 
 ## Architecture
 
@@ -185,28 +239,30 @@ critique/
 ├── LICENSE             # MIT
 ├── README.md
 ├── docs/
-│   ├── banner.png      # wide README header
-│   └── preview.png     # npm pi.dev preview card
-├── screenshot.png      # full-res master
+│   ├── banner.jpeg      # wide README header
+│   └── preview.jpeg     # npm pi.dev preview card
 └── src/
-    ├── index.ts        # /critique command surface, model picker, review UI (≈300 lines)
-    ├── config.ts       # persistence + model resolution: pinned, auto, fallback (≈90 lines)
-    ├── work-step.ts    # episode splitting + token-budgeted serialization (≈200 lines)
-    └── review.ts       # reviewer prompt + model call + injected-message builder (≈115 lines)
+    ├── index.ts             # /critique command surface, config UI, review UI, input hook
+    ├── config.ts            # persistence + model resolution: pinned, auto, fallback
+    ├── prompt-critique.ts   # automatic user-prompt critique prompt, gate, model call, questions detection
+    ├── work-step.ts         # episode splitting + token-budgeted serialization
+    └── review.ts            # reviewer prompt + model call + injected-message builder
 ```
 
-Four-file extension with zero external dependencies (only pi's bundled `@earendil-works/*` + Node built-ins):
+Five-file extension with zero external dependencies (only pi's bundled `@earendil-works/*` + Node built-ins):
 
 - **Episode splitter** — splits a session branch into user-message-bounded episodes, picks the last one with work
 - **Token budgeter** — hard caps per field, marks truncations so the reviewer knows what it didn't see
-- **Reviewer call** — tool-free `ctx.modelRegistry.complete()` with a structured system prompt
+- **Reviewer call** — tool-free `ctx.modelRegistry.complete()` with a domain-general structured prompt
 - **Advisory formatter** — wraps the review in a "non-mandatory" envelope before injecting as a follow-up user message
-- **UI** — paginated TUI model picker (SelectList) + Markdown review viewer + cancelable BorderedLoader
+- **Prompt critique** — optional input hook with local trivial-prompt gate, three challenge levels, and ultra-short JSON model output
+- **Questions detection** — heuristic ambiguity detector with three frequency levels; shows a clarifying widget with A/B/C options
+- **UI** — lazy-loaded pickers/viewers/loaders + 30-second Critique widget + 30-second Questions widget
 
 ## Notes
 
-- The critique model runs with **no tools** and never touches the filesystem. It judges purely from the serialized work step (which includes the diffs and outputs of `edit`/`write`/`bash` calls).
-- In TUI mode the review runs behind a cancelable loader (Esc aborts) and `/critique view` opens a scrollable Markdown viewer. In RPC mode reviews are surfaced through notifications; print mode logs them to stdout.
+- The critique model runs with **no tools** and never touches the filesystem. It judges purely from the serialized work step, whatever the domain.
+- In TUI mode the review runs behind a cancelable loader (Esc aborts) and `/critique view` opens a scrollable Markdown viewer. Automatic prompt critique appears as a compact `Critique` widget with a 30-second auto-discard timeout. The Questions feature appears as a `Questions` widget with the same timeout. In RPC mode reviews are surfaced through notifications/dialogs; print mode logs manual reviews to stdout and skips automatic prompt critique and questions.
 - Provider errors (bad keys, insufficient balance, rate limit) are surfaced as errors instead of silently producing empty reviews.
 - Reviews are capped at 16,000 chars to keep the injected follow-up reasonable; longer reviews are truncated with `… [review truncated]`.
 

package/package.json

@@ -1,15 +1,13 @@
 {
   "name": "pi-critique-model",
-  "version": "1.0.0",
-  "description": "A pi extension that reviews the last work step with a separate model and feeds the review back to the working model as advisory feedback. Independent reviewer with tool-free analysis, structured Markdown verdict (APPROVED / APPROVED_WITH_SUGGESTIONS / CHANGES_RECOMMENDED) and a token-budgeted context.",
+  "version": "1.1.0",
+  "description": "pi critique: work review + prompt challenge.",
   "keywords": [
     "pi-package",
     "review",
     "critique",
-    "feedback",
-    "code-review",
-    "second-opinion",
-    "advisory"
+    "adversarial-review",
+    "prompt-critique"
   ],
   "author": "Javier Noguerol <https://github.com/noguerol>",
   "license": "MIT",

package/src/config.ts

@@ -11,16 +11,38 @@ import type { Model } from "@earendil-works/pi-ai";
 import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
 import { dirname, join } from "node:path";
 
+import type {
+  AutoPromptCritiqueLevel,
+  AutoPromptCritiqueModelSource,
+} from "./prompt-critique.ts";
+
+export type QuestionsFrequency = "essential" | "normal" | "verbose";
+
 export interface CritiqueConfig {
   /** Canonical "provider/modelId" of the critique model. Empty string = auto. */
   model: string;
   /** Inject the review back into the working model automatically. */
   autoInject: boolean;
+  /** Challenge sufficiently rich user instructions before the agent starts. */
+  autoPromptCritique: boolean;
+  /** How adversarial the automatic prompt critique should be. */
+  autoPromptCritiqueLevel: AutoPromptCritiqueLevel;
+  /** Which model is used for automatic prompt critique. */
+  autoPromptCritiqueModel: AutoPromptCritiqueModelSource;
+  /** Ask clarifying questions when user input is ambiguous. */
+  questions: boolean;
+  /** Sensitivity level for the questions feature. */
+  questionsFrequency: QuestionsFrequency;
 }
 
 export const DEFAULT_CONFIG: CritiqueConfig = {
   model: "",
   autoInject: true,
+  autoPromptCritique: false,
+  autoPromptCritiqueLevel: "inconsistencies",
+  autoPromptCritiqueModel: "working",
+  questions: false,
+  questionsFrequency: "normal",
 };
 
 export function configFilePath(): string {
@@ -30,10 +52,31 @@ export function configFilePath(): string {
 export function loadConfig(): CritiqueConfig {
   try {
     const raw = JSON.parse(readFileSync(configFilePath(), "utf8")) as Partial<CritiqueConfig>;
+    const level = raw.autoPromptCritiqueLevel;
+    const modelSource = raw.autoPromptCritiqueModel;
+    const frequency = raw.questionsFrequency;
     return {
       model: typeof raw.model === "string" ? raw.model : DEFAULT_CONFIG.model,
       autoInject:
         typeof raw.autoInject === "boolean" ? raw.autoInject : DEFAULT_CONFIG.autoInject,
+      autoPromptCritique:
+        typeof raw.autoPromptCritique === "boolean"
+          ? raw.autoPromptCritique
+          : DEFAULT_CONFIG.autoPromptCritique,
+      autoPromptCritiqueLevel:
+        level === "inconsistencies" || level === "critical" || level === "corrosive"
+          ? level
+          : DEFAULT_CONFIG.autoPromptCritiqueLevel,
+      autoPromptCritiqueModel:
+        modelSource === "working" || modelSource === "critique"
+          ? modelSource
+          : DEFAULT_CONFIG.autoPromptCritiqueModel,
+      questions:
+        typeof raw.questions === "boolean" ? raw.questions : DEFAULT_CONFIG.questions,
+      questionsFrequency:
+        frequency === "essential" || frequency === "normal" || frequency === "verbose"
+          ? frequency
+          : DEFAULT_CONFIG.questionsFrequency,
     };
   } catch {
     return { ...DEFAULT_CONFIG };
@@ -46,7 +89,7 @@ export function saveConfig(config: CritiqueConfig): void {
   writeFileSync(path, JSON.stringify(config, null, 2) + "\n", "utf8");
 }
 
-export function modelLabel(model: Model): string {
+export function modelLabel(model: Model<any>): string {
   return `${model.provider}/${model.id}`;
 }
 
@@ -55,7 +98,7 @@ export function modelLabel(model: Model): string {
  * scoping is configured, otherwise the full available catalogue. Only models
  * with configured auth are offered.
  */
-export function pickableModels(ctx: ExtensionContext): Model[] {
+export function pickableModels(ctx: ExtensionContext): Model<any>[] {
   const scoped = (ctx.scopedModels ?? []).map((entry) => entry.model);
   const candidates = scoped.length > 0 ? scoped : ctx.modelRegistry.getAvailable();
   return candidates.filter((model) => ctx.modelRegistry.hasConfiguredAuth(model));
@@ -71,13 +114,13 @@ export function pickableModels(ctx: ExtensionContext): Model[] {
 export function resolveCritiqueModel(
   ctx: ExtensionContext,
   config: CritiqueConfig,
-): Model | undefined {
+): Model<any> | undefined {
   if (config.model) {
     const slash = config.model.indexOf("/");
     const provider = slash >= 0 ? config.model.slice(0, slash) : config.model;
     const id = slash >= 0 ? config.model.slice(slash + 1) : config.model;
     const model = ctx.modelRegistry.find(provider, id);
-    return model && ctx.modelRegistry.hasConfiguredAuth(model) ? model : undefined;
+    if (model && ctx.modelRegistry.hasConfiguredAuth(model)) return model;
   }
 
   const candidates = pickableModels(ctx);
@@ -87,3 +130,14 @@ export function resolveCritiqueModel(
   );
   return different ?? working ?? candidates[0];
 }
+
+/** Resolve the model used by automatic prompt critique. */
+export function resolveAutoPromptCritiqueModel(
+  ctx: ExtensionContext,
+  config: CritiqueConfig,
+): Model<any> | undefined {
+  if (config.autoPromptCritiqueModel === "working") {
+    return ctx.model ?? pickableModels(ctx)[0];
+  }
+  return resolveCritiqueModel(ctx, config) ?? ctx.model ?? pickableModels(ctx)[0];
+}