@openlines/opl-pi-sht 0.1.14 → 0.1.16

package/README.md

@@ -8,21 +8,41 @@ A portable collection of various Pi coding agent extensions. Repository director
 
 ### Pi package
 
-Install a versioned GitHub release with Pi:
+Install a versioned release from npm or GitHub. Both ship identical content; pick one source per machine, because Pi treats the npm and Git entries as separate packages and installing both loads every extension twice.
 
 ```bash
-pi install git:github.com/linellazatin/opl-pi-sht@v0.1.14
+pi install npm:@openlines/opl-pi-sht@0.1.15
+pi install git:github.com/linellazatin/opl-pi-sht@v0.1.15
 ```
 
-Pi installs the package under `~/.pi/agent/git/github.com/linellazatin/opl-pi-sht` and runs root `npm install`, so `opl-webaccess` and `opl-browser` runtime dependencies are available. Pi packages do not install optional extension config files; copy only the configs you need from that checkout to `~/.pi/agent/configs/`.
+> Omitting the version on the npm source tracks the latest published release; Git refs stay pinned, so move them with `pi install ...@v<new>`.
+
+Pi installs the package under `~/.pi/agent/npm/node_modules/@openlines/opl-pi-sht` (npm) or `~/.pi/agent/git/github.com/linellazatin/opl-pi-sht` (Git) and runs root `npm install`, so `opl-webaccess` and `opl-browser` runtime dependencies are available. Pi packages do not install optional extension config files; copy only the configs you need from that checkout's `configs/` to `~/.pi/agent/configs/`.
 
 `opl-browser` also needs Chromium once after package installation:
 
 ```bash
-cd ~/.pi/agent/git/github.com/linellazatin/opl-pi-sht
+cd ~/.pi/agent/npm/node_modules/@openlines/opl-pi-sht   # or the git checkout path
 npx playwright install chromium
 ```
 
+#### Selecting extensions
+
+`pi install` has no per-extension flag like `./install.sh --only`. After installing the package, narrow it in `~/.pi/agent/settings.json` using the object form (filters are globs relative to the package root and layer on top of the package manifest):
+
+```json
+{
+  "packages": [
+    {
+      "source": "npm:@openlines/opl-pi-sht@0.1.15",
+      "extensions": ["extensions/opl-init/index.ts", "extensions/opl-todo/index.ts"]
+    }
+  ]
+}
+```
+
+`pi config` provides the same control interactively: toggle individual extensions from an installed package without editing globs.
+
 ### Checkout installer
 
 ```bash
@@ -152,7 +172,7 @@ The fixed cost of a full install is small and paid once per session, then cached
 
 ## Configuration
 
-Copy applicable files from [`configs/`](configs/) to `~/.pi/agent/configs/`. For a Pi Git package installation, the source directory is `~/.pi/agent/git/github.com/linellazatin/opl-pi-sht/configs/`:
+Copy applicable files from [`configs/`](configs/) to `~/.pi/agent/configs/`. For a Pi package installation, the source directory is `~/.pi/agent/npm/node_modules/@openlines/opl-pi-sht/configs/` (npm) or `~/.pi/agent/git/github.com/linellazatin/opl-pi-sht/configs/` (Git):
 
 - `opl-footer.json`, `opl-input.json`, `opl-modes.json`, `opl-todo.json`, `opl-webaccess.json`
 - `opl-browser` has optional configuration (`opl-browser.json`); all fields default, so it works without any config file.
@@ -167,7 +187,7 @@ See each extension README for commands, behavior, configuration fields, runtime
 
 ## Runtime requirements
 
-All extensions use Pi's normal extension discovery. Pi installs `opl-webaccess` extraction dependencies and Playwright automatically when installed as a Git package. For the checkout installer, install nested runtime dependencies before using those extensions:
+All extensions use Pi's normal extension discovery. Pi installs `opl-webaccess` extraction dependencies and Playwright automatically when installed as an npm or Git package. For the checkout installer, install nested runtime dependencies before using those extensions:
 
 ```bash
 cd extensions/opl-webaccess
@@ -178,9 +198,9 @@ npm install
 npx playwright install chromium
 ```
 
-A Pi Git package still needs the one-time `npx playwright install chromium` command shown above.
+A Pi package (npm or Git) still needs the one-time `npx playwright install chromium` command shown above.
 
-`opl-simplebench` writes a full JSON benchmark artifact in Pi's current working directory by default. `--test-all` additionally writes `research.md` and `page.html` beside `result.json` in a result bundle. Copy `configs/opl-simplebench.json.sample` to `~/.pi/agent/configs/opl-simplebench.json` to configure DDGS/SearXNG research and optional llama-server/llamagputop metadata endpoints. Use `/simplebench --no-artifact` or `simplebench({ no_artifact: true })` when responses must not be written to disk. Provider credentials remain outside tracked configuration; configure them through Pi provider settings, environment variables, or Pi authentication.
+`opl-simplebench` writes a full JSON benchmark artifact in Pi's current working directory by default. `--test-all` additionally writes `research.md` and `page.html` beside `result.json` in a result bundle. `/simplebench --sequence` runs a templated multi-run protocol (for example a model warm-up curve) defined in the `runSequence` block, with per-iteration tags, optional llama-server/llamagputop metrics, and a configurable pause between iterations. Copy `configs/opl-simplebench.json.sample` to `~/.pi/agent/configs/opl-simplebench.json` to configure DDGS/SearXNG research, optional llama-server/llamagputop metadata endpoints, and the run sequence. Use `/simplebench --no-artifact` or `simplebench({ no_artifact: true })` when responses must not be written to disk. Provider credentials remain outside tracked configuration; configure them through Pi provider settings, environment variables, or Pi authentication.
 
 ## Tests
 

package/configs/opl-simplebench.json.sample

@@ -1,8 +1,19 @@
 {
-  "_comment": "Copy to ~/.pi/agent/configs/opl-simplebench.json. Endpoints are optional and only used by their corresponding Simplebench flags.",
+  "_comment": "Copy to ~/.pi/agent/configs/opl-simplebench.json. Endpoints are optional and only used by their corresponding Simplebench flags. runSequence powers /simplebench --sequence: each entry is a flag string; llamaMetrics appends --llama-server --llamagputop to every entry; pauseMs sleeps between iterations.",
   "researchSearchProvider": "ddgs",
   "researchSearchUrl": "http://127.0.0.1:8000",
   "researchMaxResults": 5,
   "llamaServerUrl": "http://127.0.0.1:8080/v1",
-  "llamagputopUrl": "http://127.0.0.1:4321/stats"
+  "llamagputopUrl": "http://127.0.0.1:4321/stats",
+  "runSequence": {
+    "enabled": false,
+    "llamaMetrics": true,
+    "pauseMs": 5000,
+    "sequence": [
+      "--coding-lite --tag=coldest",
+      "--3ptest --tag=colder",
+      "--coding-lite --tag=semiwarm",
+      "--test-all --research-live --tag=warm"
+    ]
+  }
 }

package/extensions/opl-footer/README.md

@@ -19,7 +19,7 @@ Row 1 right: <context bar> <pct%> / <max tokens>
 Row 2 left:  Thinking: <LEVEL> | <active mode>
 Row 2 right: T: <total> (<cached> cached) ↑ <in> ↓ <out> | $<cost>
 
-Row 3 left:  <turns> turns · <steps> steps · <model requests> mreq · <tool calls> mtool
+Row 3 left:  <prompts> prompts · <api calls> api calls · <tool calls> tool calls
 Row 3 right: LLM <time> (<last-turn TAT>) · Tool <time> | TTFT <time> · <tokens>/s | Cache <percent>%
 ```
 
@@ -31,7 +31,7 @@ The third row is populated after the first completed turn. Its session and perfo
 - **Context bar**: configurable gradient bar with percentage and context-window size
 - **Git integration**: branch plus staged, unstaged, and untracked counts, with invalidation after relevant file and Git commands
 - **Token and cost tracking**: total, cache, input/output, and accumulated cost segments
-- **Session statistics**: turns, tool steps, model requests, and model tool calls
+- **Session statistics**: prompt, API-call, and model tool-call counts
 - **Performance statistics**: cumulative LLM/tool duration, most recent user-prompt-to-completion turnaround time, average time to first token, output rate, and cache-hit percentage
 - **Thinking and mode indicators**: thinking-level colors plus caveman, plan, chat, or unified mode segments when available
 - **Nerd Font support**: automatic detection with plain-icon fallbacks
@@ -152,7 +152,7 @@ The `git` segment shows:
 
 ## Session and Performance Statistics
 
-The `session_stats` segment shows turns, tool steps, and (when available) model requests and model tool calls for the current branch; turns and steps are reconstructed from session history so they survive quit/resume.
+The `session_stats` segment shows prompt, API-call, and model tool-call counts for the current branch, reconstructed from session history so they survive quit/resume. `prompts` counts user messages and `api calls` counts completed assistant responses, so one prompt typically drives many API calls (each tool round trip is one call); `tool calls` counts the tool-call blocks the model emitted.
 
 The `perf_stats` segment shows cumulative session LLM and tool time, average time to first token, output tokens/sec, and cache-hit percentage. The `LLM` figure is followed by the most recent user-prompt-to-completion turnaround time in parentheses, e.g. `LLM 18m 22s (2m 13s)`. This turnaround reflects only fully-settled turns (the `agent_settled` signal, after any retries or compaction), so it stays blank until the first turn completes. All `perf_stats` timing values are ephemeral — they reset each session and are not reconstructed from branch history.
 

package/extensions/opl-footer/index.ts

@@ -103,11 +103,7 @@ export default function footer(pi: ExtensionAPI) {
   let cachedUsageStats: UsageStats | null = null;
   let tuiRef: TUI | null = null;
 
-  // Session stats accumulators
-  let turns = 0;
-  let steps = 0;
-  let modelRequests = 0;
-  let modelToolCalls = 0;
+  // Session stats accumulators (timing only; counts are reconstructed from the branch)
   let llmMs = 0;
   let toolMs = 0;
   let ttftSamples: number[] = [];
@@ -124,10 +120,6 @@ export default function footer(pi: ExtensionAPI) {
     currentCtx = ctx;
     lastBranchLength = 0;
     cachedUsageStats = null;
-    turns = 0;
-    steps = 0;
-    modelRequests = 0;
-    modelToolCalls = 0;
     llmMs = 0;
     toolMs = 0;
     ttftSamples = [];
@@ -165,13 +157,11 @@ export default function footer(pi: ExtensionAPI) {
   });
 
   pi.on("turn_end", async (_event: unknown, _ctx: ExtensionContext) => {
-    turns++;
     const elapsed = Date.now() - turnStartMs;
     llmMs += Math.max(0, elapsed - toolMsThisTurn);
   });
 
   pi.on("tool_execution_start", async (event: { toolCallId: string }, _ctx: ExtensionContext) => {
-    steps++;
     toolStartTimes.set(event.toolCallId, Date.now());
   });
 
@@ -278,18 +268,15 @@ export default function footer(pi: ExtensionAPI) {
     // This allows litellm proxy to correctly separate local vs cloud models
     const isLocalModel = ctx.model?.id?.startsWith?.("local.") ?? false;
 
-    // Reconstruct turns/steps from branch so they survive quit/resume.
+    // Reconstruct prompt/API-call/tool-call counts from the branch so they survive quit/resume.
     // Timing stats (llmMs, toolMs, ttftSamples) are ephemeral — not stored in messages.
-    const branchTurns = completedMessages.length;
-    const branchSteps = branch.filter(
-      (e) => e.type === "message" && (e as AssistantMessageEvent).message?.role === "toolResult",
+    const branchPrompts = branch.filter(
+      (e) => e.type === "message" && (e as AssistantMessageEvent).message?.role === "user",
     ).length;
-
-    // Count model requests (each assistant message = 1 request)
-    const branchModelRequests = completedMessages.length;
+    const branchApiCalls = completedMessages.length;
 
     // Count tool calls from assistant message content
-    const branchModelToolCalls = completedMessages.reduce((sum, msg) => {
+    const branchToolCalls = completedMessages.reduce((sum, msg) => {
       const content = (msg as any).content || [];
       return sum + content.filter((block: any) => block?.type === "toolCall").length;
     }, 0);
@@ -310,7 +297,7 @@ export default function footer(pi: ExtensionAPI) {
       theme,
       colors,
       icons: getIcons(effectiveConfig.icons),
-      sessionStats: { turns: branchTurns, steps: branchSteps, modelRequests: branchModelRequests, modelToolCalls: branchModelToolCalls, llmMs, toolMs, ttftSamples, lastTurnaroundMs },
+      sessionStats: { prompts: branchPrompts, apiCalls: branchApiCalls, toolCalls: branchToolCalls, llmMs, toolMs, ttftSamples, lastTurnaroundMs },
     };
   }
 

package/extensions/opl-footer/segments/session-stats.ts

@@ -19,22 +19,15 @@ export function formatMs(ms: number): string {
 export const sessionStatsSegment = {
   id: "session_stats" as const,
   render(ctx: SegmentContext): RenderedSegment {
-    const { turns, steps, modelRequests, modelToolCalls } = ctx.sessionStats;
-    if (turns === 0) return { content: "", visible: false };
+    const { prompts, apiCalls, toolCalls } = ctx.sessionStats;
+    if (prompts === 0) return { content: "", visible: false };
 
-    const parts: string[] = [
-      val(ctx, `${turns}`) + dim(ctx, " turns"),
-      val(ctx, `${steps}`) + dim(ctx, " steps"),
+    const parts = [
+      val(ctx, `${prompts}`) + dim(ctx, " prompts"),
+      val(ctx, `${apiCalls}`) + dim(ctx, " api calls"),
+      val(ctx, `${toolCalls}`) + dim(ctx, " tool calls"),
     ];
 
-    // Add model stats if available
-    if (modelRequests > 0) {
-      parts.push(val(ctx, `${modelRequests}`) + dim(ctx, " mreq"));
-    }
-    if (modelToolCalls > 0) {
-      parts.push(val(ctx, `${modelToolCalls}`) + dim(ctx, " mtool"));
-    }
-
     return { content: parts.join(dim(ctx, " · ")), visible: true };
   },
 };
@@ -42,8 +35,8 @@ export const sessionStatsSegment = {
 export const perfStatsSegment = {
   id: "perf_stats" as const,
   render(ctx: SegmentContext): RenderedSegment {
-    const { turns, llmMs, toolMs, ttftSamples, lastTurnaroundMs } = ctx.sessionStats;
-    if (turns === 0) return { content: "", visible: false };
+    const { prompts, llmMs, toolMs, ttftSamples, lastTurnaroundMs } = ctx.sessionStats;
+    if (prompts === 0) return { content: "", visible: false };
 
     const { input, output, cacheRead } = ctx.usageStats;
 

package/extensions/opl-footer/types.ts

@@ -97,12 +97,11 @@ export interface UsageStats {
   cost: number;
 }
 
-// Session performance stats accumulated via events
+// Session performance stats accumulated via events; counts reconstructed from the branch
 export interface SessionStats {
-  turns: number;
-  steps: number;
-  modelRequests: number;
-  modelToolCalls: number;
+  prompts: number;
+  apiCalls: number;
+  toolCalls: number;
   llmMs: number;
   toolMs: number;
   ttftSamples: number[];

package/extensions/opl-simplebench/README.md

@@ -8,12 +8,14 @@ Auditable Pi model benchmark for closed-answer contracts, JSON instruction follo
 /simplebench
 /simplebench <model>
 /simplebench <model> [--no-artifact] [--thinking-max]
+/simplebench <model> --3ptest
 /simplebench <model> --coding-lite
 /simplebench <model> --test-all
 /simplebench <model> --research-live
 /simplebench <model> --llama-server
 /simplebench <model> --llamagputop
 /simplebench <model> --tag=<word>
+/simplebench <model> --sequence
 /simplebench --all [--no-artifact]
 /simplebench --all --test-all
 /simplebench --help
@@ -39,6 +41,8 @@ simplebench({ model: "global.openai.gpt-5.6-terra", test_all: true })
 - Twenty fixed closed-answer contracts, one strict JSON instruction test, and one chained tool-call test.
 - Six opt-in execution-backed coding tasks in disposable directories (`--coding-lite`).
 - `--test-all` runs baseline, coding-lite, and deterministic grounded research; `--research-live` adds the configured live research-artifact integration smoke test. `--all` still selects every Ollama model.
+- `--3ptest` is the explicit form of the default baseline suite (reasoning, instruction following, tool calls). It changes nothing about the run; artifacts already name the baseline suite `3ptest`.
+- `--sequence` runs the templated multi-iteration benchmark stored in `opl-simplebench.json` (`runSequence`): sequential iterations, per-entry flags and `--tag`, optional `llamaMetrics` and `pauseMs` cooldown. See "Run a templated sequence" below.
 - `--llama-server` (`llama_server: true`) captures configured direct llama-server `/props` and `/metrics` without changing inference routing, so it works with a LiteLLM proxy.
 - `--llamagputop` (`llamagputop: true`) captures configured llama.cpp `/stats` metadata independently of the inference provider; its served model ID is authoritative.
 - `--tag=<word>` (`tag: "<word>"`) labels a run: stored as `benchmark.tag` and prefixed onto the artifact file or bundle name, for example `simplebench-coldrun-coding-lite-<model>-...`. The tag must be a single word (letters, digits, dot, dash, underscore).
@@ -80,6 +84,37 @@ Use this in sensitive environments. The terminal summary still reports category
 
 `--all` is intentionally limited to Ollama discovery. It writes one artifact per model unless `--no-artifact` is set.
 
+### Run a templated sequence
+
+```text
+/simplebench --sequence
+```
+
+Repeating a warm-up protocol (first-run-after-boot coding check, then baseline, then the full suite) means re-typing the same flags every session. `--sequence` reads the `runSequence` block from `~/.pi/agent/configs/opl-simplebench.json` and runs each entry as its own iteration against the current model (or the model passed with `--sequence`):
+
+```json
+{
+  "runSequence": {
+    "enabled": true,
+    "llamaMetrics": true,
+    "pauseMs": 5000,
+    "sequence": [
+      "--coding-lite --tag=coldest",
+      "--3ptest --tag=colder",
+      "--coding-lite --tag=semiwarm",
+      "--test-all --research-live --tag=warm"
+    ]
+  }
+}
+```
+
+- `enabled` gates the flag: `/simplebench --sequence` errors when it is missing or `false`. A bare `/simplebench` is unaffected.
+- Each `sequence` entry is the same flag string you would type after `/simplebench`, including its own `--tag`.
+- `llamaMetrics: true` appends `--llama-server --llamagputop` to every entry, so local server stats are captured per iteration without repeating the flags.
+- `pauseMs` sleeps between iterations (never after the last one), including after a failed iteration, so thermal/KV-cache cooldown stays consistent across the warm-up curve.
+- Iterations run sequentially and each writes its normal artifact/bundle; a failed iteration is reported and the sequence continues. A final summary line counts completed iterations.
+- Entries containing `--all` or `--sequence` are rejected before anything runs, as are invalid configurations (empty sequence, negative `pauseMs`) and invalid per-entry tags. The outer `--tag` is ignored; each entry carries its own.
+
 ### Grounded research in `--test-all`
 
 `--test-all` runs a deterministic agent workflow after coding-lite. `web_search` returns three benchmark-local source cards. The model must write every required finding as a Markdown bullet with its inline source ID (`[S1]`, `[S2]`) and list each cited source under `## Sources` as a Markdown link, for example `- [S1](https://research.fixture/S1)`. The verifier checks each exact claim-to-source relationship, source-list URL, tool trace, and the responsive minimalist `page.html` contract. It invokes no LLM judge, external search service, embedding model, or entailment model.
@@ -94,7 +129,7 @@ When coding-lite is enabled, each coding task is recorded as a `kind: "coding"`
 
 ### Configured metadata (`--llama-server`, `--llamagputop`)
 
-Copy `configs/opl-simplebench.json.sample` to `~/.pi/agent/configs/opl-simplebench.json`. It uses camelCase fields: `llamaServerUrl`, `llamagputopUrl`, `researchSearchProvider`, `researchSearchUrl`, and `researchMaxResults`. `--llama-server` captures `GET /props` and `/metrics` from configured `llamaServerUrl`; inference remains on the configured provider route, including LiteLLM. `--llamagputop` captures configured `llamagputopUrl` stats.
+Copy `configs/opl-simplebench.json.sample` to `~/.pi/agent/configs/opl-simplebench.json`. It uses camelCase fields: `llamaServerUrl`, `llamagputopUrl`, `researchSearchProvider`, `researchSearchUrl`, `researchMaxResults`, and `runSequence` (see "Run a templated sequence"). `--llama-server` captures `GET /props` and `/metrics` from configured `llamaServerUrl`; inference remains on the configured provider route, including LiteLLM. `--llamagputop` captures configured `llamagputopUrl` stats.
 
 `serverStats` is omitted entirely for runs without the flag. Its shape:
 

package/extensions/opl-simplebench/index.ts

@@ -1,7 +1,7 @@
 import type { ExtensionAPI, AgentToolResult } from "@earendil-works/pi-coding-agent";
 import { debugLog } from "./util/debug";
 import { detectProvider } from "./util/providers";
-import { TOOL_SUPPORT_CACHE_PATH } from "./util/config";
+import { readTestConfig, TOOL_SUPPORT_CACHE_PATH, type ModelTestUserConfig } from "./util/config";
 import type { SimplebenchOptions } from "./types";
 import { createBenchmark } from "./benchmark";
 
@@ -13,7 +13,25 @@ export function parseCommandArgs(args: string): SimplebenchOptions {
     tag = tagToken.slice("--tag=".length);
     if (!/^[A-Za-z0-9][A-Za-z0-9._-]*$/.test(tag)) throw new Error(`--tag must be a single word (letters, digits, dot, dash, underscore): got "${tag}"`);
   }
-  return { model: tokens.find(token => !token.startsWith("--")), allModels: tokens.includes("--all"), writeArtifact: !tokens.includes("--no-artifact"), thinkingMax: tokens.includes("--thinking-max"), codingLite: tokens.includes("--coding-lite"), testAll: tokens.includes("--test-all"), researchLive: tokens.includes("--research-live"), llamaServer: tokens.includes("--llama-server"), llamagputop: tokens.includes("--llamagputop"), ...(tag ? { tag } : {}) };
+  return { model: tokens.find(token => !token.startsWith("--")), allModels: tokens.includes("--all"), writeArtifact: !tokens.includes("--no-artifact"), thinkingMax: tokens.includes("--thinking-max"), codingLite: tokens.includes("--coding-lite"), testAll: tokens.includes("--test-all"), researchLive: tokens.includes("--research-live"), llamaServer: tokens.includes("--llama-server"), llamagputop: tokens.includes("--llamagputop"), ...(tag ? { tag } : {}), ...(tokens.includes("--3ptest") ? { threePTest: true } : {}), ...(tokens.includes("--sequence") ? { sequence: true } : {}) };
+}
+
+export interface ResolvedRunSequence { runs: Array<{ entry: string; options: SimplebenchOptions }>; pauseMs: number }
+
+/** Expand the configured runSequence into per-iteration benchmark options. Throws on disabled/invalid configuration. */
+export function resolveRunSequence(userConfig: ModelTestUserConfig): ResolvedRunSequence {
+  const rs = userConfig.runSequence;
+  if (!rs?.enabled) throw new Error("runSequence is not enabled in opl-simplebench.json");
+  if (!Array.isArray(rs.sequence) || rs.sequence.length === 0) throw new Error("runSequence.sequence must be a non-empty array of flag strings");
+  const pauseMs = rs.pauseMs ?? 0;
+  if (typeof pauseMs !== "number" || !Number.isFinite(pauseMs) || pauseMs < 0) throw new Error(`runSequence.pauseMs must be a non-negative number: got ${JSON.stringify(rs.pauseMs)}`);
+  const runs = rs.sequence.map((entry: string) => {
+    const tokens = String(entry).trim().split(/\s+/);
+    if (tokens.includes("--all")) throw new Error(`runSequence entry cannot contain --all: "${entry}"`);
+    if (tokens.includes("--sequence")) throw new Error(`runSequence entry cannot contain --sequence: "${entry}"`);
+    return { entry: String(entry), options: parseCommandArgs(`${entry}${rs.llamaMetrics ? " --llama-server --llamagputop" : ""}`) };
+  });
+  return { runs, pauseMs };
 }
 
 export default function (pi: ExtensionAPI) {
@@ -22,7 +40,7 @@ export default function (pi: ExtensionAPI) {
 
 pi.registerCommand("simplebench", {
   description: "Benchmark a model with auditable closed-answer, instruction, and tool-use tests.",
-  detailedHelp: "\n\n🔍 Simplebench Extension\n\nThis extension tests AI models across multiple dimensions:\n• Closed-answer contract: 20 deterministic final-line answers (logic, math, spatial, commonsense)\n• Tool Usage: Ability to use available tools effectively\n• Instruction Following: How well the model follows complex JSON instructions\n• Coding Lite: Six isolated, execution-backed coding tasks\n\n📋 Usage Examples:\n  /simplebench                    # Test current model\n  /simplebench <model>           # Test a specific model\n  /simplebench --all             # Test all Ollama models\n  /simplebench <model> --coding-lite # Run only coding tasks\n  /simplebench <model> --test-all # Run baseline, coding, and grounded research\n  /simplebench <model> --research-live # Add live search integration smoke test\n  /simplebench --all --test-all  # Run complete suite for every Ollama model\n  /simplebench <model> --thinking-max # Request max reasoning\n  /simplebench <model> --llama-server # Capture configured /props and /metrics\n  /simplebench <model> --llamagputop # Capture configured llama.cpp stats\n  /simplebench <model> --tag=coldrun # Label the run: benchmark.tag plus artifact filename prefix\n  /simplebench --help            # Show this help\n  /simplebench --clear-cache     # Clear tool support cache\n\nCoding tasks run in disposable directories and never access the user repository.\n",
+  detailedHelp: "\n\n🔍 Simplebench Extension\n\nThis extension tests AI models across multiple dimensions:\n• Closed-answer contract: 20 deterministic final-line answers (logic, math, spatial, commonsense)\n• Tool Usage: Ability to use available tools effectively\n• Instruction Following: How well the model follows complex JSON instructions\n• Coding Lite: Six isolated, execution-backed coding tasks\n\n📋 Usage Examples:\n  /simplebench                    # Test current model\n  /simplebench <model>           # Test a specific model\n  /simplebench --all             # Test all Ollama models\n  /simplebench <model> --3ptest  # Run the default 3ptest baseline explicitly\n  /simplebench <model> --coding-lite # Run only coding tasks\n  /simplebench <model> --test-all # Run baseline, coding, and grounded research\n  /simplebench <model> --research-live # Add live search integration smoke test\n  /simplebench --all --test-all  # Run complete suite for every Ollama model\n  /simplebench <model> --sequence # Run the configured runSequence template\n  /simplebench <model> --thinking-max # Request max reasoning\n  /simplebench <model> --llama-server # Capture configured /props and /metrics\n  /simplebench <model> --llamagputop # Capture configured llama.cpp stats\n  /simplebench <model> --tag=coldrun # Label the run: benchmark.tag plus artifact filename prefix\n  /simplebench --help            # Show this help\n  /simplebench --clear-cache     # Clear tool support cache\n\nCoding tasks run in disposable directories and never access the user repository.\n",
   getArgumentCompletions: async (prefix) => {
     try {
       const models = await getOllamaModels();
@@ -54,6 +72,8 @@ pi.registerCommand("simplebench", {
         "  /simplebench [model] --test-all - Run baseline, coding, and deterministic grounded research\n" +
         "  /simplebench [model] --research-live - Add live-search integration smoke test\n" +
         "  /simplebench [model] --tag=<word> - Label the run (single word; added to benchmark.tag and the artifact name)\n" +
+        "  /simplebench [model] --3ptest - Run the default baseline suite explicitly\n" +
+        "  /simplebench [model] --sequence - Run the templated multi-iteration sequence from opl-simplebench.json (runSequence; per-entry --tag, llamaMetrics and pauseMs supported; the outer --tag is ignored)\n" +
         "  /simplebench --all --test-all - Run complete suite for all Ollama models\n" +
         "  /simplebench --clear-cache - Clear tool support cache\n",
         "info"
@@ -76,6 +96,46 @@ pi.registerCommand("simplebench", {
       return;
     }
 
+    if (parsedArgs.sequence) {
+      let resolved: ResolvedRunSequence;
+      try {
+        resolved = resolveRunSequence(readTestConfig());
+      } catch (e: any) {
+        ctx.ui.notify(e?.message || String(e), "error");
+        return;
+      }
+      const sequenceModel = parsedArgs.model || ctx.model?.id;
+      if (!sequenceModel) {
+        ctx.ui.notify("No model specified and no model currently selected", "error");
+        return;
+      }
+      ctx.ui.notify(`Running ${resolved.runs.length}-iteration sequence (${resolved.runs.map(r => r.entry).join(" | ")})...`, "info");
+      let completed = 0;
+      for (let i = 0; i < resolved.runs.length; i++) {
+        if (i > 0 && resolved.pauseMs > 0) {
+          ctx.ui.notify(`Cooling down ${resolved.pauseMs / 1000}s before iteration ${i + 1}/${resolved.runs.length}...`, "info");
+          await new Promise(resolve => setTimeout(resolve, resolved.pauseMs));
+        }
+        const { entry, options } = resolved.runs[i];
+        const model = options.model || sequenceModel;
+        ctx.ui.notify(`Iteration ${i + 1}/${resolved.runs.length} (${entry}) on ${model}...`, "info");
+        try {
+          const report = await testModel(model, ctx, { ...options, model });
+          completed += 1;
+          pi.sendMessage({
+            customType: "simplebench-report",
+            content: report,
+            display: { type: "content", content: report },
+            details: { model, timestamp: new Date().toISOString() },
+          });
+        } catch (e: any) {
+          ctx.ui.notify(`Iteration ${i + 1} (${entry}) failed: ${e.message}`, "error");
+        }
+      }
+      ctx.ui.notify(`Sequence done: ${completed}/${resolved.runs.length} iterations completed`, "info");
+      return;
+    }
+
     if (parsedArgs.allModels) {
       const providerInfo = detectProvider(ctx);
       if (providerInfo.kind !== "ollama") {

package/extensions/opl-simplebench/types.ts

@@ -1,6 +1,6 @@
 import type { ServerStats } from "./llama-server";
 
-export interface SimplebenchOptions { model?: string; allModels: boolean; writeArtifact: boolean; thinkingMax?: boolean; codingLite?: boolean; testAll?: boolean; researchLive?: boolean; llamaServer?: boolean; llamagputop?: boolean; tag?: string; }
+export interface SimplebenchOptions { model?: string; allModels: boolean; writeArtifact: boolean; thinkingMax?: boolean; codingLite?: boolean; testAll?: boolean; researchLive?: boolean; llamaServer?: boolean; llamagputop?: boolean; tag?: string; threePTest?: boolean; sequence?: boolean; }
 export interface RequestMetrics { requestCount: number; retryCount: number; wallTimeMs: number; timeToAnswerMs: number; timeToFirstTokenMs: number | null; startedAt: string | null; finishedAt: string | null; inputTokens: number | null; outputTokens: number | null; totalTokens: number | null; outputTokensPerSecond: number | null; toolCalls: Array<{ name: string; arguments?: unknown }>; }
 export interface TestRecord { id: string; kind: "reasoning" | "instructions" | "tools" | "coding" | "research-grounded" | "research-live"; category?: string; prompt: string; response: string | null; expectedAnswer?: string | string[]; extractedAnswer?: string; score: string; passed: boolean; error: string | null; metrics: RequestMetrics; coding?: { publicPassed: boolean; hiddenPassed: boolean; verifiedAfterEdit: boolean; unrelatedFiles: string[]; toolCalls: number; turns: number; efficiency?: "STRONG" | "MODERATE" | "WEAK" | "FAIL"; }; research?: { toolCalls: string[]; files: string[]; }; }
 export interface RunArtifact { schemaVersion: 1; benchmark: { name: "opl-simplebench"; suite?: "baseline" | "coding-lite" | "test-all"; tag?: string; model: string; provider: string; providerKind: string; thinking: { requested: "default" | "max"; effective: "provider-default" | "openai-reasoning-effort" | "pi-bedrock-reasoning"; level: "max" | null; modelMetadataSource: "active-context" | "scoped-model" | null; }; startedAt: string; finishedAt: string; wallTimeMs: number; artifactEnabled: boolean; }; tests: TestRecord[]; summary: Record<string, unknown> & { serverStats?: ServerStats }; }

package/extensions/opl-simplebench/util/config.ts

@@ -79,6 +79,15 @@ export interface ModelTestUserConfig {
   researchMaxResults?: number;
   llamaServerUrl?: string;
   llamagputopUrl?: string;
+  runSequence?: RunSequenceConfig;
+}
+
+/** Templated multi-run benchmark sequence used by `/simplebench --sequence`. */
+export interface RunSequenceConfig {
+  enabled?: boolean;
+  llamaMetrics?: boolean;
+  pauseMs?: number;
+  sequence?: string[];
 }
 
 /**

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@openlines/opl-pi-sht",
-  "version": "0.1.14",
+  "version": "0.1.16",
   "description": "Pi coding-agent extension kit: token-efficient context, safe modes, fingerprinted /init, model benchmarking, web access, and browser automation.",
   "repository": {
     "type": "git",

package/configs/opl-footer.json

@@ -1,48 +0,0 @@
-{
-  "_comment": " \ue22c Available segment IDs: pi, model, path, git, thinking, caveman, plan_mode, chat_mode, mode_switcher, separator, token_in, token_out, token_total, cache_read, cache_write, cost, context_pct, context_total, session_stats, perf_stats, text:<literal>",
-  "row1LeftSegments": ["pi", "separator", "model", "separator", "path", "git"],
-  "row1RightSegments": ["context_pct"],
-  "row2LeftSegments": [
-    "thinking",
-    "separator",
-    "mode_switcher"
-  ],
-  "row2RightSegments": ["token_total", "separator", "cost"],
-  "row3LeftSegments": ["session_stats"],
-  "row3RightSegments": ["perf_stats"],
-  "icons": {
-    "pi": "\udb80\udfff",
-    "folder": "\uf115",
-    "branch": "\uf126",
-    "git": "\uf1d3",
-    "separator": "|"
-  },
-  "colors": {
-    "model": "#c07898",
-    "thinkingHigh": "#afb9fe",
-    "thinkingXhigh": "#9575cd",
-    "thinkingMax": "#ce93d8",
-    "separator": "#87827a"
-  },
-  "segmentOptions": {
-    "path": {
-      "mode": "full"
-    },
-    "git": {
-      "showBranch": true,
-      "showStaged": true,
-      "showUnstaged": true,
-      "showUntracked": true
-    },
-    "contextBar": {
-      "barWidth": 18,
-      "filledChar": "▋",
-      "unfilledChar": "▋",
-      "unfilledColor": "#4e4c49",
-      "gradientStart": "#f29373",
-      "gradientMid": "#d67858",
-      "gradientEnd": "#ae4f2f",
-      "gradientMidPoint": 0.55
-    }
-  }
-}

package/configs/opl-input.json

@@ -1,22 +0,0 @@
-{
-  "boxedView": false,
-  "companion": {
-    "enabled": true,
-    "color": "accent",
-    "type": "dog",
-    "types": [
-      {
-        "typeName": "cat",
-        "top": " /\\,/\\ "
-      },
-      {
-        "typeName": "dog",
-        "top": " /),(\\ "
-      },
-      {
-        "typeName": "bunny",
-        "top": " (\\,/) "
-      }
-    ]
-  }
-}

package/configs/opl-modes.json

@@ -1,291 +0,0 @@
-{
-  "_comment": "Every field below is optional; values shown are the extension's built-in defaults for reference.",
-  "ui": {
-    "hideNotify": false,
-    "hideWidget": true
-  },
-  "shortcuts": {
-    "_comment": "Shortcut that cycles through all enabled, visible modes in registry order. Default: ctrl+alt+m.",
-    "cycleMode": "shift+tab"
-  },
-  "cleanup": {
-    "_comment": "Set to true to delete the plan file automatically after plan_complete is called.",
-    "cleanupOnComplete": false
-  },
-  "defaultNotifyTemplate": "✓ {Name} mode ON",
-  "_comment_defaultNotifyTemplate": "Default notify text for any custom mode that doesn't set its own labels.notify. {Name} is replaced with the mode name, capitalized.",
-  "_comment_lazyTools": "Tool names withheld from the active set at rest and enabled on demand via the load_tools tool. Only affects modes that would otherwise include them (off/execute and custom modes that list them). Core built-ins, plan_complete, and load_tools are protected and ignored. Omit or leave empty to keep all tools always active.",
-  "lazyTools": [
-    "subagent", 
-    "subagent_wait", 
-    "subagent_supervisor", 
-    "browser", 
-    "simplebench"
-  ],
-  "bashPatterns": {
-    "_comment_safe": "Replace-only: providing safePatterns replaces ALL defaults for both built-in chat/plan modes (shared base). Omit to keep built-in safe patterns. Shown here with the built-in list.",
-    "_comment_destructive": "Replace-only: providing destructivePatterns replaces ALL defaults for both built-in chat/plan modes (shared base). Omit to keep built-in destructive patterns. Shown here with the built-in list.",
-    "safePatterns": [
-      "^\\s*cat\\b",
-      "^\\s*head\\b",
-      "^\\s*tail\\b",
-      "^\\s*less\\b",
-      "^\\s*more\\b",
-      "^\\s*grep\\b",
-      "^\\s*find\\b",
-      "^\\s*ls\\b",
-      "^\\s*pwd\\b",
-      "^\\s*cd\\b",
-      "^\\s*echo\\b",
-      "^\\s*printf\\b",
-      "^\\s*wc\\b",
-      "^\\s*sort\\b",
-      "^\\s*diff\\b",
-      "^\\s*file\\b",
-      "^\\s*stat\\b",
-      "^\\s*du\\b",
-      "^\\s*df\\b",
-      "^\\s*tree\\b",
-      "^\\s*which\\b",
-      "^\\s*whereis\\b",
-      "^\\s*type\\b",
-      "^\\s*env\\b",
-      "^\\s*printenv\\b",
-      "^\\s*uname\\b",
-      "^\\s*whoami\\b",
-      "^\\s*date\\b",
-      "^\\s*uptime\\b",
-      "^\\s*ps\\b",
-      "^\\s*free\\b",
-      "^\\s*rg\\b",
-      "^\\s*fd\\b",
-      "^\\s*bat\\b",
-      "^\\s*jq\\b",
-      "^\\s*git\\s+(status|log|diff|show|branch|remote)",
-      "^\\s*node\\s+--version",
-      "^\\s*python\\s+--version",
-      "^\\s*(npx\\s+)?tsc\\b.*--noEmit",
-      "^\\s*npm\\s+(list|ls|view|info|outdated|audit)",
-      "^\\s*yarn\\s+(list|info|why|audit)"
-    ],
-    "destructivePatterns": [
-      "\\brm\\b",
-      "\\brmdir\\b",
-      "\\bmv\\b",
-      "\\bcp\\b",
-      "\\bmkdir\\b",
-      "\\btouch\\b",
-      "\\bchmod\\b",
-      "\\bchown\\b",
-      "\\btee\\b",
-      "\\bdd\\b",
-      "\\bshred\\b",
-      "(^|[^<])>(?!>|&)",
-      ">>",
-      "\\bnpm\\s+(install|uninstall|update|ci)",
-      "\\byarn\\s+(add|remove|install)",
-      "\\bpip\\s+(install|uninstall)",
-      "\\bgit\\s+(add|commit|push|merge|rebase|reset|checkout|branch\\s+-)",
-      "\\bsudo\\b",
-      "\\bsu\\b",
-      "\\bkill\\b",
-      "\\bpkill\\b",
-      "\\b(sh|bash|zsh)\\b",
-      "\\b(vim?|nano|emacs|code|subl)\\b"
-    ]
-  },
-  "_comment_tools": "Set modes.chat.tools or modes.plan.tools to replace their default tools. The legacy top-level chatAllowedTools and planAllowedTools keys remain supported but are deprecated.",
-  "_comment_widgetColor": "available colors - accent, border, borderAccent, borderMuted, success, error, warning, muted, dim, text, thinkingText, searchMatchText, userMessageText, customMessageText, customMessageLabel, toolTitle, toolOutput, mdHeading, mdLink, mdLinkUrl, mdCode, mdCodeBlock, mdCodeBlockBorder, mdQuote, mdQuoteBorder, mdHr, mdListBullet, toolDiffAdded, toolDiffRemoved, toolDiffContext, syntaxComment, syntaxKeyword, syntaxFunction, syntaxVariable, syntaxString, syntaxNumber, syntaxType, syntaxOperator, syntaxPunctuation, thinkingOff, thinkingMinimal, thinkingLow, thinkingMedium, thinkingHigh, thinkingXhigh, thinkingMax, bashMode",
-  "_comment_appearance": "Every mode owns appearance here. prefix/prefixColor/borderColor style opl-input; modeColor styles opl-footer's Mode value. All color values accept Pi theme tokens or #RRGGBB.",
-  "modes": {
-    "off": {
-      "_comment": "Built-in mode — no restrictions, normal agent behavior.",
-      "prompt": "",
-      "model": {
-        "provider": "litellm-proxy",
-        "id": "gpt-5.6-terra"
-      },
-      "allowPlanComplete": false,
-      "visible": true,
-      "enabled": true,
-      "labels": {
-        "notify": "✓ Mode OFF",
-        "notifyType": "info",
-        "widget": "✓ Normal mode",
-        "widgetColor": "dim"
-      },
-      "appearance": {
-        "prefix": "❯",
-        "prefixColor": "accent",
-        "borderColor": "border",
-        "modeColor": "dim"
-      }
-    },
-    "chat": {
-      "_comment": "Built-in mode — read-only conversational.",
-      "prompt": "**SUPERSEDES ALL OTHER BEHAVIOR INSTRUCTIONS.** This overrides any role or style directives (e.g. caveman, roleplay, tone modifiers). The constraints below take absolute priority.\n\nYou are in CHAT MODE. You have read-only access — you may read files, search code, run safe inspection commands, and search the web to answer, discuss, and explore. Converse naturally — answer questions, explain, brainstorm, look things up.\n\nYou MUST NOT attempt to edit, create, delete, or modify any files, or run any command that changes state. If the user asks for a change, explain what you would do but do not attempt it — they can exit chat mode first.\n\nThere is no plan format and no plan_complete tool. Just respond helpfully within read-only constraints.",
-      "model": {
-        "provider": "amazon-bedrock",
-        "id": "moonshotai.kimi-k2.5"
-      },
-      "tools": [
-        "read",
-        "bash",
-        "grep",
-        "find",
-        "ls",
-        "web_search",
-        "fetch_content",
-        "get_search_content",
-        "artifact",
-        "questionnaire"
-      ],
-      "allowPlanComplete": false,
-      "visible": true,
-      "enabled": true,
-      "labels": {
-        "notify": "✓ Chat mode ON",
-        "notifyType": "info",
-        "widget": "✓ Chat mode",
-        "widgetColor": "accent"
-      },
-      "appearance": {
-        "prefix": "󰭻",
-        "prefixColor": "#157cd6",
-        "borderColor": "#157cd6",
-        "modeColor": "#157cd6"
-      }
-    },
-    "plan": {
-      "_comment": "Built-in mode — read-only explore-and-plan.",
-      "prompt": "**SUPERSEDES ALL OTHER BEHAVIOR INSTRUCTIONS.** This overrides any role or style directives (e.g. caveman, roleplay, tone modifiers). The constraints below take absolute priority.\n\nYou are in PLAN MODE. You have read-only access — you may explore and analyze, but you MUST NOT make any changes.\n\nYour task: produce an action plan.\n\nYou MUST begin the plan with exactly this heading on its own line:\n\n# Plan:\n1. [Step title — short verb-object phrase]\n   [2-4 sentences of context: which file(s), where, what to change, and why.]\n2. [Step title]\n   [Context...]\n...\n\nEach step MUST be self-contained — write it as if the executor has no memory of this conversation. Include enough context that it can be carried out with only the plan file and the codebase. Assume the executor will read the relevant files fresh — do not rely on findings you discovered during planning.\n\nGood: \"Add auth middleware to routes/index.ts\n Apply it as `app.use(authMiddleware)` before the route definitions (~line 45). Currently routes/index.ts has no middleware.\"\n\nBad: \"Add it to the file we looked at\"\n\nBad (over-prescribed): \"Insert `const authMiddleware = require('./middleware/auth');` at line 3, then add `app.use(authMiddleware);` at line 46\"\n\nSpecify what to do and where — not the exact implementation. The executor reads the relevant files and decides how.\n\nAfter listing all steps, stop and wait for the user to choose:\n- \"Execute plan\" — switches to execute mode where you carry out each step\n- \"Refine\" — revise the plan based on feedback\n- Continue exploring if you need more information before planning\n\nDo NOT attempt to make any file changes, run destructive commands, or modify anything.",
-      "model": {
-        "provider": "amazon-bedrock",
-        "id": "global.anthropic.claude-sonnet-5"
-      },
-      "tools": [
-        "read",
-        "bash",
-        "grep",
-        "find",
-        "ls",
-        "web_search",
-        "fetch_content",
-        "get_search_content",
-        "artifact",
-        "questionnaire"
-      ],
-      "allowPlanComplete": false,
-      "visible": true,
-      "enabled": true,
-      "labels": {
-        "notify": "✓ Plan mode ON",
-        "notifyType": "info",
-        "notifyWithTitle": "✓ Active plan: {title}",
-        "notifyLoaded": "✓ Active plan: {title}",
-        "widget": "✓ Plan mode active",
-        "widgetWithTitle": "✓ Active plan: {title}",
-        "widgetColor": "accent"
-      },
-      "appearance": {
-        "prefix": "⏸",
-        "prefixColor": "#52d90f",
-        "borderColor": "#52d90f",
-        "modeColor": "#52d90f"
-      }
-    },
-    "execute": {
-      "_comment": "Built-in mode — entered only via an 'execute:<plan-file>' picker item, never a bare menu choice (visible: false). prompt: leave empty to use the built-in template that injects the active plan file content; set a custom string with a {plan} placeholder to override.",
-      "prompt": "",
-      "model": {
-        "provider": "amazon-bedrock",
-        "id": "global.anthropic.claude-opus-4-8"
-      },
-      "allowPlanComplete": true,
-      "visible": false,
-      "enabled": true,
-      "labels": {
-        "notify": "✓ Executing plan",
-        "notifyWithTitle": "✓ Executing plan: {title}",
-        "notifyType": "info",
-        "widget": "✓ Executing plan",
-        "widgetWithTitle": "✓ Executing plan: {title}",
-        "widgetColor": "muted"
-      },
-      "appearance": {
-        "prefix": "⏸",
-        "prefixColor": "#52d90f",
-        "borderColor": "#52d90f",
-        "modeColor": "#52d90f"
-      }
-    },
-    "review": {
-      "enabled": true,
-      "allowExecute": false,
-      "prompt": "**SUPERSEDES ALL OTHER BEHAVIOR INSTRUCTIONS.** This overrides any role or style directives (e.g. caveman, roleplay, tone modifiers). The constraints below take absolute priority.\n\nYou are in REVIEW MODE. Analyze pull requests, code diffs, and codebases. Focus on correctness, style, test coverage, security, and architecture. Suggest improvements but do not apply them. Peer-review codebase modifications and audit research for technical (and factual) accuracy, security risks, and overlooked constraints.\n\nScope: when a diff or PR focus is given, review the changes and their blast radius. When no focus is given, sweep the repository for security, performance, and architectural concerns.\n\n## 1. Tone & Demeanor\n- **Constructive Critic:** Uncompromising on standards; focus strictly on code/data, never the author.\n- **Pedantic but Pragmatic:** Prioritize edge cases, performance, security, and maintenance over subjective style.\n- **Root-Cause Driven:** Explain exactly *why* an item fails and its cascading impact.\n\n## 2. Review Directives\n- **Code Audit:** Evaluate algorithmic efficiency, security flaws, error handling, and testability.\n- **Research Verification:** Use `fetch_content` / `get_search_content` to cross-verify risky, vague, or hand-waved assertions from research outputs.\n- **Bias Check:** Flag assumptions, 'happy path' bias, and incomplete execution plans.\n\n## 3. Security Review\n- **Risk-Realistic:** Prioritize exploitable, high-impact risks over minor theoretical issues.\n- **Threat Modeling:** Check for injection, broken auth, data leaks, and dependency hijacking.\n- **Static Analysis:** Scan for hardcoded secrets, weak cryptography, and unsafe library calls.\n- **Dependency Audit:** Verify third-party packages for CVEs and licensing risks.\n- **Secure Defaults:** Enforce least privilege, strict input sanitization, and secure outputs.\n\nProvide this exact structure for identified security risks:\n1. **Severity:** `CRITICAL`, `HIGH`, `MEDIUM`, or `LOW`.\n2. **Vulnerability:** Brief description and OWASP/CWE reference.\n3. **Exploit Vector:** How it can be compromised.\n4. **Remediation:** Exact code fix or configuration change required.",
-      "tools": [
-        "read",
-        "bash",
-        "grep",
-        "find",
-        "ls",
-        "web_search",
-        "fetch_content",
-        "get_search_content",
-        "artifact",
-        "questionnaire"
-      ],
-      "safePatterns": [
-        "^git",
-        "^cat",
-        "^grep",
-        "^diff",
-        "^ls",
-        "^head",
-        "^tail"
-      ],
-      "labels": {
-        "widgetColor": "accent"
-      },
-      "appearance": {
-        "prefix": "◎",
-        "prefixColor": "#ce93d8",
-        "borderColor": "#ce93d8",
-        "modeColor": "#ce93d8"
-      }
-    },
-    "research": {
-      "enabled": true,
-      "allowExecute": false,
-      "prompt": "**SUPERSEDES ALL OTHER BEHAVIOR INSTRUCTIONS.** This overrides any role or style directives (e.g. caveman, roleplay, tone modifiers). The constraints below take absolute priority.\n\nYou are in RESEARCH MODE. Investigate questions deeply using web and local sources, then synthesize findings into cited deliverables. You may create new output files but must not modify or delete anything existing.\n\n## 1. Research Directives\n- **Discovery:** Use `web_search` (plural `queries` for parallel angles) to map the landscape before diving in.\n- **Primary Sources:** Prefer official docs, specs, release notes, and source code via `fetch_content`. Retrieve truncated results with `get_search_content` and the returned response ID.\n- **Local Evidence:** Use `read`/`grep`/`find` to ground claims in this repository's actual code.\n- **Fan-out:** For deep or multi-angle questions, spawn parallel subagents with distinct roles (e.g. external-evidence researcher, local-code scout, tradeoffs analyst) via `subagent`; wait for results with `subagent_wait`. Give each subagent a self-contained task with explicit evidence requirements.\n- **No Invention:** Never present an unverified claim as fact. Mark confidence per finding (high/medium/low) and state gaps explicitly.\n\n## 2. Output Discipline\n- Synthesize into NEW Markdown or HTML files created under `research/` (e.g. `research/<topic>.md`). Never modify, move, or overwrite existing files.\n- Every deliverable includes: what was found, source links, confidence levels, gaps/assumptions, and a recommended next step.\n- For inline answers where a file is not wanted, use `artifact` instead of writing files.\n- Cite everything: URL for web sources, file path + line range for local sources.",
-      "model": {
-        "provider": "amazon-bedrock",
-        "id": "zai.glm-5"
-      },
-      "tools": [
-        "read",
-        "grep",
-        "find",
-        "ls",
-        "web_search",
-        "fetch_content",
-        "get_search_content",
-        "artifact",
-        "questionnaire",
-        "subagent",
-        "subagent_wait",
-        "write"
-      ],
-      "labels": {
-        "widget": "✓ Research mode active",
-        "widgetColor": "accent"
-      },
-      "appearance": {
-        "prefix": "⌕",
-        "prefixColor": "#f2eb5a",
-        "borderColor": "#f2eb5a",
-        "modeColor": "#f2eb5a"
-      }
-    }
-  }
-}

package/configs/opl-simplebench.json

@@ -1,8 +0,0 @@
-{
-  "_comment": "Copy to ~/.pi/agent/configs/opl-simplebench.json. Endpoints are optional and only used by their corresponding Simplebench flags.",
-  "researchSearchProvider": "ddgs",
-  "researchSearchUrl": "http://192.168.1.100:8000",
-  "researchMaxResults": 5,
-  "llamaServerUrl": "http://192.168.1.106:7679/v1",
-  "llamagputopUrl": "http://192.168.1.106:4321/stats"
-}

package/configs/opl-todo.json

@@ -1,12 +0,0 @@
-{
-  "allDoneHideMs": 10000,
-  "shortcuts": {
-    "toggleWidget": "ctrl+alt+t",
-    "resetDone": "ctrl+alt+r"
-  },
-  "widget": {
-    "widthPercent": 33,
-    "maxHeightPercent": 50,
-    "minWidth": 32
-  }
-}

package/configs/opl-webaccess.json

@@ -1,31 +0,0 @@
-{
-  "provider": "ddgs",
-  "providers": {
-    "gemini": {
-      "apiKeyEnv": "GEMINI_API_KEY",
-      "baseUrl": "https://generativelanguage.googleapis.com/v1beta",
-      "model": "gemini-3.5-flash-lite",
-      "_available_models": ["gemini-3.6-flash", "gemini-3.5-flash-lite", "gemini-2.5-flash", "gemini-2.5-pro"]
-    },
-    "tavily": {
-      "apiKeyEnv": "TAVILY_API_KEY",
-      "maxResults": 5
-    },
-    "ddgs": {
-      "apiUrl": "http://192.168.1.100:8000",
-      "maxResults": 12
-    },
-    "searxng": {
-      "instanceUrl": "http://192.168.1.100:8888",
-      "maxResults": 12,
-      "categories": "general",
-      "safeSearch": 0
-    },
-    "exa": {
-      "apiKeyEnv": "EXA_API_KEY",
-      "maxResults": 5,
-      "searchType": "auto",
-      "includeSummary": true
-    }
-  }
-}