@demigodmode/pi-web-agent 0.2.1 → 0.3.1

package/README.md

@@ -1,199 +1,119 @@
 # pi-web-agent
 
-`@demigodmode/pi-web-agent` is a Pi package for reliable web access.
+[![CI](https://github.com/demigodmode/pi-web-agent/actions/workflows/ci.yml/badge.svg)](https://github.com/demigodmode/pi-web-agent/actions/workflows/ci.yml)
+[![npm version](https://img.shields.io/npm/v/@demigodmode/pi-web-agent)](https://www.npmjs.com/package/@demigodmode/pi-web-agent)
+[![Docs](https://img.shields.io/badge/docs-github%20pages-blue)](https://demigodmode.github.io/pi-web-agent/)
 
-It is built around a simple rule: searching for a page is not the same thing as reading it. This package keeps those steps separate, prefers plain HTTP by default, and is designed to say "I couldn't read this reliably" instead of making something up.
+`@demigodmode/pi-web-agent` is a Pi package for web access.
 
-## What it does
+The whole point is keeping the boundaries straight:
 
-The package is built around three tools:
+- `web_search` is for discovery
+- `web_fetch` is for plain HTTP reads
+- `web_fetch_headless` is the explicit browser path
+- `web_explore` is the bounded research path
 
-- `web_search` finds relevant pages and returns titles, URLs, and snippets
-- `web_fetch` fetches a specific page over plain HTTP and tries to extract readable content
-- `web_fetch_headless` is the explicit browser-based path for pages that need rendering
-
-The boundary between those tools is intentional.
-
-`web_search` is for discovery. It should not imply that a page was fetched.
-
-`web_fetch` is for reading a page over HTTP. If the result looks weak, incomplete, blocked, or too script-heavy, it should return `needs_headless` instead of bluffing.
-
-`web_fetch_headless` exists for the cases where a browser really is required. It is opt-in only.
-
-## Why this exists
-
-A lot of web tooling in coding agents gets fuzzy in exactly the wrong places. Search results get treated like page reads. Browser fallback happens behind the scenes. Failures get softened into fake confidence.
-
-This package is trying to do the opposite.
-
-The rules are straightforward:
-
-- no hidden browser launch
-- no automatic HTTP-to-headless fallback
-- no claiming a page was read when only snippets were available
-- explicit structured failure when the result is incomplete or blocked
-
-## What makes it different
-
-The main thing is the contract.
-
-`web_search` discovers sources.
-
-`web_fetch` reads over HTTP only.
-
-`web_fetch_headless` is the explicit browser path.
-
-That separation is the whole point. It makes failures easier to reason about and avoids the weird behavior where a tool quietly changes execution mode behind your back.
+That sounds obvious, but a lot of agent tooling gets fuzzy right there. This package is meant to be stricter about what it actually did and more willing to say when a read was not good enough to trust.
 
 ## Install
 
-Install it through Pi:
-
 ```bash
 pi install npm:@demigodmode/pi-web-agent
 ```
 
-Update installed packages later with:
+Later on, update installed packages with:
 
 ```bash
 pi update
 ```
 
-If you just want to inspect the package from npm directly, the package name is:
+## Docs
 
-```bash
-npm view @demigodmode/pi-web-agent
-```
+Docs site:
 
-## Current status
+- https://demigodmode.github.io/pi-web-agent/
 
-This repo is in early MVP shape, but it is no longer just a design doc.
+Work on the docs locally:
 
-Right now it has:
+```bash
+npm run docs:dev
+```
 
-- a TypeScript project scaffold
-- shared result and status contracts
-- a DuckDuckGo HTML parser for `web_search`
-- an HTTP fetch path with readability-based extraction and conservative escalation to `needs_headless`
-- a real browser-backed `web_fetch_headless` implementation with local browser resolution
-- repo-local Pi extension wiring for development
-- a test suite around parser behavior, contracts, extraction, caching, and tool adapters
-- optional smoke coverage for local installed browsers
+Build the docs:
 
-So the project is real and usable, but still early.
+```bash
+npm run docs:build
+```
 
-## Example behavior
+## Presentation modes
 
-These are conceptual examples of the contract the package is aiming to expose.
+`pi-web-agent` renders web tool output in one visible mode at a time:
 
-### Search
+- `compact` — short summary, default everywhere
+- `preview` — slightly richer bounded view
+- `verbose` — fuller bounded view
 
-`web_search("pi coding agent")`
+## Settings
 
-Returns discovery results like:
+Primary UI:
 
-- title
-- URL
-- snippet
+```text
+/web-agent settings
+```
 
-It does not imply the page was fetched.
+Helper commands:
 
-### HTTP fetch
+```text
+/web-agent show
+/web-agent reset project
+/web-agent reset global
+/web-agent mode preview
+/web-agent mode web_search verbose
+/web-agent mode web_search inherit
+```
 
-`web_fetch("https://example.com/article")`
+Config files:
 
-If the page is readable over plain HTTP, it should return extracted content.
+```text
+Global:  ~/.pi/agent/extensions/pi-web-agent/config.json
+Project: .pi/extensions/pi-web-agent/config.json
+```
 
-If the page looks too script-heavy, too thin, blocked, or otherwise unreliable, it should return `needs_headless` instead of pretending the extraction is good enough.
+Precedence:
 
-### Explicit headless fetch
+- built-in defaults
+- global config
+- project config
 
-`web_fetch_headless("https://example.com/app")`
+Project config overrides global config.
 
-This is the browser-based path for pages that really need rendering.
+Example:
 
-This path now launches a local browser explicitly, waits for the rendered page to settle, and then extracts readable content from the rendered HTML.
+```json
+{
+  "presentation": {
+    "defaultMode": "compact",
+    "tools": {
+      "web_search": { "mode": "preview" },
+      "web_explore": { "mode": "verbose" }
+    }
+  }
+}
+```
 
 ## Local development
 
-Install dependencies:
-
 ```bash
 npm install
-```
-
-Run tests with coverage:
-
-```bash
 npm test
-```
-
-Run the typecheck used as lint:
-
-```bash
 npm run lint
-```
-
-Build the project:
-
-```bash
 npm run build
 ```
 
-To run the optional real-browser smoke test for headless fetch, set `PI_HEADLESS_SMOKE=1` before running Vitest. It stays skipped by default so local browser install differences do not make the normal test suite flaky.
-
-Coverage is now part of the normal `npm test` flow. Vitest prints a text summary in the terminal and writes the full HTML report to `coverage/`.
-
-### Trying it in Pi locally
-
-This repo includes a project-local Pi extension entrypoint at `.pi/extensions/pi-web-agent.ts` for development and hot reload.
+For local Pi work, this repo includes `.pi/extensions/pi-web-agent.ts`.
 
-For the published npm package, Pi loads the compiled runtime from `dist/extension.js` via the `pi.extensions` entry in `package.json`.
-
-After starting Pi in this project, use `/reload` if you change the extension code and want Pi to pick up the latest version.
-
-## Project layout
-
-The code is split into small modules on purpose.
-
-- `src/extension.ts` - package entry surface
-- `src/tools/` - thin tool adapters
-- `src/search/` - search backend logic
-- `src/fetch/` - HTTP and headless fetch logic
-- `src/extract/` - readable-content extraction
-- `src/cache/` - small cache utilities
-- `src/types.ts` - shared contracts
-- `tests/` - parser, contract, extraction, fetch, and adapter tests
+If Pi is already running, use `/reload` after changes.
 
 ## License
 
 AGPL-3.0-only. See `LICENSE`.
-
-## Release process
-
-1. Update `CHANGELOG.md` under `## Unreleased`.
-2. Run `npm run release:dry-run` to preview the next version.
-3. Run `npm run release` to bump version, rewrite the changelog release heading, create a release commit, and create a tag.
-4. Push the branch and tag.
-5. GitHub Actions publishes the tagged release to npm.
-
-## Maintainer release notes
-
-This repo is set up for npm Trusted Publishing from GitHub Actions.
-
-In npm package settings, add a trusted publisher for:
-- package: `@demigodmode/pi-web-agent`
-- provider: GitHub Actions
-- repository: `demigodmode/pi-web-agent`
-
-That replaces the old `NPM_TOKEN` secret flow.
-
-## Near-term next steps
-
-The next chunk of work is pretty clear:
-
-- keep tightening weak-content escalation on tricky HTTP targets
-- improve cleanup of noisy rendered-page extraction on busy sites
-- expand fixtures and end-to-end coverage
-- add alternate search backends behind a first-class provider abstraction
-

package/dist/commands/web-agent-config.d.ts

@@ -0,0 +1,23 @@
+import { type ExtensionAPI } from '@mariozechner/pi-coding-agent';
+import { loadPresentationConfigLayers, type LoadedPresentationConfig } from '../presentation/config-store.js';
+import type { PresentationConfig, PresentationConfigOverride, PresentationScope } from '../presentation/types.js';
+type CommandDeps = {
+    load?: () => ReturnType<typeof loadPresentationConfigLayers>;
+    save?: (scope: PresentationScope, config: PresentationConfigOverride) => Promise<void>;
+    reset?: (scope: PresentationScope) => Promise<void>;
+};
+export type SettingsDraftState = {
+    scope: PresentationScope;
+    drafts: Record<PresentationScope, PresentationConfig>;
+    config: PresentationConfig;
+};
+export declare function getInheritedConfigForScope(loaded: Awaited<LoadedPresentationConfig>, scope: PresentationScope): PresentationConfig;
+export declare function getScopeDisplayConfig(loaded: Awaited<LoadedPresentationConfig>, scope: PresentationScope): PresentationConfig;
+export declare function createSettingsDraftState(loaded: Awaited<LoadedPresentationConfig>, initialScope: PresentationScope): SettingsDraftState;
+export declare function applySettingsValue(state: SettingsDraftState, id: string, newValue: string): SettingsDraftState;
+export declare function collapsePresentationConfigToOverride(config: PresentationConfig, inheritedConfig: PresentationConfig): PresentationConfigOverride;
+export declare function handleSettingsShortcut(data: string): {
+    action: 'cancel' | 'reset' | 'save';
+} | undefined;
+export declare function registerWebAgentConfigCommands(pi: ExtensionAPI, deps?: CommandDeps): void;
+export {};

package/dist/commands/web-agent-config.js

@@ -0,0 +1,254 @@
+import { getSettingsListTheme } from '@mariozechner/pi-coding-agent';
+import { Container, SettingsList, Text } from '@mariozechner/pi-tui';
+import { DEFAULT_PRESENTATION_CONFIG, mergePresentationConfigLayers, resolvePresentationMode } from '../presentation/config.js';
+import { loadPresentationConfigLayers, resetPresentationConfigScope, savePresentationConfigScope } from '../presentation/config-store.js';
+const PRESENTATION_TOOL_NAMES = [
+    'web_search',
+    'web_fetch',
+    'web_fetch_headless',
+    'web_explore'
+];
+function parseScopeToken(token) {
+    return token === 'global' || token === 'project' ? token : undefined;
+}
+function clonePresentationConfig(config) {
+    return {
+        defaultMode: config.defaultMode,
+        tools: { ...config.tools }
+    };
+}
+function formatConfigSummary(config) {
+    const lines = [`defaultMode: ${config.defaultMode}`];
+    for (const toolName of PRESENTATION_TOOL_NAMES) {
+        lines.push(`${toolName}: ${config.tools[toolName]?.mode ?? 'inherit'}`);
+    }
+    return lines.join('\n');
+}
+function buildSettingsItems(scope, config) {
+    return [
+        {
+            id: 'scope',
+            label: 'Write scope',
+            currentValue: scope,
+            values: ['project', 'global']
+        },
+        {
+            id: 'defaultMode',
+            label: 'Default mode',
+            currentValue: config.defaultMode,
+            values: ['compact', 'preview', 'verbose']
+        },
+        ...PRESENTATION_TOOL_NAMES.map((toolName) => ({
+            id: `tool:${toolName}`,
+            label: toolName,
+            currentValue: config.tools[toolName]?.mode ?? 'inherit',
+            values: ['inherit', 'compact', 'preview', 'verbose']
+        }))
+    ];
+}
+function isToolName(value) {
+    return PRESENTATION_TOOL_NAMES.includes(value);
+}
+function isModeOrInherit(value) {
+    return ['inherit', 'compact', 'preview', 'verbose'].includes(value);
+}
+export function getInheritedConfigForScope(loaded, scope) {
+    if (scope === 'global') {
+        return DEFAULT_PRESENTATION_CONFIG;
+    }
+    return mergePresentationConfigLayers(DEFAULT_PRESENTATION_CONFIG, loaded.global.rawConfig);
+}
+export function getScopeDisplayConfig(loaded, scope) {
+    if (scope === 'global') {
+        return mergePresentationConfigLayers(DEFAULT_PRESENTATION_CONFIG, loaded.global.rawConfig);
+    }
+    return mergePresentationConfigLayers(DEFAULT_PRESENTATION_CONFIG, loaded.global.rawConfig, loaded.project.rawConfig);
+}
+export function createSettingsDraftState(loaded, initialScope) {
+    const drafts = {
+        global: getScopeDisplayConfig(loaded, 'global'),
+        project: getScopeDisplayConfig(loaded, 'project')
+    };
+    return {
+        scope: initialScope,
+        drafts,
+        config: clonePresentationConfig(drafts[initialScope])
+    };
+}
+export function applySettingsValue(state, id, newValue) {
+    const nextDrafts = {
+        global: clonePresentationConfig(state.drafts.global),
+        project: clonePresentationConfig(state.drafts.project)
+    };
+    let nextScope = state.scope;
+    if (id === 'scope' && (newValue === 'project' || newValue === 'global')) {
+        nextScope = newValue;
+        return {
+            scope: nextScope,
+            drafts: nextDrafts,
+            config: clonePresentationConfig(nextDrafts[nextScope])
+        };
+    }
+    const currentDraft = clonePresentationConfig(nextDrafts[nextScope]);
+    if (id === 'defaultMode' && (newValue === 'compact' || newValue === 'preview' || newValue === 'verbose')) {
+        currentDraft.defaultMode = newValue;
+    }
+    if (id.startsWith('tool:')) {
+        const toolName = id.slice('tool:'.length);
+        const nextTools = { ...currentDraft.tools };
+        if (newValue === 'inherit') {
+            delete nextTools[toolName];
+        }
+        else if (newValue === 'compact' ||
+            newValue === 'preview' ||
+            newValue === 'verbose') {
+            nextTools[toolName] = { mode: newValue };
+        }
+        currentDraft.tools = nextTools;
+    }
+    nextDrafts[nextScope] = currentDraft;
+    return {
+        scope: nextScope,
+        drafts: nextDrafts,
+        config: clonePresentationConfig(nextDrafts[nextScope])
+    };
+}
+export function collapsePresentationConfigToOverride(config, inheritedConfig) {
+    const tools = Object.fromEntries(PRESENTATION_TOOL_NAMES.flatMap((toolName) => {
+        const configuredMode = config.tools[toolName]?.mode;
+        if (!configuredMode) {
+            return [];
+        }
+        const inheritedMode = resolvePresentationMode(toolName, inheritedConfig);
+        if (configuredMode === inheritedMode) {
+            return [];
+        }
+        return [[toolName, { mode: configuredMode }]];
+    }));
+    return {
+        defaultMode: config.defaultMode === inheritedConfig.defaultMode ? undefined : config.defaultMode,
+        tools
+    };
+}
+export function handleSettingsShortcut(data) {
+    if (data === '\u001b') {
+        return { action: 'cancel' };
+    }
+    if (data === '\u0012') {
+        return { action: 'reset' };
+    }
+    if (data === '\u0013') {
+        return { action: 'save' };
+    }
+    return undefined;
+}
+async function openSettingsUi(ctx, loaded, initialScope) {
+    return ctx.ui.custom((_tui, theme, _kb, done) => {
+        let state = createSettingsDraftState(loaded, initialScope);
+        let settingsList;
+        const container = new Container();
+        container.addChild(new Text(theme.fg('accent', theme.bold('pi-web-agent settings')), 1, 1));
+        container.addChild(new Text(theme.fg('muted', 'Ctrl+S save · Ctrl+R reset scope · Esc cancel'), 1, 2));
+        const rebuildSettingsList = () => {
+            if (settingsList) {
+                container.removeChild(settingsList);
+            }
+            settingsList = new SettingsList(buildSettingsItems(state.scope, state.config), Math.min(PRESENTATION_TOOL_NAMES.length + 8, 18), getSettingsListTheme(), (id, newValue) => {
+                state = applySettingsValue(state, id, newValue);
+                rebuildSettingsList();
+                container.invalidate();
+            }, () => done({ action: 'save', scope: state.scope, config: state.config }), { enableSearch: true });
+            container.addChild(settingsList);
+        };
+        rebuildSettingsList();
+        return {
+            render: (width) => container.render(width),
+            invalidate: () => container.invalidate(),
+            handleInput: (data) => {
+                const shortcut = handleSettingsShortcut(JSON.stringify(data).slice(1, -1));
+                if (shortcut?.action === 'cancel') {
+                    done({ action: 'cancel' });
+                    return;
+                }
+                if (shortcut?.action === 'reset') {
+                    done({ action: 'reset', scope: state.scope });
+                    return;
+                }
+                if (shortcut?.action === 'save') {
+                    done({ action: 'save', scope: state.scope, config: state.config });
+                    return;
+                }
+                settingsList.handleInput?.(data);
+            }
+        };
+    });
+}
+export function registerWebAgentConfigCommands(pi, deps = {}) {
+    const load = deps.load ?? (() => loadPresentationConfigLayers());
+    const save = deps.save ?? ((scope, config) => savePresentationConfigScope({}, scope, config));
+    const reset = deps.reset ?? ((scope) => resetPresentationConfigScope({}, scope));
+    pi.registerCommand('web-agent', {
+        description: 'Open settings or manage pi-web-agent presentation config',
+        handler: async (args, ctx) => {
+            const [action, maybeScope] = (args ?? '').trim().split(/\s+/).filter(Boolean);
+            if (action === 'show') {
+                const loaded = await load();
+                ctx.ui.notify([
+                    formatConfigSummary(loaded.effectiveConfig),
+                    `global: ${loaded.global.path}${loaded.global.exists ? '' : ' (missing)'}`,
+                    `project: ${loaded.project.path}${loaded.project.exists ? '' : ' (missing)'}`
+                ].join('\n'), 'info');
+                return;
+            }
+            if (action === 'reset') {
+                const scope = parseScopeToken(maybeScope) ?? 'project';
+                await reset(scope);
+                ctx.ui.notify(`Reset ${scope} config`, 'info');
+                return;
+            }
+            if (action === 'mode') {
+                const [, first, second] = (args ?? '').trim().split(/\s+/).filter(Boolean);
+                const loaded = await load();
+                const scope = 'project';
+                const baseConfig = getScopeDisplayConfig(loaded, scope);
+                const inheritedConfig = getInheritedConfigForScope(loaded, scope);
+                if (first && isModeOrInherit(first) && first !== 'inherit') {
+                    await save(scope, collapsePresentationConfigToOverride({ ...baseConfig, defaultMode: first }, inheritedConfig));
+                    ctx.ui.notify(`Saved project default mode = ${first}`, 'info');
+                    return;
+                }
+                if (first && second && isToolName(first) && isModeOrInherit(second)) {
+                    const nextTools = { ...baseConfig.tools };
+                    if (second === 'inherit') {
+                        delete nextTools[first];
+                    }
+                    else {
+                        nextTools[first] = { mode: second };
+                    }
+                    await save(scope, collapsePresentationConfigToOverride({ ...baseConfig, tools: nextTools }, inheritedConfig));
+                    ctx.ui.notify(`Saved project ${first} = ${second}`, 'info');
+                    return;
+                }
+                ctx.ui.notify('Usage: /web-agent mode <compact|preview|verbose> or /web-agent mode <tool> <inherit|compact|preview|verbose>', 'info');
+                return;
+            }
+            if (!action || action === 'settings') {
+                const loaded = await load();
+                const initialScope = 'project';
+                const result = await openSettingsUi(ctx, loaded, initialScope);
+                if (!result || result.action === 'cancel') {
+                    return;
+                }
+                if (result.action === 'reset') {
+                    await reset(result.scope);
+                    ctx.ui.notify(`Reset ${result.scope} config`, 'info');
+                    return;
+                }
+                await save(result.scope, collapsePresentationConfigToOverride(result.config, getInheritedConfigForScope(loaded, result.scope)));
+                ctx.ui.notify(`Saved ${result.scope} config`, 'info');
+                return;
+            }
+            ctx.ui.notify('Use /web-agent, /web-agent show, /web-agent reset project, or /web-agent settings', 'info');
+        }
+    });
+}