@hachej/boring-workspace 0.1.17 → 0.1.20

package/docs/plans/archive/HOT_RELOADABLE_AGENT_PLUGINS_PLAN.md

@@ -0,0 +1,218 @@
+# Hot-Reloadable Boring Plugins Plan
+
+Last updated: 2026-05-12
+Status: current architecture snapshot for PR #18
+
+This file is the single active plan for the hot-reloadable plugin/agent layer.
+Older split plans for agent docs, plugin-agent layering, and derivation path A
+were consolidated here so implementation guidance matches the code now in the
+branch.
+
+## Goals
+
+- `/reload` reloads both Pi agent assets and Boring workspace plugin assets.
+- Plugin metadata is package-shaped and easy to author.
+- UI registration is runtime code, not JSON-driven wiring.
+- Pi-specific runtime/reload behavior stays behind the Pi harness adapter.
+- The agent harness remains pluggable for non-Pi runtimes.
+- Bad plugin metadata is observable and recoverable: no process crash, stable
+  error events, and `.error` files.
+
+## Package model
+
+A Boring plugin is a package/directory with `package.json` metadata and optional
+runtime entrypoints:
+
+```txt
+plugin/
+  package.json
+  front/index.tsx      # browser-only BoringFrontFactory
+  agent/index.ts       # optional Pi ExtensionFactory
+  agent/skills/        # optional Pi skills
+  server/index.ts      # optional trusted workspace/server routes
+  shared/              # optional platform-neutral types/constants
+```
+
+### `package.json#boring`
+
+`boring` is workspace/UI discovery metadata only:
+
+```json
+{
+  "name": "example-plugin",
+  "boring": {
+    "label": "Example Plugin",
+    "front": "front/index.tsx",
+    "server": "server/index.ts",
+    "derivesFrom": "data-catalog"
+  }
+}
+```
+
+Allowed runtime semantics:
+
+- `id` / derived package name selects the stable plugin id.
+- `label` is display metadata.
+- `front` points at the front factory entrypoint.
+- `server` points at trusted Node routes/helpers, or `false` to opt out.
+- `derivesFrom` is discovery metadata for templates/catalogs.
+
+Not allowed in `boring`: panels, commands, left tabs, surface resolvers, agent
+tools, skills, extensions, packages, system prompts. Those old JSON registration
+arrays were intentionally removed.
+
+### `package.json#pi`
+
+`pi` owns agent/Pi contributions:
+
+```json
+{
+  "pi": {
+    "extensions": ["agent/index.ts"],
+    "skills": ["agent/skills"],
+    "packages": [{ "source": "file:.", "extensions": ["agent/index.ts"] }],
+    "systemPrompt": "Use this plugin's tools when working with example data."
+  }
+}
+```
+
+`extensions`, `skills`, `packages`, and `systemPrompt` are consumed by the Pi
+adapter. Workspace code only discovers and forwards them; it does not implement
+Pi loading directly.
+
+## Runtime registration
+
+### Front/UI
+
+`BoringFrontFactory` is the single runtime UI registration source:
+
+```ts
+import type { BoringFrontFactory } from "@hachej/boring-workspace/plugin"
+
+const plugin: BoringFrontFactory = (api) => {
+  api.registerPanel({ id: "example.panel", title: "Example", component: ExamplePane })
+  api.registerCommand({ id: "example.open", title: "Open Example", run: () => {} })
+  api.registerLeftTab({ id: "example.left", title: "Example", component: ExampleLeft })
+  api.registerSurfaceResolver({ id: "example.surface", resolve: () => ({ component: "example.panel" }) })
+}
+
+export default plugin
+```
+
+Front plugin code is browser code. It must not contribute executable agent tools.
+Hot-loaded panels are ordinary React function components: hooks such as
+`useState`, `useEffect`, and `useMemo` are supported. The host Vite dev server
+must resolve `react`, `react-dom`, `react/jsx-runtime`, and
+`react/jsx-dev-runtime` to the same singleton modules used by the workspace
+shell; duplicate React copies are a host configuration bug, not a plugin author
+contract. The legacy front-side `agentTools` / `agent-tool` output path is
+removed.
+
+### Server/workspace
+
+`server/index.ts` is trusted host-process code for workspace routes or support
+helpers. Hot-reloadable package plugins should put new tool capabilities in
+`pi.extensions` via `agent/index.ts`. Programmatic host/server plugin APIs may
+still adapt legacy `extraTools` during migration, but package metadata does not
+carry front-side or JSON-declared tools.
+
+### Agent/Pi
+
+`agent/index.ts` exports native Pi extension factories. The Pi harness adapter
+owns conversion into `DefaultResourceLoader`, dynamic package metadata refresh,
+skill loading, and `piSession.reload()`.
+
+## Reload flow
+
+1. User sends `/reload` in chat.
+2. Front slash-command calls `POST /api/v1/agent/reload`.
+3. Generic agent route calls the workspace-provided `beforeReload` hook.
+4. Workspace composition refreshes package.json `pi` inputs and reloads
+   `BoringPluginAssetManager`.
+5. Manager emits `boring.plugin.load`, `boring.plugin.unload`, or
+   `boring.plugin.error` SSE events.
+6. Generic agent route calls the configured harness `reloadSession` method.
+7. Pi harness implementation reloads Pi extension/skill/package resources for
+   the session.
+8. Front plugin runtime hot-swaps successful `front` modules and keeps the last
+   good UI alive on malformed/error events.
+
+The generic agent package exposes only the harness seam; Pi-specific knobs live
+under `pi` options.
+
+## Validation and safety
+
+Plugin discovery/preflight must:
+
+- reject invalid ids and duplicate effective plugin ids;
+- reject `.` paths, absolute paths, backslash paths, null-byte paths, and
+  traversal (`../`) paths;
+- validate explicit `front`, `server`, `pi.extensions`, `pi.skills`, and nested
+  `pi.packages[*]` resource filters;
+- perform realpath containment checks for existing paths;
+- check the nearest existing ancestor for missing paths under symlinked parents;
+- allow empty collection directories such as `.pi/extensions`;
+- report explicit plugin dirs without `package.json` as `MISSING_PACKAGE_JSON`;
+- surface invalid JSON/metadata through preflight errors rather than crashing
+  startup.
+
+Error reporting:
+
+- `BoringPluginAssetManager.load()` returns errors.
+- SSE emits `boring.plugin.error`.
+- `.error` files are written under the configured error root.
+- If a plugin id cannot be safely derived, use a stable `preflight-<hash>` id.
+
+## Agent docs for plugin creation
+
+The agent should learn the plugin API from workspace-owned Markdown docs:
+
+- `packages/workspace/src/server/docs/plugins.md`
+- `packages/workspace/src/server/docs/panels.md`
+- `packages/workspace/src/server/docs/bridge.md`
+
+`buildBoringSystemPrompt()` embeds those docs into the agent prompt when the
+workspace app has strong filesystem capability. This replaces the older separate
+agent-doc-embedding plan.
+
+## Current asset-serving caveat
+
+Hot-loaded front entries currently use Vite-style `/@fs/<absolute-path>` module
+URLs. `WorkspaceProvider` gates that path behind `frontPluginHotReload="vite"`,
+which defaults on only in dev, because Vite transforms TypeScript/TSX and React
+imports for the browser. Vite hosts that enable this mode must alias/dedupe the
+React family imports to the host app singleton so hook-based plugin panels run
+under the same dispatcher as the workspace shell.
+
+A production Fastify-only host needs a workspace-owned authenticated module
+asset endpoint/bundler before front plugin hot-loading can work without Vite.
+Until that endpoint exists, document front plugin hot-reload as development /
+workspace-dev-server scope.
+
+## Public API boundaries
+
+- `@hachej/boring-workspace/plugin` exports front authoring helpers,
+  `BoringFrontFactory`, and package metadata types.
+- Public barrels export `BoringPluginPackageJson`, not the server runtime
+  manifest shape.
+- Server-only runtime manifests stay under `server/agentPlugins`.
+- Shared/browser code must not import `@hachej/boring-agent` values.
+- `UiBridge.postCommand` remains the single command dispatch source.
+
+## Done in PR #18
+
+- Generic `/api/v1/agent/reload` route and `/reload` slash command.
+- Pluggable `AgentHarnessFactory` and Pi-scoped adapter options.
+- Workspace plugin scanner, asset manager, routes, and SSE front reload client.
+- `package.json#pi` / `package.json#boring` split.
+- `BoringFrontFactory` as the only runtime UI registration source.
+- Removal of obsolete front-side `agentTools` registration path.
+- Realpath-aware path validation and observable preflight errors.
+- Consolidated plan docs into this current architecture plan.
+
+## Not in scope for PR #18
+
+- `apps/boring-macro-v2`.
+- Password-reset smoke tests.
+- Production Fastify asset bundling for front plugin modules.
+- Cloud/multi-tenant plugin provisioning.

package/docs/plans/archive/RELOAD_PLUGGABILITY_PLAN.md

@@ -0,0 +1,174 @@
+# Reload pluggability — small, harness-agnostic seams
+
+Status: proposal, builds on PR #18.
+Scope: cut workspace ↔ Pi coupling in the reload path. Keep Pi as the default
+harness; let a future custom harness slot in without forking the workspace.
+
+## Two-track reload, no callback between tracks
+
+```
+POST /api/v1/agent/reload
+  ├─ Track A — workspace owns it
+  │    BoringPluginAssetManager.load() → SSE → front hot-swap
+  │    (errors → 422 with details; preserves last-good UI)
+  └─ Track B — harness owns it (optional)
+       harness.reloadSession?(sessionId)
+       Pi today; custom harness later; missing method = skip.
+```
+
+The two tracks do not call into each other. The workspace stops registering Pi
+extensions and stops mutating arrays that Pi later reads.
+
+## Single new seam: dynamic prompt provider
+
+```ts
+// packages/agent/src/shared/harness.ts
+interface AgentHarnessFactoryInput {
+  // ...existing
+  /**
+   * Optional source of additional system prompt content. Harness reads it
+   * each time it builds/rebuilds a session prompt. Returning `undefined`
+   * means "nothing to add right now". Workspace plugin layer supplies it;
+   * harness decides when to call it.
+   */
+  systemPromptDynamic?: () => string | undefined | Promise<string | undefined>
+}
+```
+
+- Workspace passes `systemPromptDynamic: () => aggregatePluginPrompts(boringAssetManager)`.
+- Pi adapter, internally, registers a `before_agent_start` extension that calls
+  the provider and appends to `event.systemPrompt`. Pi's native
+  `reloadSession` already re-fires `before_agent_start`, so /reload picks up
+  fresh plugin prompts with no extra lifecycle plumbing.
+- Other harnesses call the provider during their own session-init flow, or
+  ignore it.
+
+## Audit — other leaky abstractions in today's reload path
+
+### 1. Workspace mutates Pi-owned arrays in place (high)
+
+`createWorkspaceAgentServer.ts → syncPackageJsonPiOptions()` calls
+`piAdditionalSkillPaths.splice(...)`, `piPackages.splice(...)`,
+`piExtensionPaths.splice(...)` to mutate the *same* arrays that were passed
+into `pi: { ... }` on `createAgentApp`. This relies on Pi re-reading those
+arrays inside `reloadSession`. Two bad properties:
+
+- It's invisible coupling — neither side declares the contract.
+- A non-Pi harness has no reason to read those arrays again, so the mutation
+  is silently meaningless.
+
+**Fix:** replace the snapshot arrays with a single getter:
+
+```ts
+// On PiHarnessOptions:
+interface PiHarnessOptions {
+  // ...existing
+  getResources?: () => {
+    additionalSkillPaths?: string[]
+    packages?: WorkspacePiPackageSource[]
+    extensionPaths?: string[]
+    extensionFactories?: ExtensionFactory[]
+  }
+}
+```
+
+Pi's resource-loader rebuild reads from `getResources()` each time. Workspace
+provides one getter that merges static + package.json-discovered values. No
+splices. No shared array references.
+
+This is Pi-internal and doesn't change the workspace's public surface much,
+but it removes the most surprising piece of coupling in the file.
+
+### 2. Pi-shaped helpers live in the workspace package (medium)
+
+`createBoringPiPackageSource(workspaceRoot)` builds `{ source, skills:
+["skills/boring-plugin-authoring"] }` — a Pi `WorkspacePiPackageSource` shape.
+It lives in `createWorkspaceAgentServer.ts` and gets prepended to `piPackages`.
+
+A non-Pi harness has no use for this shape and shouldn't have to pretend it
+does. **Fix:** move the helper into the Pi-default composition path:
+
+```ts
+// pseudo
+const harnessFactory = opts.harnessFactory ?? composePiHarnessFactory({
+  workspaceRoot,
+  pi: opts.pi,
+  bundledSkillPackage: createBoringPiPackageSource(workspaceRoot),
+})
+```
+
+Workspace stops constructing Pi-shaped values when a non-default harness is
+in use.
+
+### 3. Pi's bundled skill package is provisioned unconditionally (medium)
+
+`createBoringPiPackageProvisioningContribution()` materializes
+`@hachej/boring-pi` into the child workspace's `node_modules` regardless of
+which harness is active. It should only run when the Pi adapter is the
+selected harness.
+
+**Fix:** make provisioning contributions a *harness adapter* responsibility
+too:
+
+```ts
+interface AgentHarnessFactory {
+  contributeProvisioning?(workspaceRoot: string): WorkspaceProvisioningContribution[]
+}
+```
+
+`composePiHarnessFactory` returns the Pi-bundled provisioning entry from
+`contributeProvisioning`. Workspace collects them generically. Custom
+harnesses contribute their own asset packages, or nothing.
+
+### 4. `/reload` response shape is harness-shaped (low)
+
+`reloadRoutes` returns `{ ok, sessionId, reloaded }`. `reloaded: boolean` is
+fine for Pi but doesn't carry boring-plugin reload outcomes (loaded count,
+errors-but-still-usable, last-good UI kept), so the front always renders the
+same "Agent plugins reloaded." message regardless of what really happened.
+
+**Fix:** widen the response without breaking the existing field:
+
+```ts
+{
+  ok: true,
+  sessionId,
+  reloaded,                          // keep — harness reload result
+  boring?: {                         // new — workspace track
+    loaded: number,
+    errors: { id: string; message: string }[],
+  }
+}
+```
+
+Front `/reload` handler uses `boring.errors` to surface compile failures
+without forcing a 422 (today's `throw` model conflates "reload broken"
+with "some plugins broken").
+
+### 5. Naming creep — `pi` in the workspace public API (defer)
+
+`WorkspaceAgentServerOptions.pi`, `WorkspacePiPackageSource`,
+`compactPiPackages` are exported by `@hachej/boring-workspace`. Hard to fix
+without a breaking change for plugin authors. Leave for the day a second
+harness lands; rename then under one umbrella (`harness?: { pi?: ..., ...}`)
+with one release of aliases.
+
+## Tradeoff acknowledged once
+
+Track A and Track B don't share a callback. If a custom harness wants
+plugin `systemPrompt`s to refresh inside a running session, it implements
+the `systemPromptDynamic` getter call into its own session-init flow. Pi
+already does this for free via `before_agent_start`. Other harnesses opt
+in or accept static prompts.
+
+## Execution order
+
+| # | Change | Risk | Notes |
+|---|--------|------|-------|
+| 1 | Add `systemPromptDynamic` on `AgentHarnessFactoryInput`. Pi adapter consumes it via an internal `before_agent_start` extension. Delete `packages/workspace/src/server/agentPlugins/boringPiExtension.ts` and its test. Workspace passes the getter. | Low — same runtime behavior, code moves. | Step 1 keeps every existing test green. |
+| 2 | Replace `syncPackageJsonPiOptions` array splices with a single `getResources()` getter on `PiHarnessOptions`. Pi reads it on every session rebuild. Workspace stops mutating shared arrays. | Low | Drops 30+ lines of mutation glue. |
+| 3 | Move `createBoringPiPackageSource` and Pi-bundled provisioning into a `composePiHarnessFactory` helper. Workspace constructs them only on the Pi-default path. | Low | Custom harness path now allocates zero Pi-shaped data. |
+| 4 | Widen `/api/v1/agent/reload` response with optional `boring` block. Update front `/reload` handler to surface compile errors without 422. | Low | Strictly additive on the wire. |
+| 5 (later) | Rename workspace public types to drop `pi` prefix; ship `harness: { pi?: ... }` namespacing with deprecated aliases. | Medium | Wait until a second harness justifies the churn. |
+
+Steps 1–4 are independent commits. Each can land on its own.