@osolmaz/pi-workflows 0.9.1 → 0.11.0

package/docs/plans/2026-08-19-human-decision-gates-plan.md

@@ -0,0 +1,296 @@
+---
+title: Add reusable human decision gates
+author: Onur Solmaz <2453968+osolmaz@users.noreply.github.com>
+date: 2026-08-19
+---
+
+# Add reusable human decision gates
+
+Pi Workflows must let any workflow stop after a proposal, ask the operator in Pi and Telegram, and continue from a verified human choice. A `replan` choice must collect the operator's exact alternative text, send it back to planning, and present the revised plan for another decision.
+
+The canonical behavior and public contracts are in [Human decisions](../HUMAN_DECISIONS.md). This plan covers the practical implementation in Pi Workflows without changing Pi core or adding a persistent operating-system service.
+
+## Outcome
+
+Workflow authors get three reusable levels:
+
+1. `humanDecision()` adds a custom typed decision gate to any graph.
+2. The included `plan-approval` workflow provides standard `continue`, `stop`, and `replan` exits.
+3. The standalone `autodoc` workflow records selected plans and is composed into autoimplement and monitor repair.
+
+Pi and Telegram implement one channel interface. Workflows address a named audience and contain no transport details. The first valid human response wins, stale and conflicting responses fail, and crash recovery creates one continuation.
+
+## Scope
+
+### Planning workflow contracts
+
+- Add a standalone typed `autodoc` built-in with `ready` and `blocked` exits.
+- Make autodoc locate an already selected plan, update or adopt canonical documents, run documentation checks, and return a documented-plan record.
+- Keep autodoc separate from solution selection and implementation.
+- Make autoimplement locate and record an existing plan from input, conversation context, or referenced documents.
+- Block autoimplement when no clear plan exists.
+- Skip autodoc only when plan discovery verifies current canonical documents or an explicit documentation receipt has a matching plan digest.
+- Route every evidence-driven revision through `autoplan`, then `autodoc`, before implementation resumes.
+- Never run initial autoplan only because the structured plan input is absent.
+
+### Public authoring API
+
+- Add `defineHumanChoices()`, `choice()`, and `textInput()`.
+- Add `humanDecision()` as a typed checkpoint helper.
+- Add `humanDecisionEdge()` with exhaustive TypeScript cases.
+- Infer a discriminated response union from the choice contract.
+- Validate the request and selected choice at runtime, including any choice input.
+- Export a reusable typed `plan-approval` workflow.
+
+### Engine and storage
+
+- Add a human decision contract to checkpoint definitions and snapshots.
+- Preserve the original workflow input when a human decision continues.
+- Make the accepted response the gate's output in the continuation.
+- Keep legacy checkpoint continuation unchanged.
+- Add immutable records for requests, deliveries, answer attempts, accepted answers, settlement results, and continuation results.
+- Accept the first valid response with a no-replace write.
+- Adopt identical retries and reject conflicting responses.
+- Derive one continuation identity from the accepted decision.
+- Reject stale responses by decision ID and canonical request digest.
+- Rebuild the pending-decision index from immutable records.
+
+### Host and channels
+
+- Add a `HumanDecisionChannel` interface outside the engine layer.
+- Add a Pi channel using documented TUI APIs.
+- Add a Telegram channel using the Bot API and private profiles.
+- Add named audience resolution and private channel configuration.
+- Add a setup command that writes private configuration with mode `600`, references an existing mode-`0600` token file, and verifies the Telegram bot without reading the token into a prompt or printing it.
+- Keep Telegram optional. Pi Workflows must start and run normal workflows without Telegram configuration.
+- Use one leased long-poll owner per Telegram profile across active Pi processes.
+
+### Documentation and display
+
+- Add the authoring API to `docs/workflows.md`.
+- Add human decision state to `docs/run-bundles.md`.
+- Add channel setup and recovery instructions.
+- Add a custom gate example and a composed plan approval example.
+- Show pending and accepted choices in TypeScript and Rust viewers.
+- Redact credentials, chat IDs, user IDs, message IDs, and private profile names from model-visible and public presentation.
+
+## Non-goals
+
+- Change Pi core or use undocumented Pi APIs.
+- Create a Telegram bot or copy a token without explicit source and destination approval.
+- Add Telegram-specific logic to monitor, autoimplement, autoplan, or user workflows.
+- Run an always-on service when no Pi process is active.
+- Promise exactly-once Telegram message creation after an ambiguous Bot API response.
+- Add arbitrary forms in the first release. Choice buttons and one text input cover the required flows.
+- Change existing checkpoints or historical run bundles.
+- Enable human approval by default in existing built-in workflows.
+
+## Design decisions
+
+### Extend checkpoints instead of the engine graph
+
+`humanDecision()` returns `nodeType: "checkpoint"`. The engine already knows how to park a run and create a continuation. The added contract defines who can answer, how to validate the answer, and how to deliver the request.
+
+The new behavior applies only when a checkpoint carries a human decision contract. Ordinary checkpoints keep using the continuation input supplied by `/workflow answer`.
+
+### Separate workflow intent from delivery
+
+A workflow declares its question and choices together with the subject and audience. Private host configuration maps the audience to Pi, Telegram, or later channels. This keeps workflow files portable and lets one Telegram adapter serve every workflow.
+
+### Verify the answer source in the host
+
+The model-facing workflow tool is not a human channel. It must reject `answer` while a human decision is waiting. The Pi UI path and Telegram adapter call an internal submission API that assigns the source and actor. Callers cannot supply a trusted source label themselves.
+
+### Keep exact response text
+
+The Telegram adapter accepts replan text only as a reply to the exact `ForceReply` prompt created for the pending decision and choice. It validates the numeric user and chat IDs and preserves the received text. Planning receives that string without a choice-classification model call.
+
+### Make answer acceptance exact
+
+Channel delivery can be retried or fail independently. Decision acceptance is one atomic no-replace operation. The accepted response and deterministic continuation identity prevent two channels from starting two continuations.
+
+### Handle Telegram delivery limits honestly
+
+Telegram Bot API `sendMessage` has no client idempotency key. The adapter writes an intent before sending. A confirmed response records the Telegram message ID. A timeout after a possible send becomes `unknown` and is not retried automatically. This prevents an automatic duplicate at the cost of a possibly missing Telegram notification. Pi remains available, and an operator can request a new delivery attempt.
+
+## Data model
+
+Add versioned JSON schemas for:
+
+- `human-decision-request-v1`;
+- `human-decision-delivery-v1`;
+- `human-decision-answer-attempt-v1`;
+- `human-decision-accepted-v1`;
+- `human-decision-cancellation-v1`;
+- `human-decision-resolution-v1`;
+- `human-decision-receipt-v1` for redacted continuation state;
+- `human-decision-settlement-v1`; and
+- `human-decision-continuation-v1`.
+
+Use canonical JSON for request and answer digests. Keep credential values and private channel configuration outside every schema.
+
+Store decision records under the existing workflow state root:
+
+```text
+decisions/<decision-id>/
+  request.json
+  deliveries/<channel>/<attempt-id>.json
+  answers/<attempt-id>.json
+  resolution.json
+  accepted.json
+  cancelled.json
+  settlements/<channel>/<attempt-id>.json
+  continuation.json
+```
+
+All final records use no-replace creation and adopt only byte-identical content. Temporary files remain in the same filesystem so rename and exclusive-create rules stay valid.
+
+A private SQLite index may track pending decisions, channel leases, Telegram update offsets, opaque callback IDs, and known message IDs. It is a projection and can be deleted and rebuilt from immutable records plus current private channel state.
+
+## Work plan
+
+### Contracts and compile-time API
+
+- Add generic choice and input types.
+- Build the response union from choice keys.
+- Add `humanDecision()` and exhaustive `humanDecisionEdge()`.
+- Extend workflow validation and definition snapshots.
+- Add compile-time fixtures for every invalid route shape.
+
+### Durable decision storage
+
+- Add schemas and generated types.
+- Add canonical request and response digest helpers.
+- Add the decision store with no-replace writes and identical-byte adoption.
+- Add a rebuildable pending-decision projection.
+- Test concurrent writers and partial records.
+
+### Safe human decision continuation
+
+- Detect a human decision at `continueRun()`.
+- Validate the accepted response through the node contract.
+- Preserve the parent's original workflow input.
+- Expose the response as the checkpoint output.
+- derive and adopt the continuation identity; and
+- leave the legacy checkpoint path unchanged.
+
+Test crashes before acceptance, after acceptance, during continuation creation, and after continuation completion.
+
+### Channel management
+
+- Add the channel interface and audience resolver in the extension layer.
+- Start channels during documented session lifecycle hooks.
+- Stop them idempotently during shutdown and reload.
+- Fan out one decision to all audience channels.
+- Send every channel response through the same acceptance method.
+- Settle all confirmed channel messages after acceptance, cancellation, or expiry.
+
+### Pi channel
+
+- Render pending decisions in the workflow widget.
+- Open a TUI choice view from the checkpoint notification and workflow command.
+- Open a text editor for text choices.
+- Mark answers from this path as Pi interactive answers.
+- Reject model-tool answers for human decisions.
+- Restore pending decisions after restart.
+
+### Telegram channel
+
+- Add private profile and credential-reference parsing.
+- Add a configuration command that accepts a private token-file reference and performs `getMe` verification without copying or displaying the token.
+- Add the shared long-poll lease and durable update offset.
+- Render choice buttons with opaque callback IDs.
+- Implement the bound `ForceReply` text flow.
+- Verify every bound reply field, including the actor, chat, reply message, decision ID, selected choice, and request digest.
+- Record every delivery outcome without leaking request URLs or tokens.
+- Edit confirmed messages after a decision settles.
+
+Use a fake Bot API in all automated tests. Do not require a real token or network access.
+
+### Autodoc and autoimplement
+
+- Define the typed standalone `autodoc` built-in workflow.
+- Add a durable documented-plan result shared by autodoc, autoimplement, and monitor.
+- Replace autoimplement's missing-input redesign route with existing-plan discovery.
+- Block when discovery finds no clear plan.
+- Skip autodoc for current canonical documents.
+- Route undocumented and revised plans through autodoc.
+- Keep autoplan only on evidence-driven redesign edges.
+
+### Reusable plan approval
+
+- Define the typed `plan-approval` built-in workflow.
+- Return named `continue`, `stop`, and `replan` exits.
+- Include the accepted decision receipt in every exit.
+- Return exact instructions from `replan`.
+- Add an example that loops through `autoplan`, plan approval, and implementation.
+- Keep monitor and autoimplement behavior unchanged unless their input requests the approval workflow.
+
+### Viewers and documentation
+
+- Display the audience label, choice labels, waiting state, and accepted choice.
+- Keep actor and transport details out of public or model-visible views.
+- Document setup, authoring, cancellation, recovery, Telegram delivery ambiguity, and the no-service limitation.
+- Update the plan with meaningful implementation departures.
+
+## Implementation note
+
+The setup command uses an existing mode-`0600` token file instead of collecting and copying a token. This keeps credential ownership with the existing private store and follows the no-copy credential boundary while still verifying the bot with `getMe`.
+
+## Acceptance criteria
+
+- Autodoc runs alone, adopts current documents, updates stale documents, and blocks without a selected plan.
+- Autoimplement uses the plan already present in context or documentation and never devises merely because `input.plan` is absent.
+- Every revised plan is documented before implementation resumes.
+- A TypeScript workflow can define custom human choices and route them exhaustively.
+- A text choice returns the exact submitted text.
+- The engine and run snapshot still identify the node as a checkpoint.
+- The workflow tool cannot answer a protected human decision.
+- Pi and Telegram can receive the same decision through one audience profile.
+- The first concurrent valid answer wins and creates one continuation.
+- An identical retry is adopted; a conflicting or stale answer is rejected.
+- The original workflow input survives a human-decision continuation.
+- Old checkpoints and old bundles pass their existing tests unchanged.
+- The Telegram adapter accepts text only from the bound reply and approved numeric actor.
+- A missing Telegram profile leaves Pi decisions usable.
+- A Telegram timeout does not cause an automatic duplicate send.
+- Restart recovery adopts confirmed deliveries and accepted continuations.
+- The standard plan approval workflow loops through replan and requires approval for the revised digest.
+- No credential or private channel value appears in source, fixtures, run bundles, logs, screenshots, or model-visible output.
+
+## Verification
+
+Run focused tests while implementing:
+
+```bash
+npx vitest run test/human-decision-api.test.ts test/human-decision-store.test.ts
+npx vitest run test/human-decision-engine.test.ts test/run-resume.test.ts
+npx vitest run test/pi-decision-channel.test.ts test/telegram-decision-channel.test.ts
+npx vitest run test/plan-approval.test.ts test/composition.test.ts
+```
+
+Run all repository gates before review:
+
+```bash
+npm run check
+npm run test:e2e
+cargo test --manifest-path tui/Cargo.toml
+npx slophammer-ts@latest dry .
+npx slophammer-ts@latest check . --only ts.dependency-boundaries-required
+npx -y @simpledoc/simpledoc check
+git diff --check
+```
+
+Run Pi Reviewer against the pushed branch. Fix every P0 and P1 finding and rerun it. Address valid P2 findings, but do not rerun review solely because of a P2-only change. Open or update a pull request and leave it unmerged unless merge is separately authorized.
+
+## Release
+
+This work adds compatible public APIs and additive persisted records. Release it as the next minor pre-1.0 version after the compatibility checks, viewer tests, real-Pi tests, and recovery tests pass.
+
+## Contract impact
+
+- **Session state:** normal workflow messages and interactive decision results.
+- **Other persistent data:** additive decision records, a rebuildable private channel index, and private channel configuration.
+- **Pi internals:** none.
+- **Public Pi API:** documented extension lifecycle plus command and UI methods only.
+- **Public Pi Workflows API:** typed human choices, `humanDecision()`, `humanDecisionEdge()`, the channel interface, and the `plan-approval` workflow.

package/docs/plans/2026-08-19-human-decision-presentations-plan.md

@@ -0,0 +1,172 @@
+---
+title: Add readable human decision presentations
+author: Onur Solmaz <2453968+osolmaz@users.noreply.github.com>
+date: 2026-08-19
+---
+
+# Add readable human decision presentations
+
+Pi Workflows currently sends a structured human decision body to Telegram with
+`JSON.stringify()`. A plan approval therefore reaches the operator as machine
+JSON. Pi also shows only the decision title in its basic selection prompt.
+
+The operator must receive clear text in every channel. Structured data must
+remain the durable source for validation, routing, and audit evidence. The
+[human decision presentations specification](../HUMAN_DECISION_PRESENTATIONS.md)
+defines the selected contract.
+
+## Outcome
+
+Add one versioned, channel-independent presentation format beside the canonical
+subject. Pi and Telegram render the same complete content with their own native
+layout. The visible presentation and machine subject are both bound to the
+accepted decision.
+
+The design uses the existing checkpoint engine, decision store, channel
+interface, and documented Pi extension APIs. It does not require Pi core,
+another service, or a Telegram change.
+
+## Scope
+
+- Add `DecisionPresentation v1` and `HumanDecisionRequest v2` schemas and types.
+- Add deterministic presentation normalization and digests.
+- Keep the current typed subject as machine truth.
+- Require an explicit presentation for new structured subjects.
+- Add a reusable plan presenter.
+- Render complete readable messages in Pi and Telegram.
+- Add durable multipart delivery evidence and safe recovery.
+- Update TypeScript and Rust viewers.
+- Preserve v1 definitions and immutable run bundles.
+
+## Non-goals
+
+- Do not modify Pi core or use undocumented Pi APIs.
+- Do not change Telegram or create a delivery relay.
+- Do not add a template language or general rich text system.
+- Do not derive new operator text from arbitrary v2 subject data.
+- Do not rewrite v1 requests, accepted answers, or run bundles.
+- Do not store Telegram identifiers or credentials in public durable records.
+
+## Implementation
+
+### 1. Add the durable contracts
+
+Create schemas and TypeScript types for the presentation, v2 request, and
+multipart delivery evidence. Keep all v1 schemas and readers unchanged.
+
+Generate and validate canonical examples. Reject unknown blocks, object-valued
+display fields, missing summaries, missing presentations, and excessive
+content.
+
+### 2. Add deterministic normalization
+
+Add a workflow-core module that validates the flat presentation blocks,
+normalizes line endings, enforces the specification limits, and computes the
+presentation digest. Keep it independent of Pi and Telegram.
+
+Add a v1 compatibility presenter. A string body becomes a paragraph. An object
+body becomes readable sections, labels, fields, and lists in stable key order.
+The adapter reads only the historical display body.
+
+### 3. Update the authoring API and request identity
+
+Add the preferred `subject` plus `presentation` request form. Keep the current
+`body` form as a deprecated compatibility overload. The preferred form fails
+validation when either value is missing.
+
+For v2, compute separate subject and presentation digests and bind the request
+revision, subject, visible presentation, choices, and input prompts into the
+overall request digest. Select the v1 or v2 digest algorithm from the stored
+request schema.
+
+### 4. Add the plan presenter
+
+Create one reusable presenter for plan approvals. Derive its summary and
+sections from the same typed plan stored in the subject. Use it in the built-in
+plan approval path and composed workflows. Do not serialize missing or unknown
+fields.
+
+### 5. Narrow the channel boundary
+
+Normalize the stored request before channel dispatch. Give each channel only
+the normalized presentation and typed choices. Remove direct channel access to
+the subject and remove the `JSON.stringify()` fallback.
+
+### 6. Render complete Telegram messages
+
+Render plain text with no parse mode. Precompute all message parts, split at
+readable boundaries, add deterministic part numbers and a short fingerprint,
+and put buttons on the final part only.
+
+Write intent before sending and one durable content receipt for each confirmed
+part. Keep remote IDs in the private projection. Stop automatic delivery after
+an ambiguous send. Never clip approval content.
+
+### 7. Add the full Pi decision view
+
+Use documented `ctx.ui.custom()` APIs and existing TUI components to show the
+complete presentation, fingerprint, choices, and text prompt. Support wrapping,
+scrolling, resizing, cancellation, and theme changes. Preserve the abort signal
+that closes Pi when another channel wins.
+
+### 8. Update recovery and viewers
+
+Recover confirmed multipart sends without duplication. Resume only when the
+next part is provably unsent. Keep one accepted answer and one continuation.
+
+Update the TypeScript and Rust viewers to show the human presentation by
+default and machine data as separate detail. Continue to render v1 through the
+compatibility presenter without changing stored bytes.
+
+### 9. Verify compatibility and failure handling
+
+Add tests for:
+
+- presentation schemas, normalization, bounds, Unicode, and canonical digests;
+- subject and presentation changes that reject stale answers;
+- old definitions, pending v1 decisions, and accepted v1 decisions;
+- complete and sparse plan presentations;
+- Pi and Telegram content parity;
+- unsafe markup-like text and terminal control characters;
+- single-part and multipart Telegram messages;
+- partial, ambiguous, resumed, and settled delivery;
+- Pi scrolling, input, resize, cancellation, and external settlement;
+- one answer and one continuation under concurrency and crash injection;
+- TypeScript and Rust viewer parity; and
+- absence of raw JSON subject serialization in channel output.
+
+## Rollout
+
+The preferred authoring path starts emitting v2 after release. Existing
+body-based workflows continue to create and read v1 until a separate removal is
+approved. No migration runs. Historical records remain immutable.
+
+A live Telegram smoke can use an already configured private audience profile.
+The test must not print, copy, or persist credentials. Fake Bot API and real-Pi
+end-to-end tests remain the required automated evidence.
+
+## Acceptance criteria
+
+- New structured decisions cannot reach a channel without an explicit readable
+  presentation.
+- Telegram and Pi show the same complete decision meaning.
+- Telegram never shows raw JSON or silently truncates approval content.
+- Pi shows the full decision instead of only its title.
+- The accepted record binds the exact subject and visible presentation.
+- V1 definitions and bundles remain readable without rewrites.
+- Ambiguous Telegram sends fail closed without duplicate messages or
+  continuation.
+- All repository, documentation, privacy, TypeScript, Rust, and real-Pi checks
+  pass.
+
+## Verification
+
+```bash
+npm run check
+npm run test:e2e
+cargo test --manifest-path tui/Cargo.toml
+npx slophammer-ts@latest dry .
+npx slophammer-ts@latest check . --only ts.dependency-boundaries-required
+npx -y @simpledoc/simpledoc check
+npm pack --dry-run --json
+```

package/docs/plans/2026-08-19-provider-compatible-workflow-tool-schema-plan.md

@@ -0,0 +1,45 @@
+---
+title: Provider-compatible workflow tool schema plan
+author: Onur Solmaz <2453968+osolmaz@users.noreply.github.com>
+date: 2026-08-19
+---
+
+# Provider-compatible workflow tool schema plan
+
+## Goal
+
+Fix the workflow tool schema so strict OpenAI-compatible providers accept it without weakening action validation. Apply the fix to the interactive Pi extension and the headless RPC bridge in PR #33.
+
+## Requirements
+
+- Every provider-facing tool schema has `type: "object"` at its root.
+- Provider-facing schemas do not use root unions or conditional object schemas.
+- Each action keeps its current required fields and rejects fields for other actions.
+- The exact action schemas remain the source for TypeScript types and runtime validation.
+- Invalid input is rejected before workflow state changes or RPC output.
+- The existing action names and input fields do not change.
+
+## Implementation
+
+1. Define the exact action object schemas in a shared, Pi-independent workflow module.
+2. Build the interactive and submission-only schemas from those action objects.
+3. Generate flat provider-facing object schemas from the same action definitions.
+4. Parse each tool call against the exact action schema before dispatch.
+5. Use the shared submission schema in the RPC bridge.
+6. Add unit tests for valid actions, missing fields, wrong-action fields, nested update data, and both provider schema roots.
+7. Add an integration test that sends the schemas through Pi's OpenAI Completions adapter to a strict local endpoint.
+
+## Non-goals
+
+- Do not change Pi core or provider adapters.
+- Do not add provider-specific branches.
+- Do not add a schema version, new action format, or compatibility path.
+- Do not change persisted workflow data.
+
+## Acceptance
+
+- The strict local endpoint accepts both workflow tool schemas.
+- Runtime parsing rejects malformed calls before execution.
+- `npm run check`, `npm run test:e2e`, Rust tests, Slophammer checks, and `git diff --check` pass.
+- Pi Reviewer reports no P0 or P1 findings.
+- PR #33 CI passes before merge.