pentesting 0.100.3 → 0.100.5

package/ARCHITECTURE.md

@@ -27,6 +27,9 @@ stays intentionally short and usage-focused.
   access (policy-based today; OS sandboxing is roadmap).
 - **Intelligent Queuing** — Queued inputs are deduplicated and prioritized live;
   obsolete tasks are preempted, not FIFO-replayed.
+- **Prompt Ownership** — Prompt mode owns Enter submission. Buffered replay may
+  preserve typed-ahead input, but live terminal control responses, redraws, and
+  resize noise are not allowed to turn a normal Enter into a pasted newline.
 - **Lab Session Multiplexing** — `shell-listener` runs multiple authorized TCP
   sessions with per-session routing, identity tags, verified PTY upgrade state,
   transcripts, command ledgers, replay, and orphaned evidence recovery.
@@ -66,6 +69,7 @@ Plain view:
 ```text
 user input
   -> CLI / interactive prompt loop
+  -> terminal-session prompt ownership + buffered replay boundary
   -> BuilderAPI facade
   -> BuilderApp classification and strategy
   -> active agent or delegated subagent
@@ -85,6 +89,27 @@ There is no fixed persona. The runtime derives a profile per request, optionally
 overlays a named autonomy profile, then reuses tool scope, phase, and memory
 weighting.
 
+CTF frontier exploration keeps the global `[exploration]` default off, then
+enables the effective CTF profile when `engagement.kind = "ctf"` or a
+`flag_format` is configured. That profile selects `ctf-competition` autonomy,
+turns on exploration, requires verified `flag_check` evidence before completion,
+and auto-continues the TUI loop after `/goal`. See `docs/config/ctf.toml`,
+`docs/plans/2026/07/08/PLAN_CtfFrontierExploration_2026-07-08.md`, and the
+decision record beside it.
+
+The shipped design is advisor-guided, not scheduler-owned: `decide_next` is a
+pure traversal recommendation over `EvidenceKind::Lead` events, while dispatch
+uses the existing delegated-task fan-out. There is no detached task registry or
+second runtime scheduler in the default implementation. Phase 10 is pinned by
+`orch_spec::exploration_e2e`, which exercises the flag-off path and the
+breadth/depth/pivot/reseed/merge flow without launching real subagents.
+
+Reliability guards sit below the profile layer. Provider protocol capabilities
+default native tool support for OpenAI/Anthropic-compatible responses on
+registry misses, unparsed tool-shaped markup becomes retryable instead of a
+false completion, raw control markup is scrubbed from the stream, and the
+markdown renderer bounds very large no-newline buffers.
+
 ```mermaid
 flowchart LR
     R[Request] --> GEN["Derive<br/>dynamic profile"]
@@ -145,8 +170,9 @@ definition under `crates/builder_repo/src/agents/`.
 ## Named Autonomy Profiles
 
 Optionally pin the top-level profile (`autonomy_profile = "ctf-competition"`);
-leave unset for classifier-driven defaults. Delegated subagent contracts stay
-intact. Defined in `crates/builder_domain/src/autonomy_profile.rs`.
+leave unset for classifier-driven defaults. CTF engagement config auto-selects
+the CTF profile when no explicit profile is set. Delegated subagent contracts
+stay intact. Defined in `crates/builder_domain/src/autonomy_profile.rs`.
 
 | Profile | Purpose |
 | --- | --- |
@@ -157,9 +183,10 @@ intact. Defined in `crates/builder_domain/src/autonomy_profile.rs`.
 
 ## Engagement Metadata
 
-Runs can attach typed engagement context — scope, phase, tags, and standard refs
-(PTES, MITRE ATT&CK, OWASP, CWE/CAPEC, NIST CSF, CIS Controls) — to workflow
-metadata. `/workflow report` exports it as a Markdown handoff.
+Runs can attach typed engagement context — scope, phase, tags, flag format,
+flag-evidence requirement, and standard refs (PTES, MITRE ATT&CK, OWASP,
+CWE/CAPEC, NIST CSF, CIS Controls) — to workflow metadata. `/workflow report`
+exports it as a Markdown handoff.
 
 ## Memory & Knowledge
 
@@ -316,14 +343,14 @@ npm install -g pentesting
 
 ## Supported Runtime Targets
 
-The npm launcher currently resolves managed native release assets for the
-targets below. Other operating systems can run the Docker image or use
+The npm launcher currently resolves a managed native release asset for Linux
+x64. Other operating systems, including Apple Silicon Macs, Linux arm64,
+Windows, and Android, can run the multi-architecture Docker image or use
 `PENTESTING_BIN` to point at a locally built compatible binary.
 
 | OS | CPU | Release asset |
 | --- | --- | --- |
 | Linux | x64 | `pentesting-x86_64-unknown-linux-musl` |
-| Android | arm64 | `pentesting-aarch64-linux-android` |
 
 `/update` (and the npm postinstall) resolves the matching asset for the current
 supported OS/CPU target from `agnusdei1207/pentesting-public`.
@@ -332,22 +359,19 @@ supported OS/CPU target from `agnusdei1207/pentesting-public`.
 
 Pentesting reads a user-global config from `~/.pentesting/.pentesting.toml`,
 overridden by project-local `.pentesting.toml` files discovered by walking up
-from the working directory. The storage config is intentionally small:
-
-```toml
-[storage]
-backend = "local"                            # Local md/fs runtime state
-```
+from the working directory. Local storage is the default; normal setup only
+needs the public model endpoint environment.
 
 The same shape is represented in `builder.schema.json`, `.pentesting.toml`, and
-the interactive `/config` UI.
-
-Environment variables use the `PENTESTING_` prefix, with `__` for nested keys —
-e.g. `PENTESTING_SESSION__MODEL_ID=...`. Set `PENTESTING_CONFIG` to override the
-global config directory.
+the interactive `/config` UI. Set `PENTESTING_CONFIG` to override the global
+config directory.
 
 | Variable | Description |
 | --- | --- |
+| `OPENAI_API_KEY` | API key or gateway token for OpenAI-compatible providers. |
+| `OPENAI_BASE_URL` | OpenAI-compatible base URL, for example `https://api.openai.com/v1` or a self-hosted gateway. |
+| `OPENAI_MODEL` | Default model id. |
+| `OPENAI_MAX_TOKENS` | Optional output-token override; accepts values like `4096`, `32k`, or `1M`. |
 | `PENTESTING_BIN` | Use an already-installed Builder binary instead of the managed download. |
 | `PENTESTING_PRODUCT_NAME` | Runtime banner label. The `pentesting` launcher sets this to `pentesting` automatically. |
 | `PENTESTING_REPO` | Override the public release repo used for binary downloads. Defaults to `agnusdei1207/pentesting-public`. |

package/README.md

@@ -17,7 +17,7 @@ The runtime—not the model—owns routing, tool sandboxing, verification, compl
 
 ## ✅ Capabilities
 
-Agent-loop essentials at parity with leading coding agents, plus a security-focused layer on top.
+Agent-loop essentials plus a security-focused runtime layer.
 
 | Area | What it does |
 | --- | --- |
@@ -25,62 +25,63 @@ Agent-loop essentials at parity with leading coding agents, plus a security-focu
 | Tool guardrails | Read-before-edit enforcement, repeat-call guard, policy-gated file/shell/network |
 | Permissions | Allow / Deny / Confirm, with once/always decisions persisted |
 | Context compaction | Automatic micro-pruning + model-triggered summarization, pre/post hooks |
-| Interrupt & steering | ESC to interrupt mid-turn, queue follow-ups while it works |
+| Interrupt & steering | ESC to interrupt mid-turn, queue follow-ups while it works, and keep Enter submissions deterministic across PTY redraw/control-response noise |
 | Reverse-shell control plane | Accept and track multiple authorized callbacks, tag/probe sessions, send/observe/run commands, verify PTY upgrades, and keep transcript/ledger/evidence records |
 | PTY session control | Treat upgraded shells as managed PTY sessions with resize gating, transcript capture, FD accounting, and deterministic close/revoke/gc reclamation |
-| Subagents | Parallel delegated tasks, budget-capped, role-specific personas |
+| Frontier-guided exploration | Persistent, deduplicated lead frontier: the runtime enforces pivots and caps, and recommends breadth-first enumeration vs depth-first exploitation while the model drives |
+| Subagents | Broad-clone delegates with category focus overlays; independent deep-specialist leads (crypto/rev/pwn) run in the same parallel batch so the main line keeps moving |
 | MCP & skills | Dynamic MCP tool discovery, layered markdown skills |
 | Memory | Ebbinghaus-style decaying notes + hybrid lexical/semantic/graph retrieval |
 
-**Beyond the baseline:** evidence-gated completion adjudication (Complete / Replan / Blocked), a CTF-oriented reverse-shell/PTY control plane (multi-session tracking, helper-only `pty_upgrade` plus verified `upgrade`, transcript/history/replay, lifecycle/FD management), and CTF-grade verification tools (flag check, best-of-N verify, findings ledger).
-
 ---
 
 ## 🚀 Quick Start
 
-Install globally, or run without installing via your package manager of choice:
+Install globally, or run once with a Node package runner:
 
 ```bash
 npm install -g pentesting && pentesting   # install
 npx pentesting                            # or run once, no install
 pnpm dlx pentesting
 yarn dlx pentesting
-bunx pentesting
-deno run npm:pentesting
 ```
 
+Requires Node.js 18.18+. Use npm/npx/pnpm/yarn; Bun and Deno runners are not supported.
+
 ### 🐳 Run with Docker
 
-No local Rust toolchain or native binary needed — the image ships the full
-runtime plus the offensive testing toolchain:
+No local Rust toolchain needed:
 
 ```bash
-export ANTHROPIC_AUTH_TOKEN="sk-..."                    # provider token
-export ANTHROPIC_API_KEY="$ANTHROPIC_AUTH_TOKEN"        # keep both for Anthropic-compatible gateways
-export ANTHROPIC_BASE_URL="https://api.minimax.io/anthropic"
-export ANTHROPIC_MODEL="MiniMax-M2.7"
-export PENTESTING_SESSION__MODEL_ID="$ANTHROPIC_MODEL"
+export OPENAI_API_KEY="sk-..."                 # provider API key or gateway token
+export OPENAI_BASE_URL="https://api.openai.com/v1"
+export OPENAI_MODEL="gpt-4o"
+export OPENAI_MAX_TOKENS="4096"
 
 docker run -it --rm \
-  -v "$(pwd):/workspace" -w /workspace \
-  -e ANTHROPIC_AUTH_TOKEN \
-  -e ANTHROPIC_API_KEY \
-  -e ANTHROPIC_BASE_URL \
-  -e ANTHROPIC_MODEL \
-  -e PENTESTING_SESSION__MODEL_ID \
+  -v "$(pwd):/workspace" \
+  -v pentesting-config:/root/.pentesting \
+  -w /workspace \
+  -e OPENAI_API_KEY \
+  -e OPENAI_BASE_URL \
+  -e OPENAI_MODEL \
+  -e OPENAI_MAX_TOKENS \
   agnusdei1207/pentesting:latest
 ```
 
+Provider environment variables are optional for startup. If they are omitted,
+the interactive TUI still launches and lets you configure/login inside the
+persistent `pentesting-config` volume.
+
 Or via Docker Compose:
 
 ```bash
 PENTESTING_PROJECT_DIR=/path/to/project \
-ANTHROPIC_AUTH_TOKEN=sk-... \
-ANTHROPIC_API_KEY=sk-... \
-ANTHROPIC_BASE_URL=https://api.minimax.io/anthropic \
-ANTHROPIC_MODEL=MiniMax-M2.7 \
-PENTESTING_SESSION__MODEL_ID=MiniMax-M2.7 \
-docker compose run pentesting
+OPENAI_API_KEY=sk-... \
+OPENAI_BASE_URL=https://api.openai.com/v1 \
+OPENAI_MODEL=gpt-4o \
+OPENAI_MAX_TOKENS=4096 \
+docker compose run --rm pentesting
 ```
 
 ### Common commands
@@ -92,7 +93,7 @@ pentesting shell-listener --bind 127.0.0.1 --port 4444   # Authorized reverse-sh
 pentesting --version
 ```
 
-Use `shell-listener` in authorized CTF/lab work to accept multiple callbacks, label/tag sessions, probe remote identity, run sentinel-tracked commands, verify dumb-shell PTY upgrades, attach/detach without closing, and retain raw logs, transcripts, command history, replay data, and evidence manifests.
+Use `shell-listener` in authorized CTF/lab work to accept callbacks, tag sessions, run sentinel-tracked commands, verify PTY upgrades, and retain transcripts/evidence.
 
 Short version: the agent can catch a dumb reverse shell, upgrade it to a verified PTY, pivot to a second callback, and reclaim both sessions cleanly with `close` / `revoke` / `gc`.
 
@@ -100,7 +101,7 @@ Inside a session, `/help` lists every command and `/update` pulls the latest rel
 
 ### 🔌 Reverse-shell & PTY sessions
 
-Catch and manage authorized shell callbacks like a session manager — **one listener, many targets, full audit trail, no leaked file descriptors.**
+Catch and manage authorized shell callbacks: one listener, many targets, audited cleanup.
 
 ```bash
 pentesting shell-listener --port 4444             # 1. Catch callbacks (tracks many at once)
@@ -120,45 +121,82 @@ pentesting shell-session revoke --session 1       # 5. Hard-close: reclaim FDs +
 
 ## ⚙️ Configuration
 
-Pentesting reads `~/.pentesting/.pentesting.toml`, overridden by a project-local `.pentesting.toml`:
-
-```toml
-[storage]
-backend = "local"   # Local md/fs runtime state
-```
-
-Environment variables use the `PENTESTING_` prefix (`__` for nested keys, e.g. `PENTESTING_SESSION__MODEL_ID=...`). Full variable list and the legacy `BUILDER_*` aliases are documented in [`ARCHITECTURE.md`](ARCHITECTURE.md#configuration).
+Pentesting uses local storage by default. For normal use, configure only the model endpoint:
 
 ### Example `.env` / shell setup
 
 ```bash
-# Model provider credentials — set whichever provider(s) you use
-export ANTHROPIC_API_KEY="sk-ant-..."
-export ANTHROPIC_AUTH_TOKEN="$ANTHROPIC_API_KEY"         # token used by Anthropic-compatible providers
-export ANTHROPIC_BASE_URL="https://api.minimax.io/anthropic"
-export ANTHROPIC_MODEL="MiniMax-M2.7"
+# OpenAI-compatible provider or gateway
 export OPENAI_API_KEY="sk-..."
-export OPENROUTER_API_KEY="sk-or-..."
+export OPENAI_BASE_URL="https://api.openai.com/v1"
+export OPENAI_MODEL="gpt-4o"
+export OPENAI_MAX_TOKENS="4096"
 
 # Runtime overrides (optional)
-export PENTESTING_SESSION__MODEL_ID="$ANTHROPIC_MODEL"  # default model for new sessions
 export PENTESTING_CONFIG="$HOME/.config/pentesting"      # move the global config dir
 export PENTESTING_SKIP_DOWNLOAD=1                        # skip postinstall binary download (CI)
 ```
 
-With Docker, pass the same variables through with `-e`:
+The TUI pre-fills output max tokens from known model names/provider metadata. `OPENAI_MAX_TOKENS` and the TUI field are explicit overrides for custom/self-hosted models.
+
+### Frontier-guided CTF exploration
+
+The global frontier engine default stays off, but CTF-shaped engagement config
+auto-applies the CTF runtime profile: `ctf-competition` autonomy, runtime
+exploration, autonomous continuation, and a completion gate that requires
+verified `flag_check` evidence. A ready-to-copy profile lives at
+`docs/config/ctf.toml`:
+
+```toml
+[engagement]
+kind = "ctf"
+flag_format = "flag\\{[^}]+\\}"
+require_flag_evidence = true
+
+[exploration]
+enabled = true
+max_frontier_nodes = 256
+max_breadth = 4
+```
+
+With the profile active, the runtime stores `explore` leads as
+`EvidenceKind::Lead` events, carries prior-run frontier/findings/verified flags
+forward, deduplicates leads, auto-pivots dead probes, enforces graph/cap limits,
+and injects a breadth/depth recommendation into mission control. The model still
+drives the work through `explore` and `task`; heavy independent leads such as
+crypto/rev/pwn ride the same parallel delegation batch rather than a detached
+background scheduler. Operators can inspect the current graph with `/frontier`,
+and the status row shows compact frontier progress during dispatch.
+
+Malformed tool-call output is fail-loud: unparsed tool-shaped markup is retried
+instead of accepted as a final answer, the TUI suppresses raw control markup, and
+large no-newline streams are bounded so rendering cannot hang behind usage-only
+progress ticks.
+
+The shipped contract is covered by
+`cargo test -p builder_app orch_spec::exploration_e2e`: it verifies the
+flag-off regression path plus breadth enumeration, depth descent, sibling
+breadth, dead-probe pivot, global reseed, seed digest, broad-clone overlay, and
+child result promotion.
+
+With Docker, mount both your project and the persistent runtime config volume, then pass the same variables through with `-e`:
 
 ```bash
 docker run -it --rm \
-  -v "$(pwd):/workspace" -w /workspace \
-  -e ANTHROPIC_AUTH_TOKEN \
-  -e ANTHROPIC_API_KEY \
-  -e ANTHROPIC_BASE_URL \
-  -e ANTHROPIC_MODEL \
-  -e PENTESTING_SESSION__MODEL_ID \
+  -v "$(pwd):/workspace" \
+  -v pentesting-config:/root/.pentesting \
+  -w /workspace \
+  -e OPENAI_API_KEY \
+  -e OPENAI_BASE_URL \
+  -e OPENAI_MODEL \
+  -e OPENAI_MAX_TOKENS \
   agnusdei1207/pentesting:latest
 ```
 
+Those variables only pre-fill provider settings. Leaving them unset must not
+block the container from launching the TUI; configure the provider interactively
+inside the container instead.
+
 > **Note:** The command you run is always **`pentesting`**. The internal engine is **`builder`** — `pentesting` downloads and runs it under the hood; the engine name never surfaces in normal use.
 
 ---
@@ -166,6 +204,7 @@ docker run -it --rm \
 ## 📖 Documentation
 
 * [`ARCHITECTURE.md`](ARCHITECTURE.md) — Runtime flow, agent team, memory model, crate map, tool surface, and supported targets.
+* Internal layout: service traits are split under `builder_app/src/services/*`; OpenAI request conversion is isolated under `dto/openai/request/*`.
 * [Public site](https://agnusdei1207.github.io/pentesting-public/) — Landing page and public runtime entry surface.
 * [`compose.yaml`](https://github.com/agnusdei1207/pentesting-public) — Docker Compose facade for pentesting sessions.
 

package/lib/runtime.mjs

@@ -14,15 +14,9 @@ const MANAGED_BINARY_DIR = path.join(PACKAGE_ROOT, "vendor");
 const MANAGED_BINARY_MANIFEST = path.join(MANAGED_BINARY_DIR, "pentesting-manifest.json");
 
 const RELEASE_TARGETS = {
-  // Native binaries published as release assets. Linux uses musl (static, covers
-  // x64/arm64); android is the cross-built target. macOS/Windows are not
-  // cross-buildable from the Linux release host, so they are served by the
-  // Docker image (see the Unsupported-platform error below).
-  "android:arm64": {
-    assetName: "pentesting-aarch64-linux-android",
-    binaryFileName: "builder",
-    label: "android-arm64",
-  },
+  // Native binaries published as release assets. Linux x64 uses musl (static).
+  // macOS, Windows, Linux arm64, and Android are served by the multi-arch Docker
+  // image (see the unsupported-platform error below).
   "linux:x64": {
     assetName: "pentesting-x86_64-unknown-linux-musl",
     binaryFileName: "builder",
@@ -58,10 +52,10 @@ export function resolveReleaseAsset(platform = process.platform, arch = process.
   if (!target) {
     const supported = Object.keys(RELEASE_TARGETS).join(", ");
     throw new Error(
-      `No native pentesting binary for '${platform}/${arch}'. ` +
+        `No native pentesting binary for '${platform}/${arch}'. ` +
         `Native targets: ${supported}. ` +
-        `On macOS/Windows run it via Docker instead: ` +
-        `docker run -it --rm -v "$(pwd):/workspace" agnusdei1207/pentesting`,
+        `On macOS/Windows/Linux arm64/Android run it via Docker instead: ` +
+        `docker run -it --rm -v "$(pwd):/workspace" -v pentesting-config:/root/.pentesting -w /workspace -e OPENAI_API_KEY -e OPENAI_BASE_URL -e OPENAI_MODEL -e OPENAI_MAX_TOKENS agnusdei1207/pentesting`,
     );
   }
   return target;

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "pentesting",
-  "version": "0.100.3",
-  "builderReleaseTag": "v0.100.0",
+  "version": "0.100.5",
+  "builderReleaseTag": "v0.100.5",
   "description": "pentesting — coding agent runtime with audited reverse-shell capture, verified PTY session upgrades, pivot control, evidence transcripts, and safe session reclamation.",
   "license": "MIT",
   "author": "agnusdei1207",
@@ -52,8 +52,9 @@
     "audit": "bash scripts/audit-project.sh",
     "test": "node --test tests/*.test.mjs",
     "preflight:local": "bash scripts/preflight-local.sh",
+    "refactor:stability": "bash scripts/refactor-stability.sh",
     "verify": "npm run consistency && npm run test",
-    "check": "npm run verify",
+    "check": "npm run check:docker",
     "pentesting:help": "bash scripts/pentesting-release-help.sh",
     "pentesting:status": "bash scripts/pentesting-release-status.sh",
     "pentesting:test": "npm run test",
@@ -71,7 +72,7 @@
     "public:sync": "bash scripts/sync-public-repo.sh",
     "public:mirror-release": "bash scripts/mirror-public-release.sh",
     "release:backfill": "bash scripts/backfill-release-local.sh",
-    "check:docker": "sh -c 'npm run docker:build && docker run -it --rm -v builder-workspace:/workspace -e ANTHROPIC_BASE_URL -e ANTHROPIC_AUTH_TOKEN -e ANTHROPIC_MODEL -e ANTHROPIC_API_KEY -e MINIMAX_API_KEY -e OPENAI_API_KEY -e OPENAI_BASE_URL -e GEMINI_API_KEY -e DEEPSEEK_API_KEY agnusdei1207/pentesting:latest'",
+    "check:docker": "npm run docker:build && docker run -it --rm -v builder-workspace:/workspace -v pentesting-config:/root/.pentesting -e OPENAI_API_KEY -e OPENAI_BASE_URL -e OPENAI_MODEL -e OPENAI_MAX_TOKENS agnusdei1207/pentesting:latest",
     "docker:build": "(docker image inspect agnusdei1207/pentesting-build-base:1.96 >/dev/null 2>&1 && docker image inspect agnusdei1207/pentesting-runtime-base:26.04 >/dev/null 2>&1 || npm run docker:base:build) && docker build --build-arg APP_VERSION=$(git describe --tags --abbrev=0 2>/dev/null || echo dev) -t agnusdei1207/pentesting:latest .",
     "docker:base:build": "docker build -t agnusdei1207/pentesting-build-base:1.96 -f docker/build-base.Dockerfile . && docker build -t agnusdei1207/pentesting-runtime-base:26.04 -f docker/runtime-base.Dockerfile .",
     "release:patch": "./scripts/release-all.sh patch",