rea-agents 1.6.0 → 1.7.0

package/README.md

@@ -30,7 +30,7 @@
 
 See a feature in an app that you want in your own product? Give the app to your agent—even without its source code. With REA, the agent can investigate the feature, explain how it works, show its evidence, and build a version adapted to your stack and requirements.
 
-REA gives agents one consistent way to investigate software. Today that includes deep native analysis through Hopper, complete function dossiers, reproducible Evidence v2 records, controlled process capture, passive website and Electron observation, and bounded JavaScript/source-map reconstruction. The longer-term toolkit extends the same agent workflow to APIs, protocols, mobile artifacts, firmware, richer runtime behavior, and differences between versions.
+REA gives agents one consistent way to investigate software. Today that includes deep native analysis and function dossiers through Hopper or bring-your-own Ghidra on Linux, reproducible Evidence v2 records, controlled process capture, passive website and Electron observation, bounded JavaScript/source-map reconstruction, and a versioned domain graph for connecting JavaScript application layers without confusing static inference with runtime observation. The longer-term toolkit extends the same agent workflow to APIs, protocols, mobile artifacts, firmware, richer runtime behavior, and differences between versions.
 
 Reverse engineering normally makes the operator choose a tool, learn its API, move evidence between programs, and decide what to inspect next. REA gives that work to the agent through commands, skills, structured results, and repeatable investigation workflows.
 
@@ -74,14 +74,14 @@ REA shows how it reached its conclusions. It does not claim to recover original
 
 ## Why REA
 
-|                          |                                                                                      |
-| ------------------------ | ------------------------------------------------------------------------------------ |
-| **Built for agents**     | Ask what an app does and let your agent inspect it instead of guessing.              |
-| **CLI and MCP**          | Run the same reverse-engineering capabilities from your terminal or agent.           |
-| **Complexity handled**   | REA installs and manages the reverse-engineering tools behind the scenes.            |
-| **From insight to code** | Understand a feature, then build your own version in the same coding session.        |
-| **Local by design**      | Analysis runs on your Mac. REA does not upload the app to a hosted analysis service. |
-| **Keeps context**        | Investigate several apps without starting over for every question.                   |
+|                          |                                                                                                       |
+| ------------------------ | ----------------------------------------------------------------------------------------------------- |
+| **Built for agents**     | Ask what an app does and let your agent inspect it instead of guessing.                               |
+| **CLI and MCP**          | Run the same reverse-engineering capabilities from your terminal or agent.                            |
+| **Complexity handled**   | REA installs and manages the reverse-engineering tools behind the scenes.                             |
+| **From insight to code** | Understand a feature, then build your own version in the same coding session.                         |
+| **Local by design**      | Analysis runs on your supported local host. REA does not upload the app to a hosted analysis service. |
+| **Keeps context**        | Investigate several apps without starting over for every question.                                    |
 
 ## Quick start
 
@@ -94,7 +94,7 @@ rea setup
 
 Installing the CLI does not update Homebrew, Node.js, npm, Hopper, or agent configuration. `rea setup` detects what is already present, prints every proposed change, and asks before applying it.
 
-REA detects Claude Code, Claude Desktop, Codex, Cursor, Gemini CLI, Windsurf, and Devin. Registrations are additive, backup-first, and read back after writing. You can safely rerun setup.
+REA detects Claude Code, Claude Desktop, Codex, Cursor, Gemini CLI, Windsurf, and Devin. It configures the first six when detected; Devin is reported but left unchanged because it has no documented local MCP configuration boundary. Registrations are additive, backup-first, and read back after writing. You can safely rerun setup.
 
 An optional curl wrapper installs the same CLI package and starts setup only when a terminal is available:
 
@@ -110,7 +110,7 @@ Pass installer options after `bash -s --`, for example `--dry-run`, `--no-setup`
 npx skills add morluto/rea
 ```
 
-Ask your agent to set up REA. It will check your Mac, explain anything it needs to install, ask for approval, and guide you through system prompts. After setup, restart the agent if it asks you to load the full REA toolset.
+Ask your agent to set up REA. It will check your supported host, explain anything it needs to install, ask for approval, and guide you through system prompts. After setup, restart the agent if it asks you to load the full REA toolset.
 
 Review the setup plan, approve it if appropriate, then describe the app or feature you want to understand. Hopper can run in its free demo mode; if it shows a first-run prompt, choose the demo or enter an existing license.
 
@@ -153,7 +153,7 @@ Choose either the no-install commands or the global installation. You do not nee
 - Node.js 22.19+ or 24.11+ (including newer releases)
 - npm; REA does not require or install a particular npm version
 
-Deep binary analysis currently uses [Hopper](https://www.hopperapp.com/), a separate desktop application with its own license. Setup reuses an existing installation. If Hopper is missing, interactive setup proposes the official package and includes it in the confirmation plan. Unattended installation requires `rea setup --yes --install-hopper`.
+Deep binary operations use [Hopper](https://www.hopperapp.com/), a separate desktop application with its own license, or a caller-selected Ghidra provider. Ghidra supplies read-only inventory, function metadata, decompilation, assembly, resolved calls, typed references, xrefs, CFG, and function dossiers; GUI state and mutations remain unavailable through that provider. Setup reuses an existing Hopper installation or an operator-supplied Ghidra installation. It never downloads Ghidra or installs Java. If neither provider is ready, interactive setup proposes Hopper; unattended Hopper installation requires `rea setup --yes --install-hopper`.
 
 If something is not working, run:
 
@@ -186,6 +186,28 @@ Install the missing distribution packages and rerun `rea setup`. Linux demo auto
 
 REA defaults `HOPPER_LAUNCHER_PATH` to `/Applications/Hopper Disassembler.app/Contents/MacOS/hopper` on macOS and `/opt/hopper/bin/Hopper` on Linux. Explicit configuration always takes precedence.
 
+### Ghidra read-only analysis provider
+
+The current Ghidra adapter supports the exact official Ghidra 12.1.2 release on Linux x64 with a 64-bit full JDK 21. Download and extract those projects yourself, then configure absolute paths:
+
+```bash
+export GHIDRA_INSTALL_DIR=/absolute/path/to/ghidra_12.1.2_PUBLIC
+export JAVA_HOME=/absolute/path/to/jdk-21 # optional when java and javac resolve from PATH
+rea doctor --json
+rea setup
+rea providers --json
+```
+
+Doctor distinguishes missing configuration, a bad installation root, the wrong Ghidra or Java version, a JRE without `javac`, a missing `support/analyzeHeadless`, and an unsupported platform or architecture. Approved setup only copies the verified non-secret paths into detected MCP registrations; it does not modify the Ghidra installation or install/download Ghidra or Java.
+
+REA loads its packaged Java `HeadlessScript` with `-scriptPath`, imports the target into a mode-0700 temporary project, enables `-readOnly` and `-deleteProject`, caps auto-analysis at 300 seconds with two CPUs and a 2 GiB Java heap, and authenticates every request over a mode-0600 Unix socket. It isolates Ghidra home/cache/config/temp paths and removes the owned project, socket, process group, and runtime root on close, cancellation, timeout, or process exit.
+
+The Ghidra adapter declares 18 direct and enhanced operations. Its ten inventory operations are `list_documents`, `list_procedures`, `list_strings`, `list_names`, `list_segments`, `address_name`, `procedure_address`, `resolve_containing_procedure`, `search_procedures`, and `search_strings`. It also admits `procedure_info`, `procedure_pseudo_code`, `procedure_assembly`, `procedure_callers`, `procedure_callees`, `procedure_references`, `xrefs`, and `analyze_function`. These capabilities enable the shared Swift/Objective-C inventory workflows, `binary_overview`, `batch_decompile`, `get_call_graph`, `find_xrefs_to_name`, `trace_feature`, and complete function dossiers. Default-space addresses are lowercase `0x` hexadecimal. Other spaces, including `EXTERNAL`, use `<percent-encoded-space>:0x<hex>`. Symbol results identify primary, dynamic, external, type, and source facts; procedures distinguish external functions and thunks; strings identify charset, missing-terminator state, byte length, and value truncation; memory-block ends are exclusive and permissions come directly from Ghidra.
+
+The bridge serves operations only after auto-analysis completes. Each Program owns one persistent `DecompInterface`; a bounded 32-request FIFO serializes Ghidra API access, and every decompile has a 30-second native deadline. Reference results preserve Ghidra's call/jump/data/read/write/indirect/computed/external facts, while unresolved targetless flows remain explicitly unknown. Synthetic entry-point references without actionable memory sources are omitted. Pseudocode and assembly are provider-specific observations, not original source or Hopper-equivalent text. An analysis timeout, scan or inventory safety limit, request timeout, or oversized response fails explicitly instead of returning a partial result labeled complete.
+
+`npm run verify:ghidra` compiles source-owned x86-64 debug and stripped ELF, AArch64 ELF, x86-64 PE, and x86-64 Mach-O fixtures. Against real Ghidra 12.1.2 it validates every admitted operation, direct and indirect calls, imports/exports/thunks, typed references, strings/xrefs, multi-block CFG, cancellation, deadlines, concurrency, malformed inputs, and complete process/project cleanup. Set `REA_CC`, `REA_CLANG`, or `REA_LLD_LINK` only when the corresponding compiler command is not on `PATH`.
+
 To remove only REA-owned MCP registrations and the managed skill:
 
 ```bash
@@ -240,13 +262,15 @@ Exports never replace an existing file unless `--overwrite` is explicit. Imports
 
 Provider-neutral analysis snapshots persist successful, immutable REA calls and
 their Evidence v2 records. They are exact caches rather than Hopper databases:
-REA reuses an entry only when the binary digest, format, architecture, operation
-parameters, loader arguments, and provider identity match. Custom Hopper loader
-overrides disable snapshots because their provider configuration cannot be
-replayed safely. Cursor-dependent and mutating calls are never cached. Snapshot
-files can contain proprietary analysis results and local
-paths, so REA keeps them local, writes them with owner-only permissions, and
-requires a separate approved root:
+REA reuses a v2 entry only when the binary digest, kind, format, architecture,
+operation parameters, concrete provider build, and canonical analysis-profile
+digest match. Hopper loader defaults and configured overrides are normalized by
+the Hopper adapter and committed to that profile, so overrides occupy a distinct
+safe cache partition instead of disabling snapshots. Snapshot v1 cannot prove
+those semantics and is rejected with recapture guidance. Cursor-dependent and
+mutating calls are never cached. Snapshot files can contain proprietary analysis
+results and local paths, so REA keeps them local, writes them with owner-only
+permissions, and requires a separate approved root:
 
 ```bash
 export REA_ANALYSIS_SNAPSHOT_ROOTS_JSON='["/absolute/path/to/analysis"]'
@@ -255,7 +279,7 @@ rea analyze /absolute/path/to/app --snapshot /absolute/path/to/analysis/app.json
 rea analyze /absolute/path/to/app --snapshot /absolute/path/to/analysis/app.json
 ```
 
-Exact CLI evidence replays happen before any provider starts. In MCP sessions,
+Exact CLI evidence replays happen before any provider process starts. In MCP sessions,
 pass `snapshot_path` to `open_binary` to import a snapshot atomically while
 opening its matching target; MCP providers may still start before a cached call
 is replayed. Pass `snapshot_path` and, when required, `overwrite: true` to
@@ -310,11 +334,14 @@ The public interface describes what the agent is trying to learn. Providers deci
 
 ## Current status
 
-REA is already useful for native application investigation on macOS:
+REA is already useful for native application, browser, and Electron investigation on supported macOS and Linux hosts:
 
-- Open Mach-O, ELF, PE, `.app`, ZIP, APK, IPA, ASAR, plist, JavaScript, source-map, and Hopper database targets.
+- Open Mach-O, ELF, PE, `.app`, ZIP, APK, IPA, ASAR, plist, JavaScript, source-map, and generic analysis-database targets; Hopper remains the only adapter that accepts legacy `.hop` databases.
+- Discover deep-analysis candidates without starting them, choose deterministically, and retain one immutable provider/profile binding until an explicit switch or close; provider failures never trigger transparent fallback.
 - Attach to a user-owned Chrome-family browser over a configured loopback CDP endpoint; capture exact-origin web structure, safe metadata, approved value-free payload shapes, bundle/source-map evidence, WebMCP declarations, user-action timelines, capture diffs, and explicitly approved screenshots without navigation or JavaScript evaluation.
 - Inspect Electron `file://` renderer pages through a separate canonical-root permission boundary without invoking Electron APIs; script contents remain separately approved and byte bounded.
+- Validate and canonically serialize a provider-neutral [JavaScript Application Graph v1](docs/javascript-application-graph.md) spanning packages, ASAR entries, Electron roles, JavaScript/source-map entities, browser/runtime instances, IPC, endpoints, storage, and native add-ons. This shipped domain contract performs no extraction or I/O by itself.
+- Reconstruct bounded static package, entrypoint, Webpack/Rspack module, import, worker, endpoint, storage, source-map, and native-add-on structure from a local directory or ASAR through an AST-only [application service](docs/javascript-artifact-reconstruction.md). The service does not execute bootstrap code and does not add a standalone CLI/MCP tool yet.
 - Traverse content-addressed artifact graphs without extraction; on macOS, read-only DMG traversal additionally requires `native_mount_approved: true` and `REA_ARTIFACT_NATIVE_MOUNT_ENABLED=true`. Materialize only approved occurrences into absent output roots.
 - Build bounded function dossiers with pseudocode, assembly, CFG edges, comments, calls, references, strings, and names.
 - Search and trace features across symbols, strings, metadata, references, and call paths.
@@ -350,26 +377,37 @@ rea inspect-web-page http://127.0.0.1:9222 TARGET_ID --approved --json
 
 All eight browser tools expose the same Evidence v2 contracts over CLI and MCP. Inspection is passive: REA does not evaluate page JavaScript, navigate, click, close the page, or close the browser. Query values, credentials, cookies, authorization headers, storage values, and raw JSON or WebSocket values are never retained. Separately approved captures can retain bounded redacted console primitives, value-free JSON/WebSocket shapes, script sources, accessibility text, or screenshot pixels. Existing activity before attach is explicitly unavailable. See [Website observation with CDP](docs/browser-observation.md) for browser startup, schemas, limits, and the threat model.
 
+Exact package, tool-family, provider, setup-client, public schema-version, and CLI facts are generated from source in [`docs/product-catalog.json`](docs/product-catalog.json). `npm run docs:check` verifies that this catalog, the narrative documentation, TypeDoc, and generated schemas have not drifted.
+
 ## Roadmap
 
-REA is growing into a toolkit for understanding software across static artifacts and observed behavior. The next capability families are:
+REA is growing into a toolkit for understanding software across static artifacts and observed behavior. The [current status](#current-status) above is the shipped baseline; the items below are planned work.
+
+### Now
+
+1. **Maintain truthful product metadata** — extend the shipped canonical catalog and drift checks whenever versions, tools, providers, schemas, setup clients, or CLI capabilities change.
+2. **Electron boundary extraction and high-level surface** — extend the shipped static artifact projector with contextBridge, IPC, utility-process, and native-export boundaries, then expose one provider-neutral CLI/MCP workflow without widening authority.
+3. **Cross-provider conformance growth** — add source-owned architectures and difficult indirect/thunk cases while preserving semantic comparison and provider-specific text boundaries.
+
+### Next
+
+1. **Cross-layer tracing and version comparison** — trace renderer behavior through preload, IPC, main-process, storage, network, and native boundaries, then compare those paths across application versions.
+2. **Deeper JavaScript and source recovery** — add historical-source matching, rechunked/minified cross-version matching, and stronger static/runtime reconciliation on top of the shipped AST-only Webpack/Rspack reconstruction.
+3. **Deterministic behavior harnesses** — extend process ownership, protocol fixtures, filesystem observation, reconnects, and cross-version behavioral comparison.
+
+### Later
 
-1. **Artifact decomposition** — DMG, ASAR, ZIP, packages, universal-binary slices, application resources, embedded frameworks, mobile packages, and artifact graphs.
-2. **Web and Electron investigation** — extend the shipped passive CDP, bundle, visual-diff, and file-page observation with approved interaction, controlled replay, Electron IPC observation, semantic UI differences, and deeper JavaScript reconstruction.
-3. **Deterministic behavior harnesses** — stronger process-tree ownership, protocol fixtures, network policy, filesystem tracing, signals, reconnects, and cross-version comparison.
-4. **JavaScript and source recovery** — bundle indexing, AST/module reconstruction, source-map discovery, historical-source matching, and CodeDB-backed cross-references.
-5. **Runtime observation** — approval-gated LLDB, Frida, system logs, process and filesystem observers, and native API tracing.
-6. **More static-analysis providers** — native platform utilities first, followed by Ghidra, IDA/Hex-Rays, Binary Ninja, Rizin, LIEF, and other engines behind provider-neutral capabilities.
-7. **More targets and platforms** — Windows-native providers and ConPTY verification, Linux parity, websites and APIs, mobile artifacts, firmware, document formats, and other software-defined systems.
-8. **Differential reconstruction expansion** — add automatic function matching, protocol/UI comparison, controlled replay, residual-unknown planning, and reconstruction verification to persistent version runs.
+1. **Controlled interaction and replay** — keep approved JavaScript execution, browser interaction, Electron instrumentation, and fuzzing behind a separate authority from passive observation.
+2. **Native runtime observation** — approval-gated LLDB, Frida, system logs, process/filesystem observers, and native API tracing.
+3. **Additional providers and targets** — evaluate IDA/Hex-Rays, Binary Ninja, Rizin, LIEF, Windows-native providers, mobile artifacts, firmware, document formats, and other software-defined systems.
 
-Roadmap items describe direction, not shipped support. New providers must produce the same evidence and safety metadata as existing capabilities before they become part of the public workflow. Once REA has multiple optional toolchains, setup can become capability-selective; the consent rules for that future work are recorded in the [installation roadmap](docs/roadmap.md).
+New providers must produce the same evidence and safety metadata as existing capabilities before they become part of the public workflow. Once REA has multiple optional toolchains, setup can become capability-selective; the consent rules for that future work are recorded in the [installation roadmap](docs/roadmap.md).
 
-See the [static-analysis provider evaluation](docs/provider-evaluation.md) for the current research matrix and admission gate.
+See the [static-analysis provider evaluation](docs/provider-evaluation.md) for the shipped Ghidra function-analysis boundary, remaining admission gates, and provider comparison matrix, and [ADR-0001](docs/adr/0001-provider-selection-and-analysis-profiles.md) for the binding, selection, profile, snapshot, and compatibility decisions.
 
 ## Using REA with other agents
 
-Setup currently configures Claude Desktop and Cursor automatically. Any agent that supports local MCP servers can use REA with the configuration below.
+Setup detects Claude Code, Claude Desktop, Codex, Cursor, Gemini CLI, Windsurf, and Devin. It automatically configures the first six when present; detected Devin installations are reported but left unchanged. Any agent that supports local MCP servers can use REA with the configuration below.
 
 ### Manual MCP configuration
 
@@ -396,14 +434,17 @@ flowchart LR
     Agent["Agent"] --> REA["REA<br/>CLI + MCP"]
     Terminal --> REA
     REA --> Workspace["Investigation workspace<br/>evidence + artifacts + captures"]
-    Workspace --> Router["Capability router"]
-    Router --> Hopper["Hopper provider"]
-    Router --> Native["Native macOS provider"]
-    Router --> Artifact["Artifact graph provider"]
-    Router --> Browser["Browser CDP provider"]
-    Router --> Process["Process capture provider"]
-    Router -. roadmap .-> More["Dynamic and additional<br/>static providers"]
-    Hopper --> Target["Target software"]
+    REA --> Session["Target-bound session router"]
+    Session --> Registry["Deep-provider registry<br/>deterministic selection"]
+    Registry --> Hopper["Hopper provider"]
+    Registry --> Ghidra["Ghidra provider<br/>read-only inventory + function analysis"]
+    Hopper --> Runtime["Owned provider runtime<br/>deadline + bounded diagnostics + cleanup"]
+    Ghidra --> Runtime
+    Session --> Native["Native macOS provider"]
+    Session --> Artifact["Artifact graph provider"]
+    REA --> Browser["Browser CDP provider"]
+    REA --> Process["Process capture provider"]
+    Runtime --> Target["Target software"]
     Process --> Target
     Native --> Target
     Artifact --> Target
@@ -444,6 +485,42 @@ rea mcp
 
 REA accepts a Mac `.app` folder directly. If an agent cannot find an app by name, tell it where the app is installed.
 
+### Choosing a deep-analysis provider
+
+Every deep-analysis open resolves a provider before creating its client. The
+same selector and precedence apply to the CLI, MCP, and startup configuration:
+
+```bash
+rea providers --json
+rea analyze /absolute/path/to/program --provider hopper
+REA_ANALYSIS_PROVIDER=hopper rea decompile /absolute/path/to/program 0x1000
+```
+
+For MCP, pass the optional selector on `open_binary`:
+
+```json
+{
+  "path": "/absolute/path/to/program",
+  "provider_id": "hopper"
+}
+```
+
+The request-level `provider_id` or `--provider` wins over
+`REA_ANALYSIS_PROVIDER`; all accept a provider ID or `auto`. Automatic selection
+binds the sole usable deep candidate, reports `ambiguous` when several are
+usable, and can leave an artifact-only target unbound so its disjoint artifact
+operations still work. An explicit unknown, unavailable, or unsupported
+provider fails with candidate IDs, stable rejection codes, and actionable local
+diagnostics. `binary_session`, `rea providers`, and `rea capabilities` expose
+the authoritative `analysis_provider_candidates` and
+`analysis_provider_binding` fields. Reopening the same target without a selector
+keeps its binding; runtime failure never selects another provider silently.
+Ghidra can appear as an available, target-compatible candidate after doctor
+validates its exact installation. Its capability list contains the 18 admitted
+read-only inventory and function-analysis operations; selecting it still does
+not make Hopper-only GUI or mutation operations available and never triggers a
+silent fallback.
+
 ### CLI exit status
 
 | Status    | Meaning                                                                                                                                                                                                                                                                  |
@@ -506,7 +583,7 @@ unpacked; REA does not silently accept the mismatched artifact.
 
 ## Security model
 
-REA does not provide a hosted analysis service. Hopper communication uses an authenticated private local socket. Dynamic capabilities are disabled by default and require both operator policy and explicit per-call approval. REA is not a security sandbox: providers and launched targets run with the current user's permissions, and each capability reports its side effects and limitations. Report vulnerabilities through the private process in [SECURITY.md](SECURITY.md).
+REA does not provide a hosted analysis service. Hopper and Ghidra bridge communication uses authenticated private local sockets. Dynamic capabilities are disabled by default and require both operator policy and explicit per-call approval. REA is not a security sandbox: providers and launched targets run with the current user's permissions, and each capability reports its side effects and limitations. Report vulnerabilities through the private process in [SECURITY.md](SECURITY.md).
 
 ## FAQ
 
@@ -531,6 +608,13 @@ No. Setup can install Hopper for you, but Hopper remains separate software with
 
 </details>
 
+<details>
+<summary><strong>Does REA install or include Ghidra or Java?</strong></summary>
+
+No. Ghidra support is bring-your-own. REA packages only its Java bridge source, validates the exact supported Ghidra 12.1.2 and 64-bit JDK 21 installation, and loads that bridge through Ghidra's external script path after setup approval records the paths.
+
+</details>
+
 <details>
 <summary><strong>Does REA upload the app?</strong></summary>
 
@@ -548,7 +632,7 @@ No decompiler can guarantee the original source. REA gives an agent pseudocode,
 <details>
 <summary><strong>Which agents can use REA?</strong></summary>
 
-Any agent that can run a local MCP server can use the manual configuration. Setup currently detects and configures Claude Desktop and Cursor automatically.
+Any agent that can run a local MCP server can use the manual configuration. Setup detects Claude Code, Claude Desktop, Codex, Cursor, Gemini CLI, Windsurf, and Devin; it automatically configures the first six when present and reports Devin without modifying it.
 
 </details>
 
@@ -558,7 +642,7 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for setup, architecture, tests, and relea
 
 ## Project links
 
-[npm](https://www.npmjs.com/package/rea-agents) · [Issues](https://github.com/morluto/rea/issues) · [Security](SECURITY.md) · [Contributing](CONTRIBUTING.md) · [Hopper](https://www.hopperapp.com/)
+[npm](https://www.npmjs.com/package/rea-agents) · [Issues](https://github.com/morluto/rea/issues) · [Security](SECURITY.md) · [Contributing](CONTRIBUTING.md) · [Hopper](https://www.hopperapp.com/) · [Ghidra](https://github.com/NationalSecurityAgency/ghidra)
 
 ## License