open-research-protocol 0.4.25 → 0.4.26

package/docs/RESEARCH_COUNCIL.md

@@ -0,0 +1,123 @@
+# ORP Research Council
+
+ORP research council runs turn one question into a durable, tool-callable research artifact. The default profile is OpenAI-only right now, so one saved ORP key can power the full loop:
+
+```bash
+orp research ask "Where should this system live?" --json
+```
+
+By default this is a dry run. ORP writes the decomposition, profile, lane plan, lane JSON files, synthesized planning answer, and summary under:
+
+```text
+orp/research/<run_id>/
+```
+
+Live provider calls require an explicit flag:
+
+```bash
+orp research ask "Where should this system live?" --execute --json
+```
+
+## Lanes
+
+The built-in `openai-council` profile defines three OpenAI API lanes:
+
+- `openai_reasoning_high`: `gpt-5.4` with `reasoning.effort=high` for the deliberate thinking pass.
+- `openai_web_synthesis`: `gpt-5.4` with high reasoning plus Responses API web search for current public evidence and citations.
+- `openai_deep_research`: `o3-deep-research-2025-06-26` with background execution and web search preview for Pro/Deep Research style investigation.
+
+This follows OpenAI's current model guidance: `gpt-5.4` is the default for general-purpose, coding, reasoning, and agentic workflows; web search is enabled through the Responses API `tools` array when current information is needed; and Deep Research is available through the Responses endpoint with `o3-deep-research-2025-06-26`.
+
+## API Call Moments
+
+ORP records when API keys are intended to be used:
+
+- `plan`: local decomposition only. No API key is resolved.
+- `thinking_reasoning_high`: resolve `openai-primary` immediately before the `openai_reasoning_high` lane.
+- `web_synthesis`: resolve `openai-primary` immediately before the `openai_web_synthesis` lane.
+- `pro_deep_research`: resolve `openai-primary` immediately before the `openai_deep_research` lane.
+
+Dry runs write every lane with `api_call.called=false`. Live runs require `--execute`; even then, secret values are read only at the lane call moment and are not written to artifacts.
+
+Secret values are read from environment variables first. If an env var is missing and a matching ORP Keychain secret is available, ORP can use it at execution time. Secret values are not persisted in artifacts.
+
+The default live profile expects this ORP secret alias or env var:
+
+- `openai-primary` / `OPENAI_API_KEY`
+
+Store a local machine copy without the hosted secret API like this:
+
+```bash
+printf '%s' '<openai-key>' | orp secrets keychain-add \
+  --alias openai-primary \
+  --label "OpenAI Primary" \
+  --provider openai \
+  --value-stdin \
+  --json
+```
+
+## Fixtures
+
+Provider outputs can be attached without spending live calls:
+
+```bash
+orp research ask "Where should this live?" \
+  --lane-fixture openai_reasoning_high=reports/reasoning.json \
+  --lane-fixture openai_web_synthesis=reports/web.txt \
+  --json
+```
+
+Fixtures are useful when an OpenAI run happened outside ORP, when you are comparing model settings manually, or when tests need deterministic lane outputs.
+
+## OpenAI API Notes
+
+ORP uses the Responses API for these lanes. Useful knobs in profile JSON:
+
+- `model`: for example `gpt-5.4` or `o3-deep-research-2025-06-26`.
+- `call_moment`: the named research-loop moment when this lane may resolve a key.
+- `reasoning_effort`: `none`, `low`, `medium`, `high`, or `xhigh` for supported models.
+- `reasoning_summary`: `auto` or `detailed` for Deep Research reasoning summaries.
+- `text_verbosity`: `low`, `medium`, or `high`.
+- `web_search`: `true` to add the Responses API web-search tool.
+- `search_context_size`: `low`, `medium`, or `high` for web search.
+- `background`: `true` for long-running Deep Research calls.
+- `max_output_tokens`: hard cap for a lane response.
+
+The default profile deliberately avoids Anthropic, xAI, and local-model lanes so a single OpenAI key is enough.
+
+## Project Context Timing
+
+`orp init` creates `orp/project.json`, a process-only project context lens for the current directory. It records the authority surfaces ORP can see, the directory signals it should route on, and the default research timing policy:
+
+- decompose locally first
+- use high-reasoning API calls when a decision gate or ambiguous next action needs outside reasoning
+- use web synthesis when current public facts, docs, papers, project status, or citations matter
+- use Deep Research only after reasoning/web lanes expose a research-heavy gap, disagreement, source-quality issue, or literature-scale synthesis need
+
+Run `orp project refresh --json` after adding or changing roadmap, spec, agent-guidance, docs, manifest, or command-surface files. Refreshing project context does not call a provider; live provider calls remain explicit through `orp research ask --execute`.
+
+## Follow-Up Commands
+
+```bash
+orp project show --json
+orp project refresh --json
+orp research status latest --json
+orp research show latest --json
+```
+
+## Codex MCP Tool
+
+ORP also ships a tiny stdio MCP wrapper for the research commands:
+
+```toml
+[mcp_servers.orp-research]
+command = "/path/to/orp/scripts/orp-mcp"
+```
+
+It exposes:
+
+- `orp_research_ask`
+- `orp_research_status`
+- `orp_research_show`
+
+Research council files are ORP process artifacts. They record decomposition, provider lane outputs, and synthesis. Canonical evidence still belongs in source repositories, linked reports, cited URLs, datasets, papers, or other primary artifacts.

package/docs/START_HERE.md

@@ -43,6 +43,7 @@ orp home
 orp agents root set /absolute/path/to/projects
 orp init
 orp agents audit
+orp project show --json
 orp workspace create main-cody-1
 orp workspace tabs main
 orp agenda refresh --json
@@ -64,6 +65,7 @@ That gets you:
 - an optional umbrella projects root for parent/child agent guidance
 - repo governance initialized
 - repo-level AGENTS.md and CLAUDE.md scaffolded or refreshed
+- `orp/project.json` created as the local project context lens
 - a local workspace ledger
 - the main recovery surface
 - a local operating agenda
@@ -73,6 +75,8 @@ That gets you:
 - a clean repo-governance read
 - a first intentional checkpoint
 
+`orp/project.json` records the current directory's authority surfaces, directory signals, and default research call timing. It is refreshed by `orp init` and can evolve with the directory through `orp project refresh --json`, especially after adding or changing roadmap, spec, agent-guidance, docs, manifest, or command-surface files.
+
 ## Beginner Flow
 
 This is the zero-assumption path for a new user.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "open-research-protocol",
-  "version": "0.4.25",
+  "version": "0.4.26",
   "description": "ORP CLI (Open Research Protocol): workspace ledgers, secrets, scheduling, governed execution, and agent-friendly research workflows.",
   "license": "MIT",
   "author": "Fractal Research Group <cody@frg.earth>",

package/scripts/orp-mcp

@@ -0,0 +1,205 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import json
+from pathlib import Path
+import subprocess
+import sys
+from typing import Any
+
+
+ROOT = Path(__file__).resolve().parents[1]
+CLI = ROOT / "cli" / "orp.py"
+
+
+TOOLS: list[dict[str, Any]] = [
+    {
+        "name": "orp_research_ask",
+        "description": "Create an ORP OpenAI research-loop run with high-reasoning, web-synthesis, and deep-research call moments. Dry-run by default; set execute=true for live provider lanes.",
+        "inputSchema": {
+            "type": "object",
+            "required": ["question"],
+            "properties": {
+                "question": {"type": "string", "minLength": 1},
+                "repo_root": {"type": "string", "description": "Repository root. Defaults to current directory."},
+                "run_id": {"type": "string"},
+                "profile": {"type": "string", "default": "openai-council"},
+                "profile_file": {"type": "string"},
+                "execute": {"type": "boolean", "default": False},
+                "lane_fixtures": {
+                    "type": "object",
+                    "description": "Mapping of lane_id to fixture path.",
+                    "additionalProperties": {"type": "string"},
+                },
+                "chimera_bin": {"type": "string", "default": "chimera"},
+                "timeout_sec": {"type": "integer", "minimum": 1, "default": 120},
+            },
+        },
+    },
+    {
+        "name": "orp_research_status",
+        "description": "Show status and lane summary for an ORP research council run.",
+        "inputSchema": {
+            "type": "object",
+            "properties": {
+                "repo_root": {"type": "string", "description": "Repository root. Defaults to current directory."},
+                "run_id": {"type": "string", "default": "latest"},
+            },
+        },
+    },
+    {
+        "name": "orp_research_show",
+        "description": "Show an ORP research council answer payload.",
+        "inputSchema": {
+            "type": "object",
+            "properties": {
+                "repo_root": {"type": "string", "description": "Repository root. Defaults to current directory."},
+                "run_id": {"type": "string", "default": "latest"},
+            },
+        },
+    },
+]
+
+
+def _json_response(request_id: Any, result: Any | None = None, error: Any | None = None) -> dict[str, Any]:
+    response: dict[str, Any] = {"jsonrpc": "2.0", "id": request_id}
+    if error is not None:
+        response["error"] = error
+    else:
+        response["result"] = result if result is not None else {}
+    return response
+
+
+def _write(payload: dict[str, Any]) -> None:
+    sys.stdout.write(json.dumps(payload, separators=(",", ":")) + "\n")
+    sys.stdout.flush()
+
+
+def _text_result(text: str, *, is_error: bool = False) -> dict[str, Any]:
+    return {
+        "content": [{"type": "text", "text": text}],
+        "isError": is_error,
+    }
+
+
+def _repo_root(args: dict[str, Any]) -> str:
+    raw = str(args.get("repo_root", "") or "").strip()
+    return raw or "."
+
+
+def _run_orp(args: list[str]) -> tuple[int, str, str]:
+    proc = subprocess.run(
+        [sys.executable, str(CLI), *args],
+        capture_output=True,
+        text=True,
+        cwd=str(ROOT),
+    )
+    return proc.returncode, proc.stdout, proc.stderr
+
+
+def _call_research_ask(args: dict[str, Any]) -> dict[str, Any]:
+    question = str(args.get("question", "") or "").strip()
+    if not question:
+        return _text_result("question is required", is_error=True)
+    cmd = ["--repo-root", _repo_root(args), "research", "ask", question]
+    profile = str(args.get("profile", "") or "").strip()
+    if profile:
+        cmd.extend(["--profile", profile])
+    profile_file = str(args.get("profile_file", "") or "").strip()
+    if profile_file:
+        cmd.extend(["--profile-file", profile_file])
+    run_id = str(args.get("run_id", "") or "").strip()
+    if run_id:
+        cmd.extend(["--run-id", run_id])
+    if bool(args.get("execute", False)):
+        cmd.append("--execute")
+    lane_fixtures = args.get("lane_fixtures")
+    if isinstance(lane_fixtures, dict):
+        for lane_id, path in lane_fixtures.items():
+            cmd.extend(["--lane-fixture", f"{lane_id}={path}"])
+    chimera_bin = str(args.get("chimera_bin", "") or "").strip()
+    if chimera_bin:
+        cmd.extend(["--chimera-bin", chimera_bin])
+    timeout_sec = args.get("timeout_sec")
+    if timeout_sec is not None:
+        cmd.extend(["--timeout-sec", str(timeout_sec)])
+    cmd.append("--json")
+    code, stdout, stderr = _run_orp(cmd)
+    if code != 0:
+        return _text_result((stderr or stdout or f"orp exited {code}").strip(), is_error=True)
+    return _text_result(stdout.strip())
+
+
+def _call_research_status(args: dict[str, Any]) -> dict[str, Any]:
+    run_id = str(args.get("run_id", "") or "latest").strip() or "latest"
+    cmd = ["--repo-root", _repo_root(args), "research", "status", run_id, "--json"]
+    code, stdout, stderr = _run_orp(cmd)
+    if code != 0:
+        return _text_result((stderr or stdout or f"orp exited {code}").strip(), is_error=True)
+    return _text_result(stdout.strip())
+
+
+def _call_research_show(args: dict[str, Any]) -> dict[str, Any]:
+    run_id = str(args.get("run_id", "") or "latest").strip() or "latest"
+    cmd = ["--repo-root", _repo_root(args), "research", "show", run_id, "--json"]
+    code, stdout, stderr = _run_orp(cmd)
+    if code != 0:
+        return _text_result((stderr or stdout or f"orp exited {code}").strip(), is_error=True)
+    return _text_result(stdout.strip())
+
+
+def _handle(request: dict[str, Any]) -> dict[str, Any] | None:
+    request_id = request.get("id")
+    method = str(request.get("method", "") or "")
+    if method.startswith("notifications/"):
+        return None
+    if method == "initialize":
+        return _json_response(
+            request_id,
+            {
+                "protocolVersion": str(request.get("params", {}).get("protocolVersion", "2024-11-05")),
+                "capabilities": {"tools": {}},
+                "serverInfo": {"name": "orp-research", "version": "0.1.0"},
+            },
+        )
+    if method == "tools/list":
+        return _json_response(request_id, {"tools": TOOLS})
+    if method == "tools/call":
+        params = request.get("params")
+        if not isinstance(params, dict):
+            return _json_response(request_id, error={"code": -32602, "message": "params must be an object"})
+        name = str(params.get("name", "") or "")
+        arguments = params.get("arguments")
+        if not isinstance(arguments, dict):
+            arguments = {}
+        if name == "orp_research_ask":
+            return _json_response(request_id, _call_research_ask(arguments))
+        if name == "orp_research_status":
+            return _json_response(request_id, _call_research_status(arguments))
+        if name == "orp_research_show":
+            return _json_response(request_id, _call_research_show(arguments))
+        return _json_response(request_id, error={"code": -32601, "message": f"unknown tool: {name}"})
+    return _json_response(request_id, error={"code": -32601, "message": f"unknown method: {method}"})
+
+
+def main() -> int:
+    for line in sys.stdin:
+        text = line.strip()
+        if not text:
+            continue
+        try:
+            request = json.loads(text)
+        except Exception as exc:
+            _write(_json_response(None, error={"code": -32700, "message": str(exc)}))
+            continue
+        if not isinstance(request, dict):
+            _write(_json_response(None, error={"code": -32600, "message": "request must be an object"}))
+            continue
+        response = _handle(request)
+        if response is not None:
+            _write(response)
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

package/spec/v1/project-context.schema.json

@@ -0,0 +1,223 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://openresearchprotocol.com/spec/v1/project-context.schema.json",
+  "title": "ORP Project Context",
+  "description": "Machine-readable ORP process artifact describing a local directory's authority surfaces, directory signals, research timing policy, and evolution policy.",
+  "type": "object",
+  "additionalProperties": true,
+  "required": [
+    "schema_version",
+    "kind",
+    "project",
+    "initialized_at_utc",
+    "refreshed_at_utc",
+    "refresh_source",
+    "authority_surfaces",
+    "directory_signals",
+    "research_policy",
+    "evolution_policy",
+    "next_actions",
+    "notes"
+  ],
+  "properties": {
+    "schema_version": {
+      "const": "1.0.0"
+    },
+    "kind": {
+      "const": "orp_project_context"
+    },
+    "project": {
+      "type": "object",
+      "required": [
+        "name",
+        "root"
+      ],
+      "properties": {
+        "name": {
+          "type": "string",
+          "minLength": 1
+        },
+        "root": {
+          "type": "string",
+          "minLength": 1
+        }
+      }
+    },
+    "initialized_at_utc": {
+      "type": "string",
+      "minLength": 1
+    },
+    "refreshed_at_utc": {
+      "type": "string",
+      "minLength": 1
+    },
+    "refresh_source": {
+      "type": "string",
+      "minLength": 1
+    },
+    "authority_surfaces": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "required": [
+          "path",
+          "kind",
+          "role",
+          "exists",
+          "size_bytes"
+        ],
+        "properties": {
+          "path": {
+            "type": "string",
+            "minLength": 1
+          },
+          "kind": {
+            "type": "string",
+            "minLength": 1
+          },
+          "role": {
+            "type": "string",
+            "minLength": 1
+          },
+          "exists": {
+            "type": "boolean"
+          },
+          "size_bytes": {
+            "type": "integer",
+            "minimum": 0
+          }
+        }
+      }
+    },
+    "directory_signals": {
+      "type": "object",
+      "required": [
+        "source_dirs",
+        "languages_or_stacks",
+        "has_tests",
+        "has_docs",
+        "has_orp_config",
+        "authority_surface_count"
+      ],
+      "properties": {
+        "source_dirs": {
+          "type": "array",
+          "items": {
+            "type": "string"
+          }
+        },
+        "languages_or_stacks": {
+          "type": "array",
+          "items": {
+            "type": "string"
+          }
+        },
+        "has_tests": {
+          "type": "boolean"
+        },
+        "has_docs": {
+          "type": "boolean"
+        },
+        "has_orp_config": {
+          "type": "boolean"
+        },
+        "authority_surface_count": {
+          "type": "integer",
+          "minimum": 0
+        }
+      }
+    },
+    "research_policy": {
+      "type": "object",
+      "required": [
+        "default_timing",
+        "provider_calls_are_explicit",
+        "live_calls_require_execute",
+        "call_moments"
+      ],
+      "properties": {
+        "default_timing": {
+          "type": "string",
+          "minLength": 1
+        },
+        "provider_calls_are_explicit": {
+          "type": "boolean"
+        },
+        "live_calls_require_execute": {
+          "type": "boolean"
+        },
+        "secret_alias": {
+          "type": "string"
+        },
+        "env_var": {
+          "type": "string"
+        },
+        "call_moments": {
+          "type": "array",
+          "items": {
+            "type": "object",
+            "required": [
+              "moment_id",
+              "calls_api",
+              "when"
+            ],
+            "properties": {
+              "moment_id": {
+                "type": "string",
+                "minLength": 1
+              },
+              "calls_api": {
+                "type": "boolean"
+              },
+              "lane": {
+                "type": "string"
+              },
+              "model": {
+                "type": "string"
+              },
+              "when": {
+                "type": "string",
+                "minLength": 1
+              },
+              "capability_note": {
+                "type": "string"
+              }
+            }
+          }
+        },
+        "skip_research_when": {
+          "type": "array",
+          "items": {
+            "type": "string"
+          }
+        },
+        "escalate_to_deep_research_when": {
+          "type": "array",
+          "items": {
+            "type": "string"
+          }
+        }
+      }
+    },
+    "evolution_policy": {
+      "type": "object",
+      "required": [
+        "refresh_surfaces",
+        "evolution_loop",
+        "boundary"
+      ]
+    },
+    "next_actions": {
+      "type": "array",
+      "items": {
+        "type": "string"
+      }
+    },
+    "notes": {
+      "type": "array",
+      "items": {
+        "type": "string"
+      }
+    }
+  }
+}