open-research-protocol 0.4.19 → 0.4.21

package/docs/AGENT_LOOP.md

@@ -2,6 +2,12 @@
 
 Use this loop when an AI agent is the primary operator of an ORP-enabled repo.
 
+For the longer-horizon model where ORP scans a project's specs, roadmaps, and
+codebase, compiles what is autonomous, and runs until a true human gate, also
+read:
+
+- `docs/ORP_AUTONOMY_PROJECT_COMPILATION_MODEL.md`
+
 ## 1. Discover
 
 - Read `llms.txt`.
@@ -12,6 +18,7 @@ Use this loop when an AI agent is the primary operator of an ORP-enabled repo.
   - Treat it as an optional lens for deeper, wider, top-down, or rotated perspective shifts.
 - If packs matter, run `orp pack list --json`.
 - Read `PROTOCOL.md` before making claims.
+- If the repo uses parent/child agent guidance, run `orp agents audit --json` so you know `AGENTS.md` and `CLAUDE.md` are aligned before taking a long-running path.
 - If the task is external OSS contribution workflow or PR governance, read
   `docs/EXTERNAL_CONTRIBUTION_GOVERNANCE.md` before selecting work.
 - For the more detailed external-contribution operator rhythm, then read
@@ -20,9 +27,41 @@ Use this loop when an AI agent is the primary operator of an ORP-enabled repo.
 ## 2. Select Work
 
 - Identify the target profile and canonical artifact paths.
+- If the task depends on the current highest-leverage action slice, refresh ORP's agenda first:
+  - `orp agenda refresh --json`
+  - `orp agenda refresh-status --json`
+  - `orp agenda enable-refreshes --json`
+  - `orp agenda enable-refreshes --morning 08:30 --afternoon 13:00 --evening 18:30 --json`
+  - `orp agenda disable-refreshes --json`
+  - `orp agenda actions --json`
+  - `orp agenda suggestions --json`
+  - `orp agenda focus --json`
+  - `orp agenda set-north-star "Advance the ocular controller and ORP ecosystems" --json`
+- If the task depends on umbrella guidance or project-specific agent instructions, inspect or refresh the managed files first:
+  - `orp agents root show --json`
+  - `orp agents sync --json`
+  - `orp agents audit --json`
+- If the task depends on an external service, deployment target, dataset platform, or publishing destination, inspect the saved connections first:
+  - `orp connections providers --json`
+  - `orp connections list --json`
+  - `orp connections show <connection> --json`
+  - `orp connections add <connection-id> --provider github --auth-secret-alias <alias> --secret-binding releases=<alias> --json`
+  - `orp connections add <connection-id> --provider custom --label "Custom Service" --url https://example.org --secret-binding primary=<alias> --json`
+  - `orp connections update <connection> --status paused --json`
+  - `orp connections sync --json`
+  - `orp connections pull --json`
+- If the task is shaped by contests, programs, grants, or similar openings, inspect the saved board first:
+  - `orp opportunities list --json`
+  - `orp opportunities show <board> --json`
+  - `orp opportunities focus <board> --limit 5 --json`
+  - `orp opportunities add <board> --title "<title>" --kind contest --section <section> --priority high --json`
+  - `orp opportunities update <board> <item-id> --status submitted --json`
+  - `orp opportunities sync <board> --json`
+  - `orp opportunities pull <board> --json`
 - If the task needs an API key or token that is not already available, save it first:
   - human interactive path:
     - `orp secrets add --alias <alias> --label "<label>" --provider <provider>`
+    - `orp secrets add --alias <alias> --label "<label>" --provider <provider> --kind password --username <login>`
   - agent/script path:
     - `printf '%s' '<secret>' | orp secrets add --alias <alias> --label "<label>" --provider <provider> --value-stdin`
   - convenience path:

package/docs/ORP_AUTONOMY_PROJECT_COMPILATION_MODEL.md

@@ -0,0 +1,252 @@
+# ORP Autonomy Project Compilation Model
+
+## Purpose
+
+Define the product model for turning a real repo with specs, roadmap docs, and
+code into an autonomous ORP lane that runs until a true human gate.
+
+This is the missing bridge between:
+
+- "the roadmap exists"
+- and "the agent can actually keep going unattended"
+
+## Core Claim
+
+ORP should not require users to manually hand-build a runtime registry before
+autonomy becomes useful.
+
+Instead, ORP should:
+
+1. scan the project's authority surfaces
+2. infer what work is already executable
+3. identify what still requires human judgment or real-world action
+4. scaffold the missing runtime contracts
+5. run the admissible work until a true stop gate
+
+## The Problem This Solves
+
+In real projects, a capable agent can often continue from:
+
+- `AGENTS.md`
+- roadmap/spec documents
+- active notes
+- codebase scripts and commands
+
+But an unattended runtime cannot rely on implicit operator judgment.
+
+The unattended system needs explicit answers to:
+
+- what is the source of truth?
+- what is a roadmap vs a pause policy?
+- what step is next?
+- what command actually runs that step?
+- what output proves the step is done?
+- what requires a human?
+
+This model gives ORP a way to infer and scaffold those answers instead of
+making the user encode them all by hand first.
+
+## Core Concepts
+
+### 1. Authority Surfaces
+
+These are the repo objects that define what the project is allowed to treat as
+planning authority.
+
+Typical sources:
+
+- `AGENTS.md`
+- `llms.txt`
+- roadmap/spec docs
+- current frontier or "next step" notes
+- state files
+- package manifests
+- scripts and make targets
+- protocol docs
+
+### 2. Coverage Map
+
+ORP should build a row-by-row map of project nodes:
+
+- already executable
+- partially executable
+- strategic only
+- human gated
+
+This is the bridge from roadmap language to runtime truth.
+
+### 3. Runtime Unit
+
+A roadmap node becomes runnable only when ORP can associate it with:
+
+1. one recognized next-step surface
+2. one concrete command or builder
+3. expected outputs
+4. verification/closeout
+5. one stop-gate interpretation
+
+### 4. True Human Gate
+
+A true gate is not "the agent feels uncertain."
+
+A true gate is a boundary like:
+
+- spend or purchase
+- outreach or counterparty contact
+- provider/vendor selection with real consequences
+- legal/oversight/compliance judgment
+- external release
+- claim-bearing promotion
+- destructive irreversible action
+- real-world biological/physical execution
+
+## Proposed ORP Flow
+
+### `orp autonomy scan`
+
+Purpose:
+
+- inspect the repo and identify the candidate authority surfaces
+
+Outputs:
+
+- authority stack
+- route split if multiple roadmap families exist
+- likely live frontier docs
+- possible command/build surfaces
+
+### `orp autonomy coverage`
+
+Purpose:
+
+- build the roadmap-to-runtime coverage matrix
+
+Outputs:
+
+- executable nodes
+- partial nodes
+- human-gated nodes
+- missing compilation requirements
+
+### `orp autonomy compile`
+
+Purpose:
+
+- scaffold runtime units for uncovered but admissible roadmap work
+
+Outputs:
+
+- task contracts
+- mission contracts
+- gate dossiers
+- checkpoint/closeout contracts
+
+### `orp autonomy run`
+
+Purpose:
+
+- execute the compiled lane until a true gate, no-runnable boundary, or failure
+
+Behavior:
+
+- derive next admissible task
+- execute
+- checkpoint
+- refresh workspace state
+- repeat
+- stop only at true gates
+
+### `orp autonomy gates`
+
+Purpose:
+
+- show active gates and what human decisions they require
+
+Outputs:
+
+- gate id
+- exact trigger
+- current packet/dossier
+- safe parallel work
+
+### `orp autonomy resume`
+
+Purpose:
+
+- reopen a paused lane after a human gate is explicitly approved
+
+Outputs:
+
+- resumed runtime context
+- reopened next step
+- continuation receipt
+
+## Compilation Heuristics
+
+ORP should infer aggressively but conservatively.
+
+Good candidates for automatic compilation:
+
+- builder scripts that emit canonical artifacts
+- docs that use exact next-step language
+- repeated checkpoint patterns
+- task-specific notes with strong input/output structure
+
+Bad candidates for automatic compilation:
+
+- vague strategic narratives with no runnable command
+- tasks that imply counterparty contact
+- tasks that imply money
+- steps that promote support-only outputs into authority
+
+## What ORP Should Emit
+
+For a real project, the compiled autonomy packet should include:
+
+- authority surfaces
+- roadmap hierarchy
+- coverage matrix
+- task registry
+- mission queue
+- stop-gate policy
+- human dossier templates
+- run-until-gate launch surface
+- resume-after-gate surface
+
+## Boundary Rule
+
+ORP remains process/governance infrastructure.
+
+It can compile, route, checkpoint, and pause.
+
+It must not treat its own packets as scientific or operational evidence.
+Evidence still lives in the repo's canonical artifacts.
+
+## Example Pattern
+
+The controller benchmark experiment surfaced the exact shape:
+
+1. exploit public/support-only work fully
+2. compile the remaining pre-outreach tasks
+3. keep drafts unsent
+4. stop only when the next step would actually contact a counterparty or spend
+5. emit a gate dossier
+6. resume only after the human opens that gate
+
+That pattern is generalizable well beyond one biotech project.
+
+## Success Criteria
+
+This model is working when ORP can take a repo with real specs and roadmaps and
+produce:
+
+- one honest "what can be done autonomously?" map
+- one honest "where do humans still matter?" map
+- one runnable autonomous lane
+- one clean pause packet at the first real human boundary
+
+## One-Sentence Version
+
+ORP should compile a project's real planning surfaces into an executable
+autonomous lane that keeps going until the work reaches a true human gate,
+instead of requiring users to hand-author the whole runtime first.

package/docs/START_HERE.md

@@ -7,6 +7,10 @@ The design goal is simple:
 - local-first by default
 - hosted sync is optional
 - the workspace ledger remembers where ongoing work lives
+- `AGENTS.md` and `CLAUDE.md` keep umbrella and child guidance aligned
+- the agenda keeps two ranked lists for what matters now and what to expand next
+- the connections registry remembers which services, accounts, and secret aliases power your tooling
+- opportunity boards remember contests, programs, grants, and similar openings
 - repo governance keeps work intentional
 - the agent uses ORP as its operating lens, not as an afterthought
 
@@ -17,6 +21,7 @@ Do not let ORP process scaffolding masquerade as evidence or repository truth.
 ORP is there to:
 
 - keep the workspace ledger honest
+- keep connection identities and auth references organized
 - keep repo governance explicit
 - keep secrets and continuity usable
 - help the agent stay aligned
@@ -35,9 +40,19 @@ If you already know you want the standard local-first ORP loop in a repo, the fa
 
 ```bash
 orp home
+orp agents root set /absolute/path/to/projects
 orp init
+orp agents audit
 orp workspace create main-cody-1
 orp workspace tabs main
+orp agenda refresh --json
+orp agenda focus
+orp connections providers
+orp connections add github-main --provider github --label "GitHub Main" --auth-secret-alias github-main
+orp connections add huggingface-main --provider huggingface --label "Hugging Face" --secret-binding primary=hf-main --secret-binding publish=hf-publish
+orp connections add my-science-portal --provider custom --label "My Science Portal" --url https://example.org --secret-binding primary=my-science-token
+orp opportunities create main-opportunities --label "Main Opportunities"
+orp opportunities list
 orp secrets ensure --alias openai-primary --provider openai --current-project --json
 orp status --json
 orp checkpoint create -m "capture loop state" --json
@@ -46,9 +61,14 @@ orp checkpoint create -m "capture loop state" --json
 That gets you:
 
 - the discovery screen
+- an optional umbrella projects root for parent/child agent guidance
 - repo governance initialized
+- repo-level AGENTS.md and CLAUDE.md scaffolded or refreshed
 - a local workspace ledger
 - the main recovery surface
+- a local operating agenda
+- a saved connection record for a real integration
+- a separate opportunities board for contests/programs/grants
 - secrets setup
 - a clean repo-governance read
 - a first intentional checkpoint
@@ -97,6 +117,9 @@ More concretely, `orp init` does these jobs:
   - `orp/HANDOFF.md`
   - `orp/checkpoints/CHECKPOINT_LOG.md`
   - `analysis/orp.kernel.task.yml`
+- scaffolds or updates:
+  - `AGENTS.md`
+  - `CLAUDE.md`
 - writes governance metadata like:
   - `orp/governance.json`
   - `orp/agent-policy.json`
@@ -104,6 +127,31 @@ More concretely, `orp init` does these jobs:
 
 So `orp init` is not "start doing the work." It is "make this repo legible and governable for humans and agents."
 
+The important subtlety is that ORP does not own those agent files outright:
+
+- if `AGENTS.md` or `CLAUDE.md` already exists, ORP preserves the human-written content and only refreshes its own marked blocks
+- if the file is missing, ORP scaffolds it with a sensible starter structure
+- the ORP-managed blocks can point a child repo back to a higher-level umbrella projects root when you configure one
+
+If you keep many repos under one shared parent directory, set that umbrella root once:
+
+```bash
+orp agents root set /absolute/path/to/projects
+```
+
+Then initialize or resync a child repo with the parent explicitly linked:
+
+```bash
+orp init --projects-root /absolute/path/to/projects
+orp agents audit
+```
+
+If you want to refresh the files later without rerunning the full repo bootstrap:
+
+```bash
+orp agents sync
+```
+
 After that, check the repo state:
 
 ```bash
@@ -115,7 +163,7 @@ orp status --json
 This is the local-first memory layer for ongoing work.
 
 ```bash
-orp workspace create main-cody-1
+orp workspace create mac-main --machine-label "Mac Studio"
 orp workspace list
 orp workspace tabs main
 ```
@@ -126,15 +174,21 @@ The important idea is:
 
 - the workspace ledger is not a GUI tab manager
 - it is the saved memory of where work lives
-- it stores paths plus exact resumable command lines
+- it stores machine-specific paths plus exact resumable command lines
+- it can also store portable remote repo URLs and bootstrap commands for another machine
 - it is the crash-recovery layer for you and the agent
 
+This is separate from the agent-instruction hierarchy:
+
+- `workspace` remembers concrete working continuity like paths and resume commands
+- `agents` keeps the higher-level parent/child guidance files aligned
+
 ### 5. Save ongoing tabs and sessions
 
 If you want ORP to remember an ongoing repo path plus a resumable session, add it explicitly:
 
 ```bash
-orp workspace add-tab main --path /absolute/path/to/project --resume-command "codex resume <session-id>"
+orp workspace add-tab main --path /absolute/path/to/project --remote-url git@github.com:org/project.git --bootstrap-command "npm install" --resume-command "codex resume <session-id>"
 ```
 
 For Claude:
@@ -161,6 +215,17 @@ or:
 resume: cd '/absolute/path/to/project' && claude --resume <session-id>
 ```
 
+If you also save a remote repo and bootstrap command, `orp workspace tabs main` will show extra lines like:
+
+```text
+remote: git@github.com:org/project.git [branch main]
+clone: git clone 'git@github.com:org/project.git' 'project'
+bootstrap: npm install
+setup: git clone 'git@github.com:org/project.git' 'project' && cd 'project' && npm install
+```
+
+That is the portable part a second machine can use. The local `path` and the local `resume:` line still belong to the machine that saved the session.
+
 To remove something from the saved ledger:
 
 ```bash
@@ -170,12 +235,168 @@ orp workspace remove-tab main --path /absolute/path/to/project
 So the `workspace` lane is really:
 
 - `create` a ledger
-- `add-tab` to remember a path/session
+- `add-tab` to remember a path/session and, optionally, the remote/bootstrap details needed on another rig
 - `remove-tab` to prune old work
 - `tabs` to inspect exact saved recovery commands
 - `list` to see all saved workspaces
 
-### 6. Set up secrets
+### 6. Create one agenda lane
+
+This is the separate memory lane for ranked action pressure and ranked suggestions. It is intentionally distinct from `workspace`, `connections`, and `opportunities`.
+
+Run one refresh:
+
+```bash
+orp agenda refresh --json
+```
+
+Check whether recurring refreshes are enabled:
+
+```bash
+orp agenda refresh-status --json
+```
+
+If you want ORP to refresh the agenda on a schedule, you must opt in explicitly. Nothing starts making recurring Codex calls until you enable it yourself. The starter preset is morning, afternoon, and evening:
+
+```bash
+orp agenda enable-refreshes --json
+```
+
+If you want your own times instead, override them directly:
+
+```bash
+orp agenda enable-refreshes --morning 08:30 --afternoon 13:00 --evening 18:30 --json
+```
+
+Disable the recurring refreshes any time:
+
+```bash
+orp agenda disable-refreshes --json
+```
+
+Inspect the saved outputs:
+
+```bash
+orp agenda actions
+orp agenda suggestions
+orp agenda focus
+```
+
+If you want ORP to optimize around one explicit north star instead of inferring it from current context:
+
+```bash
+orp agenda set-north-star "Advance the ocular controller and ORP ecosystems"
+```
+
+The practical model is:
+
+- `agenda refresh` is a real Codex reasoning pass
+- it reads current main-workspace context, GitHub pressure, opportunities, and connections
+- it writes two saved ranked lists locally:
+  - `actions`
+  - `suggestions`
+- recurring agenda refreshes are disabled by default until the user enables them
+- the built-in default schedule is morning, afternoon, and evening, but user-chosen times win
+- `focus` is the fastest operator and agent recall surface
+
+### 7. Create one connections registry
+
+This is the separate memory lane for service accounts, data sources, deploy targets, databases, archives, and public research destinations. It is intentionally distinct from both `workspace` and `secrets`.
+
+Inspect the built-in provider templates first if they help:
+
+```bash
+orp connections providers
+```
+
+Add one saved connection that references a saved ORP secret alias:
+
+```bash
+orp connections add github-main --provider github --label "GitHub Main" --account cody --organization sproutseeds --auth-secret-alias github-main
+```
+
+If the same service needs multiple tokens, keep them together under named secret bindings:
+
+```bash
+orp connections add huggingface-main --provider huggingface --label "Hugging Face" --account cody --secret-binding primary=hf-main --secret-binding publish=hf-publish --secret-binding inference=hf-inference
+```
+
+If ORP has never seen the service before, use `custom` and keep going:
+
+```bash
+orp connections add my-science-portal --provider custom --label "My Science Portal" --url https://example.org --secret-binding primary=my-science-token
+```
+
+Inspect or update it later:
+
+```bash
+orp connections show github-main
+orp connections list
+orp connections update github-main --status paused
+```
+
+Mirror the same registry to hosted ORP when you want the same setup on another machine:
+
+```bash
+orp auth login
+orp connections sync --json
+orp connections pull --json
+```
+
+The important idea is:
+
+- `secrets` stores the actual sensitive value
+- `connections` stores the provider/account/capability record and which secret alias or named secret bindings power it
+- `workspace` stores where current work lives
+- built-in providers are optional; `custom` is the straight path for anything new or specialized
+
+### 8. Create one opportunities board
+
+This is the separate memory lane for contests, programs, grants, fellowships, and similar openings. It is intentionally distinct from `workspace`.
+
+Create one board:
+
+```bash
+orp opportunities create main-opportunities --label "Main Opportunities"
+```
+
+Add one tracked item:
+
+```bash
+orp opportunities add main-opportunities --title "vision-prize" --kind contest --section ocular-longevity --priority high --url https://example.com/vision-prize
+```
+
+Inspect the board:
+
+```bash
+orp opportunities show main-opportunities
+orp opportunities focus main-opportunities --limit 5
+orp opportunities list
+```
+
+Update or remove one item:
+
+```bash
+orp opportunities update main-opportunities vision-prize --status submitted
+orp opportunities remove main-opportunities vision-prize
+```
+
+If you authenticate later and want the same board on another machine, mirror it:
+
+```bash
+orp auth login
+orp opportunities sync main-opportunities --json
+orp opportunities pull main-opportunities --json
+```
+
+The important idea is:
+
+- `workspace` remembers where active sessions live
+- `opportunities` remembers what external openings matter
+- agents can read and update both, but they should stay separate
+- local-first is the default; hosted sync is optional
+
+### 9. Set up secrets
 
 Today, secrets are the one part of ORP that still starts from the hosted ORP account layer. So before you save or resolve secrets, log in:
 
@@ -220,6 +441,17 @@ That is the clearest beginner flow:
 4. list or show it later
 5. resolve it when you need to use it
 
+If the service needs a username too, save that alongside the secret:
+
+```bash
+orp secrets add --alias huggingface-login --label "Hugging Face Login" --provider huggingface --kind password --username cody
+```
+
+That means ORP can remember both:
+
+- the secret value
+- the related username/login identifier
+
 #### Agent or script flow
 
 If an agent or script needs to save a key non-interactively, use stdin:
@@ -320,7 +552,7 @@ The right mental model is:
 - if the key is missing and you did not pass a value, ORP prompts you for it
 - `add` is the clearest first command for a brand-new human user
 
-### 7. Start working safely
+### 10. Start working safely
 
 ORP's default repo-governance loop is:
 
@@ -396,7 +628,7 @@ So the checkpoint governance loop is really:
 
 That is the part of ORP that keeps the human, the agent, and the repo in check together.
 
-### 8. Keep the program context visible
+### 11. Keep the program context visible
 
 If the work belongs to a longer-running research or product trajectory:
 
@@ -412,7 +644,7 @@ The distinction is:
 - `workspace` answers: where is the work physically happening?
 - `frontier` answers: where are we logically in the larger program?
 
-### 9. Use optional perspective support
+### 12. Use optional perspective support
 
 When the work feels too linear, too trapped, or too narrow:
 
@@ -429,7 +661,7 @@ It is there to help with:
 - rotating the angle of attack
 - keeping the work fresh without making the workflow sloppy
 
-### 10. Optional hosted sync
+### 13. Optional hosted sync
 
 Hosted ORP is not required for the local workspace ledger or repo-governance loop.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "open-research-protocol",
-  "version": "0.4.19",
+  "version": "0.4.21",
   "description": "ORP CLI (Open Research Protocol): workspace ledgers, secrets, scheduling, governed execution, and agent-friendly research workflows.",
   "license": "MIT",
   "author": "Fractal Research Group <cody@frg.earth>",