open-research-protocol 0.3.0

package/docs/EXTERNAL_CONTRIBUTION_GOVERNANCE.md

@@ -0,0 +1,275 @@
+# External Contribution Governance
+
+Date: 2026-03-07
+
+This document explains how ORP should represent external OSS contribution work such as:
+
+- contributing from a local research repo into `leanprover-community/mathlib4`
+- contributing into `google-deepmind/formal-conjectures`
+- contributing into any other upstream repo where etiquette, coordination, and review discipline matter
+
+It is the ORP-side entry point for PR-governance workflow design.
+
+## What This Document Is
+
+This is not a mathlib document, a formal-conjectures document, or a repo-specific checklist.
+
+It is the ORP contract for external contribution workflow:
+
+- what ORP should standardize
+- what should stay adapter-specific
+- what lifecycle an agent or operator should preserve
+- how to keep governance artifacts process-only while real evidence stays in the working repo
+
+If someone asks, "How should ORP model external PR work without hardcoding one ecosystem's rules?",
+this is the document they should read first.
+
+## What ORP Is Standardizing
+
+ORP should standardize:
+
+- lifecycle states for external contribution work
+- machine-readable gate outcomes
+- packet and summary structure for governance runs
+- the boundary between process metadata and evidence
+- the adapter model for ecosystem-specific commands and etiquette
+
+ORP should not hardwire:
+
+- one project's PR-body wording
+- one project's branch naming rules
+- one project's build commands
+- one project's disclosure or queue policy
+- one project's social norms as if they were universal protocol semantics
+
+Those belong in adapters, rendered configs, or host-repo docs.
+
+## Boundary
+
+ORP remains process-only.
+
+- ORP docs, packets, summaries, and governance reports are not evidence.
+- Evidence remains in canonical artifact paths of the working repo.
+- Governance profiles track workflow quality, review discipline, and readiness state.
+- They do not prove mathematical truth or replace host-repo verification artifacts.
+
+This matters because external contribution work often mixes:
+
+- real repo evidence such as test logs, proof artifacts, CI results, and source diffs
+- process coordination such as overlap checks, PR readiness, draft CI watch, and review-thread handling
+
+ORP is responsible for the second category and should point clearly at the first.
+
+## When To Use This
+
+Use this governance layer when the primary task is:
+
+- selecting upstream work
+- coordinating with an existing contributor
+- validating whether a PR is appropriate to open
+- moving from local branch to draft PR
+- deciding whether a draft is truly ready for review
+- responding to maintainer feedback in a disciplined loop
+
+Do not use this document as the main guide for:
+
+- pure research-claim verification
+- theorem/proof evidence review
+- repo-internal task orchestration that does not involve upstream etiquette
+
+## First Read
+
+Read these in order:
+
+1. `docs/EXTERNAL_CONTRIBUTION_GOVERNANCE.md`
+2. `PROTOCOL.md`
+3. `AGENT_INTEGRATION.md`
+4. `docs/AGENT_LOOP.md`
+5. `docs/OSS_CONTRIBUTION_AGENT_LOOP.md`
+6. `docs/SUNFLOWER_CODA_PR_GOVERNANCE_MAPPING.md`
+7. the current host-repo handoff or contribution-governance notes
+
+Use the current `sunflower-coda` workflow as the reference implementation, but do not copy its
+repo-specific details into ORP core semantics.
+
+## Standard Lifecycle
+
+This is the portable lifecycle ORP should represent.
+
+1. `watch_and_select`
+   Identify a contribution target and confirm it is worth pursuing.
+2. `viability_gate`
+   Check whether the issue or idea is actually appropriate for contribution.
+3. `overlap_and_coordination`
+   Avoid duplicating active work and coordinate before competing.
+4. `isolated_local_work`
+   Work in an isolated branch or worktree.
+5. `local_first_verification`
+   Prefer local validation before spending CI or reviewer attention.
+6. `ready_to_draft_freeze`
+   Pause, tighten, and confirm the branch is genuinely ready to be shown.
+7. `draft_pr_open_or_update`
+   Open or update the PR in draft state first.
+8. `draft_ci_watch`
+   Watch draft CI and fix issues before promotion.
+9. `ready_for_review_decision`
+   Promote only when the full gate chain is satisfied.
+10. `feedback_hardening_loop`
+   Convert maintainer feedback into concrete fixes and new guards when possible.
+
+This lifecycle is the heart of the ORP representation.
+
+## Generic Rules To Preserve
+
+These are the portable norms the ORP side should preserve across ecosystems.
+
+1. Coordinate before competing with an active contributor.
+2. Prefer local verification over CI churn.
+3. Open or update PRs as draft first.
+4. Keep public PR text portable and concise.
+5. Only mark ready for review when the full gate chain is satisfied.
+6. Resolve review threads only after the corresponding fixes are pushed and rechecked.
+7. Thank reviewers directly and specifically after fixes land.
+8. When maintainer feedback reveals a missed check, convert it into a guard if machine-checkable.
+
+## Where This Lives In ORP
+
+The intended ORP layout is:
+
+- canonical governance doc:
+  - `docs/EXTERNAL_CONTRIBUTION_GOVERNANCE.md`
+- operator-facing loop:
+  - `docs/OSS_CONTRIBUTION_AGENT_LOOP.md`
+- generic pack root:
+  - `packs/external-pr-governance/`
+- adapters:
+  - `packs/external-pr-governance/adapters/mathlib/`
+  - `packs/external-pr-governance/adapters/formal-conjectures/`
+- generic profile templates:
+  - `packs/external-pr-governance/profiles/oss-pr-governance.yml.tmpl`
+  - `packs/external-pr-governance/profiles/oss-feedback-hardening.yml.tmpl`
+- illustrative example:
+  - `examples/orp.external-pr-governance.yml`
+
+## Reference Implementation
+
+Right now, the strongest live reference implementation is in `sunflower-coda`.
+
+Treat these as the model for behavior, not as ORP-core files to copy directly:
+
+- host-repo docs for submission checklist, viability gate, PR template, feedback loop, and upstream lane
+- host-repo analysis artifacts for issue watchlists and watch status
+- host-repo scripts for viability, overlap, local gate, tighten, ready-to-draft, PR opening, and PR-body preflight
+
+ORP's job is to extract the generic shape from that implementation without freezing `sunflower-coda`
+details into core semantics.
+
+## What Belongs Where
+
+Use this split when porting an external contribution workflow into ORP.
+
+### ORP core
+
+Belongs here:
+
+- generic lifecycle semantics
+- packet/report schema
+- machine-readable run outcomes
+- process/evidence boundary
+
+Does not belong here:
+
+- mathlib-specific branch naming
+- formal-conjectures-specific build commands
+- repo-specific PR templates
+- repo-specific queue policy text
+
+### Generic governance pack
+
+Belongs here:
+
+- generic pre-open, local-readiness, draft-lifecycle, and feedback-hardening profiles
+- generic variable names
+- generic gate naming
+- generic lifecycle ordering
+
+### Adapters
+
+Belongs here:
+
+- exact commands
+- exact pass tokens
+- exact host-repo file paths
+- exact etiquette overlays that are specific to one ecosystem
+
+### Host repo
+
+Belongs here:
+
+- actual source changes
+- actual test and verification artifacts
+- actual PR text and issue references
+- actual evidence and reviewer-visible outputs
+
+## Definition Of Done
+
+Use this checklist to decide whether the ORP-side port is actually complete.
+
+### Coverage
+
+- `PASS` if the ORP workflow covers:
+  - watch
+  - viability
+  - overlap
+  - local gate
+  - ready-to-draft
+  - draft PR
+  - draft CI
+  - ready-for-review
+  - feedback hardening
+
+### Separation
+
+- `PASS` if ORP core remains generic.
+- `FAIL` if ecosystem-specific rules leak into ORP core semantics.
+
+### Contracts
+
+- `PASS` if decisions and gate outcomes are machine-readable.
+- `PASS` if lifecycle states map cleanly into ORP `workflow_state`.
+
+### Etiquette
+
+- `PASS` if the ORP representation preserves:
+  - coordination-first behavior
+  - draft-first behavior
+  - no-ready-before-green
+  - concise public replies
+  - review-thread discipline
+
+### Validation
+
+- `PASS` if the ORP side includes at least:
+  - one mathlib example run
+  - one formal-conjectures example run
+  - one blocked or coordinate-first example
+  - one full green local-to-ready chain
+
+### Hardening
+
+- `PASS` if the ORP port explicitly requires:
+  - missed check -> new guard when machine-checkable
+  - docs/template/instruction update in the same pass
+  - PASS + FAIL evidence for the new guard behavior
+
+## Recommended Next Move
+
+Treat mathlib as the reference implementation and formal-conjectures as the first non-mathlib
+adapter.
+
+The next ORP implementation step should be:
+
+1. define the generic external-contribution pack contract
+2. keep the existing mathlib implementation as the reference adapter
+3. add a formal-conjectures adapter with the same lifecycle shape
+4. only then tighten prompts and operator instructions around it

package/docs/MATHLIB_COLLABORATION_FLOW_PROMPT.md

@@ -0,0 +1,112 @@
+# ORP Agent Prompt, Mathlib Collaboration Flow
+
+You are onboarding into ORP with no prior chat context.
+
+Goal:
+Encode a proven, local-first mathlib collaboration workflow into ORP in a way that is reusable, auditable, and maintainer-friendly.
+
+## Primary source of truth to ingest first
+
+Read these files from this path:
+`/path/to/sunflower-coda/repo`
+
+- `docs/MATHLIB_SUBMISSION_CHECKLIST.md`
+- `docs/MATHLIB_ISSUE_VIABILITY_GATE.md`
+- `docs/MATHLIB_DRAFT_PR_TEMPLATE.md`
+- `docs/MATHLIB_NATURALITY_SNIPPET.md`
+- `docs/WORKTREE_LANES.md`
+- `scripts/mathlib-issue-local-gate.sh`
+- `scripts/mathlib-issue-viability-gate.py`
+- `scripts/mathlib-overlap-precheck.sh`
+- `scripts/mathlib-pr-body-preflight.py`
+- `scripts/install-mathlib-local-hook.sh`
+- `scripts/issue-watch.py`
+- `analysis/OSS_ISSUE_WATCHLIST.json`
+- `analysis/OSS_ISSUE_WATCH_STATUS.md`
+
+## Non-negotiable constraints
+
+1. ORP docs/templates are process, not evidence.
+2. Evidence remains in canonical artifact paths, packets only reference those paths.
+3. Local-first gateing comes before draft CI.
+4. Coordination-first etiquette applies for assigned or already-active issues.
+5. Public PR text must be concise and portable, never machine-local.
+6. No auto-merge behavior, ORP standardizes evidence and reviewability.
+
+## Required mathlib flow to represent in ORP
+
+### A. Pre-work viability and overlap gates
+
+- Viability decisions must include:
+  - `PASS`
+  - `COORDINATE`
+  - `CLOSE_CANDIDATE`
+  - `BLOCKED_NOT_OPEN`
+- Include overlap precheck against open PRs before drafting.
+- Require a natural-generality check before implementation.
+
+### B. Local implementation gates
+
+- Branch naming includes issue number.
+- Targeted build and targeted linter run.
+- Local gate runner is canonical pass/fail decision.
+- Support local-hold mode where work remains local with no draft PR.
+
+### C. Draft and review lifecycle
+
+Use this lifecycle exactly:
+`local_green -> draft_open -> draft_ci_green -> ready_for_review`
+
+Track at least:
+- action owner
+- last checked timestamp
+- last reviewed timestamp
+- work stage
+- whether review is needed
+
+### D. PR body hygiene gate
+
+Require:
+- concise summary
+- portable validation wording
+- no absolute local paths
+- no machine-only helper commands in public PR text
+
+## What to produce in ORP
+
+1. A proposal to update ORP positioning language to:
+   `spec-first core, optional runtime, optional profiles`
+   while preserving current protocol guarantees.
+2. ORP schema mapping for collaborator states and gate outcomes.
+3. A mathlib profile config with command mappings for the gates above.
+4. A minimal runtime plan that wraps existing scripts first.
+5. Migration notes from current scripts to ORP profile adapters.
+6. A short risk register with mitigations.
+
+## Delivery format
+
+Return all of the following:
+
+1. Proposed repo tree changes.
+2. Profile schema snippet.
+3. State machine table.
+4. Sample packet JSON that references canonical evidence paths.
+5. Phased rollout plan:
+   - Phase 1: spec + profile
+   - Phase 2: runtime wrappers
+   - Phase 3: reviewer/collaborator UX modules
+
+## Decision defaults
+
+Unless blocked, default to:
+- Language: TypeScript/Node for v1 CLI speed
+- Config: `orp.yml` with explicit gate command contracts
+
+If you propose a different default, include a concrete tradeoff table.
+
+## Scope discipline
+
+- Do not redesign ORP from scratch.
+- Preserve ORP epistemic boundaries from `PROTOCOL.md`.
+- Keep MVP small, local-first, deterministic.
+- Optimize for maintainers adopting the least-scary slice first.

package/docs/NPM_RELEASE_CHECKLIST.md

@@ -0,0 +1,55 @@
+# npm Release Checklist (`open-research-protocol`)
+
+Use this checklist to publish professional, versioned ORP CLI releases.
+
+## One-time setup
+
+1. Confirm npm package ownership for `open-research-protocol`.
+2. Add repository secret `NPM_TOKEN` in GitHub Actions:
+   - token should have permission to publish this package.
+3. Confirm package metadata in `package.json`:
+   - `name`, `version`, `repository`, `bin`.
+
+## Per-release flow
+
+1. Ensure `main` is green and local tests pass:
+   - `python3 -m unittest discover -s tests -v`
+   - `npm pack --dry-run --cache /tmp/orp-npm-cache`
+2. Bump version in `package.json` (for example `0.1.1`).
+3. Commit and push the version bump to `main`.
+4. Create and push a matching tag:
+   - `git tag v0.1.1`
+   - `git push origin v0.1.1`
+5. Watch workflow:
+   - `.github/workflows/npm-publish.yml`
+   - tag push is the normal publish trigger
+6. Verify npm install:
+   - `npm i -g open-research-protocol`
+   - `orp -h`
+
+## Important guardrail
+
+Tag version must match `package.json` exactly.
+
+- Example:
+  - tag: `v0.1.1`
+  - package version: `0.1.1`
+
+The publish workflow hard-fails if these differ.
+
+Manual workflow dispatch is available as a recovery path, but it still requires the same exact version string.
+
+## Manual publish fallback
+
+If automation is temporarily unavailable:
+
+1. Checkout intended commit locally.
+2. Run release validations above.
+3. Publish:
+   - `npm publish --access public`
+4. Create and push the matching tag:
+   - `git tag v0.1.1`
+   - `git push origin v0.1.1`
+5. Add release notes.
+
+The tag-triggered workflow will still validate the version and will skip `npm publish` if that exact npm version already exists.

package/docs/ORP_V1_ATOMIC_DISCOVERY_EVOLUTION.md

@@ -0,0 +1,186 @@
+# ORP v1 Evolution: Atomic Discovery + Team Collaboration
+
+This document extends ORP from a docs-first protocol into a local-first runtime model that still preserves ORP core boundaries.
+
+## Why this evolution
+
+ORP cannot be only a GitHub/PR collaboration wrapper. In sunflower-coda style workflows, truth advances through:
+
+- atomic board states (`tickets -> gates -> atoms`),
+- scope-lock and spec-faithfulness checks,
+- deterministic theorem-cycle gates,
+- route-progress verification (strict and loose).
+
+So ORP v1 must model discovery loops and collaboration loops as first-class peers.
+
+## Core model
+
+1. ORP Spec (stable core)
+- Protocol rules, claim levels, downgrade rules.
+- Versioned schemas for packets/config.
+- Lifecycle mapping for claim and atomic states.
+
+2. ORP Runtime (optional executor)
+- Runs configured gates.
+- Captures stdout/stderr/timings/evidence pointers.
+- Emits packet JSON + packet markdown + artifact logs.
+
+3. ORP Profiles (optional apps)
+- `discovery` profile for atomic problem-scope loops.
+- `collaboration` profile for PR prep and handoff.
+- `review` profile for triage and verification.
+
+## Proposed repo tree
+
+```text
+orp/
+  PROTOCOL.md
+  templates/
+  spec/
+    v1/
+      packet.schema.json
+      orp.config.schema.json
+      LIFECYCLE_MAPPING.md
+  examples/
+    orp.sunflower-coda.atomic.yml
+    packet.problem_scope.example.json
+  docs/
+    ORP_V1_ATOMIC_DISCOVERY_EVOLUTION.md
+  scripts/
+    orp-init.sh
+    orp-checkpoint.sh
+    orp-agent-integrate.sh
+  cli/                         # planned runtime package
+    src/
+      commands/
+        init.ts
+        gate-run.ts
+        packet-emit.ts
+      profiles/
+        discovery.ts
+        collaboration.ts
+        review.ts
+      packet/
+        writer-json.ts
+        writer-markdown.ts
+      gates/
+        runner.ts
+        evaluator.ts
+```
+
+## Lifecycle compatibility contract
+
+Claim workflow stays compatible with existing templates:
+
+- `Draft -> draft`
+- `In review -> ready`
+- `Verified -> reviewed`
+- `Blocked -> blocked`
+- `Retracted -> retracted`
+
+Atomic discovery workflow adds board-native mappings:
+
+- `todo -> draft`
+- `in_progress -> ready`
+- `blocked -> blocked`
+- `done -> reviewed`
+
+`accepted` remains explicit; do not auto-promote without recorded acceptance criteria.
+
+## Packet kinds and minimum outputs
+
+ORP v1 packet kinds:
+
+- `pr`
+- `claim`
+- `verification`
+- `problem_scope`
+- `atom_pass`
+
+Minimum outputs per run:
+
+- `orp/packets/<id>.json`
+- `orp/packets/<id>.md`
+- `orp/artifacts/<run_id>/...`
+
+For discovery packets, include board context:
+
+- `board_id`, `problem_id`, `ticket_id`, `gate_id`, `atom_id`
+- atom dependencies and ready-queue size
+- route status snapshot
+- board JSON snapshot path
+
+## Sunflower-coda profile mapping
+
+The example profile (`examples/orp.sunflower-coda.atomic.yml`) treats existing board operations as canonical gates:
+
+- board refresh gate (`problem857_ops_board.py refresh`)
+- ready-queue gate (`problem857_ops_board.py ready`)
+- spec-faithfulness gate (`orchestrator/problem857_public_spec_check.py`)
+- Lean build gate (`lake build SunflowerLean.Balance`)
+- frontier/route gate (`scripts/frontier_status.py --problem 857`)
+
+This means ORP wraps your proven local gates instead of replacing them.
+
+## Migration path for this repo
+
+1. Keep `PROTOCOL.md` and templates as the constitution.
+2. Keep current shell scripts as stable compatibility layer.
+3. Add runtime CLI as optional (not mandatory for adoption).
+4. Ship discovery profile first for atomic board users.
+5. Keep reviewer/collaborator profiles optional modules.
+
+This keeps ORP core stable while letting high-rigor workflows (like sunflower-coda) adopt runtime enforcement immediately.
+
+## Pack Distribution Model
+
+To keep ORP core repo-agnostic and still support specialized domains:
+
+- publish domain workflows as profile packs under `packs/` (or external repos),
+- define pack metadata in `pack.yml` (schema: `spec/v1/profile-pack.schema.json`),
+- ship templates with placeholders (for example `{{TARGET_REPO_ROOT}}`),
+- render to concrete config via `scripts/orp-pack-render.py`.
+
+This makes profiles portable and installable without hardcoding one team's repo paths into ORP core.
+
+Collaboration governance example is now captured as a separate pack template:
+
+- `packs/erdos-open-problems/profiles/sunflower-mathlib-pr-governance.yml.tmpl`
+
+It models pre-open viability/policy checks and draft-readiness gates as optional collaboration profiles, parallel to discovery profiles.
+
+## Minimal CLI Now Available
+
+A minimal runtime skeleton now exists at:
+
+- `cli/orp.py`
+- `scripts/orp` (launcher)
+
+Supported commands:
+
+```bash
+./scripts/orp init
+./scripts/orp gate run --profile default
+./scripts/orp packet emit --profile default
+./scripts/orp report summary --run-id <run_id>
+./scripts/orp pack install --pack-id erdos-open-problems --target-repo-root /path/to/repo
+./scripts/orp erdos sync
+```
+
+For sunflower-style discovery profiles:
+
+```bash
+./scripts/orp --repo-root /path/to/repo --config examples/orp.sunflower-coda.atomic.yml \
+  gate run --profile sunflower_problem857_discovery
+
+./scripts/orp --repo-root /path/to/repo --config examples/orp.sunflower-coda.atomic.yml \
+  packet emit --profile sunflower_problem857_discovery --kind problem_scope
+```
+
+Current scope of this runtime:
+
+- sequential gate execution with pass/fail rule checks,
+- captured stdout/stderr logs in `orp/artifacts/<run_id>/`,
+- one-page run summaries (`orp/artifacts/<run_id>/RUN_SUMMARY.md`) via `orp report summary`,
+- run state in `orp/state.json`,
+- packet JSON + markdown emission to `orp/packets/`.

package/docs/OSS_CONTRIBUTION_AGENT_LOOP.md

@@ -0,0 +1,69 @@
+# OSS Contribution Agent Loop
+
+Use this loop when the task is external OSS contribution workflow rather than a research claim.
+
+## 1. Orient
+
+- Read `docs/EXTERNAL_CONTRIBUTION_GOVERNANCE.md`.
+- Read `PROTOCOL.md`.
+- Keep the ORP boundary in force: governance artifacts are process-only, not evidence.
+
+## 2. Choose The Adapter
+
+- If the target is mathlib, inspect:
+  - `packs/external-pr-governance/adapters/mathlib/README.md`
+- If the target is formal-conjectures, inspect:
+  - `packs/external-pr-governance/adapters/formal-conjectures/README.md`
+
+If the target repo is new, model its adapter on those two.
+
+## 3. Render Or Draft A Governance Config
+
+- Start from:
+  - `packs/external-pr-governance/pack.yml`
+- Render or adapt:
+  - `packs/external-pr-governance/profiles/oss-pr-governance.yml.tmpl`
+  - `packs/external-pr-governance/profiles/oss-feedback-hardening.yml.tmpl`
+
+Fastest path:
+
+```bash
+orp pack install --pack-id external-pr-governance
+```
+
+That gives you install-and-adapt configs with explicit placeholder commands. Replace those adapter hooks before running meaningful governance passes.
+
+If a host repo already has a stronger adapter, prefer that adapter over the generic template.
+
+## 4. Run The Lifecycle In Order
+
+1. `external_watch_select`
+2. `external_pre_open`
+3. `external_local_readiness`
+4. `external_draft_transition`
+5. `external_draft_lifecycle`
+6. `external_feedback_hardening` when maintainer feedback reveals a missed check
+
+Do not skip directly from local work to ready-for-review.
+
+## 5. Preserve Etiquette
+
+- Coordinate before competing with active contributors.
+- Keep PR text portable and concise.
+- Open or update as draft first.
+- Only mark ready when local checks, freeze/readiness checks, and draft CI are all green.
+- Resolve review threads only after the corresponding fix is pushed and rechecked.
+
+## 6. Checkpoint
+
+Before handoff, compaction, `git commit`, or `git push`:
+
+```sh
+scripts/orp-checkpoint.sh --sync --agent-file /path/to/agent/instructions.md "oss contribution governance checkpoint"
+```
+
+## 7. Respect The Boundary
+
+- Governance profiles and packets describe workflow quality.
+- The underlying repo artifacts remain the evidence source.
+- If a guard fails, downgrade workflow confidence immediately and preserve the failure state.