open-research-protocol 0.4.6 → 0.4.8

package/docs/ORP_REASONING_KERNEL_EVALUATION_PLAN.md

@@ -0,0 +1,261 @@
+# ORP Reasoning Kernel Evaluation Plan
+
+This document turns the remaining kernel evidence gaps into concrete next
+experiments.
+
+The goal is to upgrade the kernel from:
+
+- technically valid and operationally useful
+
+to:
+
+- comparatively validated against real alternatives and real project outcomes
+
+Supporting references:
+
+- [docs/ORP_REASONING_KERNEL_COMPARISON_PILOT.md](/Volumes/Code_2TB/code/orp/docs/ORP_REASONING_KERNEL_COMPARISON_PILOT.md)
+- [docs/ORP_REASONING_KERNEL_PICKUP_PILOT.md](/Volumes/Code_2TB/code/orp/docs/ORP_REASONING_KERNEL_PICKUP_PILOT.md)
+- [docs/ORP_REASONING_KERNEL_AGENT_PILOT.md](/Volumes/Code_2TB/code/orp/docs/ORP_REASONING_KERNEL_AGENT_PILOT.md)
+- [docs/ORP_REASONING_KERNEL_AGENT_REPLICATION.md](/Volumes/Code_2TB/code/orp/docs/ORP_REASONING_KERNEL_AGENT_REPLICATION.md)
+- [docs/ORP_REASONING_KERNEL_CONTINUATION_PILOT.md](/Volumes/Code_2TB/code/orp/docs/ORP_REASONING_KERNEL_CONTINUATION_PILOT.md)
+- [docs/ORP_REASONING_KERNEL_CANONICAL_CONTINUATION_PILOT.md](/Volumes/Code_2TB/code/orp/docs/ORP_REASONING_KERNEL_CANONICAL_CONTINUATION_PILOT.md)
+- [docs/ORP_REASONING_KERNEL_EVIDENCE_MATRIX.md](/Volumes/Code_2TB/code/orp/docs/ORP_REASONING_KERNEL_EVIDENCE_MATRIX.md)
+- [docs/ORP_REASONING_KERNEL_TECHNICAL_VALIDATION.md](/Volumes/Code_2TB/code/orp/docs/ORP_REASONING_KERNEL_TECHNICAL_VALIDATION.md)
+
+## Evaluation Principles
+
+Every kernel evaluation should be:
+
+- comparative, not just descriptive
+- cross-domain where possible
+- judged by downstream usefulness, not only schema validity
+- reproducible and artifact-backed
+
+The main alternatives to compare against are:
+
+1. Free-form artifact writing
+2. Generic checklist artifact writing
+3. ORP typed kernel artifact writing
+
+The current internal package now includes:
+
+- deterministic structural comparison
+- deterministic pickup proxy
+- live fresh-agent recoverability simulation
+- live fresh-agent replication
+- softer downstream continuation
+- harder canonical downstream continuation
+- CI threshold checks over the committed benchmark package
+
+## Experiment 1: Artifact Clarity And Completeness
+
+Status:
+
+- a first deterministic internal comparison harness now exists
+- what remains is blinded human scoring and a larger prompt set
+
+### Question
+
+Does the ORP kernel produce more complete and legible promotable artifacts than
+free-form writing or a generic checklist?
+
+### Setup
+
+- Select 20 prompts spread across:
+  - software
+  - research
+  - product/design
+  - operations
+  - writing/knowledge work
+- For each prompt, produce three artifact versions:
+  - free-form
+  - generic checklist
+  - ORP kernel
+
+### Scoring
+
+Blind-review each artifact against:
+
+- artifact type clarity
+- boundary clarity
+- constraint clarity
+- evaluation clarity
+- handoff readiness
+- ambiguity remaining
+
+### Primary metric
+
+- mean reviewer score by condition
+
+### Success criterion
+
+- kernel condition beats free-form and generic checklist on at least four of
+  six scoring dimensions
+
+## Experiment 2: Handoff Pickup Study
+
+Status:
+
+- a first deterministic pickup proxy now exists
+- a first live Codex pickup simulation now exists
+- a `10`-repeat live Codex replication layer now exists with per-field stability tables
+- what remains is live human pickup measurement and broader cross-model replication
+
+### Question
+
+Does the ORP kernel improve pickup quality for another human or agent?
+
+### Setup
+
+- Use matched artifacts from Experiment 1
+- Give a second operator one artifact at a time and ask them to:
+  - explain the task
+  - state the constraints
+  - identify success criteria
+  - identify next action
+
+### Scoring
+
+- time to correct interpretation
+- interpretation accuracy
+- number of clarifying questions required
+
+### Primary metric
+
+- successful pickup rate without clarification
+
+### Success criterion
+
+- kernel artifacts reduce clarifying questions and increase correct pickup rate
+
+## Experiment 3: Downstream Execution Quality
+
+Status:
+
+- a softer live continuation benchmark exists
+- a harder canonical continuation benchmark now exists
+- what remains is repeated canonical continuation, cross-model continuation,
+  and true downstream execution quality beyond artifact production
+
+### Question
+
+Does kernel-structured promotion improve downstream execution or review
+success?
+
+### Setup
+
+- Choose a fixed set of implementation or research tasks
+- Feed matched task artifacts to agents or operators
+- Compare execution using:
+  - free-form task definition
+  - generic checklist task definition
+  - kernel task artifact
+
+### Scoring
+
+- completion rate
+- rework rate
+- reviewer acceptance
+- alignment with stated constraints
+- mismatch between claimed and delivered outcome
+
+### Primary metric
+
+- accepted completion rate with minimal rework
+
+### Success criterion
+
+- kernel condition improves acceptance or reduces rework materially
+
+## Experiment 4: Cross-Domain Corpus Fit
+
+### Question
+
+Do the current kernel artifact classes fit real project work across domains?
+
+### Setup
+
+- Collect 50 to 100 real artifacts across:
+  - software
+  - research
+  - design/product
+  - ops/reliability
+  - writing/editorial
+- Map each artifact into the current kernel classes
+
+### Scoring
+
+- clean fit
+- awkward fit
+- no fit
+- missing field pressure
+- repeated need for new field or class
+
+### Primary metric
+
+- percent of artifacts that map cleanly without schema strain
+
+### Success criterion
+
+- at least 80 percent clean fit across the corpus
+
+## Experiment 5: Operational Warning Value
+
+### Question
+
+Do soft-mode kernel warnings predict later rework or low-quality promotion?
+
+### Setup
+
+- instrument real ORP repos using soft-mode kernel gates
+- log:
+  - warning presence
+  - later edits to the same artifact
+  - eventual hard-mode pass/fail
+  - downstream rework indicators
+
+### Primary metric
+
+- correlation between early warnings and later rework
+
+### Success criterion
+
+- warnings show predictive value strong enough to justify continued soft-mode
+  emphasis
+
+## Suggested Order
+
+The best order is:
+
+1. Artifact clarity and completeness
+2. Handoff pickup
+3. Cross-domain corpus fit
+4. Downstream execution quality
+5. Operational warning value
+
+That sequence gives the fastest evidence on whether the kernel is genuinely
+useful before investing in longer operational studies.
+
+## Minimal Evidence Package To Upgrade v0.1 Claims
+
+If we want to move from “strong implementation” to “serious comparative
+validation,” the smallest next package is:
+
+1. a 20-prompt comparative artifact study
+2. a pickup study on those same artifacts
+3. a cross-domain corpus fit table
+
+That would be enough to substantially strengthen the kernel’s public claims.
+
+## Bottom Line
+
+The kernel is already technically real and operationally validated.
+
+What remains is comparative evidence:
+
+- better than free-form?
+- better than a generic checklist?
+- good across real domains?
+- useful in real handoffs?
+
+This plan defines how to answer those questions cleanly.

package/docs/ORP_REASONING_KERNEL_EVIDENCE_MATRIX.md

@@ -0,0 +1,131 @@
+# ORP Reasoning Kernel Evidence Matrix
+
+This document separates what the ORP Reasoning Kernel currently proves from
+what it only suggests.
+
+Its purpose is to prevent the kernel from being over-claimed. The kernel is
+stronger when we can say, precisely:
+
+- what is already validated
+- what is only partially supported
+- what is still unproven
+- what experiment would close the gap
+
+Supporting references:
+
+- [docs/ORP_REASONING_KERNEL_V0_1.md](/Volumes/Code_2TB/code/orp/docs/ORP_REASONING_KERNEL_V0_1.md)
+- [docs/ORP_REASONING_KERNEL_TECHNICAL_VALIDATION.md](/Volumes/Code_2TB/code/orp/docs/ORP_REASONING_KERNEL_TECHNICAL_VALIDATION.md)
+- [docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json](/Volumes/Code_2TB/code/orp/docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json)
+- [docs/ORP_REASONING_KERNEL_AGENT_PILOT.md](/Volumes/Code_2TB/code/orp/docs/ORP_REASONING_KERNEL_AGENT_PILOT.md)
+- [docs/ORP_REASONING_KERNEL_AGENT_REPLICATION.md](/Volumes/Code_2TB/code/orp/docs/ORP_REASONING_KERNEL_AGENT_REPLICATION.md)
+- [docs/ORP_REASONING_KERNEL_CONTINUATION_PILOT.md](/Volumes/Code_2TB/code/orp/docs/ORP_REASONING_KERNEL_CONTINUATION_PILOT.md)
+- [docs/ORP_REASONING_KERNEL_CANONICAL_CONTINUATION_PILOT.md](/Volumes/Code_2TB/code/orp/docs/ORP_REASONING_KERNEL_CANONICAL_CONTINUATION_PILOT.md)
+
+## Evidence Grades
+
+- `A`
+  Directly supported by shipped implementation, tests, and repeatable benchmark
+  evidence in this repo.
+- `B`
+  Strongly supported by implementation behavior and design logic, but still
+  missing comparative or external validation.
+- `C`
+  Plausible and well-motivated, but not yet measured directly.
+- `D`
+  Strategic aspiration only. No meaningful validation yet.
+
+## What Is Sealed For v0.1
+
+These claims are strong enough to treat as validated implementation truths for
+the current kernel release.
+
+| Claim | Grade | Current Evidence | Why It Matters |
+| --- | --- | --- | --- |
+| ORP has a real typed kernel artifact surface. | A | [spec/v1/kernel.schema.json](/Volumes/Code_2TB/code/orp/spec/v1/kernel.schema.json), [cli/orp.py](/Volumes/Code_2TB/code/orp/cli/orp.py) | The kernel is not just prose. It is an enforceable CLI surface. |
+| `orp init` seeds a valid starter kernel artifact and validates it in the default flow. | A | [tests/test_orp_init.py](/Volumes/Code_2TB/code/orp/tests/test_orp_init.py), [docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json](/Volumes/Code_2TB/code/orp/docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json) | New repos get the kernel by default instead of needing manual adoption. |
+| All seven v0.1 artifact classes can scaffold and validate successfully. | A | [tests/test_orp_kernel.py](/Volumes/Code_2TB/code/orp/tests/test_orp_kernel.py), [docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json](/Volumes/Code_2TB/code/orp/docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json) | The kernel is broad enough for multiple project artifact types. |
+| Hard mode blocks invalid promotable artifacts. | A | [tests/test_orp_kernel.py](/Volumes/Code_2TB/code/orp/tests/test_orp_kernel.py), [docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json](/Volumes/Code_2TB/code/orp/docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json) | ORP can enforce structural promotion standards rather than only advising. |
+| Soft mode records invalidity without blocking work. | A | [tests/test_orp_kernel.py](/Volumes/Code_2TB/code/orp/tests/test_orp_kernel.py), [docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json](/Volumes/Code_2TB/code/orp/docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json) | ORP can stay fluid at intake while still surfacing missing structure. |
+| Existing `structure_kernel` gates remain compatible when no explicit kernel config is present. | A | [tests/test_orp_kernel.py](/Volumes/Code_2TB/code/orp/tests/test_orp_kernel.py), [docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json](/Volumes/Code_2TB/code/orp/docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json) | The kernel does not silently break earlier ORP configurations. |
+| One-shot local kernel CLI operations are within human-scale latency on the reference machine. | A | [scripts/orp-kernel-benchmark.py](/Volumes/Code_2TB/code/orp/scripts/orp-kernel-benchmark.py), [docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json](/Volumes/Code_2TB/code/orp/docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json) | The kernel is operationally lightweight enough to use during normal work. |
+| A small cross-domain reference corpus fits the current class set cleanly. | A | [examples/kernel/corpus](/Volumes/Code_2TB/code/orp/examples/kernel/corpus), [tests/test_orp_kernel_corpus.py](/Volumes/Code_2TB/code/orp/tests/test_orp_kernel_corpus.py), [docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json](/Volumes/Code_2TB/code/orp/docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json) | The kernel now has explicit cross-domain fit evidence, not only rationale. |
+| Each artifact class rejects a candidate when a required field is removed. | A | [tests/test_orp_kernel_corpus.py](/Volumes/Code_2TB/code/orp/tests/test_orp_kernel_corpus.py), [docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json](/Volumes/Code_2TB/code/orp/docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json) | Class-specific enforcement is directly proven instead of inferred from a subset of cases. |
+| The CLI validator stays aligned with the published kernel schema. | A | [tests/test_orp_kernel_corpus.py](/Volumes/Code_2TB/code/orp/tests/test_orp_kernel_corpus.py), [docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json](/Volumes/Code_2TB/code/orp/docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json) | The kernel no longer relies on an undocumented validator rule set drifting away from the schema. |
+| Equivalent YAML and JSON artifacts validate to the same semantic result. | A | [tests/test_orp_kernel_corpus.py](/Volumes/Code_2TB/code/orp/tests/test_orp_kernel_corpus.py), [docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json](/Volumes/Code_2TB/code/orp/docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json) | The protocol is representation-stable rather than format-sensitive. |
+| The validator rejects adversarial near-miss artifacts. | A | [tests/test_orp_kernel_corpus.py](/Volumes/Code_2TB/code/orp/tests/test_orp_kernel_corpus.py), [docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json](/Volumes/Code_2TB/code/orp/docs/benchmarks/orp_reasoning_kernel_v0_1_validation.json) | The kernel is stronger against malformed or gameable inputs than before. |
+| On a matched internal comparison corpus, kernel artifacts outperform both free-form and generic checklist artifacts on structural scoring. | A | [docs/ORP_REASONING_KERNEL_COMPARISON_PILOT.md](/Volumes/Code_2TB/code/orp/docs/ORP_REASONING_KERNEL_COMPARISON_PILOT.md), [docs/benchmarks/orp_reasoning_kernel_comparison_v0_1.json](/Volumes/Code_2TB/code/orp/docs/benchmarks/orp_reasoning_kernel_comparison_v0_1.json), [scripts/orp-kernel-comparison.py](/Volumes/Code_2TB/code/orp/scripts/orp-kernel-comparison.py) | ORP now has direct comparative evidence for structural artifact quality on a matched internal corpus, not only rationale. |
+| On a matched internal pickup proxy, kernel artifacts preserve more explicit handoff-critical information than both free-form and generic checklist artifacts. | A | [docs/ORP_REASONING_KERNEL_PICKUP_PILOT.md](/Volumes/Code_2TB/code/orp/docs/ORP_REASONING_KERNEL_PICKUP_PILOT.md), [docs/benchmarks/orp_reasoning_kernel_pickup_v0_1.json](/Volumes/Code_2TB/code/orp/docs/benchmarks/orp_reasoning_kernel_pickup_v0_1.json), [scripts/orp-kernel-pickup.py](/Volumes/Code_2TB/code/orp/scripts/orp-kernel-pickup.py) | ORP now has a second comparative signal showing that kernel structure turns into more explicit pickup value, not just fuller-looking artifacts. |
+| On a matched live Codex recoverability simulation, kernel artifacts preserve full required-field recoverability, outperform free-form artifacts on all matched cases, and outperform generic checklist artifacts on average without per-case losses. | A | [docs/ORP_REASONING_KERNEL_AGENT_PILOT.md](/Volumes/Code_2TB/code/orp/docs/ORP_REASONING_KERNEL_AGENT_PILOT.md), [docs/benchmarks/orp_reasoning_kernel_agent_pilot_v0_1.json](/Volumes/Code_2TB/code/orp/docs/benchmarks/orp_reasoning_kernel_agent_pilot_v0_1.json), [scripts/orp-kernel-agent-pilot.py](/Volumes/Code_2TB/code/orp/scripts/orp-kernel-agent-pilot.py) | ORP now has direct in-environment agent evidence that the kernel’s structural advantage survives contact with a real fresh downstream Codex session. |
+| On a `10`-repeat full-corpus live Codex replication pilot, the kernel’s recoverability advantage stays stable across fresh-session reruns, with zero invention, no run-level losses, and perfect per-field stability on required kernel fields. | A | [docs/ORP_REASONING_KERNEL_AGENT_REPLICATION.md](/Volumes/Code_2TB/code/orp/docs/ORP_REASONING_KERNEL_AGENT_REPLICATION.md), [docs/benchmarks/orp_reasoning_kernel_agent_replication_v0_2.json](/Volumes/Code_2TB/code/orp/docs/benchmarks/orp_reasoning_kernel_agent_replication_v0_2.json), [scripts/orp-kernel-agent-replication.py](/Volumes/Code_2TB/code/orp/scripts/orp-kernel-agent-replication.py) | ORP now has stronger repeatability evidence that the live agent result is not just a single-run artifact and that the structural advantage survives at field level, not only in aggregate means. |
+| On a matched full-corpus live continuation pilot, kernel artifacts support the strongest continuation score, never underperform the generic checklist baseline, and keep invention at zero. | A | [docs/ORP_REASONING_KERNEL_CONTINUATION_PILOT.md](/Volumes/Code_2TB/code/orp/docs/ORP_REASONING_KERNEL_CONTINUATION_PILOT.md), [docs/benchmarks/orp_reasoning_kernel_continuation_v0_1.json](/Volumes/Code_2TB/code/orp/docs/benchmarks/orp_reasoning_kernel_continuation_v0_1.json), [scripts/orp-kernel-continuation-pilot.py](/Volumes/Code_2TB/code/orp/scripts/orp-kernel-continuation-pilot.py) | ORP now has direct agent-first evidence that kernel artifacts are not only recoverable, but also a safe and effective base for downstream continuation. |
+| On a harder matched full-corpus canonical continuation pilot, kernel artifacts beat free-form on every case, beat checklist on average, and keep the lowest invention rate while revealing checklist as a real competitive baseline on some cases. | A | [docs/ORP_REASONING_KERNEL_CANONICAL_CONTINUATION_PILOT.md](/Volumes/Code_2TB/code/orp/docs/ORP_REASONING_KERNEL_CANONICAL_CONTINUATION_PILOT.md), [docs/benchmarks/orp_reasoning_kernel_canonical_continuation_v0_1.json](/Volumes/Code_2TB/code/orp/docs/benchmarks/orp_reasoning_kernel_canonical_continuation_v0_1.json), [scripts/orp-kernel-canonical-continuation.py](/Volumes/Code_2TB/code/orp/scripts/orp-kernel-canonical-continuation.py) | ORP now has a stricter downstream-agent benchmark where the task is not merely “continue safely,” but “produce the next canonical artifact” without inventing unsupported structure. |
+
+## What Is Strong But Not Fully Sealed
+
+These claims are directionally convincing, but still need comparative or
+broader validation before they should be presented as fully proven.
+
+| Claim | Grade | Current Evidence | Missing Evidence | Best Next Experiment |
+| --- | --- | --- | --- | --- |
+| The seven chosen artifact classes are a good universal first set. | B | Cross-domain rationale, passing roundtrip coverage, a small five-domain reference corpus, and full required-field enforcement across all classes | Larger real-world corpus coverage across multiple project types | Cross-domain corpus fit study at real-project scale |
+| The kernel reduces accidental chat-to-truth drift. | B | ORP’s promotion model and hard/soft gate behavior | Before/after repo evidence showing fewer ambiguous promotable artifacts | Comparative artifact hygiene study |
+| The kernel is better than a generic checklist. | B | Typed classes give stronger semantic requirements than one generic field list, the internal comparison pilot shows a structural scoring advantage, the live Codex pilot shows a higher mean recoverability score with no per-case losses, and the harder canonical continuation pilot still favors the kernel on aggregate while showing checklist is genuinely competitive | Blinded human review and downstream task outcomes | Structured baseline comparison |
+| The kernel is better than free-form artifact writing alone. | B | Implementation logic is strong, promotion semantics exist, the internal comparison and pickup pilots favor the kernel, and the live Codex pilot shows a `7/7` case advantage on recoverability | Outcome comparison on real artifacts and reviewers | Free-form vs kernel artifact review study |
+| The kernel can scale across agents and handoffs. | B | Machine-checkable artifacts, visible run traces, a matched internal pickup proxy, a live Codex recoverability pilot showing stronger downstream field recovery, and first replication/continuation smokes that preserve the same general pattern | Multi-operator pickup data across models and humans | Handoff pickup experiment |
+
+## What Is Still Unproven
+
+These are important claims, but they should not be stated as established fact
+yet.
+
+| Claim | Grade | Why It Is Not Yet Proven | What Would Prove It |
+| --- | --- | --- | --- |
+| Kernel use improves downstream project outcomes. | D | No controlled comparison yet between kernel and non-kernel workflows | Comparative study over real tasks with outcome scoring |
+| Kernel-aware agents produce better work than agents without kernel structure. | D | No A/B benchmark on identical prompts and acceptance criteria | Agent benchmark across matched tasks |
+| Kernel warnings correlate with later rework or quality failures. | D | No operational data collection yet | Longitudinal study on soft-mode warnings vs later edits |
+| The current class set is close to optimal. | D | No competing class-model comparison or pruning study | Corpus analysis plus ablation tests |
+| The kernel is equally suitable across software, research, ops, and writing domains. | C | Small cross-domain corpus fit is good, but no broad real-project or reviewer evidence yet | Cross-domain corpus and reviewer study |
+
+## Research Questions That Matter Most
+
+If we want to move from “technically valid” to “research-grade validated,” the
+highest-value questions are:
+
+1. Does the kernel reduce ambiguity relative to free-form project artifacts?
+2. Does the kernel improve handoff pickup quality for humans and agents?
+3. Does the kernel improve downstream implementation or review success?
+4. Is the typed class model actually better than a simpler generic checklist?
+5. Where does the current class set fail across real project domains?
+
+## Recommended Public Wording Right Now
+
+These are safe claims now:
+
+- ORP ships a typed reasoning-kernel artifact layer.
+- ORP can scaffold, validate, and gate promotable kernel artifacts.
+- ORP supports hard and soft promotion semantics for kernel validation.
+- ORP includes repeatable benchmark evidence for the current implementation.
+
+These should still be framed as goals or hypotheses:
+
+- the kernel improves project outcomes
+- the kernel is superior to other artifact-structuring approaches
+- the current class set is the right final ontology
+
+## Bottom Line
+
+The kernel is sealed for `v0.1` as an implementation and protocol surface.
+
+Its internal validity is now much stronger than a simple happy-path release:
+
+- schema and validator rules align directly
+- every required field is exercised through ablation
+- equivalent YAML and JSON artifacts behave the same
+- adversarial near-miss artifacts are rejected
+- a small cross-domain corpus fits the current class set
+- a live fresh-agent Codex pilot shows the kernel’s recoverability advantage survives beyond deterministic local scoring
+- a `10`-repeat full-corpus replication pilot and two levels of continuation pilot suggest that advantage is reasonably stable and carries into downstream agent continuation behavior
+
+It is not yet sealed as a universally outcome-superior project methodology.
+
+That is a good place to be if we stay explicit about the difference.

package/docs/ORP_REASONING_KERNEL_EVOLUTION.md

@@ -0,0 +1,123 @@
+# ORP Reasoning Kernel Evolution
+
+This document defines how the ORP kernel should get stronger over time without
+becoming slippery.
+
+## Core Rule
+
+The kernel should be:
+
+- stable as a contract
+- adaptive through governed evolution
+- never silently rewritten by agents on the fly
+
+Short form:
+
+- stable core
+- observable pressure
+- explicit evolution
+
+## What Should Stay Stable
+
+The current core kernel remains the canonical source of truth for:
+
+- artifact classes
+- required-field rules
+- hard vs soft promotion behavior
+- machine-readable kernel validation in `RUN.json`
+
+Those semantics live in:
+
+- [spec/v1/kernel.schema.json](/Volumes/Code_2TB/code/orp/spec/v1/kernel.schema.json)
+- [cli/orp.py](/Volumes/Code_2TB/code/orp/cli/orp.py)
+
+The kernel should not self-mutate from a single chat, a single agent guess, or
+one repo’s habits.
+
+## What Should Evolve
+
+The kernel should evolve from evidence about real use:
+
+- repeated missing fields
+- repeated field invention
+- repeated continuation failures
+- recurring requests for the same extra structure
+- extension fields that become broadly useful
+
+That evidence should shape proposals and migrations, not rewrite the core
+implicitly.
+
+## CLI Surfaces
+
+ORP now exposes three explicit kernel-evolution surfaces:
+
+### `orp kernel stats`
+
+Observe real kernel-validation pressure from `RUN.json` artifacts.
+
+Use it to answer questions like:
+
+- which fields are repeatedly missing?
+- which artifact classes fail most often?
+- where is the current kernel strained in live repo usage?
+
+### `orp kernel propose`
+
+Scaffold a governed kernel-evolution proposal artifact.
+
+Use it for changes like:
+
+- adding a field
+- introducing a new artifact class
+- changing a requirement
+- deprecating an old field
+
+Proposal shape is governed by:
+
+- [spec/v1/kernel-proposal.schema.json](/Volumes/Code_2TB/code/orp/spec/v1/kernel-proposal.schema.json)
+
+### `orp kernel migrate`
+
+Rewrite an artifact into the current canonical field order and schema version.
+
+Use it to:
+
+- normalize older artifacts
+- apply explicit schema-version upgrades
+- preserve stable truth while the kernel evolves
+
+## Extensions
+
+The best place for new pressure to land first is usually not the core kernel.
+It should begin as an extension or proposal before becoming universal.
+
+Extension shape is defined in:
+
+- [spec/v1/kernel-extension.schema.json](/Volumes/Code_2TB/code/orp/spec/v1/kernel-extension.schema.json)
+
+That gives ORP a place to trial domain-specific structure without forcing it
+into every project prematurely.
+
+## Recommended Kernel Evolution Loop
+
+1. Observe pressure with `orp kernel stats`
+2. Write an explicit proposal with `orp kernel propose`
+3. Test the proposal against benchmark corpus and live agent pickup/continuation
+4. If accepted, version the schema deliberately
+5. Normalize older artifacts with `orp kernel migrate`
+6. Protect the committed evidence package with CI threshold checks
+
+## Non-Goal
+
+The kernel should not become a hidden adaptive prompt system that silently
+changes what truth means.
+
+The repository should always be able to answer:
+
+- which kernel version is in effect?
+- why was it changed?
+- what evidence justified the change?
+- how do older artifacts migrate safely?
+
+That is the ORP standard for a living protocol: dynamic in evidence, explicit
+in truth.

package/docs/ORP_REASONING_KERNEL_PICKUP_PILOT.md

@@ -0,0 +1,107 @@
+# ORP Reasoning Kernel Pickup Pilot
+
+This document records the first in-repo pickup and handoff proxy for the ORP
+Reasoning Kernel.
+
+Supporting artifact:
+
+- [docs/benchmarks/orp_reasoning_kernel_pickup_v0_1.json](/Volumes/Code_2TB/code/orp/docs/benchmarks/orp_reasoning_kernel_pickup_v0_1.json)
+
+Supporting corpus and harness:
+
+- [examples/kernel/comparison/comparison-corpus.json](/Volumes/Code_2TB/code/orp/examples/kernel/comparison/comparison-corpus.json)
+- [scripts/orp-kernel-pickup.py](/Volumes/Code_2TB/code/orp/scripts/orp-kernel-pickup.py)
+
+## What This Pilot Measures
+
+This pilot measures **explicit pickup readiness** on a matched internal corpus.
+
+For each artifact class, it asks whether a downstream operator could recover
+the class-specific pickup targets directly from the artifact itself.
+
+Examples:
+
+- `task`
+  - object
+  - constraints
+  - success criteria
+- `decision`
+  - question
+  - chosen path
+  - consequences
+- `checkpoint`
+  - current state
+  - risks
+  - next handoff target
+
+The pilot compares:
+
+1. free-form artifact writing
+2. generic checklist artifact writing
+3. ORP typed kernel artifact writing
+
+## What This Is And Is Not
+
+This is a **pickup proxy**, not a full live handoff study.
+
+It does **not** prove:
+
+- time-to-understanding with real operators
+- clarification count under live review
+- downstream execution quality
+
+It **does** prove something narrower and still valuable:
+
+how much pickup-critical information remains explicitly recoverable in the
+artifact itself.
+
+## Current Result
+
+On the matched internal corpus, the current report shows:
+
+- kernel mean pickup score: `1.000`
+- generic checklist mean pickup score: `0.743`
+- free-form mean pickup score: `0.452`
+
+Pairwise result:
+
+- kernel beats generic checklist on `7/7` cases
+- kernel beats free-form on `7/7` cases
+- generic checklist beats free-form on `7/7` cases
+
+Additional result:
+
+- kernel keeps all pickup targets explicitly answerable on the matched corpus
+
+## Why This Matters
+
+This pilot strengthens the kernel story in a way the pure structure benchmark
+could not.
+
+The earlier comparison pilot showed that kernel artifacts are structurally
+fuller than the simpler alternatives.
+
+This pickup pilot shows that the added structure is not decorative. It turns
+into directly recoverable handoff value.
+
+That is important because ORP is not trying to optimize for pretty artifacts.
+It is trying to optimize for artifacts that another human or agent can pick up
+and continue without confusion.
+
+## Honest Caveat
+
+This pilot still does not seal the kernel as a universally outcome-superior
+methodology.
+
+It is stronger evidence than a rationale-only claim, but it remains an
+internal, deterministic proxy. The next step after this is still a live
+human/agent pickup study as described in
+[docs/ORP_REASONING_KERNEL_EVALUATION_PLAN.md](/Volumes/Code_2TB/code/orp/docs/ORP_REASONING_KERNEL_EVALUATION_PLAN.md).
+
+## Bottom Line
+
+The pickup pilot makes the kernel harder to dismiss.
+
+We now have evidence that on a matched internal corpus, kernel artifacts do
+not just score as more structured. They also preserve more explicit handoff
+value than free-form and generic checklist alternatives.