project-loop-harness 0.2.0__tar.gz → 0.2.1__tar.gz

{project_loop_harness-0.2.0/src/project_loop_harness.egg-info → project_loop_harness-0.2.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: project-loop-harness
-Version: 0.2.0
+Version: 0.2.1
 Summary: Local control plane for agentic project development loops
 Author: Project Loop Harness Contributors
 License: MIT

project_loop_harness-0.2.1/agent-tasks/0089-dogfood-fixture-propose.md

@@ -0,0 +1,107 @@
+# Task 0089: Dogfood-to-Fixture Propose (v0.2.1)
+
+Design source: `docs/verification-feedback-design.md` Part 4 plus the
+Approval addendum item 4 (staging location). Where this spec and the
+design doc disagree, the design doc wins. No migration, no new
+runtime dependency.
+
+## Goal
+
+An eval suite whose fixtures do not grow from real usage is not
+credible. Add `pcl eval fixture propose --from-receipt <evidence-id>`
+so real context receipts become UNLABELED fixture candidates that a
+human labels before adoption. PLH never fabricates ground-truth
+labels.
+
+## Design constraints (approved, do not relitigate)
+
+- Staging area is repo-root `fixtures/proposed/` — a reviewable,
+  Git-tracked source asset, NOT `.project-loop/` state.
+- Proposed candidates carry `labels_status: "unlabeled"` and EMPTY
+  `expected_files` / `expected_tests` / `critical_context` arrays.
+  Filling them is a human review step by definition.
+- Adoption into `tests/fixtures/` is a manual `git mv`-style step
+  after human labeling. NO auto-adoption path exists, and none may
+  be added.
+
+## Scope
+
+### 1. `pcl eval fixture propose --from-receipt <evidence-id> [--json]`
+
+- Loads the referenced evidence row; it must be `type =
+  'context_receipt'` with a readable `context-receipt/v0` artifact —
+  unknown evidence, wrong type, or unreadable artifact are distinct
+  typed errors (reuse the receipt-loading discipline from
+  `verification_feedback.py` where sensible).
+- Emits `fixtures/proposed/<receipt-evidence-id>-retrieval.json`
+  (lowercased evidence id, deterministic name; refuse to overwrite an
+  existing file unless `--force` — overwriting a half-labeled
+  candidate silently would destroy human work).
+- Candidate shape: `retrieval-fixture/v0` with ONE task:
+  - `id`: derived from the receipt evidence id (deterministic).
+  - `diff`: a minimal synthetic unified diff touching exactly the
+    receipt's `changed_files` paths (same mechanical style as
+    existing fixtures). Receipts do not store original diff text, so
+    the replay diff is synthesized; record that honestly via
+    `diff_synthesized_from_receipt: true` inside the task.
+  - `expected_files: []`, `expected_tests: []`,
+    `critical_context: []`, `labels_status: "unlabeled"`.
+  - `source_receipt` provenance block: `evidence_id`, `created_at`,
+    `diff_source`, `base_ref` (when present), and
+    `retrieved_candidate_paths` (the receipt's
+    `included_candidate_context` paths) — this is the raw material
+    the human labeler starts from, clearly provenance, not labels.
+- Record one standard `eval_fixture_proposed` event via the shared
+  `append_event` path (SQLite events table + JSONL mirror, sequential
+  event id), recording the receipt evidence id and output path. The
+  event row is the ONLY permitted SQLite write; no other tables may
+  be touched.
+  **Clarified 2026-07-06 after first acceptance pass:** the original
+  wording "JSONL event ... No SQLite mutation" was over-constrained
+  and led to a JSONL-only event with a uuid id, which violates the
+  task-0012 audit integrity invariant (`validate --strict` errors on
+  JSONL event ids missing from the DB events table). Events always go
+  through `append_event` to both stores; do not invent side-channel
+  event writers.
+
+### 2. Guard: eval refuses unlabeled fixtures
+
+- `pcl eval retrieval` on a fixture whose tasks carry
+  `labels_status: "unlabeled"` (or empty expected+critical blocks
+  alongside that marker) fails with a typed error telling the
+  operator to label the candidate and move it into `tests/fixtures/`.
+  Labeled/legacy fixtures (no `labels_status` field) evaluate as
+  today.
+
+### 3. Documentation
+
+- `docs/code-context.md` (eval section): propose workflow, staging
+  directory, human labeling step, manual adoption, and the
+  no-fabricated-labels rule.
+- Add `fixtures/proposed/.gitkeep` (or a one-line README.md in that
+  directory) so the staging area exists in Git.
+
+## Acceptance Criteria
+
+- Proposing from a real receipt fixture yields a candidate that is
+  byte-deterministic given the same receipt (no timestamps of its
+  own — reuse the receipt's `created_at`).
+- Candidate visibly unlabeled: `labels_status: "unlabeled"`, all
+  three expected/critical arrays empty (contract test).
+- `pcl eval retrieval` against the proposed candidate → typed error;
+  after a test labels it (filling expected blocks, removing/flipping
+  `labels_status`), evaluation runs and produces metrics.
+- Unknown/wrong-type/unreadable receipt → distinct typed errors,
+  nothing written.
+- Existing candidate without `--force` → typed error, file untouched.
+- `ruff check .` passes; full `python3 -m pytest` passes; `pcl init`
+  smoke against a temp dir passes.
+
+## Do Not
+
+- Do not fill or guess expected/critical labels from receipt data.
+- Do not add an auto-adoption or auto-labeling path.
+- Do not write candidates anywhere under `.project-loop/`.
+- Do not mutate SQLite; the only side effects are the candidate file
+  and one JSONL event.
+- Do not use hit/miss/usefulness vocabulary.

project_loop_harness-0.2.1/agent-tasks/0090-eval-baseline-record-compare.md

@@ -0,0 +1,134 @@
+# Task 0090: Eval Baseline Record / Compare (v0.2.1)
+
+Design source: `docs/verification-feedback-design.md` Part 3 plus the
+Approval addendum. Where this spec and the design doc disagree, the
+design doc wins. Depends on task 0089 (unlabeled-fixture guard) being
+merged. No migration, no new runtime dependency.
+
+## Goal
+
+Make retrieval eval results durable, comparable evidence with full
+provenance, so the semantic promotion gate can eventually be decided
+by evidence instead of enthusiasm. Metric thresholds stay ADVISORY;
+only broken measurement infrastructure blocks.
+
+## Scope
+
+### 1. Metrics completion (additive to `retrieval-eval/v0` output)
+
+- Add `false_positive_rate` = (retrieved_total − true_positive_total)
+  / retrieved_total (aggregate and per task; `null` on empty
+  denominator — same rule as everywhere else).
+- Add `token_cost_estimate` = deterministic `charclass/v1` estimate
+  for the retrieved candidate files (per task and aggregate). Reuse
+  the existing estimator; it is an estimate label, never a billing
+  claim. Files missing from the index contribute 0 and are listed in
+  a `token_cost_unestimated_paths` array rather than silently
+  skipped.
+  **Clarified 2026-07-06 after first acceptance pass:** the estimate
+  basis is a per-file `token_estimate` INTEGER computed at index
+  build time (file text is already in memory during scan) and stored
+  in the index detail payload. The detail artifact must NEVER store
+  file content — the index stays hashes + symbol-lite metadata
+  (architecture boundary; 0077 budget discipline; 0072 sensitive
+  surface). Old details lacking `token_estimate` route those paths
+  into `token_cost_unestimated_paths`. The first implementation
+  embedded full `indexed_content` text into the detail; do not
+  repeat that.
+- Existing precision / recall / missing_critical_context stay
+  unchanged.
+
+### 2. Fixture kind coverage (5 kinds)
+
+The approved kinds: code change, docs-only, config-only, rename/move,
+secret omission. Code change, rename/move, and secret omission
+already exist (`retrieval_v0.json`, `retrieval_adversarial_v0.json`).
+Add small labeled fixture tasks for **docs-only** and **config-only**
+changes in the same synthetic style. Keep the existing floor-test
+discipline: assert metric floors per fixture in tests so regressions
+fail loudly.
+
+### 3. `pcl eval retrieval --record-baseline`
+
+- Runs the eval, then stores the full evaluation payload as a normal
+  evidence artifact under `.project-loop/evidence/retrieval-eval/`
+  (evidence row + JSONL event, following the receipt-recording
+  pattern).
+- Baseline artifact carries a `baseline_provenance` block with ALL
+  SIX approved fields, each REQUIRED:
+  1. `fixture_path` + `fixture_content_hash` (sha256 of fixture
+     bytes),
+  2. `git_head` of the target repo,
+  3. `index_run_id` + `index_detail_hash` (hash of the index detail
+     artifact),
+  4. `code_context_config_hash` (hash of the effective `code_index`
+     config subtree from `pcl.yaml`, canonicalized),
+  5. `pcl_version` (the real `pcl.__version__`),
+  6. `eval_contract_version` (`retrieval-eval/v0`).
+- Missing provenance input (e.g., no index run exists, not a git
+  repo) → typed error, nothing recorded. Broken provenance is not a
+  baseline.
+
+### 4. `pcl eval retrieval --compare-baseline`
+
+- Finds the LATEST recorded baseline whose `fixture_content_hash`
+  matches the current fixture; reports per-metric deltas
+  (precision, recall, missing-critical-context count,
+  false_positive_rate, token_cost_estimate) plus both provenance
+  blocks side by side.
+- No baseline with a matching fixture hash → typed
+  "not comparable" error naming the nearest baseline and why it does
+  not match (hash mismatch is stated explicitly, never silently
+  compared).
+- Comparison output is a report, not a verdict: deltas and facts
+  only. No pass/fail field, no threshold evaluation.
+
+### 5. CI advisory comparison
+
+- Extend `scripts/run_advisory_retrieval_eval.py` (or add a sibling
+  script wired the same way) so CI runs the eval and, when a
+  committed baseline reference exists, prints the comparison as an
+  advisory artifact.
+- Metric regressions NEVER fail CI.
+- Eval INFRASTRUCTURE failures DO fail CI: eval command error,
+  fixture contract violation, unreadable fixture, provenance
+  computation failure. Broken measurement is not advisory.
+
+### 6. Documentation
+
+- `docs/code-context.md` eval section: baseline lifecycle, the six
+  provenance fields, the advisory-vs-blocking boundary, and the
+  semantic promotion gate linkage (evidence decides promotion).
+
+## Acceptance Criteria
+
+- Recording twice on an unchanged repo+fixture yields identical
+  metrics and identical provenance except evidence id/created_at.
+- Compare on matching hash reports per-metric deltas; compare after
+  editing the fixture → typed not-comparable error that names the
+  hash mismatch.
+- Missing index run → typed error from `--record-baseline`, no
+  evidence row, no event.
+- `false_positive_rate` and `token_cost_estimate` appear per task
+  and aggregate, `null` on empty denominators;
+  `token_cost_unestimated_paths` lists unindexed files.
+- Docs-only and config-only fixture tasks exist with floor tests.
+- CI script: a deliberately corrupted fixture fails the script
+  (blocking path test); a metric delta does not (advisory path
+  test).
+- `pcl validate --strict` untouched by baselines beyond standard
+  evidence integrity.
+- `ruff check .` passes; full `python3 -m pytest` passes; `pcl init`
+  smoke against a temp dir passes.
+
+## Do Not
+
+- Do not implement metric thresholds, pass/fail verdicts, or release
+  gates on metric values.
+- Do not compare across different fixture hashes "helpfully".
+- Do not read generated dashboard HTML, mutate state from the eval
+  path beyond the evidence row + event, or add raw SQL.
+- Do not use hit/miss/usefulness vocabulary; token cost is an
+  estimate, not a price.
+- Do not touch embeddings / Tree-sitter / call graphs — the semantic
+  promotion gate is unchanged.

{project_loop_harness-0.2.0 → project_loop_harness-0.2.1}/docs/code-context.md

@@ -55,6 +55,8 @@ Evaluate retrieval behavior against labels:
 
 ```bash
 pcl eval retrieval --fixture tests/fixtures/retrieval_v0.json --json
+pcl eval retrieval --fixture tests/fixtures/retrieval_v0.json --record-baseline --json
+pcl eval retrieval --fixture tests/fixtures/retrieval_v0.json --compare-baseline --json
 ```
 
 ## Index Contract
@@ -109,6 +111,7 @@ needs every file, ignored path, and hash-skip entry on stdout.
       "mtime": 123456789,
       "sha256": "abc123",
       "line_count": 10,
+      "token_estimate": 42,
       "symbol_summary": {
         "contract_version": "symbol-summary/v0",
         "symbols": [{"type": "function", "name": "pack_context_for_job", "line": 1}]
@@ -629,6 +632,29 @@ Task fields:
   `assertion_note`, and `must_not_retrieve`, are metadata and are ignored by
   the evaluator.
 
+Dogfood receipts become fixture candidates through a staging workflow:
+
+```bash
+pcl eval fixture propose --from-receipt E-0001 --json
+```
+
+The command reads a real `context_receipt` evidence row and writes an
+UNLABELED `retrieval-fixture/v0` candidate to repo-root `fixtures/proposed/`.
+That directory is a Git-tracked source staging area, not `.project-loop/`
+runtime state. The candidate has one task, a synthetic unified diff touching
+the receipt's `changed_files` paths, `diff_synthesized_from_receipt: true`, and
+source receipt provenance (`evidence_id`, `created_at`, `diff_source`,
+optional `base_ref`, and retrieved candidate paths). The synthetic diff is only
+a replay handle because receipts do not store original diff text.
+
+PLH never fabricates ground-truth labels. Proposed tasks carry
+`labels_status: "unlabeled"` and empty `expected_files`, `expected_tests`, and
+`critical_context` arrays. `pcl eval retrieval` refuses these candidates with a
+typed error. A human must inspect the source receipt, fill the expected and
+critical arrays, remove or change the unlabeled marker, and manually adopt the
+fixture into `tests/fixtures/` with a normal Git move. There is no auto-labeling
+or auto-adoption path.
+
 Fixture evolution is additive in v0:
 
 - New optional fields may be added without a version bump.
@@ -640,6 +666,8 @@ Fixture evolution is additive in v0:
 
 The checked-in fixture families have different purposes:
 
+- `retrieval_v0.json`: small synthetic coverage for code change, docs-only,
+  and config-only tasks.
 - `real-history`: derived from actual repository changes, currently following
   the `tests/fixtures/retrieval_real_history_v0.json` pattern. These fixtures
   measure ordinary retrieval quality against real change history.
@@ -649,23 +677,88 @@ The checked-in fixture families have different purposes:
   omission, stale-index signaling, and an annotated renamed-file baseline miss.
 
 `pcl eval retrieval --fixture <path> --json` returns `retrieval-eval/v0` with
-precision, recall, `missing_critical_context`, and per-task retrieved paths.
+precision, recall, false-positive rate, token cost estimate,
+`missing_critical_context`, and per-task retrieved paths.
+
+Metric fields:
+
+- `precision`: true positives divided by retrieved paths, preserving the v0
+  empty-denominator rule.
+- `recall`: true positives divided by expected files/tests, preserving the v0
+  empty-denominator rule.
+- `false_positive_rate`: `(retrieved - true_positives) / retrieved`; empty
+  retrieved denominators yield `null`.
+- `token_cost_estimate`: deterministic `charclass/v1` estimate over retrieved
+  paths' per-file `token_estimate` integers from
+  `.project-loop/cache/code-index-detail.json`. The estimate is computed at
+  index build time from indexed text already in memory; the detail artifact does
+  not store raw file contents. It is an estimate of indexed text volume, not a
+  price or billing signal.
+- `token_cost_unestimated_paths`: retrieved paths that are absent from the
+  detail artifact or present without a `token_estimate`, including details
+  built by older versions. They contribute `0` to the estimate and are listed
+  explicitly.
+- `missing_critical_context`: labeled critical paths not retrieved.
+
 Task output may also include additive diagnostic fields used by adversarial
 fixtures: `retrieval_source`, `staleness_warnings`,
 `staleness_affected_paths`, `sensitive_omitted_count`,
 `retrieved_snapshot_consistency`, `excluded_changed_files`, and
 `expected_misses`.
 
+### Baseline Lifecycle
+
+`pcl eval retrieval --fixture <path> --record-baseline --json` runs the eval
+and stores the full payload as normal evidence under
+`.project-loop/evidence/retrieval-eval/`. Recording writes an evidence row and
+the standard mirrored event in SQLite and `.project-loop/events.jsonl`; it does
+not use a JSONL-only side channel.
+
+Every baseline artifact carries `baseline_provenance` with these required
+fields:
+
+- `fixture_path`
+- `fixture_content_hash` (sha256 of fixture bytes)
+- `git_head`
+- `index_run_id`
+- `index_detail_hash` (sha256 of the code index detail artifact)
+- `code_context_config_hash` (canonical effective `code_index` subtree)
+- `pcl_version`
+- `eval_contract_version` (`retrieval-eval/v0`)
+
+Missing provenance inputs, such as no index run, missing index detail artifact,
+or a non-Git target, are typed errors. In those cases no baseline artifact,
+evidence row, or event is recorded.
+
+`pcl eval retrieval --fixture <path> --compare-baseline --json` compares the
+current eval with the latest recorded baseline whose `fixture_content_hash`
+matches the current fixture. It reports current metrics, baseline metrics,
+metric deltas, current provenance, and baseline provenance. The reported deltas
+cover precision, recall, missing-critical-context count, false-positive rate,
+and token cost estimate.
+
+Comparison never crosses fixture hashes. If only a different fixture hash is
+available, the command returns a typed not-comparable error naming the nearest
+baseline and the hash mismatch. The comparison output is facts and deltas only:
+no threshold logic, no verdict field, and no release gate.
+
 CI runs `python3 scripts/run_advisory_retrieval_eval.py` after pytest. The step
 initializes and indexes the checked-out project, evaluates the checked-in
 real-history fixtures, evaluates the checked-in adversarial fixture against a
-prepared temp project, and prints a compact JSON summary. This is advisory:
-metric values are not release blockers and no recall/precision threshold is set
-in v0.1.11. Crashed eval runs or broken fixture files still fail the command.
-
-Promotion condition: retrieval metrics become release blockers only after
-dogfood data from real projects exists, thresholds are chosen from that data,
-and the thresholds are documented in a later task. Until then, proposals for
-semantic retrieval, Tree-sitter parsing, or call-graph retrieval must bring
-eval evidence showing improvement on these fixtures, but this document does
-not define concrete numeric gates.
+prepared temp project, and prints a compact JSON summary. When a local
+retrieval-eval baseline exists, the script also prints the same comparison
+facts as advisory output.
+
+Advisory-vs-blocking boundary:
+
+- Metric deltas never fail CI in v0.2.1.
+- Eval infrastructure integrity failures do fail CI: eval command errors,
+  fixture contract violations, unreadable fixtures, and provenance computation
+  failures.
+- Broken measurement is not advisory.
+
+Semantic promotion gate: richer retrieval approaches such as semantic
+retrieval, Tree-sitter parsing, or call-graph retrieval remain out of scope for
+v0.2.1. Future promotion is evidence-driven: baseline history and fixture
+coverage decide whether richer retrieval is justified, not enthusiasm. This
+document deliberately defines no metric thresholds or release gates.

{project_loop_harness-0.2.0 → project_loop_harness-0.2.1}/docs/distribution.md

@@ -134,7 +134,7 @@ jobs:
       - uses: actions/setup-python@v5
         with:
           python-version: "3.11"
-      - uses: owner/project-loop-harness/.github/actions/project-loop-validate@v0.2.0
+      - uses: owner/project-loop-harness/.github/actions/project-loop-validate@v0.2.1
         with:
           root: "."
           strict: "true"

{project_loop_harness-0.2.0 → project_loop_harness-0.2.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "project-loop-harness"
-version = "0.2.0"
+version = "0.2.1"
 description = "Local control plane for agentic project development loops"
 readme = "README.md"
 requires-python = ">=3.10"

{project_loop_harness-0.2.0 → project_loop_harness-0.2.1}/scripts/run_advisory_retrieval_eval.py

@@ -1,5 +1,6 @@
 from __future__ import annotations
 
+import argparse
 import json
 from pathlib import Path
 import subprocess
@@ -23,22 +24,29 @@ ADVERSARIAL_SENSITIVE_FILES = {
 }
 
 
-def main() -> int:
+def main(argv: list[str] | None = None) -> int:
+    args = _parse_args(argv)
+    root = Path(args.root).resolve()
+    fixtures = args.fixtures or [
+        str(root / "tests/fixtures/retrieval_v0.json"),
+        str(root / "tests/fixtures/retrieval_real_history_v0.json"),
+    ]
     fixture_summaries: list[dict[str, Any]] = []
-    _init_project(REPO_ROOT)
-    _build_index(REPO_ROOT)
-    for fixture in [
-        "tests/fixtures/retrieval_v0.json",
-        "tests/fixtures/retrieval_real_history_v0.json",
-    ]:
-        fixture_summaries.append(_evaluate_fixture(REPO_ROOT, fixture))
-
-    with tempfile.TemporaryDirectory(prefix="pcl-retrieval-adversarial-") as temp_dir:
-        adversarial_root = Path(temp_dir)
-        _prepare_adversarial_project(adversarial_root)
-        fixture_summaries.append(
-            _evaluate_fixture(adversarial_root, str(REPO_ROOT / "tests/fixtures/retrieval_adversarial_v0.json"))
-        )
+    _init_project(root)
+    _build_index(root)
+    for fixture in fixtures:
+        fixture_summaries.append(_evaluate_fixture(root, fixture))
+
+    if not args.skip_adversarial:
+        with tempfile.TemporaryDirectory(prefix="pcl-retrieval-adversarial-") as temp_dir:
+            adversarial_root = Path(temp_dir)
+            _prepare_adversarial_project(adversarial_root)
+            fixture_summaries.append(
+                _evaluate_fixture(
+                    adversarial_root,
+                    str(REPO_ROOT / "tests/fixtures/retrieval_adversarial_v0.json"),
+                )
+            )
 
     print(
         json.dumps(
@@ -46,6 +54,9 @@ def main() -> int:
                 "ok": True,
                 "mode": "advisory",
                 "note": "Metric values are reported for trend review; fixture or eval crashes fail this command.",
+                "baseline_note": (
+                    "Baseline comparisons are advisory when present; metric deltas do not fail this command."
+                ),
                 "fixtures": fixture_summaries,
             },
             indent=2,
@@ -55,6 +66,14 @@ def main() -> int:
     return 0
 
 
+def _parse_args(argv: list[str] | None) -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--root", default=str(REPO_ROOT))
+    parser.add_argument("--fixture", action="append", dest="fixtures", default=None)
+    parser.add_argument("--skip-adversarial", action="store_true")
+    return parser.parse_args(argv)
+
+
 def _prepare_adversarial_project(root: Path) -> None:
     _init_project(root)
     (root / "src" / "pkg").mkdir(parents=True, exist_ok=True)
@@ -113,6 +132,9 @@ def _evaluate_fixture(root: Path, fixture: str) -> dict[str, Any]:
     adversarial = _adversarial_summary(evaluation["tasks"])
     if adversarial:
         summary["adversarial"] = adversarial
+    comparison = _compare_fixture_if_baseline_exists(root, fixture)
+    if comparison:
+        summary["baseline_comparison"] = comparison
     return summary
 
 
@@ -132,20 +154,49 @@ def _adversarial_summary(tasks: list[dict[str, Any]]) -> dict[str, Any]:
 
 
 def _run_pcl(args: list[str]) -> dict[str, Any]:
-    completed = subprocess.run(
+    completed = _run_pcl_completed(args)
+    if completed.returncode != 0:
+        _print_completed_output(completed)
+        raise SystemExit(completed.returncode)
+    return json.loads(completed.stdout)
+
+
+def _run_pcl_completed(args: list[str]) -> subprocess.CompletedProcess[str]:
+    return subprocess.run(
         [PYTHON, "-m", "pcl", *args],
         cwd=REPO_ROOT,
         capture_output=True,
         check=False,
         text=True,
     )
-    if completed.returncode != 0:
-        if completed.stdout:
-            print(completed.stdout, file=sys.stderr, end="")
-        if completed.stderr:
-            print(completed.stderr, file=sys.stderr, end="")
-        raise SystemExit(completed.returncode)
-    return json.loads(completed.stdout)
+
+
+def _compare_fixture_if_baseline_exists(root: Path, fixture: str) -> dict[str, Any] | None:
+    if not any((root / ".project-loop" / "evidence" / "retrieval-eval").glob("*.json")):
+        return None
+    completed = _run_pcl_completed(
+        [
+            "--root",
+            str(root),
+            "eval",
+            "retrieval",
+            "--fixture",
+            fixture,
+            "--compare-baseline",
+            "--json",
+        ]
+    )
+    if completed.returncode == 0:
+        return json.loads(completed.stdout)["comparison"]
+    _print_completed_output(completed)
+    raise SystemExit(completed.returncode)
+
+
+def _print_completed_output(completed: subprocess.CompletedProcess[str]) -> None:
+    if completed.stdout:
+        print(completed.stdout, file=sys.stderr, end="")
+    if completed.stderr:
+        print(completed.stderr, file=sys.stderr, end="")
 
 
 def _append_text(path: Path, text: str) -> None:

project_loop_harness-0.2.1/src/pcl/__init__.py

@@ -0,0 +1 @@
+__version__ = "0.2.1"

{project_loop_harness-0.2.0 → project_loop_harness-0.2.1}/src/pcl/cli.py

@@ -14,7 +14,10 @@ from .code_index import (
     analyze_impact,
     build_code_index,
     code_index_status,
+    compare_retrieval_baseline,
     evaluate_retrieval,
+    propose_retrieval_fixture,
+    record_retrieval_baseline,
     search_code,
 )
 from .commands import (
@@ -618,6 +621,34 @@ def build_parser() -> argparse.ArgumentParser:
     eval_sub = p_eval.add_subparsers(dest="eval_command", required=True)
     p_eval_retrieval = eval_sub.add_parser("retrieval", help="Evaluate indexed retrieval")
     p_eval_retrieval.add_argument("--fixture", required=True)
+    eval_retrieval_baseline = p_eval_retrieval.add_mutually_exclusive_group()
+    eval_retrieval_baseline.add_argument(
+        "--record-baseline",
+        action="store_true",
+        help="Store the retrieval eval payload as a provenance-bearing evidence baseline.",
+    )
+    eval_retrieval_baseline.add_argument(
+        "--compare-baseline",
+        action="store_true",
+        help="Compare against the latest recorded baseline with the same fixture hash.",
+    )
+    p_eval_fixture = eval_sub.add_parser("fixture", help="Manage retrieval fixture candidates")
+    eval_fixture_sub = p_eval_fixture.add_subparsers(dest="eval_fixture_command", required=True)
+    p_eval_fixture_propose = eval_fixture_sub.add_parser(
+        "propose",
+        help="Propose an unlabeled retrieval fixture from a context receipt.",
+    )
+    p_eval_fixture_propose.add_argument(
+        "--from-receipt",
+        required=True,
+        dest="from_receipt",
+        help="Context receipt evidence ID to stage as an unlabeled fixture candidate.",
+    )
+    p_eval_fixture_propose.add_argument(
+        "--force",
+        action="store_true",
+        help="Overwrite an existing proposed candidate after confirming no human labels will be lost.",
+    )
 
     p_verification = sub.add_parser("verification", help="Record verification results")
     verification_sub = p_verification.add_subparsers(dest="verification_command", required=True)
@@ -1878,13 +1909,38 @@ def main(argv: list[str] | None = None) -> int:
             return 0
 
         if args.command == "eval" and args.eval_command == "retrieval":
-            result = evaluate_retrieval(paths, fixture_path=args.fixture)
+            if args.record_baseline:
+                result = record_retrieval_baseline(paths, fixture_path=args.fixture)
+            elif args.compare_baseline:
+                result = compare_retrieval_baseline(paths, fixture_path=args.fixture)
+            else:
+                result = evaluate_retrieval(paths, fixture_path=args.fixture)
             if json_output:
                 _print_json(result)
+            elif args.record_baseline:
+                print(result["baseline"]["evidence_path"])
+            elif args.compare_baseline:
+                print(to_pretty_json(result["comparison"]))
             else:
                 print(to_pretty_json(result["evaluation"]))
             return 0
 
+        if (
+            args.command == "eval"
+            and args.eval_command == "fixture"
+            and args.eval_fixture_command == "propose"
+        ):
+            result = propose_retrieval_fixture(
+                paths,
+                receipt_evidence_id=args.from_receipt,
+                force=args.force,
+            )
+            if json_output:
+                _print_json(result)
+            else:
+                print(result["fixture"]["path"])
+            return 0
+
         if args.command == "verification" and args.verification_command == "record":
             result = record_verification(
                 paths,

{project_loop_harness-0.2.0 → project_loop_harness-0.2.1}/src/pcl/code_context/__init__.py

@@ -30,7 +30,10 @@ from .eval import (
     _resolve_fixture_path,
     _retrieved_paths_for_fixture_task,
     _string_set,
+    compare_retrieval_baseline,
     evaluate_retrieval,
+    propose_retrieval_fixture,
+    record_retrieval_baseline,
 )
 from .impact import (
     IMPACT_CONTRACT_VERSION,