loop-engineer 0.7.0__tar.gz → 0.8.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/.claude-plugin/plugin.json +1 -1
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/.github/workflows/ci.yml +17 -1
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/.gitignore +3 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/CHANGELOG.md +31 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/CONTRIBUTING.md +19 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/PKG-INFO +6 -6
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/README.md +5 -5
- loop_engineer-0.8.0/docs/contributing/issues/01-good-first-qw9-trigger-phrases.md +45 -0
- loop_engineer-0.8.0/docs/contributing/issues/02-good-first-qw10-self-eval-labels.md +39 -0
- loop_engineer-0.8.0/docs/contributing/issues/03-good-first-emit-metrics-vocabulary.md +44 -0
- loop_engineer-0.8.0/docs/contributing/issues/04-help-wanted-openhands-recipe.md +42 -0
- loop_engineer-0.8.0/docs/contributing/issues/05-help-wanted-ruflo-recipe.md +39 -0
- loop_engineer-0.8.0/docs/contributing/issues/06-help-wanted-gap-reports.md +42 -0
- loop_engineer-0.8.0/docs/contributing/issues/07-help-wanted-emit-scaffold-runlog-seed.md +61 -0
- loop_engineer-0.8.0/docs/gap-reports/superpowers.md +107 -0
- loop_engineer-0.8.0/docs/integrations/langgraph.md +88 -0
- loop_engineer-0.8.0/docs/integrations/temporal.md +90 -0
- loop_engineer-0.8.0/docs/superpowers/plans/2026-07-08-v0.8.0-composes-the-field.md +2114 -0
- loop_engineer-0.8.0/docs/superpowers/specs/2026-07-08-v0.8.0-composes-the-field-design.md +221 -0
- loop_engineer-0.8.0/examples/flaky-test-triage/README.md +96 -0
- loop_engineer-0.8.0/examples/flaky-test-triage/RUNLOG.md +25 -0
- loop_engineer-0.8.0/examples/flaky-test-triage/SPEC.md +55 -0
- loop_engineer-0.8.0/examples/flaky-test-triage/TASKS.json +23 -0
- loop_engineer-0.8.0/examples/flaky-test-triage/WORKFLOW.md +77 -0
- loop_engineer-0.8.0/examples/flaky-test-triage/scripts/run-example +28 -0
- loop_engineer-0.8.0/examples/flaky-test-triage/scripts/verify-fast +6 -0
- loop_engineer-0.8.0/examples/flaky-test-triage/scripts/verify-full +15 -0
- loop_engineer-0.8.0/examples/flaky-test-triage/target/jobs.py +17 -0
- loop_engineer-0.8.0/examples/flaky-test-triage/target/manifest.json +8 -0
- loop_engineer-0.8.0/examples/flaky-test-triage/target/measure_stability.py +32 -0
- loop_engineer-0.8.0/examples/flaky-test-triage/target/test_holdout.py +15 -0
- loop_engineer-0.8.0/examples/flaky-test-triage/target/test_visible.py +9 -0
- loop_engineer-0.8.0/examples/langgraph-emit/README.md +52 -0
- loop_engineer-0.8.0/examples/langgraph-emit/graph_example.py +129 -0
- loop_engineer-0.8.0/examples/superpowers-run/.superpowers/sdd/progress.md +4 -0
- loop_engineer-0.8.0/examples/superpowers-run/README.md +15 -0
- loop_engineer-0.8.0/examples/superpowers-run/docs/superpowers/plans/2026-07-08-csv-dedupe.md +17 -0
- loop_engineer-0.8.0/examples/superpowers-run/docs/superpowers/specs/2026-07-08-csv-dedupe-design.md +19 -0
- loop_engineer-0.8.0/examples/temporal-certify/README.md +67 -0
- loop_engineer-0.8.0/examples/temporal-certify/workflow_example.py +193 -0
- loop_engineer-0.8.0/loop/foreign.py +69 -0
- loop_engineer-0.8.0/loop/integrations.py +125 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/pyproject.toml +1 -1
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/inspect_loop.py +26 -4
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_docs_version.py +3 -2
- loop_engineer-0.8.0/scripts/test_flaky_example.py +55 -0
- loop_engineer-0.8.0/scripts/test_foreign_inspect.py +84 -0
- loop_engineer-0.8.0/scripts/test_integrations.py +170 -0
- loop_engineer-0.8.0/scripts/test_langgraph_recipe_st3.py +75 -0
- loop_engineer-0.8.0/scripts/test_temporal_recipe.py +156 -0
- loop_engineer-0.7.0/docs/integrations/langgraph.md +0 -30
- loop_engineer-0.7.0/examples/langgraph-emit/README.md +0 -34
- loop_engineer-0.7.0/examples/langgraph-emit/graph_example.py +0 -82
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/.claude-plugin/marketplace.json +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/.github/ISSUE_TEMPLATE/bug_report.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/.github/ISSUE_TEMPLATE/config.yml +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/.github/ISSUE_TEMPLATE/feature_request.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/.github/PULL_REQUEST_TEMPLATE.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/.github/workflows/publish.yml +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/.pre-commit-hooks.yaml +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/CODE_OF_CONDUCT.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/GLOSSARY.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/LICENSE +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/SECURITY.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/action.yml +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/docs/ROADMAP-v1.0.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/docs/demo.cast +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/docs/demo.gif +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/docs/metrics-baseline.json +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/docs/social-card.png +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/docs/superpowers/plans/2026-06-20-loop-engineer-v1.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/docs/superpowers/plans/2026-06-30-loop-engineer-v1.0-roadmap.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/docs/superpowers/plans/2026-07-03-adoption-slices.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/docs/superpowers/specs/2026-06-20-loop-engineer-design.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/docs/superpowers/specs/2026-06-30-st1-metrics-baseline.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/docs/superpowers/specs/2026-06-30-st2-portable-contract-spec.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/docs/superpowers/specs/2026-06-30-st3-integration-adapters.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/docs/superpowers/specs/2026-06-30-v04-credibility-enforcement.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/docs/superpowers/specs/2026-07-03-adoption-slices-design.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/evals/cases/structural.json +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/evals/rubric.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/examples/coverage-repair/ADR.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/examples/coverage-repair/README.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/examples/coverage-repair/RUNLOG.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/examples/coverage-repair/SPEC.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/examples/coverage-repair/TASKS.json +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/examples/coverage-repair/WORKFLOW.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/examples/coverage-repair/scripts/run-example +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/examples/coverage-repair/scripts/verify-fast +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/examples/coverage-repair/scripts/verify-full +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/examples/coverage-repair/target/manifest.json +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/examples/coverage-repair/target/measure_coverage.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/examples/coverage-repair/target/pricing.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/examples/coverage-repair/target/test_holdout.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/examples/coverage-repair/target/test_visible.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/examples/coverage-repair/terminal_state.json +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/examples/naive-loop/README.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/hooks/stop_firewall.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/loop/__init__.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/loop/__main__.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/loop/_resources.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/loop/contract.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/loop/emit.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/loop/paths.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/loop/scaffold.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/reference/architecture-matrix.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/reference/eval-suite.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/reference/loop-patterns.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/reference/model-routing.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/reference/platform-map.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/reference/prompt-templates.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/reference/repo-os-contract.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/reference/safety-and-approvals.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/schemas/manifest.schema.json +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/schemas/receipt.schema.json +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/schemas/repair-record.schema.json +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/schemas/rollout-record.schema.json +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/schemas/state.schema.json +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/schemas/tasks.schema.json +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/schemas/terminal.schema.json +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/action_scorecard.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/anticheat_scan.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/benchmark_harness.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/holdout_gate.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/metrics.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/rollout_ledger.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/runtime_monitor.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/self_eval.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_action_scorecard.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_anticheat_scan.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_benchmark_harness.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_conformance.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_contract_records.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_docs_adoption.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_docs_baseline.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_docs_claims.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_emit.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_example_runnable.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_holdout_gate.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_inspect_loop.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_langgraph_recipe.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_loop_cli.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_loop_contract_core.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_metrics.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_metrics_cli.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_precommit_hook.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_resources.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_rollout_ledger.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_runtime_monitor.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_scaffold.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_schemas_metrics.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_self_eval.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_stop_firewall.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_template_roundtrip.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_validate_frontmatter.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/test_wheel_selfcontained.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/scripts/validate_frontmatter.py +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/skills/loop-architect/SKILL.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/skills/loop-contract/SKILL.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/skills/loop-engineer/SKILL.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/skills/loop-evals/SKILL.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/skills/loop-flywheel/SKILL.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/skills/loop-inspector/SKILL.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/skills/loop-inspector/reference/patterns.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/skills/loop-repair/SKILL.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/skills/loop-run/SKILL.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/skills/loop-runtime-monitor/SKILL.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/skills/loop-runtime-monitor/reference/patterns.md +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/templates/AGENTS.md.tmpl +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/templates/EVALS-rubric.md.tmpl +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/templates/RUNLOG.md.tmpl +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/templates/SPEC.md.tmpl +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/templates/TASKS.json.tmpl +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/templates/WORKFLOW.md.tmpl +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/templates/extract-trace-metrics.sh +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/templates/judge-rubric.sh +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/templates/manifest.yaml.tmpl +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/templates/state.json.tmpl +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/templates/terminal_state.json.tmpl +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/templates/verify-fast.sh +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/templates/verify-full.sh +0 -0
- {loop_engineer-0.7.0 → loop_engineer-0.8.0}/templates/verify-safety.sh +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "loop-engineer",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.8.0",
|
|
4
4
|
"description": "Design, launch, verify, repair, and improve agent loops. A Claude-Code-native architect+operator for long-running, verifiable, self-improving agentic-coding systems.",
|
|
5
5
|
"author": { "name": "Sollan Systems", "url": "https://github.com/SollanSystems" },
|
|
6
6
|
"homepage": "https://github.com/SollanSystems/loop-engineer",
|
|
@@ -59,7 +59,23 @@ jobs:
|
|
|
59
59
|
- name: Install recipe dependencies
|
|
60
60
|
run: python -m pip install --upgrade pip pyyaml pytest jsonschema langgraph
|
|
61
61
|
- name: LangGraph recipe end-to-end
|
|
62
|
-
run: python -B -m pytest -q -p no:cacheprovider scripts/test_langgraph_recipe.py
|
|
62
|
+
run: python -B -m pytest -q -p no:cacheprovider scripts/test_langgraph_recipe.py scripts/test_langgraph_recipe_st3.py
|
|
63
|
+
|
|
64
|
+
recipe-temporal:
|
|
65
|
+
name: recipe (temporal)
|
|
66
|
+
runs-on: ubuntu-latest
|
|
67
|
+
steps:
|
|
68
|
+
- uses: actions/checkout@v4
|
|
69
|
+
- uses: actions/setup-python@v5
|
|
70
|
+
with:
|
|
71
|
+
python-version: "3.12"
|
|
72
|
+
- name: Install recipe dependencies
|
|
73
|
+
run: python -m pip install --upgrade pip pyyaml pytest jsonschema temporalio
|
|
74
|
+
- name: Temporal recipe end-to-end
|
|
75
|
+
# start_local() downloads the Temporal dev-server binary (~20MB) to
|
|
76
|
+
# $TMPDIR/temporal-sdk-python-<version> — ephemeral and quick, so (like
|
|
77
|
+
# recipe-langgraph) this job carries no cache step.
|
|
78
|
+
run: python -B -m pytest -q -p no:cacheprovider scripts/test_temporal_recipe.py
|
|
63
79
|
|
|
64
80
|
action-dogfood:
|
|
65
81
|
name: action (dogfood on flagship example)
|
|
@@ -14,6 +14,37 @@ All notable changes to `loop-engineer` are documented here.
|
|
|
14
14
|
`WORKFLOW.md` and `README.md` are reworded to describe the mechanism; the 0.3.4
|
|
15
15
|
history is left intact.
|
|
16
16
|
|
|
17
|
+
## 0.8.0 — 2026-07-09
|
|
18
|
+
|
|
19
|
+
**ST3 — integration adapters.** `loop/integrations.py`: an engine-neutral,
|
|
20
|
+
pure-stdlib projection (`EngineOutcome` + `to_terminal_state`) from any
|
|
21
|
+
engine's "the run ended" signal onto the 7 typed terminal states, with the
|
|
22
|
+
fixed precedence safety → human → blocked → budget → spec-gap → gate verdict.
|
|
23
|
+
`Succeeded` is reachable only through a green `holdout_gate.decide` verdict,
|
|
24
|
+
a clean anticheat sweep, a met criterion, and evidence; `false_completion` is
|
|
25
|
+
copied from the gate, never synthesized; missing gate/anticheat input fails
|
|
26
|
+
closed to `FailedUnverifiable`. The LangGraph recipe is upgraded in place to
|
|
27
|
+
this bar (its run now scores clean under `loop metrics` — closing the
|
|
28
|
+
recorded FCR-1.0 follow-up) and a Temporal recipe lands
|
|
29
|
+
(`examples/temporal-certify/`, certify-activity pattern, cancellation →
|
|
30
|
+
`AbortedByHuman`, retry exhaustion → `FailedBlocked`, timeout →
|
|
31
|
+
`FailedBudget`). Both recipes pin the false-completion invariant
|
|
32
|
+
(visible-green/holdout-red → `FailedUnverifiable` with
|
|
33
|
+
`false_completion: true`, never `Succeeded`) and pass the doctor round-trip.
|
|
34
|
+
|
|
35
|
+
**ST4 — contributor funnel.** `loop inspect` now recognizes a foreign
|
|
36
|
+
Superpowers-style run dir read-only (`loop/foreign.py` — a layout mapper onto
|
|
37
|
+
the existing `LoopPaths` seam; the M2/M3-hardened scorer is untouched and a
|
|
38
|
+
foreign harness with no gate and no terminal record scores honestly low).
|
|
39
|
+
The reading is checked in as `docs/gap-reports/superpowers.md` — the §14
|
|
40
|
+
conformance checklist evaluated against a vendored, sanitized fixture
|
|
41
|
+
(`examples/superpowers-run/`). A second runnable example lands:
|
|
42
|
+
`examples/flaky-test-triage/` — doctor-clean, gate-backed, and the showcase
|
|
43
|
+
for repair records (`loop metrics` derives a non-null repair-productivity of
|
|
44
|
+
1.0 from its anchored red→green repair). Seven gate-backed starter issues are
|
|
45
|
+
drafted under `docs/contributing/issues/` and filed at release;
|
|
46
|
+
CONTRIBUTING gains the start-here funnel.
|
|
47
|
+
|
|
17
48
|
## 0.7.0 — 2026-07-08
|
|
18
49
|
|
|
19
50
|
**ST2 — the portable standard.** The on-disk contract is now a documented,
|
|
@@ -20,6 +20,25 @@ python3 -m loop inspect examples/coverage-repair
|
|
|
20
20
|
|
|
21
21
|
If you don't have the deps, prefix with `uv run --with pyyaml --with pytest`.
|
|
22
22
|
|
|
23
|
+
## Start here — the contributor funnel
|
|
24
|
+
|
|
25
|
+
Every open starter issue names **the gate that proves the fix** — a
|
|
26
|
+
deterministic command that is red before your change and green after. That is
|
|
27
|
+
the whole review bar (see the ground rule above).
|
|
28
|
+
|
|
29
|
+
- [`good first issue`](https://github.com/SollanSystems/loop-engineer/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22)
|
|
30
|
+
— small, bounded, gate-verifiable fixes.
|
|
31
|
+
- [`help wanted`](https://github.com/SollanSystems/loop-engineer/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22)
|
|
32
|
+
— integration recipes (OpenHands, ruflo — designs already written in
|
|
33
|
+
`docs/superpowers/specs/2026-06-30-st3-integration-adapters.md`) and
|
|
34
|
+
foreign-harness gap reports (`docs/gap-reports/`).
|
|
35
|
+
|
|
36
|
+
The contribution target for anything that emits or consumes contract
|
|
37
|
+
artifacts is the standard: `reference/repo-os-contract.md` — a harness that
|
|
38
|
+
satisfies the §14 conformance checklist (A1–E1) may claim it emits a
|
|
39
|
+
Loop-Engineer-conformant contract v1. Drafts for the seeded issues live in
|
|
40
|
+
`docs/contributing/issues/` and are filed on GitHub at release time.
|
|
41
|
+
|
|
23
42
|
## Repository layout
|
|
24
43
|
|
|
25
44
|
| Path | What lives here |
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: loop-engineer
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.8.0
|
|
4
4
|
Summary: Portable Loop Contract Core: validate and inspect repo-native operating contracts for agent loops.
|
|
5
5
|
Project-URL: Homepage, https://github.com/SollanSystems/loop-engineer
|
|
6
6
|
Project-URL: Repository, https://github.com/SollanSystems/loop-engineer
|
|
@@ -42,7 +42,7 @@ Description-Content-Type: text/markdown
|
|
|
42
42
|
[](https://github.com/SollanSystems/loop-engineer/actions/workflows/ci.yml)
|
|
43
43
|
[](https://www.python.org/downloads/)
|
|
44
44
|
[](LICENSE)
|
|
45
|
-
[](https://github.com/SollanSystems/loop-engineer/tags)
|
|
46
46
|
|
|
47
47
|
Long-running agents commit **false completion**. After context compaction they
|
|
48
48
|
forget what "done" meant, optimize to the visible test, patch in circles, and
|
|
@@ -344,7 +344,7 @@ refuses an evidence-free `Succeeded` at write time. Recipe:
|
|
|
344
344
|
**CI** — one workflow step validates the contract and publishes a scorecard:
|
|
345
345
|
|
|
346
346
|
```yaml
|
|
347
|
-
- uses: SollanSystems/loop-engineer@v0.
|
|
347
|
+
- uses: SollanSystems/loop-engineer@v0.8.0
|
|
348
348
|
with:
|
|
349
349
|
path: "."
|
|
350
350
|
```
|
|
@@ -434,12 +434,12 @@ license, and README differentiation.
|
|
|
434
434
|
|
|
435
435
|
## Status
|
|
436
436
|
|
|
437
|
-
- Version: `0.
|
|
438
|
-
- Release tag: `v0.
|
|
437
|
+
- Version: `0.8.0`
|
|
438
|
+
- Release tag: `v0.8.0` (PyPI publish trigger; plugin tags through 0.6.0 used `loop-engineer--v<version>`)
|
|
439
439
|
- License: MIT
|
|
440
440
|
- Primary interface: Claude Code plugin
|
|
441
441
|
- Portable core: Python CLI + JSON schemas
|
|
442
|
-
- Current reference
|
|
442
|
+
- Current reference examples: `examples/coverage-repair`, `examples/flaky-test-triage`
|
|
443
443
|
|
|
444
444
|
---
|
|
445
445
|
|
|
@@ -5,7 +5,7 @@
|
|
|
5
5
|
[](https://github.com/SollanSystems/loop-engineer/actions/workflows/ci.yml)
|
|
6
6
|
[](https://www.python.org/downloads/)
|
|
7
7
|
[](LICENSE)
|
|
8
|
-
[](https://github.com/SollanSystems/loop-engineer/tags)
|
|
9
9
|
|
|
10
10
|
Long-running agents commit **false completion**. After context compaction they
|
|
11
11
|
forget what "done" meant, optimize to the visible test, patch in circles, and
|
|
@@ -307,7 +307,7 @@ refuses an evidence-free `Succeeded` at write time. Recipe:
|
|
|
307
307
|
**CI** — one workflow step validates the contract and publishes a scorecard:
|
|
308
308
|
|
|
309
309
|
```yaml
|
|
310
|
-
- uses: SollanSystems/loop-engineer@v0.
|
|
310
|
+
- uses: SollanSystems/loop-engineer@v0.8.0
|
|
311
311
|
with:
|
|
312
312
|
path: "."
|
|
313
313
|
```
|
|
@@ -397,12 +397,12 @@ license, and README differentiation.
|
|
|
397
397
|
|
|
398
398
|
## Status
|
|
399
399
|
|
|
400
|
-
- Version: `0.
|
|
401
|
-
- Release tag: `v0.
|
|
400
|
+
- Version: `0.8.0`
|
|
401
|
+
- Release tag: `v0.8.0` (PyPI publish trigger; plugin tags through 0.6.0 used `loop-engineer--v<version>`)
|
|
402
402
|
- License: MIT
|
|
403
403
|
- Primary interface: Claude Code plugin
|
|
404
404
|
- Portable core: Python CLI + JSON schemas
|
|
405
|
-
- Current reference
|
|
405
|
+
- Current reference examples: `examples/coverage-repair`, `examples/flaky-test-triage`
|
|
406
406
|
|
|
407
407
|
---
|
|
408
408
|
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
<!-- title: Trigger-phrase disambiguation batch (3 LOW fixes) -->
|
|
2
|
+
<!-- labels: good first issue -->
|
|
3
|
+
|
|
4
|
+
# Trigger-phrase disambiguation batch (3 LOW fixes)
|
|
5
|
+
|
|
6
|
+
Three small `SKILL.md` frontmatter edits that sharpen router resolution. Each is
|
|
7
|
+
independently checkable and none changes behavior — only the trigger prose.
|
|
8
|
+
|
|
9
|
+
## Problem
|
|
10
|
+
|
|
11
|
+
Three trigger-phrase weaknesses, all present at this commit:
|
|
12
|
+
|
|
13
|
+
- **Shared bare "grade" verb.** `skills/loop-evals/SKILL.md` anchors *"or grade a
|
|
14
|
+
long-running agentic run"* and `skills/loop-inspector/SKILL.md` anchors *"grade a
|
|
15
|
+
superpowers / ruflo / .loop harness"* — both hang the same verb on different
|
|
16
|
+
objects, so a bare "grade this" query has no clean winner.
|
|
17
|
+
- **`loop-evals` verbosity outlier.** Its `description` scalar runs well past the
|
|
18
|
+
sibling band (~736 chars vs a ~400–510-char peer range), because capability prose
|
|
19
|
+
(the 7-layer suite, deterministic-then-rubric, the regression harness) lives in the
|
|
20
|
+
frontmatter instead of the body.
|
|
21
|
+
- **`loop-run` weak first example.** `skills/loop-run/SKILL.md` opens its example
|
|
22
|
+
list with bare `'run the loop'` — the only one of its examples lacking a
|
|
23
|
+
qualifier.
|
|
24
|
+
|
|
25
|
+
## Proposal
|
|
26
|
+
|
|
27
|
+
- Make the noun part of each "grade" phrase: evals → *"grade a run's outcome against
|
|
28
|
+
its SPEC"*; inspector → *"grade this harness/contract's readiness"*.
|
|
29
|
+
- Trim `loop-evals`' `description` into the ~400–510-char sibling band by moving the
|
|
30
|
+
capability-summary prose into the skill body, leaving trigger phrases + a one-line
|
|
31
|
+
hook in frontmatter.
|
|
32
|
+
- Qualify `loop-run`'s opening example: *"run the agent loop"* (or *"run this loop's
|
|
33
|
+
state machine"*).
|
|
34
|
+
|
|
35
|
+
Keep every `description` a *quoted* YAML scalar (the suite quotes all of them).
|
|
36
|
+
|
|
37
|
+
## The gate that proves the fix
|
|
38
|
+
|
|
39
|
+
```bash
|
|
40
|
+
python3 scripts/validate_frontmatter.py # green
|
|
41
|
+
python3 scripts/self_eval.py # green
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
`loop-evals`' frontmatter length should land back in the sibling band; the two
|
|
45
|
+
gates above are the whole review bar.
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
<!-- title: Label self_eval terminal/repair/eval checks honestly as doc-completeness -->
|
|
2
|
+
<!-- labels: good first issue -->
|
|
3
|
+
|
|
4
|
+
# Label self_eval terminal/repair/eval checks honestly as doc-completeness
|
|
5
|
+
|
|
6
|
+
A naming/comment clarification (no behavioral change required) that keeps the gate
|
|
7
|
+
honest about its own scope.
|
|
8
|
+
|
|
9
|
+
## Problem
|
|
10
|
+
|
|
11
|
+
Three `scripts/self_eval.py` checks are substring-presence scans over a single
|
|
12
|
+
`SKILL.md` each, not behavioral enforcement:
|
|
13
|
+
|
|
14
|
+
- `check_terminal_states` (`scripts/self_eval.py:185`) — `missing = [s for s in
|
|
15
|
+
facts["terminal_states"] if s not in text]` over `skills/loop-run/SKILL.md`.
|
|
16
|
+
- `check_repair_fields` (`scripts/self_eval.py:193`) — same shape over
|
|
17
|
+
`skills/loop-repair/SKILL.md`.
|
|
18
|
+
- `check_eval_layers_and_metrics` (`scripts/self_eval.py:206`) — normalized
|
|
19
|
+
substring presence over `skills/loop-evals/SKILL.md`.
|
|
20
|
+
|
|
21
|
+
Each passes as long as the canonical words *appear in the prose*. Presenting them as
|
|
22
|
+
"the hard pass/fail gate" risks a reader mistaking documentation-completeness for
|
|
23
|
+
runtime-correctness enforcement — gaming requires only listing the canonical words.
|
|
24
|
+
|
|
25
|
+
## Proposal
|
|
26
|
+
|
|
27
|
+
Rename and/or comment the three checks as **documentation-completeness** checks (not
|
|
28
|
+
behavioral enforcement), and say so where `self_eval` is described as a gate — in
|
|
29
|
+
`CONTRIBUTING.md` (the "Ground rule" / self_eval mention) and in the README's
|
|
30
|
+
structural-check list. No behavioral change is required if the checks are
|
|
31
|
+
intentional; this is a scope-honesty clarification consistent with the suite's own
|
|
32
|
+
posture.
|
|
33
|
+
|
|
34
|
+
## The gate that proves the fix
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
python3 scripts/self_eval.py # green (13 structural invariants)
|
|
38
|
+
python3 -m pytest -q scripts/test_docs_claims.py # README accuracy assertions green
|
|
39
|
+
```
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
<!-- title: Reconcile emit's iteration-outcome vocabulary with metrics' recognized tokens -->
|
|
2
|
+
<!-- labels: good first issue -->
|
|
3
|
+
|
|
4
|
+
# Reconcile emit's iteration-outcome vocabulary with metrics' recognized tokens
|
|
5
|
+
|
|
6
|
+
The writer and the metrics reader disagree on the outcome vocabulary, so a RUNLOG
|
|
7
|
+
written entirely through the sanctioned writer can still look "dirty" to `loop
|
|
8
|
+
metrics`.
|
|
9
|
+
|
|
10
|
+
## Problem
|
|
11
|
+
|
|
12
|
+
`loop/emit.py` `_ITERATION_OUTCOMES` (`loop/emit.py:26`) accepts `approval_requested`
|
|
13
|
+
and `replanned`, but `scripts/metrics.py` `_KNOWN_OUTCOME_TOKENS`
|
|
14
|
+
(`scripts/metrics.py:101`, built from `_SUCCESS_OUTCOME_TOKENS` +
|
|
15
|
+
`_HONEST_RED_OUTCOME_TOKENS`) recognizes neither. So an iteration appended via
|
|
16
|
+
`emit.append_iteration(..., outcome="approval_requested")` — a fully valid write —
|
|
17
|
+
surfaces under `provenance.unrecognized_outcomes`.
|
|
18
|
+
|
|
19
|
+
Checkable at this commit:
|
|
20
|
+
|
|
21
|
+
```python
|
|
22
|
+
import sys; sys.path.insert(0, "scripts"); import metrics
|
|
23
|
+
from loop.emit import _ITERATION_OUTCOMES
|
|
24
|
+
print([o for o in _ITERATION_OUTCOMES if o not in metrics._KNOWN_OUTCOME_TOKENS])
|
|
25
|
+
# -> ['approval_requested', 'replanned']
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
## Proposal
|
|
29
|
+
|
|
30
|
+
Decide the canonical vocabulary and align the two ends:
|
|
31
|
+
|
|
32
|
+
- either add `approval_requested` and `replanned` to metrics' honest-red set
|
|
33
|
+
(`scripts/metrics.py` `_HONEST_RED_OUTCOME_TOKENS`) — they are known, non-success
|
|
34
|
+
outcomes, so they belong there and should not read as "unrecognized synonyms";
|
|
35
|
+
- or narrow `emit`'s accepted set to the tokens metrics already knows.
|
|
36
|
+
|
|
37
|
+
Add a round-trip regression test: `emit.append_iteration` writing every allowed
|
|
38
|
+
outcome, then `compute_metrics(...)` reports `provenance.unrecognized_outcomes == []`.
|
|
39
|
+
|
|
40
|
+
## The gate that proves the fix
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
python3 -m pytest scripts/test_metrics.py scripts/test_emit.py # green
|
|
44
|
+
```
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
<!-- title: Integration recipe: OpenHands run → FCR gate -->
|
|
2
|
+
<!-- labels: help wanted -->
|
|
3
|
+
|
|
4
|
+
# Integration recipe: OpenHands run → FCR gate
|
|
5
|
+
|
|
6
|
+
An integration recipe that layers the loop-contract's false-completion gate over an
|
|
7
|
+
OpenHands run. **Composes, doesn't compete:** OpenHands is the EXECUTE tier — it
|
|
8
|
+
writes, runs, and tests code in a sandbox; "done" is still the agent stopping. Loop
|
|
9
|
+
Engineer wraps that exit in a typed terminal + held-out gate. This adds a layer, it
|
|
10
|
+
does not replace the runtime.
|
|
11
|
+
|
|
12
|
+
## Problem / opportunity
|
|
13
|
+
|
|
14
|
+
The design is already written — `docs/superpowers/specs/2026-06-30-st3-integration-adapters.md`
|
|
15
|
+
§5.3 — but no runnable recipe ships yet. The shipped LangGraph and Temporal recipes
|
|
16
|
+
are the template to follow:
|
|
17
|
+
|
|
18
|
+
- the adapter seam: `loop/integrations.py` (`EngineOutcome` → `to_terminal_state`);
|
|
19
|
+
- an env-guarded end-to-end example under `examples/` (skips cleanly when the engine
|
|
20
|
+
isn't installed, like `examples/langgraph-emit/` and `examples/temporal-certify/`);
|
|
21
|
+
- a CI job that exercises it.
|
|
22
|
+
|
|
23
|
+
The specialization from §5.3: OpenHands' `AgentStuckError` / max-iteration stop maps
|
|
24
|
+
to `FailedBudget`; a sandbox that touched a holdout/answer-key path (HIGH anti-cheat
|
|
25
|
+
finding) maps to `FailedUnverifiable` — the exact "the runtime ran tests but the
|
|
26
|
+
agent peeked" case OpenHands can't itself catch.
|
|
27
|
+
|
|
28
|
+
## Proposal
|
|
29
|
+
|
|
30
|
+
Follow the `docs/integrations/langgraph.md` recipe end-to-end for OpenHands: read the
|
|
31
|
+
run's trajectory as the anti-cheat trajectory input, run the holdout split, and
|
|
32
|
+
project the engine outcome through `to_terminal_state` into a typed terminal.
|
|
33
|
+
|
|
34
|
+
## The gate that proves the fix
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
python3 -m loop doctor <recipe-out> # doctor round-trip is clean
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
Plus a pinned false-completion invariant test: visible-green / holdout-red must
|
|
41
|
+
project to `FailedUnverifiable` with `false_completion: true`, and **never**
|
|
42
|
+
`Succeeded`.
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
<!-- title: Integration recipe: ruflo swarm → acceptance gate -->
|
|
2
|
+
<!-- labels: help wanted -->
|
|
3
|
+
|
|
4
|
+
# Integration recipe: ruflo swarm → acceptance gate
|
|
5
|
+
|
|
6
|
+
An integration recipe that layers a single acceptance gate over a ruflo swarm.
|
|
7
|
+
**Composes, doesn't compete:** ruflo is the ORCHESTRATE tier — a multi-agent swarm
|
|
8
|
+
whose terminal is "the coordinator decided the objective is met," a self-report
|
|
9
|
+
across N agents. Loop Engineer adds one acceptance gate the swarm must pass *as a
|
|
10
|
+
whole*. This adds a layer, it does not replace the swarm.
|
|
11
|
+
|
|
12
|
+
## Problem / opportunity
|
|
13
|
+
|
|
14
|
+
The design is written — `docs/superpowers/specs/2026-06-30-st3-integration-adapters.md`
|
|
15
|
+
§5.4 — but no runnable recipe ships yet. Same shape as the OpenHands recipe (issue
|
|
16
|
+
04): follow the shipped LangGraph/Temporal recipes as the template (`loop/integrations.py`
|
|
17
|
+
adapter, an env-guarded end-to-end example, a CI job).
|
|
18
|
+
|
|
19
|
+
The seam is the swarm's terminal hook: **no individual agent may declare the swarm
|
|
20
|
+
done** — the acceptance gate does. Register the gate as the swarm's terminal hook
|
|
21
|
+
(ruflo exposes hooks / an MCP coordination server), read the merged diff and agent
|
|
22
|
+
trails as the anti-cheat inputs, run the holdout split, and project through
|
|
23
|
+
`to_terminal_state`.
|
|
24
|
+
|
|
25
|
+
## Proposal
|
|
26
|
+
|
|
27
|
+
Follow the `docs/integrations/langgraph.md` recipe end-to-end for a ruflo swarm,
|
|
28
|
+
wiring the gate as the terminal hook so the swarm's collective "done" is decided by
|
|
29
|
+
the held-out gate, not by any agent's self-report.
|
|
30
|
+
|
|
31
|
+
## The gate that proves the fix
|
|
32
|
+
|
|
33
|
+
```bash
|
|
34
|
+
python3 -m loop doctor <recipe-out> # doctor round-trip is clean
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
Plus a pinned false-completion invariant test: visible-green / holdout-red must
|
|
38
|
+
project to `FailedUnverifiable` with `false_completion: true`, and **never**
|
|
39
|
+
`Succeeded`.
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
<!-- title: Foreign-harness gap reports — the inspect scoreboard pipeline -->
|
|
2
|
+
<!-- labels: help wanted -->
|
|
3
|
+
|
|
4
|
+
# Foreign-harness gap reports — the inspect scoreboard pipeline
|
|
5
|
+
|
|
6
|
+
Contribute a read-only `loop inspect` gap report for another public harness layout —
|
|
7
|
+
the second entry in an "inspect N public harnesses" scoreboard.
|
|
8
|
+
|
|
9
|
+
## Problem / opportunity
|
|
10
|
+
|
|
11
|
+
`docs/gap-reports/superpowers.md` is the template and the first entry. **Composes,
|
|
12
|
+
doesn't compete:** a gap report reads a foreign layout read-only and names the gaps a
|
|
13
|
+
loop-contract would *close* at the finish line — it is not a criticism of the foreign
|
|
14
|
+
harness or of the (fictional) work in the fixture. `inspect` labels a foreign layout
|
|
15
|
+
`advisory: true`; a foreign layout is read for gaps, never graded as a failing
|
|
16
|
+
contract.
|
|
17
|
+
|
|
18
|
+
The template's provenance rules are load-bearing:
|
|
19
|
+
|
|
20
|
+
- every claim is checkable against **one vendored, sanitized fixture** checked into
|
|
21
|
+
`examples/` — no version-general claims about the foreign harness itself;
|
|
22
|
+
- the report carries the §14 A1–E1 conformance table (condensed from
|
|
23
|
+
`reference/repo-os-contract.md` §14), read against that fixture;
|
|
24
|
+
- complement framing throughout ("composes, doesn't compete").
|
|
25
|
+
|
|
26
|
+
## Proposal
|
|
27
|
+
|
|
28
|
+
For another public harness layout: vendor a fictional, sanitized fixture under
|
|
29
|
+
`examples/<harness>-run/`, then author `docs/gap-reports/<harness>.md` following
|
|
30
|
+
`docs/gap-reports/superpowers.md` — the provenance blockquote, the §14 A1–E1 table,
|
|
31
|
+
the `inspect` reading, and "what emitting the contract would add," with every factual
|
|
32
|
+
claim restricted to the fixture.
|
|
33
|
+
|
|
34
|
+
## The gate that proves the fix
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
python3 -m loop inspect examples/<harness>-run # produces a scored report
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
The exit criteria follow the `scripts/test_foreign_inspect.py` patterns (scored,
|
|
41
|
+
`foreign_layout` labeled, `advisory: true`), and the report follows the template's
|
|
42
|
+
provenance rules.
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
<!-- title: emit.open_contract seeds a metrics-dirty RUNLOG placeholder -->
|
|
2
|
+
<!-- labels: help wanted -->
|
|
3
|
+
|
|
4
|
+
# emit.open_contract seeds a metrics-dirty RUNLOG placeholder
|
|
5
|
+
|
|
6
|
+
A fresh scaffold is born "metrics-dirty," which forces every integration recipe to
|
|
7
|
+
work around it. Fix the root so the workaround can be retired.
|
|
8
|
+
|
|
9
|
+
## Problem
|
|
10
|
+
|
|
11
|
+
`loop.emit.open_contract` scaffolds a `RUNLOG.md` from `templates/RUNLOG.md.tmpl`,
|
|
12
|
+
whose outcome block (`templates/RUNLOG.md.tmpl:27`) is
|
|
13
|
+
`` `{{ITERATION_OUTCOME}}` — one of: `task_passed`, … ``. `loop/scaffold.py`
|
|
14
|
+
`_substitutions` has no `ITERATION_OUTCOME` key, so `_fill` renders the unknown token
|
|
15
|
+
as the literal `REPLACE` (`loop/scaffold.py:95`). `scripts/metrics.py` then parses
|
|
16
|
+
`REPLACE` as an unrecognized outcome token, so a just-scaffolded loop is
|
|
17
|
+
metrics-dirty until real iterations land.
|
|
18
|
+
|
|
19
|
+
Checkable at this commit:
|
|
20
|
+
|
|
21
|
+
```python
|
|
22
|
+
import sys, tempfile, os
|
|
23
|
+
from loop import emit
|
|
24
|
+
sys.path.insert(0, "scripts"); import metrics
|
|
25
|
+
ws = os.path.join(tempfile.mkdtemp(), "scaffold_check")
|
|
26
|
+
emit.open_contract(ws)
|
|
27
|
+
print(metrics.compute_metrics(ws)["provenance"]["unrecognized_outcomes"])
|
|
28
|
+
# -> ['replace']
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
Because of this, both engine recipes reset `<ws>/RUNLOG.md` to emit's fresh header
|
|
32
|
+
right after `open_contract`: `examples/langgraph-emit/graph_example.py:105` and
|
|
33
|
+
`examples/temporal-certify/workflow_example.py:159`. That inline reset is a
|
|
34
|
+
documented carve-out in the plan's Global Constraints
|
|
35
|
+
(`docs/superpowers/plans/2026-07-08-v0.8.0-composes-the-field.md`, "Adjudicated
|
|
36
|
+
carve-out").
|
|
37
|
+
|
|
38
|
+
## Proposal
|
|
39
|
+
|
|
40
|
+
Make the scaffold not seed a metrics-flagged placeholder. Options, pick one:
|
|
41
|
+
|
|
42
|
+
- a first-class affordance — `emit.open_contract(seed_runlog=False)` or
|
|
43
|
+
`emit.reset_runlog(ws)` — so a caller that scores from iteration 0 opts out of the
|
|
44
|
+
placeholder; or
|
|
45
|
+
- render the placeholder outcome as a value `metrics` recognizes (or omit the seeded
|
|
46
|
+
iteration block entirely) so a fresh scaffold is metrics-clean by construction.
|
|
47
|
+
|
|
48
|
+
The reset only ever strips a placeholder — it never fabricates state — so the
|
|
49
|
+
affordance must preserve that (no synthetic completed iteration).
|
|
50
|
+
|
|
51
|
+
## The gate that proves the fix / acceptance
|
|
52
|
+
|
|
53
|
+
- A freshly-scaffolded loop reports `provenance.unrecognized_outcomes == []` (the
|
|
54
|
+
snippet above returns `[]`).
|
|
55
|
+
- Both recipes drop their inline `RUNLOG.md` resets and still pass their gates.
|
|
56
|
+
- The plan's Global-Constraints carve-out text can be retired.
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
python3 -m pytest scripts/test_metrics.py scripts/test_emit.py # green
|
|
60
|
+
python3 -m loop doctor <recipe-workspace> # recipe still clean after the inline reset is removed
|
|
61
|
+
```
|
|
@@ -0,0 +1,107 @@
|
|
|
1
|
+
# Gap report — Superpowers (foreign-harness inspect)
|
|
2
|
+
|
|
3
|
+
> **Provenance.** Evaluated **only** against the vendored fixture
|
|
4
|
+
> [`examples/superpowers-run/`](../../examples/superpowers-run/) — fictional,
|
|
5
|
+
> sanitized content, checked in. No version-general claims about Superpowers
|
|
6
|
+
> itself are made or implied: every statement below is checkable against that
|
|
7
|
+
> one directory.
|
|
8
|
+
> **Date:** 2026-07-08.
|
|
9
|
+
> **Reproduce:** `python3 -m loop inspect examples/superpowers-run`
|
|
10
|
+
|
|
11
|
+
## Composes, doesn't compete
|
|
12
|
+
|
|
13
|
+
Superpowers is a skills library — it drives *how* an agent works. The fixture
|
|
14
|
+
vendors exactly the layout such a run leaves behind: a design spec under
|
|
15
|
+
`docs/superpowers/specs/`, a plan of `- [x]` checkboxes under
|
|
16
|
+
`docs/superpowers/plans/`, and a prose `progress.md` journal. Loop Engineer is
|
|
17
|
+
the contract layer — it proves *how the work ended*. The two **compose**: a
|
|
18
|
+
Superpowers-driven run can emit a Loop-Engineer contract at the finish line.
|
|
19
|
+
This report is not a criticism of Superpowers or of the (fictional) work in the
|
|
20
|
+
fixture; the low score below measures only what a spec/plan/journal layout
|
|
21
|
+
*structurally cannot prove*.
|
|
22
|
+
|
|
23
|
+
## §14 conformance — read against the fixture
|
|
24
|
+
|
|
25
|
+
The "What the standard requires" column is condensed from
|
|
26
|
+
[`reference/repo-os-contract.md`](../../reference/repo-os-contract.md) §14.
|
|
27
|
+
|
|
28
|
+
| Item | What the standard requires | Fixture status |
|
|
29
|
+
|---|---|---|
|
|
30
|
+
| A1 | `.loop/manifest.yaml` valid against `manifest@1` incl. the canonical 7 `terminal_states` | **unmet** — no `.loop/` exists |
|
|
31
|
+
| A2 | `.loop/state.json` valid against `state@1` | **unmet** |
|
|
32
|
+
| A3 | `TASKS.json` valid against `tasks@1` (no dup ids; no evidence-free `done`) | **unmet** — plan checkboxes carry no evidence field at all |
|
|
33
|
+
| A4 | `RUNLOG.md` present | **unmet** — the journal (`progress.md`) narrates but is not an iteration log |
|
|
34
|
+
| B1 | exactly-one-of: no terminal vs valid terminal pair | **structurally unprovable** — the layout has no terminal record; "marking the work complete" lives in prose |
|
|
35
|
+
| B2 | `terminal@1` with `criteria_met`/`evidence`/`false_completion`; honest `Succeeded` rules | **structurally unprovable** |
|
|
36
|
+
| C1–C3 | receipts / repair / rollout validate when present | **absent** (nothing to check — and nothing to mine for FCR/RP) |
|
|
37
|
+
| D1–D2 | versioned `schema` keys; additive tolerance | **unmet** — no artifact carries a schema id |
|
|
38
|
+
| E1 | `doctor` lifecycle report consistent with B1 | **unmet** — `doctor` (correctly) refuses: no contract |
|
|
39
|
+
|
|
40
|
+
Each row is checkable against the fixture: `examples/superpowers-run/` contains
|
|
41
|
+
only `README.md`, `docs/superpowers/specs/2026-07-08-csv-dedupe-design.md`,
|
|
42
|
+
`docs/superpowers/plans/2026-07-08-csv-dedupe.md`, and
|
|
43
|
+
`.superpowers/sdd/progress.md`. There is no `.loop/`, no `schema:` key in any
|
|
44
|
+
file, and `progress.md`'s closing line is the prose "Marking the work
|
|
45
|
+
complete." — the exact false-completion surface B1/B2 exist to make typed.
|
|
46
|
+
Running `python3 -m loop doctor examples/superpowers-run` returns `ok: false`
|
|
47
|
+
with `lifecycle: unknown` and `missing_file` issues for every contract
|
|
48
|
+
artifact (E1).
|
|
49
|
+
|
|
50
|
+
## `inspect` reading
|
|
51
|
+
|
|
52
|
+
```json
|
|
53
|
+
{
|
|
54
|
+
"target": "examples/superpowers-run",
|
|
55
|
+
"score": 12,
|
|
56
|
+
"terminal_states_covered": 0,
|
|
57
|
+
"present": [
|
|
58
|
+
"defines verifiable success criteria"
|
|
59
|
+
],
|
|
60
|
+
"gaps": [
|
|
61
|
+
"no independent verification (verify-* script / TASKS verify command) — success is self-asserted",
|
|
62
|
+
"no approval gates declared for side-effects (destructive / secret / production / money)",
|
|
63
|
+
"no false-completion defense: no recorded holdout/anti-cheat invocation (a self-asserted false_completion flag or prose mention earns no credit)",
|
|
64
|
+
"no plan-then-execute discipline for untrusted/web reads (prompt-injection surface)",
|
|
65
|
+
"0/7 terminal states present — missing Succeeded, FailedUnverifiable, FailedBlocked, FailedBudget, FailedSafety, FailedSpecGap, AbortedByHuman (loop can end in a silent 'completed')"
|
|
66
|
+
],
|
|
67
|
+
"verdict": "weak",
|
|
68
|
+
"foreign_layout": "superpowers",
|
|
69
|
+
"advisory": true
|
|
70
|
+
}
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
The fixture scores **weak** (`score: 12`, `terminal_states_covered: 0`),
|
|
74
|
+
labeled `foreign_layout: superpowers` and `advisory: true`. Read this plainly:
|
|
75
|
+
the low score measures what the layout *cannot prove*, not the quality of the
|
|
76
|
+
work, nor of Superpowers. `advisory: true` is the whole point — a foreign
|
|
77
|
+
layout is read for gaps, never graded as a failing contract.
|
|
78
|
+
|
|
79
|
+
## What emitting the contract would add
|
|
80
|
+
|
|
81
|
+
- **A typed terminal instead of prose "complete."** The `Succeeded`/`Failed*`
|
|
82
|
+
record replaces `progress.md`'s free-text "Marking the work complete." with
|
|
83
|
+
one of the canonical 7 states plus a `false_completion` boolean.
|
|
84
|
+
- **A held-out gate that makes false completion *measurable*.** An
|
|
85
|
+
anti-cheat/holdout invocation is what turns "tests pass locally" into a
|
|
86
|
+
third-party-checkable verdict — the missing `false-completion defense` gap.
|
|
87
|
+
- **An evidence trail.** `criteria_met` maps each success criterion to a check;
|
|
88
|
+
verify bundles under `.loop/artifacts/` back the terminal instead of a
|
|
89
|
+
self-assertion.
|
|
90
|
+
- **FCR / RP derivable by `loop metrics`.** With receipts and repair records on
|
|
91
|
+
disk, false-completion-rate and repair-productivity are *computed*, not
|
|
92
|
+
claimed (the C1–C3 "nothing to mine for FCR/RP" gap).
|
|
93
|
+
|
|
94
|
+
The *how* is small: four `loop.emit` calls — `open_contract`,
|
|
95
|
+
`append_iteration`, `append_receipt`, `terminate` — at the run's finish line.
|
|
96
|
+
That path is worked end-to-end for a real engine in
|
|
97
|
+
[`docs/integrations/langgraph.md`](../integrations/langgraph.md).
|
|
98
|
+
|
|
99
|
+
## This is a seed
|
|
100
|
+
|
|
101
|
+
This is the first entry in an "inspect N public harnesses" scoreboard: read a
|
|
102
|
+
foreign layout read-only, name the gaps a contract would close, keep every
|
|
103
|
+
claim checkable against a vendored fixture. Contributions of further gap
|
|
104
|
+
reports are welcome — this file is the template. See the drafted contributor
|
|
105
|
+
issue
|
|
106
|
+
[`docs/contributing/issues/06-help-wanted-gap-reports.md`](../contributing/issues/06-help-wanted-gap-reports.md)
|
|
107
|
+
(`help wanted: gap reports`, filed on GitHub at release).
|