constraintloop 0.3.1__tar.gz → 0.5.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (80) hide show
  1. {constraintloop-0.3.1 → constraintloop-0.5.0}/.gitignore +3 -0
  2. {constraintloop-0.3.1 → constraintloop-0.5.0}/CHANGELOG.md +54 -0
  3. {constraintloop-0.3.1 → constraintloop-0.5.0}/CONTRIBUTING.md +2 -2
  4. {constraintloop-0.3.1 → constraintloop-0.5.0}/PKG-INFO +58 -19
  5. {constraintloop-0.3.1 → constraintloop-0.5.0}/README.md +57 -18
  6. constraintloop-0.5.0/docs/completion-policy.md +74 -0
  7. {constraintloop-0.3.1 → constraintloop-0.5.0}/docs/configuration.md +40 -4
  8. constraintloop-0.5.0/docs/convergence-loops.md +370 -0
  9. {constraintloop-0.3.1 → constraintloop-0.5.0}/docs/faq.md +22 -3
  10. constraintloop-0.5.0/docs/pre-release-review-2026-09-06.md +236 -0
  11. {constraintloop-0.3.1 → constraintloop-0.5.0}/docs/recipes.md +12 -4
  12. constraintloop-0.5.0/docs/release-readiness.md +75 -0
  13. {constraintloop-0.3.1 → constraintloop-0.5.0}/docs/threat-model.md +9 -0
  14. {constraintloop-0.3.1 → constraintloop-0.5.0}/pyproject.toml +1 -1
  15. {constraintloop-0.3.1 → constraintloop-0.5.0}/schema/constraintloop.schema.json +62 -0
  16. {constraintloop-0.3.1 → constraintloop-0.5.0}/scripts/check_coverage.py +2 -2
  17. constraintloop-0.5.0/scripts/wheel_failure_smoke.py +173 -0
  18. {constraintloop-0.3.1 → constraintloop-0.5.0}/src/constraintloop/__init__.py +2 -2
  19. constraintloop-0.5.0/src/constraintloop/_numbers.py +18 -0
  20. constraintloop-0.5.0/src/constraintloop/_process.py +121 -0
  21. constraintloop-0.5.0/src/constraintloop/challenges.py +248 -0
  22. {constraintloop-0.3.1 → constraintloop-0.5.0}/src/constraintloop/cli.py +87 -11
  23. {constraintloop-0.3.1 → constraintloop-0.5.0}/src/constraintloop/config.py +23 -2
  24. {constraintloop-0.3.1 → constraintloop-0.5.0}/src/constraintloop/digest.py +21 -33
  25. {constraintloop-0.3.1 → constraintloop-0.5.0}/src/constraintloop/engine.py +92 -15
  26. {constraintloop-0.3.1 → constraintloop-0.5.0}/src/constraintloop/evaluators.py +12 -9
  27. {constraintloop-0.3.1 → constraintloop-0.5.0}/src/constraintloop/hooks.py +188 -59
  28. {constraintloop-0.3.1 → constraintloop-0.5.0}/src/constraintloop/hygiene.py +4 -0
  29. {constraintloop-0.3.1 → constraintloop-0.5.0}/src/constraintloop/loops.py +201 -13
  30. {constraintloop-0.3.1 → constraintloop-0.5.0}/src/constraintloop/models.py +135 -2
  31. {constraintloop-0.3.1 → constraintloop-0.5.0}/src/constraintloop/native_cli_evaluator.py +3 -6
  32. constraintloop-0.5.0/src/constraintloop/redaction.py +38 -0
  33. {constraintloop-0.3.1 → constraintloop-0.5.0}/src/constraintloop/runners.py +22 -15
  34. {constraintloop-0.3.1 → constraintloop-0.5.0}/src/constraintloop/scaffold.py +1 -0
  35. {constraintloop-0.3.1 → constraintloop-0.5.0}/src/constraintloop/setup_hooks.py +126 -7
  36. {constraintloop-0.3.1 → constraintloop-0.5.0}/src/constraintloop/state.py +7 -3
  37. constraintloop-0.5.0/tests/test_challenges.py +601 -0
  38. {constraintloop-0.3.1 → constraintloop-0.5.0}/tests/test_cli.py +61 -1
  39. {constraintloop-0.3.1 → constraintloop-0.5.0}/tests/test_cli_commands.py +279 -1
  40. {constraintloop-0.3.1 → constraintloop-0.5.0}/tests/test_digest.py +22 -0
  41. {constraintloop-0.3.1 → constraintloop-0.5.0}/tests/test_engine.py +35 -0
  42. {constraintloop-0.3.1 → constraintloop-0.5.0}/tests/test_environment.py +13 -0
  43. {constraintloop-0.3.1 → constraintloop-0.5.0}/tests/test_eval_corpus.py +33 -0
  44. {constraintloop-0.3.1 → constraintloop-0.5.0}/tests/test_failure_lab.py +2 -1
  45. {constraintloop-0.3.1 → constraintloop-0.5.0}/tests/test_hooks.py +214 -8
  46. {constraintloop-0.3.1 → constraintloop-0.5.0}/tests/test_loops.py +2 -2
  47. {constraintloop-0.3.1 → constraintloop-0.5.0}/tests/test_models.py +57 -0
  48. {constraintloop-0.3.1 → constraintloop-0.5.0}/tests/test_native_cli_evaluator.py +109 -8
  49. constraintloop-0.5.0/tests/test_process.py +99 -0
  50. constraintloop-0.5.0/tests/test_review_regressions.py +499 -0
  51. {constraintloop-0.3.1 → constraintloop-0.5.0}/tests/test_runners.py +38 -0
  52. constraintloop-0.3.1/docs/convergence-loops.md +0 -219
  53. constraintloop-0.3.1/docs/release-readiness.md +0 -271
  54. constraintloop-0.3.1/scripts/wheel_failure_smoke.py +0 -47
  55. {constraintloop-0.3.1 → constraintloop-0.5.0}/GOVERNANCE.md +0 -0
  56. {constraintloop-0.3.1 → constraintloop-0.5.0}/LICENSE +0 -0
  57. {constraintloop-0.3.1 → constraintloop-0.5.0}/RELEASE.md +0 -0
  58. {constraintloop-0.3.1 → constraintloop-0.5.0}/SECURITY.md +0 -0
  59. {constraintloop-0.3.1 → constraintloop-0.5.0}/SUPPORT.md +0 -0
  60. {constraintloop-0.3.1 → constraintloop-0.5.0}/docs/native-cli-evaluators.md +0 -0
  61. {constraintloop-0.3.1 → constraintloop-0.5.0}/docs/openai-evaluation.md +0 -0
  62. {constraintloop-0.3.1 → constraintloop-0.5.0}/docs/provider-privacy.md +0 -0
  63. {constraintloop-0.3.1 → constraintloop-0.5.0}/scripts/check_anthropic_sdk.py +0 -0
  64. {constraintloop-0.3.1 → constraintloop-0.5.0}/scripts/check_openai_sdk.py +0 -0
  65. {constraintloop-0.3.1 → constraintloop-0.5.0}/scripts/check_sdist_contents.py +0 -0
  66. {constraintloop-0.3.1 → constraintloop-0.5.0}/scripts/generate_schema.py +0 -0
  67. {constraintloop-0.3.1 → constraintloop-0.5.0}/scripts/openai_eval_canary.py +0 -0
  68. {constraintloop-0.3.1 → constraintloop-0.5.0}/src/constraintloop/__main__.py +0 -0
  69. {constraintloop-0.3.1 → constraintloop-0.5.0}/src/constraintloop/diagnostics.py +0 -0
  70. {constraintloop-0.3.1 → constraintloop-0.5.0}/src/constraintloop/environment.py +0 -0
  71. {constraintloop-0.3.1 → constraintloop-0.5.0}/src/constraintloop/eval_corpus.py +0 -0
  72. {constraintloop-0.3.1 → constraintloop-0.5.0}/src/constraintloop/py.typed +0 -0
  73. {constraintloop-0.3.1 → constraintloop-0.5.0}/tests/__init__.py +0 -0
  74. {constraintloop-0.3.1 → constraintloop-0.5.0}/tests/failure_lab.py +0 -0
  75. {constraintloop-0.3.1 → constraintloop-0.5.0}/tests/fixtures/openai_eval_corpus_v1.yml +0 -0
  76. {constraintloop-0.3.1 → constraintloop-0.5.0}/tests/test_diagnostics.py +0 -0
  77. {constraintloop-0.3.1 → constraintloop-0.5.0}/tests/test_evaluators.py +0 -0
  78. {constraintloop-0.3.1 → constraintloop-0.5.0}/tests/test_release_metadata.py +0 -0
  79. {constraintloop-0.3.1 → constraintloop-0.5.0}/tests/test_security.py +0 -0
  80. {constraintloop-0.3.1 → constraintloop-0.5.0}/tests/test_state.py +0 -0
@@ -16,3 +16,6 @@ dist/
16
16
  .constraintloop/
17
17
  constraintloop.local.yml
18
18
  constraintloop.local.yaml
19
+ .claude/settings.local.json
20
+ .codex/hooks.json
21
+ .gemini/settings.json
@@ -5,6 +5,60 @@ Versioning, with the usual initial-development flexibility for `0.y.z`.
5
5
 
6
6
  ## [Unreleased]
7
7
 
8
+ ## [0.5.0] - 2026-09-06
9
+
10
+ - Add optional session challenge gates for Claude Code, Codex, and Gemini CLI:
11
+ generate configurable domain-grounded scenarios, record immutable plans,
12
+ verify outcomes against fresh deterministic evidence, and repair defects
13
+ without launching a model evaluator.
14
+ - Add challenge show/submit commands, challenge/verify cycle states, Gemini
15
+ loop prompts, and persisted continuation budgets for all three adapters.
16
+ - Preserve task goals through generated continuation prompts and stop Gemini
17
+ sessions on exhausted budgets instead of requesting another retry.
18
+ - Fix all ten pre-release findings: frame content hashes unambiguously; bound
19
+ ordinary recursive Stop hooks and advisory continuations; fail closed on
20
+ malformed evidence and unexpected hook errors; reject push/CI waivers in
21
+ every engine entry point; report repairable dependency failures by root
22
+ cause; renew budgets for completed tasks; refresh pending hook evidence on
23
+ the shared cycle interval; reject non-finite metrics, thresholds, and
24
+ baselines; redact retained evidence; and deny observed agent baseline
25
+ weakening and direct baseline edits.
26
+ - Reject disabled or phase-incompatible prerequisites, keep CI loops free of
27
+ local overlays, bound subprocess collection to 8 MiB, and renew supervisor
28
+ leases during long evaluations and polling waits.
29
+ - Add regression matrices for adapters, phase policy, malformed numbers, digest
30
+ boundaries, lifecycle sequences, and installed-package challenge workflows.
31
+ - Cover every changed executable line, including submission rejection,
32
+ missing requests, unreadable files, and lost leases. Report malformed YAML
33
+ by location without raw source excerpts that could disclose credentials.
34
+ - Migration: existing evidence and waivers become stale under the new input
35
+ digest. Re-run checks; review incompatible prerequisite phases and invalid
36
+ numeric baselines. Challenge gates remain opt-in. Re-run hook setup after
37
+ upgrading if hooks use a pinned ephemeral executable.
38
+
39
+ ## [0.4.1] - 2026-09-04
40
+
41
+ - Raise enforced statement coverage from 90% to 95% and branch coverage from
42
+ 80% to 90%.
43
+ - Add behavioral coverage for lifecycle CLI boundaries, hook configuration
44
+ failures, native evaluator entrypoints, timeout escalation, project
45
+ scaffolding, environment parsing, and contract validation.
46
+
47
+ ## [0.4.0] - 2026-09-04
48
+
49
+ - Enforce finite command timeouts across complete POSIX process groups so
50
+ leaked Docker/TestContainers descendants cannot hold Stop gates open.
51
+ - Add an explicit `push` phase and opt-in managed Git pre-push hook for heavy
52
+ integration gates while keeping frequent Stop checks fast.
53
+ - Install Claude hooks in gitignored local settings, persist explicit uninstall
54
+ tombstones, and remove ConstraintLoop's own committed agent settings files.
55
+ - Defer lifecycle evaluation for subagents and turns paused on background or
56
+ scheduled work.
57
+ - Compact hook failure output to counts, failing checks, and high-signal error
58
+ lines while retaining full evidence for `constraintloop debug`.
59
+ - Run commands and command evaluators from a stable project context with the
60
+ project root on `PYTHONPATH`.
61
+
8
62
  ## [0.3.1] - 2026-08-31
9
63
 
10
64
  - Prevent recursive Claude Stop-hook invocations from repeatedly blocking completion.
@@ -5,7 +5,7 @@ By participating, you agree to follow the
5
5
  the contact method documented there.
6
6
 
7
7
  ConstraintLoop accepts focused changes that strengthen evidence, state,
8
- budgeting, locks, and stopping. It is not a general agent runtime and v0.1 must
8
+ budgeting, locks, and stopping. It is not a general agent runtime and must
9
9
  not launch provider CLIs for repair turns or offer unbounded repair loops.
10
10
  Opt-in native CLI evaluators are limited to isolated, tool-disabled reviews.
11
11
 
@@ -43,5 +43,5 @@ Releases are prepared through a focused release pull request and published only
43
43
  through GitHub Trusted Publishing. See `RELEASE.md`. Contributors and agents
44
44
  must not run local package upload commands or add long-lived registry tokens.
45
45
 
46
- The supported v0.1 platforms are macOS and Linux. Windows is not supported
46
+ The supported platforms are macOS and Linux. Windows is not supported
47
47
  until hook command generation and clean-wheel tests are implemented there.
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.5
2
2
  Name: constraintloop
3
- Version: 0.3.1
3
+ Version: 0.5.0
4
4
  Summary: Evidence-based completion gates for AI coding agents
5
5
  Project-URL: Homepage, https://github.com/mauhpr/constraintloop
6
6
  Project-URL: Documentation, https://github.com/mauhpr/constraintloop/tree/main/docs
@@ -69,11 +69,17 @@ The central distinction is deliberate:
69
69
  ConstraintLoop supports Claude Code, Codex, and Gemini CLI through their hook
70
70
  lifecycles. CI is the final authority: it ignores local caches and human waivers.
71
71
 
72
- Bounded convergence loops are included in the v0.1 release scope. The design
72
+ Bounded convergence loops are implemented. The design
73
73
  keeps ConstraintLoop in control of evidence, budgets, and stopping while native
74
74
  Claude or Codex loops perform at most one requested repair per transition. See
75
75
  [docs/convergence-loops.md](docs/convergence-loops.md).
76
76
 
77
+ Completion loops can also require the current coding session to generate and
78
+ investigate N domain-grounded failure scenarios before stopping. Claude Code,
79
+ Codex, and Gemini CLI use their existing session context and tools; this
80
+ challenge gate requires no separate model evaluator. See
81
+ [session challenge gates](docs/convergence-loops.md#session-challenge-gates).
82
+
77
83
  ## At a glance
78
84
 
79
85
  | Question | ConstraintLoop answer |
@@ -123,6 +129,7 @@ uv tool install constraintloop
123
129
 
124
130
  constraintloop init
125
131
  constraintloop setup --adapter all
132
+ # Or add --pre-push above to wire heavyweight push gates locally.
126
133
  constraintloop run
127
134
  constraintloop ci
128
135
  ```
@@ -137,7 +144,7 @@ If you intentionally run setup through `uvx`, generated hooks pin the current
137
144
  ConstraintLoop version. You can choose another persistent invocation with, for
138
145
  example, `constraintloop setup --hook-executable "pipx run constraintloop"`.
139
146
 
140
- The five commands above establish this flow:
147
+ The five core commands above establish this flow:
141
148
 
142
149
  ```mermaid
143
150
  sequenceDiagram
@@ -147,7 +154,7 @@ sequenceDiagram
147
154
  participant T as Project tools
148
155
  U->>CL: init + review contract
149
156
  U->>CL: setup hooks
150
- A->>CL: run change/stop phase
157
+ A->>CL: run change/stop/push phase
151
158
  CL->>T: execute ready constraints
152
159
  T-->>CL: exit codes, metrics, artifacts
153
160
  CL-->>A: pass, repair, wait, or escalate
@@ -160,18 +167,27 @@ sequenceDiagram
160
167
  version: 1
161
168
  settings:
162
169
  max_auto_retries: 2
170
+ hook_output_limit: 4096
163
171
 
164
172
  constraints:
165
173
  tests:
166
174
  kind: command
167
175
  command: [python, -m, pytest, -q]
168
- phases: [stop, ci]
176
+ phases: [stop, push, ci]
169
177
  watch: ["src/**/*.py", "tests/**/*.py", pyproject.toml]
170
178
  retry:
171
179
  max_attempts: 3
172
180
  exit_codes: [1]
173
181
  delay_seconds: 2
174
182
 
183
+ integration_tests:
184
+ kind: command
185
+ command: [python, -m, pytest, -q, -m, integration]
186
+ phases: [push, ci]
187
+ watch: ["src/**/*.py", "tests/**/*.py", pyproject.toml]
188
+ needs: [tests]
189
+ timeout_seconds: 900
190
+
175
191
  coverage:
176
192
  kind: metric
177
193
  command: [python, -m, pytest, --cov, "--cov-report=json:coverage.json"]
@@ -266,16 +282,22 @@ advisory until their false-positive and false-negative rates are measured.
266
282
  | --- | --- | --- |
267
283
  | `change` | After a file-changing tool action | Fast syntax, formatting, or diff checks |
268
284
  | `stop` | When the agent attempts to finish | Tests, build checks, and advisory review |
285
+ | `push` | Explicit local run or opt-in Git pre-push hook | Full integration and platform suites |
269
286
  | `ci` | Protected hosted workflow | Authoritative uncached and waiver-free verification |
270
287
 
271
288
  1. `SessionStart` tells the coding agent which required gates exist.
272
289
  2. The prompt hook records the user's goal as review evidence.
273
290
  3. Before tool execution, agent attempts to edit the contract or create a
274
291
  waiver are denied.
275
- 4. After tool execution, `change` gates run and fresh results are injected.
292
+ 4. After a main-agent tool execution, `change` gates run and fresh results are
293
+ injected. Subagent tool and stop events are ignored because their working
294
+ tree may be intentionally transient.
276
295
  5. Before compaction, the completion policy is restated.
277
296
  6. At `Stop` / `AfterAgent`, required `stop` gates block completion. The agent
278
- receives precise evidence and may repair the code a bounded number of times.
297
+ receives compact evidence and may repair the code a bounded number of times.
298
+ ConstraintLoop defers this evaluation while background tasks or scheduled
299
+ wakeups are active. Full retained output remains available through
300
+ `constraintloop debug ID`.
279
301
  7. Advisory failures require either passing fresh evidence or an explicit
280
302
  snapshot-bound explanation; delivery alone never counts as review.
281
303
  8. Repeated required failure stops autonomous repair and requests a human
@@ -285,6 +307,12 @@ advisory until their false-positive and false-negative rates are measured.
285
307
  is human.
286
308
  9. `constraintloop ci` reruns every CI gate without local evidence or waivers.
287
309
 
310
+ Every command constraint has a finite timeout (300 seconds by default). On POSIX,
311
+ timeout cleanup terminates the entire spawned process group so TestContainers,
312
+ Docker clients, and other descendants cannot keep inherited output pipes open.
313
+ Commands run from their configured project-contained `cwd`, with the selected
314
+ project root prepended to `PYTHONPATH`.
315
+
288
316
  Evidence is keyed by the constraint definition and the bytes of every file
289
317
  matched by `watch`. A source change therefore makes old evidence and waivers
290
318
  stale without a mutable invalidation list. Local state lives under the
@@ -295,8 +323,12 @@ For stronger machine-local gates, create a gitignored
295
323
  `constraintloop.local.yml`. ConstraintLoop recursively merges mappings over the
296
324
  repository contract and rejects changes that could weaken committed gates.
297
325
  The authoritative `constraintloop ci` command ignores this overlay.
298
- `init` and `setup` add the overlay names and `.constraintloop/state/` to the
299
- selected project's `.gitignore`, and warn if state is already tracked.
326
+ `init` and `setup` add local state, the uninstall tombstone, generated agent hook
327
+ settings, and overlay names to the selected project's `.gitignore`, and warn if
328
+ state is already tracked. Claude uses its dedicated
329
+ `.claude/settings.local.json` path. Explicit uninstall records a local tombstone,
330
+ so a checkout that restores old committed hook wiring does not silently
331
+ reactivate ConstraintLoop; setup clears the tombstone.
300
332
 
301
333
  ### Verdicts and what they mean
302
334
 
@@ -313,19 +345,25 @@ selected project's `.gitignore`, and warn if state is already tracked.
313
345
  ## Commands
314
346
 
315
347
  - `constraintloop init` — generate a reviewable initial contract.
316
- - `constraintloop setup --adapter claude|codex|gemini|all` — merge hook entries
317
- while preserving existing hooks.
348
+ - `constraintloop setup --adapter claude|codex|gemini|all [--pre-push]` — merge
349
+ agent hook entries while preserving existing hooks; optionally install an
350
+ owned Git pre-push hook for `push` gates.
318
351
  - `constraintloop uninstall --adapter claude|codex|gemini|all` — remove only
319
- ConstraintLoop hook entries while preserving unrelated settings.
320
- - `constraintloop run --phase change|stop` run local gates with fresh caching.
352
+ ConstraintLoop hook entries while preserving unrelated settings; pass
353
+ `--pre-push` to remove an owned Git hook too.
354
+ - `constraintloop run --phase change|stop|push` — run local gates with fresh
355
+ caching. Push gates do not honor local waivers.
321
356
  - `constraintloop ci` — authoritative, uncached, waiver-free run.
322
357
  - `constraintloop cycle NAME --json` — execute one journaled loop transition.
323
358
  - `constraintloop supervise NAME` — poll pending evidence under a recoverable
324
359
  single-writer lease and exit whenever repair or termination is required.
325
- - `constraintloop loop-prompt NAME --adapter claude|codex` — print the bounded
326
- native-agent repair protocol without launching an agent.
360
+ - `constraintloop loop-prompt NAME --adapter claude|codex|gemini` — print the bounded
361
+ native-agent repair and challenge protocol without launching an agent.
362
+ - `constraintloop challenge show NAME` — inspect saved scenarios and the submission schema.
363
+ - `constraintloop challenge submit NAME --file PATH` — record session-authored
364
+ discovery or verification for the current request and input snapshot.
327
365
  - `constraintloop status` — inspect evidence without executing commands.
328
- - `constraintloop explain --phase change|stop|ci` — show why each constraint
366
+ - `constraintloop explain --phase change|stop|push|ci` — show why each constraint
329
367
  runs or is skipped, including matched and changed watch paths, cache state,
330
368
  and dependency chains.
331
369
  - `constraintloop baseline update ID|--all` — initialize or strengthen native
@@ -346,7 +384,7 @@ selected project's `.gitignore`, and warn if state is already tracked.
346
384
  - `constraintloop author` — write a review-only QA/test-authoring proposal.
347
385
 
348
386
  `enhance` and `author` intentionally do not install dependencies or modify the
349
- active contract in v0.1. Their proposal files make the future self-improvement
387
+ active contract. Their proposal files make the future self-improvement
350
388
  path auditable.
351
389
 
352
390
  ## Documentation
@@ -381,7 +419,7 @@ responses become `uncertain`; a required rubric therefore fails closed.
381
419
 
382
420
  ## Compatibility boundary
383
421
 
384
- The supported v0.3 surfaces are the CLI and exit codes, configuration schema,
422
+ The supported v0.4 surfaces are the CLI and exit codes, configuration schema,
385
423
  evaluator command protocol, native hook responses, and schema-versioned
386
424
  evidence and cycle JSON. Python submodules are internal during initial
387
425
  development and are not covered by semantic-versioning compatibility promises.
@@ -401,7 +439,8 @@ policy. ConstraintLoop records which contract ran, which inputs it covered, and
401
439
  whether the evidence is still fresh.
402
440
 
403
441
  **Why do some constraints run after every action?** Put only fast feedback in
404
- the `change` phase. Expensive tests and reviews belong in `stop` and `ci`.
442
+ the `change` phase. Keep unit checks in `stop`; put heavyweight integration
443
+ suites in `push` and `ci`.
405
444
 
406
445
  **Can I use Codex or Claude Code instead of an API evaluator?** Yes. The native
407
446
  evaluator adapter prefers the active supported CLI and remains read-only.
@@ -26,11 +26,17 @@ The central distinction is deliberate:
26
26
  ConstraintLoop supports Claude Code, Codex, and Gemini CLI through their hook
27
27
  lifecycles. CI is the final authority: it ignores local caches and human waivers.
28
28
 
29
- Bounded convergence loops are included in the v0.1 release scope. The design
29
+ Bounded convergence loops are implemented. The design
30
30
  keeps ConstraintLoop in control of evidence, budgets, and stopping while native
31
31
  Claude or Codex loops perform at most one requested repair per transition. See
32
32
  [docs/convergence-loops.md](docs/convergence-loops.md).
33
33
 
34
+ Completion loops can also require the current coding session to generate and
35
+ investigate N domain-grounded failure scenarios before stopping. Claude Code,
36
+ Codex, and Gemini CLI use their existing session context and tools; this
37
+ challenge gate requires no separate model evaluator. See
38
+ [session challenge gates](docs/convergence-loops.md#session-challenge-gates).
39
+
34
40
  ## At a glance
35
41
 
36
42
  | Question | ConstraintLoop answer |
@@ -80,6 +86,7 @@ uv tool install constraintloop
80
86
 
81
87
  constraintloop init
82
88
  constraintloop setup --adapter all
89
+ # Or add --pre-push above to wire heavyweight push gates locally.
83
90
  constraintloop run
84
91
  constraintloop ci
85
92
  ```
@@ -94,7 +101,7 @@ If you intentionally run setup through `uvx`, generated hooks pin the current
94
101
  ConstraintLoop version. You can choose another persistent invocation with, for
95
102
  example, `constraintloop setup --hook-executable "pipx run constraintloop"`.
96
103
 
97
- The five commands above establish this flow:
104
+ The five core commands above establish this flow:
98
105
 
99
106
  ```mermaid
100
107
  sequenceDiagram
@@ -104,7 +111,7 @@ sequenceDiagram
104
111
  participant T as Project tools
105
112
  U->>CL: init + review contract
106
113
  U->>CL: setup hooks
107
- A->>CL: run change/stop phase
114
+ A->>CL: run change/stop/push phase
108
115
  CL->>T: execute ready constraints
109
116
  T-->>CL: exit codes, metrics, artifacts
110
117
  CL-->>A: pass, repair, wait, or escalate
@@ -117,18 +124,27 @@ sequenceDiagram
117
124
  version: 1
118
125
  settings:
119
126
  max_auto_retries: 2
127
+ hook_output_limit: 4096
120
128
 
121
129
  constraints:
122
130
  tests:
123
131
  kind: command
124
132
  command: [python, -m, pytest, -q]
125
- phases: [stop, ci]
133
+ phases: [stop, push, ci]
126
134
  watch: ["src/**/*.py", "tests/**/*.py", pyproject.toml]
127
135
  retry:
128
136
  max_attempts: 3
129
137
  exit_codes: [1]
130
138
  delay_seconds: 2
131
139
 
140
+ integration_tests:
141
+ kind: command
142
+ command: [python, -m, pytest, -q, -m, integration]
143
+ phases: [push, ci]
144
+ watch: ["src/**/*.py", "tests/**/*.py", pyproject.toml]
145
+ needs: [tests]
146
+ timeout_seconds: 900
147
+
132
148
  coverage:
133
149
  kind: metric
134
150
  command: [python, -m, pytest, --cov, "--cov-report=json:coverage.json"]
@@ -223,16 +239,22 @@ advisory until their false-positive and false-negative rates are measured.
223
239
  | --- | --- | --- |
224
240
  | `change` | After a file-changing tool action | Fast syntax, formatting, or diff checks |
225
241
  | `stop` | When the agent attempts to finish | Tests, build checks, and advisory review |
242
+ | `push` | Explicit local run or opt-in Git pre-push hook | Full integration and platform suites |
226
243
  | `ci` | Protected hosted workflow | Authoritative uncached and waiver-free verification |
227
244
 
228
245
  1. `SessionStart` tells the coding agent which required gates exist.
229
246
  2. The prompt hook records the user's goal as review evidence.
230
247
  3. Before tool execution, agent attempts to edit the contract or create a
231
248
  waiver are denied.
232
- 4. After tool execution, `change` gates run and fresh results are injected.
249
+ 4. After a main-agent tool execution, `change` gates run and fresh results are
250
+ injected. Subagent tool and stop events are ignored because their working
251
+ tree may be intentionally transient.
233
252
  5. Before compaction, the completion policy is restated.
234
253
  6. At `Stop` / `AfterAgent`, required `stop` gates block completion. The agent
235
- receives precise evidence and may repair the code a bounded number of times.
254
+ receives compact evidence and may repair the code a bounded number of times.
255
+ ConstraintLoop defers this evaluation while background tasks or scheduled
256
+ wakeups are active. Full retained output remains available through
257
+ `constraintloop debug ID`.
236
258
  7. Advisory failures require either passing fresh evidence or an explicit
237
259
  snapshot-bound explanation; delivery alone never counts as review.
238
260
  8. Repeated required failure stops autonomous repair and requests a human
@@ -242,6 +264,12 @@ advisory until their false-positive and false-negative rates are measured.
242
264
  is human.
243
265
  9. `constraintloop ci` reruns every CI gate without local evidence or waivers.
244
266
 
267
+ Every command constraint has a finite timeout (300 seconds by default). On POSIX,
268
+ timeout cleanup terminates the entire spawned process group so TestContainers,
269
+ Docker clients, and other descendants cannot keep inherited output pipes open.
270
+ Commands run from their configured project-contained `cwd`, with the selected
271
+ project root prepended to `PYTHONPATH`.
272
+
245
273
  Evidence is keyed by the constraint definition and the bytes of every file
246
274
  matched by `watch`. A source change therefore makes old evidence and waivers
247
275
  stale without a mutable invalidation list. Local state lives under the
@@ -252,8 +280,12 @@ For stronger machine-local gates, create a gitignored
252
280
  `constraintloop.local.yml`. ConstraintLoop recursively merges mappings over the
253
281
  repository contract and rejects changes that could weaken committed gates.
254
282
  The authoritative `constraintloop ci` command ignores this overlay.
255
- `init` and `setup` add the overlay names and `.constraintloop/state/` to the
256
- selected project's `.gitignore`, and warn if state is already tracked.
283
+ `init` and `setup` add local state, the uninstall tombstone, generated agent hook
284
+ settings, and overlay names to the selected project's `.gitignore`, and warn if
285
+ state is already tracked. Claude uses its dedicated
286
+ `.claude/settings.local.json` path. Explicit uninstall records a local tombstone,
287
+ so a checkout that restores old committed hook wiring does not silently
288
+ reactivate ConstraintLoop; setup clears the tombstone.
257
289
 
258
290
  ### Verdicts and what they mean
259
291
 
@@ -270,19 +302,25 @@ selected project's `.gitignore`, and warn if state is already tracked.
270
302
  ## Commands
271
303
 
272
304
  - `constraintloop init` — generate a reviewable initial contract.
273
- - `constraintloop setup --adapter claude|codex|gemini|all` — merge hook entries
274
- while preserving existing hooks.
305
+ - `constraintloop setup --adapter claude|codex|gemini|all [--pre-push]` — merge
306
+ agent hook entries while preserving existing hooks; optionally install an
307
+ owned Git pre-push hook for `push` gates.
275
308
  - `constraintloop uninstall --adapter claude|codex|gemini|all` — remove only
276
- ConstraintLoop hook entries while preserving unrelated settings.
277
- - `constraintloop run --phase change|stop` run local gates with fresh caching.
309
+ ConstraintLoop hook entries while preserving unrelated settings; pass
310
+ `--pre-push` to remove an owned Git hook too.
311
+ - `constraintloop run --phase change|stop|push` — run local gates with fresh
312
+ caching. Push gates do not honor local waivers.
278
313
  - `constraintloop ci` — authoritative, uncached, waiver-free run.
279
314
  - `constraintloop cycle NAME --json` — execute one journaled loop transition.
280
315
  - `constraintloop supervise NAME` — poll pending evidence under a recoverable
281
316
  single-writer lease and exit whenever repair or termination is required.
282
- - `constraintloop loop-prompt NAME --adapter claude|codex` — print the bounded
283
- native-agent repair protocol without launching an agent.
317
+ - `constraintloop loop-prompt NAME --adapter claude|codex|gemini` — print the bounded
318
+ native-agent repair and challenge protocol without launching an agent.
319
+ - `constraintloop challenge show NAME` — inspect saved scenarios and the submission schema.
320
+ - `constraintloop challenge submit NAME --file PATH` — record session-authored
321
+ discovery or verification for the current request and input snapshot.
284
322
  - `constraintloop status` — inspect evidence without executing commands.
285
- - `constraintloop explain --phase change|stop|ci` — show why each constraint
323
+ - `constraintloop explain --phase change|stop|push|ci` — show why each constraint
286
324
  runs or is skipped, including matched and changed watch paths, cache state,
287
325
  and dependency chains.
288
326
  - `constraintloop baseline update ID|--all` — initialize or strengthen native
@@ -303,7 +341,7 @@ selected project's `.gitignore`, and warn if state is already tracked.
303
341
  - `constraintloop author` — write a review-only QA/test-authoring proposal.
304
342
 
305
343
  `enhance` and `author` intentionally do not install dependencies or modify the
306
- active contract in v0.1. Their proposal files make the future self-improvement
344
+ active contract. Their proposal files make the future self-improvement
307
345
  path auditable.
308
346
 
309
347
  ## Documentation
@@ -338,7 +376,7 @@ responses become `uncertain`; a required rubric therefore fails closed.
338
376
 
339
377
  ## Compatibility boundary
340
378
 
341
- The supported v0.3 surfaces are the CLI and exit codes, configuration schema,
379
+ The supported v0.4 surfaces are the CLI and exit codes, configuration schema,
342
380
  evaluator command protocol, native hook responses, and schema-versioned
343
381
  evidence and cycle JSON. Python submodules are internal during initial
344
382
  development and are not covered by semantic-versioning compatibility promises.
@@ -358,7 +396,8 @@ policy. ConstraintLoop records which contract ran, which inputs it covered, and
358
396
  whether the evidence is still fresh.
359
397
 
360
398
  **Why do some constraints run after every action?** Put only fast feedback in
361
- the `change` phase. Expensive tests and reviews belong in `stop` and `ci`.
399
+ the `change` phase. Keep unit checks in `stop`; put heavyweight integration
400
+ suites in `push` and `ci`.
362
401
 
363
402
  **Can I use Codex or Claude Code instead of an API evaluator?** Yes. The native
364
403
  evaluator adapter prefers the active supported CLI and remains read-only.
@@ -0,0 +1,74 @@
1
+ # Completion policy and v0.5 migration
2
+
3
+ ## Entry points
4
+
5
+ | Entry point | Local overlays | Local waivers | Pending refresh | Task completion |
6
+ | --- | --- | --- | --- | --- |
7
+ | `run --phase change/stop` | Strengthening only | Deterministic gates only | Use `--no-cache` for a fresh observation | Constraint report only |
8
+ | `run --phase push` | Strengthening only | Never | Use `--no-cache` | Constraint report only |
9
+ | `ci` / `run --phase ci` | Never | Never | Always uncached | Independent CI evidence |
10
+ | `cycle` / `supervise` | Except CI phase | Except push/CI | Refresh after loop interval | Persisted bounded transition; includes configured challenge work |
11
+ | Stop / AfterAgent hooks | Strengthening only | Deterministic gates only | Shared cycle interval, or refresh each event without a loop | Required evidence, configured challenges, and advisory dispositions |
12
+
13
+ The engine enforces waiver and CI-cache policy regardless of caller defaults.
14
+ `run` does not finish a session challenge gate. Use `cycle` and the native Stop
15
+ hook for completion. CI verifies committed deterministic/rubric constraints;
16
+ it does not trust a local challenge journal or start an interactive session.
17
+
18
+ Failed prerequisites produce blocked dependents with `blocked_by` IDs, not
19
+ spurious evaluation errors. A cycle's `blocking_constraints` names the root
20
+ causes, including advisory prerequisites needed by required gates. Missing
21
+ tools, parser failures, and uncertain evaluations remain blocking errors.
22
+
23
+ ## Task lifecycle
24
+
25
+ After a loop passes, a changed watched input or task goal starts a new run with
26
+ a fresh budget. Restarting an unfinished run, changing its code, or changing
27
+ its goal does not reset its limits. Unchanged completed evidence stays passed
28
+ even after the old time budget elapses. One project loop has one journal;
29
+ use separate worktrees for independent concurrent tasks.
30
+
31
+ Recursive Stop hooks are completion boundaries too. Ordinary loops use their
32
+ repair and duration budgets. Challenge loops also bound session continuations.
33
+ Without a loop, `max_auto_retries` bounds repair and advisory-disposition
34
+ continuations across changing evidence until completion succeeds. Exhaustion
35
+ returns `continue: false`, including Gemini, rather than silently allowing
36
+ completion or asking the agent to retry indefinitely.
37
+
38
+ `supervise` renews its lease during checks and waits, and checks ownership
39
+ before yielding a transition. It does not launch an agent or perform repairs.
40
+
41
+ ## Evidence and redaction
42
+
43
+ The input-digest format is versioned and length-framed: filename/content
44
+ boundaries, unreadable files, and missing baselines cannot alias ordinary
45
+ content. Upgrading invalidates old evidence and snapshot-bound waivers.
46
+
47
+ Known sensitive environment values of at least eight characters are scrubbed,
48
+ as are credential assignments such as `password=...`, `api_key=...`, and
49
+ `access_token=...`. Structured sensitive keys are scrubbed recursively.
50
+ Scrubbing applies to evidence messages, findings, structured artifact fields,
51
+ retained command output, cached reads, and native-hook feedback. Command output
52
+ is scrubbed before tail truncation so truncation cannot sever the credential
53
+ label from its value.
54
+
55
+ This is best-effort redaction, not a data-loss-prevention boundary. Unknown,
56
+ encoded, split, or unlabeled secrets may escape detection. Raw command output
57
+ exists transiently in memory for parsing, bounded to 8 MiB per subprocess.
58
+ Pre-existing files from older releases are not retroactively erased. Avoid
59
+ printing credentials and restrict access to local state. Repository artifacts
60
+ and challenge submissions remain author-controlled data; do not put secrets
61
+ in them. No new remote model API is used by the session challenge gate.
62
+
63
+ ## Upgrade checklist
64
+
65
+ 1. Upgrade ConstraintLoop to 0.5.0 and re-run `constraintloop setup --adapter all`
66
+ in projects whose hooks pin an ephemeral package version.
67
+ 2. Re-run checks to replace stale cached evidence. Revisit any intentional
68
+ local waiver against the new exact evidence; CI and push cannot use it.
69
+ 3. Ensure prerequisites are enabled in every dependent phase. Correct invalid
70
+ numeric baselines; finite numeric strings are still accepted.
71
+ 4. Keep noisy tool output below 8 MiB or write detailed reports to artifacts
72
+ and emit a compact command summary.
73
+ 5. Enable `loops.NAME.challenge` explicitly where domain-driven self-review is
74
+ desired. Omitted challenge configuration preserves ordinary gating.
@@ -22,6 +22,7 @@ Settings default to:
22
22
  | `max_auto_retries` | 2 | 0–20 |
23
23
  | `concurrency` | 4 | 1–32 |
24
24
  | `evidence_output_limit` | 65536 | 1024–1048576 bytes |
25
+ | `hook_output_limit` | 4096 | 512–32768 bytes |
25
26
  | `evaluation_bundle_limit` | 102400 | 4096–2097152 bytes |
26
27
  | `progress_interval_seconds` | 15 | 0.1–300 seconds |
27
28
 
@@ -37,9 +38,13 @@ authoritative `constraintloop ci` command ignores local overlays and always
37
38
  evaluates the committed repository contract.
38
39
 
39
40
  Every constraint supports `description`, `enforcement` (`required` or
40
- `advisory`), `phases` (`change`, `stop`, `ci`), `watch` globs, dependency IDs
41
+ `advisory`), `phases` (`change`, `stop`, `push`, `ci`), `watch` globs, dependency IDs
41
42
  in `needs`, `timeout_seconds`, and `enabled`. Dependencies must exist and the
42
43
  graph must be acyclic.
44
+ Every enabled dependent must have its prerequisites enabled in all of its
45
+ phases. A failed prerequisite blocks its dependents without running them;
46
+ cycles report the root cause for repair, even if that prerequisite is advisory.
47
+ Genuine evaluation errors still require human inspection.
43
48
  Identifiers may contain letters, numbers, dots, underscores, and hyphens.
44
49
  `watch` and `include` values must be nonempty project-relative POSIX globs.
45
50
 
@@ -63,18 +68,29 @@ retry:
63
68
 
64
69
  No retries occur when `retry` is absent. A configured policy retries only the
65
70
  listed exit codes and, when enabled, timeouts or process startup failures.
66
- Every attempt is capped by the constraint's normal timeout. Retry and periodic
71
+ Every attempt is capped by the constraint's normal timeout, which defaults to
72
+ 300 seconds and is always finite. On POSIX, a timeout terminates the entire
73
+ spawned process group, including descendants that inherited the command's output
74
+ pipes. Retry and periodic
67
75
  running status lines are emitted during human-readable runs; `--json` remains a
68
76
  single machine-readable document. `timeout_seconds` bounds each attempt and,
69
77
  unless overridden, the complete retry sequence including delays. Timeout
70
78
  retries require an explicit `total_timeout_seconds` greater than the per-attempt
71
79
  timeout. This keeps the total bound visible while leaving enough budget for a
72
- second attempt.
80
+ second attempt. Command and command-evaluator processes run from the selected
81
+ project root by default, and ConstraintLoop prepends that root to `PYTHONPATH`.
82
+ Collection of combined stdout/stderr has a separate hard 8 MiB limit per
83
+ process. Exceeding it terminates the process group and produces an error;
84
+ truncated output is never treated as a complete metric or evaluator response.
85
+ `evidence_output_limit` controls the smaller, redacted tail retained afterward.
73
86
 
74
87
  Metric constraints add `parser` and `threshold`. A parser has type `json` or
75
88
  `regex`, reads `stdout`, `stderr`, or a project-contained `file`, and selects a
76
89
  dotted JSON `path` or regex `pattern` and `group`. Threshold operators are
77
90
  `gt`, `gte`, `lt`, `lte`, and `eq`.
91
+ Measurements, thresholds, and baselines must be finite numbers. Finite numeric
92
+ strings remain supported; booleans, nulls, containers, NaN, and infinities are
93
+ rejected. `--allow-regression` never overrides numeric validation.
78
94
 
79
95
  Ratchet constraints use `kind: ratchet` with the same command and parser fields
80
96
  as a metric. Their default `mode: must_not_increase` compares the current value
@@ -89,7 +105,11 @@ constraintloop baseline update --all
89
105
  Updates that would weaken an existing baseline are rejected. Use
90
106
  `--allow-regression` only for a reviewed, intentional reset, then commit the
91
107
  baseline artifact with the contract. `baseline_file` can select another
92
- project-relative JSON file. Each baseline entry records both the numeric value
108
+ project-relative JSON file. Observed agent commands using `--allow-regression`
109
+ and direct edits to baseline files are denied by pre-tool hooks; ask the human
110
+ to perform intentional policy changes outside the hooked session. Strengthening
111
+ through the ordinary baseline-update command remains allowed. Each baseline
112
+ entry records both the numeric value
93
113
  and the SHA-256 digest of the parsed evidence source, replacing the separate
94
114
  count-and-hash bookkeeping commonly used for migration inventories.
95
115
 
@@ -133,4 +153,20 @@ state, and dependency chains without executing any gate. Human-readable final
133
153
  summaries label concrete policy failures as `constraint` and startup,
134
154
  prerequisite, or evaluation errors as `environment`; the same
135
155
  `failure_category` is retained in JSON evidence.
156
+ Hook responses use `hook_output_limit` to retain failing test names and the first
157
+ useful traceback line without injecting the complete test log. The unabridged
158
+ retained tail remains available with `constraintloop debug CONSTRAINT`.
136
159
  See `docs/convergence-loops.md` for the cycle protocol and stable exit codes.
160
+
161
+ Stop-phase loops optionally accept `challenge` for discovery and verification
162
+ performed in the active Claude Code, Codex, or Gemini CLI session. Defaults are
163
+ `count: 10` (1–100), `max_rounds: 2` (1–10), `max_continuations: 8` (1–100),
164
+ `watch: ["**/*"]`, and `domain_context: []`. Context and watch entries are
165
+ project-relative globs. The existing loop duration and repair budgets also
166
+ apply; challenge work uses no model evaluator configuration. Omit `challenge`
167
+ to retain the ordinary completion behavior. See
168
+ [session challenge gates](convergence-loops.md#session-challenge-gates) for
169
+ submission examples, evidence requirements, and adapter behavior.
170
+
171
+ See [completion policy](completion-policy.md) for the cross-entry-point matrix,
172
+ task lifecycle, redaction policy, and upgrade notes.