agentic-engineering-harness 0.4.16

package/docs/ARCHITECTURE.md

@@ -0,0 +1,47 @@
+# Architecture
+
+## Goal
+
+Provide a reusable control layer between LLM coding agents and the source repository so software quality depends on explicit contracts and executable evidence rather than model self-confidence.
+
+## Layers
+
+### 1. Agent control plane
+
+Paseo is the reference adapter. It owns process/session/worktree/mobile control, not product semantics.
+
+### 2. Semantic authority
+
+A lead agent (reference: Codex) owns requirement interpretation, architecture, SDD and review.
+
+### 3. Implementation workers
+
+Workers (reference: OpenCode + cost-efficient model) implement frozen, scoped tasks. They have no authority to redefine acceptance.
+
+### 4. Normative truth
+
+Git-versioned specs, ADRs, TaskContracts and executable acceptance criteria define intended behavior.
+
+### 5. Structural truth
+
+Graphify is the first code-intelligence adapter. Extracted graph relationships may support deterministic gates; inferred/ambiguous relationships should default to warnings until explicitly promoted.
+
+### 6. Historical memory
+
+Engram is the first memory adapter. The interface is deliberately replaceable by Cognee/Graphiti or another backend.
+
+### 7. Gate authority
+
+The deterministic harness evaluates build/type/lint/tests, scope, immutable files, API/schema policies, architecture, security and other machine-verifiable constraints.
+
+### 8. Policy
+
+OPA/Rego centralizes reusable allow/deny decisions instead of spreading policy across shell scripts.
+
+### 9. Isolation
+
+Worker execution should evolve toward an ephemeral rootless Podman sandbox with only the intended workspace writable, validation artifacts read-only, no SSH/private credentials and restricted network access.
+
+### 10. Measurement
+
+Every task should emit machine-readable reports and telemetry. Historical tasks become eval cases to measure harness improvements objectively.

package/docs/EVALS.md

@@ -0,0 +1,28 @@
+# Engineering Evals
+
+The harness itself must be evaluated.
+
+Each historical task can become a frozen eval case containing:
+
+```text
+evals/corpus/EVAL-001/
+├── metadata.yaml
+├── task.md
+├── base-commit.txt
+├── expected-invariants.yaml
+└── frozen-tests/
+```
+
+Compare harness/model/config variants on identical base commits and tasks.
+
+Primary metrics:
+
+- deterministic task success;
+- first-pass pass rate;
+- repair count;
+- human intervention rate;
+- scope/architecture/security violation rate;
+- lead-model and worker-model usage;
+- elapsed time.
+
+Production bugs should be converted into permanent regression/eval cases when practical.

package/docs/MEMORY.md

@@ -0,0 +1,28 @@
+# Memory Model
+
+## Rule
+
+Memory informs; it does not authorize.
+
+The provider abstraction starts with Engram but must remain replaceable.
+
+## Store
+
+- architectural decisions and rationale;
+- important discoveries/gotchas;
+- bugs and root causes;
+- conventions;
+- post-change implementation summaries;
+- links back to authoritative Git artifacts.
+
+## Do not use memory as source of truth for
+
+- current requirements;
+- current API/schema;
+- current acceptance criteria;
+- current migration state;
+- anything contradicted by the checked-out repository.
+
+## Future backend evaluation
+
+Compare Engram, Cognee and Graphiti using the engineering eval corpus. Measure retrieval precision, stale-memory rate, latency, operational complexity and effect on first-pass validation success.

package/docs/OBSERVABILITY.md

@@ -0,0 +1,18 @@
+# Observability
+
+The harness writes local NDJSON lifecycle events under `.harness/telemetry/` and creates OpenTelemetry spans through the OTel API.
+
+Recommended task metrics:
+
+- deterministic success rate;
+- first-pass success rate;
+- repair attempts;
+- human interventions;
+- scope and architecture violations;
+- lead/worker token consumption (when available from provider telemetry);
+- wall-clock duration;
+- validation duration;
+- memory retrieval/usefulness;
+- cost when available.
+
+A later collector/exporter integration can forward OTel data to any compatible open-source or hosted backend without changing core logic.

package/docs/OSS_STACK.md

@@ -0,0 +1,27 @@
+# OSS-first Technology Map
+
+The default policy is **zero mandatory SaaS and zero mandatory commercial-license dependency for private repositories**.
+
+Reference components:
+
+| Concern | Default | Role |
+|---|---|---|
+| Agent runtime | Paseo | Cross-provider process/session/mobile control |
+| Lead agent | Codex | Requirements, architecture, planning, review |
+| Worker | OpenCode | Routine implementation using the configured workhorse model |
+| Persistent memory | Engram | Advisory historical memory; adapter is replaceable |
+| Code topology | Graphify | Structural graph and future architecture/blast-radius gates |
+| Specification | SDD + Git | Versioned normative intent |
+| Acceptance | Gherkin + Reqnroll | Executable business behavior for .NET consumers |
+| Policy | OPA/Rego | Centralized allow/deny decisions |
+| Static/security analysis | Opengrep | OSS rules/dataflow checks |
+| Vulnerability/SBOM | Trivy | Dependency, secret, IaC and SBOM scanning |
+| Integration dependencies | Testcontainers | Real ephemeral databases/services in tests |
+| Web E2E | Playwright | Deterministic browser acceptance and traces |
+| Consumer contracts | Pact/OpenAPI checks | API compatibility |
+| Worker isolation | Podman rootless | Ephemeral least-privilege execution |
+| Telemetry | OpenTelemetry | Portable traces/metrics/log semantics |
+| Provenance | Cosign + in-toto | Artifact signing and attestations |
+| Harness quality | Engineering eval corpus | Reproducible comparison of system variants |
+
+Do not add a mandatory SaaS because it is convenient. Any hosted integration must be optional behind an interface and have a documented local/OSS path.

package/docs/PASEO.md

@@ -0,0 +1,9 @@
+# Paseo Integration
+
+Paseo is the default orchestration control plane because an existing lead agent can spawn an OpenCode subagent in the same workspace and keep controlling it remotely.
+
+The executor uses the scriptable flow `paseo run --background --quiet`, `paseo wait`, `paseo logs`, and `paseo send` for repairs. When the lead itself is a Paseo agent, Paseo supplies parent/workspace defaults automatically.
+
+The lead remains responsible for architecture and final semantic review. Paseo is an execution/control primitive, not the source of engineering truth.
+
+For a directly container-isolated worker, configure `orchestration.provider: podman`. This trades Paseo child-session visibility for stronger process isolation. Advanced setups can instead configure a Paseo custom provider binary/wrapper around a containerized OpenCode CLI.

package/docs/PUBLISHING.md

@@ -0,0 +1,110 @@
+# Publishing Agentic Engineering Harness to npm
+
+The package name is `agentic-engineering-harness` and the public CLI commands are `aeh` and `engineering-harness`.
+
+The repository is prepared for npm Trusted Publishing through GitHub Actions OIDC. No long-lived npm publish token is stored in the repository workflow.
+
+## Preflight
+
+Before any publication:
+
+```bash
+npm run release:check
+```
+
+This must pass typecheck, tests, build and `npm pack --dry-run`.
+
+Check whether the desired package name already exists:
+
+```bash
+npm view agentic-engineering-harness version
+```
+
+An npm `E404` means the name is not currently published. If another owner controls the name, choose a scoped package name before publishing rather than changing package identity after adoption.
+
+## One-time first publication
+
+npm Trusted Publisher configuration requires an npm package to exist first. The first release therefore needs one deliberate maintainer-authenticated publish.
+
+From a clean checkout of the exact release commit:
+
+```bash
+npm login
+npm run release:check
+npm publish --access public
+```
+
+Complete the npm account's required 2FA/interactive authentication. Do not create a persistent automation token solely for this bootstrap.
+
+After the first package exists, open the package settings on npmjs.com and configure a Trusted Publisher with:
+
+```text
+Provider: GitHub Actions
+Organization/user: JamesMorales04
+Repository: agentic-engineering-harness
+Workflow filename: publish.yml
+Allowed action: npm publish
+```
+
+The workflow file lives at `.github/workflows/publish.yml`; npm expects only the filename in the Trusted Publisher configuration.
+
+For the strongest steady-state posture, after the OIDC flow has been proven once, disallow traditional publish tokens for the package and retain 2FA on the maintainer account.
+
+## Steady-state release
+
+1. Change `package.json` to the intended semantic version.
+2. Ensure the CLI dispatcher reports the same version.
+3. Merge only after CI `release:check` passes.
+4. Create/publish a GitHub Release tagged exactly:
+
+```text
+v<package.json version>
+```
+
+For example:
+
+```text
+v0.4.16
+```
+
+5. The `publish-npm` workflow will:
+   - check out the release commit;
+   - use Node 24 on a GitHub-hosted runner;
+   - verify that the release tag exactly matches `package.json`;
+   - run `npm run release:check` again;
+   - execute `npm publish` using npm Trusted Publishing/OIDC.
+
+If the tag/version check fails, no publish is attempted.
+
+## What enters the npm tarball
+
+The `files` allowlist in `package.json` publishes only:
+
+```text
+dist/
+templates/
+presets/
+policies/
+schemas/
+skills/
+docs/
+```
+
+plus npm-required package metadata such as `package.json`, README and LICENSE.
+
+This is why CI runs `npm pack --dry-run`: bootstrap templates, default agents, skills and toolchain schemas are runtime assets for `aeh init`, not merely repository documentation.
+
+## Consumer installation
+
+Normal projects should pin AEH as a development dependency:
+
+```bash
+npm install --save-dev agentic-engineering-harness
+npm exec aeh -- init --setup
+```
+
+AEH should not be imported into the product runtime merely to use the engineering workflow.
+
+## Failure policy
+
+Publication is a delivery operation, not an engineering-quality gate. A registry/OIDC/permission failure must not cause the source commit to be rewritten or force-pushed. Fix the external publishing configuration and re-run/recreate the release process as appropriate while preserving the already-validated source commit.

package/docs/SDD.md

@@ -0,0 +1,35 @@
+# SDD Operating Model
+
+A change is incomplete until the following chain is coherent:
+
+```text
+Explore → Proposal → Spec → Design → Acceptance → Tasks → Frozen Contract → Apply → Verify → Archive
+```
+
+## Gherkin boundary
+
+Use Gherkin for **observable business behavior**, not implementation details.
+
+Good:
+
+```gherkin
+Rule: Staff cannot access another tenant's medical records
+
+  Scenario: Veterinarian requests a pet from another organization
+    Given Alice is a veterinarian in organization A
+    And Luna belongs to organization B
+    When Alice requests Luna's medical record
+    Then access is denied
+```
+
+Bad:
+
+```gherkin
+Scenario: Repository calls DbContext once
+```
+
+Architecture/unit tests belong elsewhere.
+
+## Phase gates
+
+A future version of the harness should validate requirement IDs across phases so later artifacts cannot silently add/remove requirements without an explicit decision.

package/docs/SECURITY.md

@@ -0,0 +1,21 @@
+# Security and Isolation
+
+A Git worktree is not a security sandbox.
+
+The target model is an ephemeral rootless container per worker:
+
+- repository workspace writable;
+- frozen contracts/acceptance validators read-only;
+- no host SSH keys;
+- no production credentials;
+- network denied or allow-listed where possible;
+- CPU/RAM/time limits;
+- extract resulting diff, then destroy sandbox.
+
+Recommended OSS tools:
+
+- Podman rootless for worker isolation;
+- Opengrep for deterministic pattern/dataflow checks;
+- Trivy for vulnerabilities, secrets, IaC and SBOM scanning;
+- OPA for policy-as-code;
+- Cosign/in-toto for later provenance/attestations.

package/docs/V0.2.md

@@ -0,0 +1,42 @@
+# v0.2 Operating Model
+
+v0.2 turns the foundation into an executable engineering control loop.
+
+```text
+Human / lead agent
+  -> SDD artifacts
+  -> traceability validation
+  -> frozen TaskContract + SHA-256 seal
+  -> Graphify before snapshot (when available)
+  -> worker executor
+       - Paseo -> OpenCode (default control-plane mode)
+       - Podman -> OpenCode (direct sandbox mode)
+  -> deterministic validation registry
+  -> structured repair packet on failure
+  -> bounded repair attempts
+  -> final ValidationReport + run artifact
+```
+
+`aeh run TASK-ID` executes that flow. Architecture/requirements remain owned by the lead agent; workers only implement the frozen contract.
+
+## Traceability
+
+A requirement is valid only when its canonical ID appears in `proposal.md`, `spec.md` as an `### ID` heading, `design.md`, `acceptance.feature` as `@ID`, `tasks.yaml`, the TaskContract and at least one known validator reference.
+
+## Executable acceptance
+
+`adapter: gherkin` discovers a `.csproj` referencing Reqnroll and runs `dotnet test` filtered by the task tag. Non-.NET systems can provide `command:` explicitly.
+
+## Worker execution
+
+Paseo mode preserves mobile/remote visibility and cross-provider orchestration. Direct Podman mode provides a stronger process boundary but is intentionally a separate execution mode; the harness does not claim Paseo itself is a container security boundary.
+
+Paseo also supports custom provider binaries, so advanced users can point a Paseo provider at their own wrapper/container strategy outside the harness.
+
+## Repairs
+
+Each failed deterministic check becomes a machine-readable repair packet. The worker receives only failing evidence and is asked for the smallest correction. The loop terminates after the configured budget.
+
+## Hard gates
+
+Built-ins: Graphify, Opengrep, Trivy, Playwright, OpenAPI, Pact/custom command and Gherkin. Graphify refresh is not fabricated by the harness: the harness consumes `graphify-out/graph.json`, refreshed through Graphify's skill or a project-specific command.

package/docs/V0.3.md

@@ -0,0 +1,40 @@
+# v0.3 — Measurement and Provenance
+
+v0.3 turns the harness from an execution gate into a measurable engineering platform.
+
+## Engineering evals
+
+An eval case lives at `evals/corpus/<case>/eval.yaml` and names a frozen `baseRef`, a task, optional fixture/setup, variants and deterministic expectations. `aeh eval run` creates a detached worktree, applies the fixture, runs the chosen variant, reads the harness run/report artifacts, scores the outcome, and destroys the worktree.
+
+The score rewards deterministic success, first-pass success, low repair count, zero human intervention and bounded cost. `aeh eval compare` ranks historical results for the same case.
+
+## Runtime metrics
+
+Every `aeh run` now records first-pass success, repair count, human interventions, wall-clock duration and any token/cost fields discoverable from structured worker output. Human interventions are explicit events recorded with `aeh intervention` so they cannot be hidden inside model prose.
+
+## OpenTelemetry
+
+Local NDJSON remains the audit trail. Set `telemetry.exporter: otlp-http-json` and an endpoint (or standard `OTEL_EXPORTER_OTLP_ENDPOINT` / `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`) to send trace-shaped events to an OTLP collector. `aeh init` installs `.harness/otel-collector.yaml` as a minimal OSS collector configuration.
+
+## Memory benchmark
+
+`aeh memory-benchmark` runs the same YAML retrieval cases against every configured command provider. This intentionally does not make Engram, Cognee, Graphiti, or any memory store authoritative. Providers compete on recall, stale-answer contamination and latency.
+
+## Test quality
+
+`mutation` and `property` are first-class validator adapters. Their commands are explicit because the correct implementation depends on the consumer stack (for example Stryker.NET vs StrykerJS or FsCheck vs fast-check).
+
+## Provenance
+
+`aeh provenance generate --artifact <file>` creates:
+
+- a SLSA v1 provenance predicate;
+- an in-toto Statement v1 with the artifact SHA-256 as subject;
+- an optional CycloneDX SBOM via Trivy;
+- an optional Sigstore bundle via `cosign sign-blob` when `--sign` is requested.
+
+The predicate links the artifact to the Git commit and, when a task ID is supplied, hashes of the run and validation reports.
+
+## Release hardening
+
+The release workflow reruns the complete harness check and `npm pack --dry-run`. Publishing requires either a published GitHub Release or explicit manual dispatch plus `NPM_TOKEN`, and uses npm provenance.

package/docs/V0.4.11.md

@@ -0,0 +1,30 @@
+# v0.4.9–v0.4.11 — Quick Workflow and Review Lifecycle
+
+## v0.4.9 — QuickContract and deterministic triage
+
+`aeh triage` accepts evidence collected by the lead (request, explicit file scope, domains, risk and escalation flags). QUICK is allowed only for bounded low-risk work with no architecture/security/auth/schema/migration/public-API/dependency escalation signal. Everything else is SPEC.
+
+A QuickContract is stored in the normal contracts directory with `mode: quick`, explicit acceptance statements, scope and immutable constraints (`breakingApiChanges=false`, `newDependencies=false`, `schemaChanges=false`). It is SHA-256 sealed like a normal TaskContract but does not require SDD artifacts.
+
+Commands:
+
+```bash
+aeh triage "Change button padding" --file src/Button.tsx --domain frontend --risk low
+aeh quick new QUICK-001 --title "Adjust button padding" --request "Change button padding" --scope src/Button.tsx --acceptance "Button uses 16px padding" --domain frontend
+aeh quick validate QUICK-001
+aeh run QUICK-001
+```
+
+## v0.4.10 — Automatic review lifecycle
+
+After deterministic validation passes for a SPEC task, the Harness executes routed reviewer agents concurrently. Reviewers must emit a typed `AEH_RESULT_JSON` payload. Findings are normalized and deduplicated. Configured blocking severities trigger bounded remediation followed by deterministic revalidation and another review round.
+
+When reviewer blockers are cleared, an orchestrator/lead performs final semantic acceptance and must return a valid orchestrator output with `finalizationSafe=true` and no unresolved items.
+
+Default policy skips agent reviewers and lead acceptance for QUICK changes to keep small changes inexpensive. Projects can enable them with `workflow.reviews.reviewQuick` and `leadAcceptanceQuick`.
+
+## v0.4.11 — Engineering workflow skill
+
+`engineering-workflow` is installed by `aeh init` and attached to the default lead agent. It teaches a Codex lead—especially one started from Paseo mobile—to inspect the repo, gather triage evidence, obey QUICK/SPEC classification, create the correct contract/spec, invoke `aeh run`, remain the parent owner and surface only meaningful failures/approvals/final acceptance.
+
+The skill also defines self-modification safety: control-plane changes produced during a run do not govern that same run; the new controller/topology becomes active on a later validated run.

package/docs/V0.4.12.md

@@ -0,0 +1,88 @@
+# v0.4.12 — Autonomous Quality Convergence
+
+v0.4.12 replaces the bounded reviewer-remediation loop with an autonomous quality convergence engine. Review remediation no longer succeeds or fails because a fixed number of rounds has elapsed. It continues until the Final Quality Gate passes or a genuine human-on-exception condition is discovered.
+
+## Final Quality Gate
+
+The default policy uses integer DebtPoints so the configured ratios are exact:
+
+| Severity | DebtPoints | DebtScore equivalent | Final maximum |
+| --- | ---: | ---: | ---: |
+| critical | 300 | 100 | 0 |
+| high | 75 | 25 | 0 |
+| medium | 24 | 8 | 0 |
+| low | 3 | 1 | 3 |
+| note | 1 | 1/3 | governed by aggregate score |
+
+Three notes equal one low exactly. `DebtScore = DebtPoints / 3`. The default final gate requires `critical=0`, `high=0`, `medium=0`, `low<=3`, and `DebtPoints<=9` (`DebtScore<=3`). Thus nine notes can pass, ten notes cannot, and three lows plus one note cannot pass.
+
+## Convergence states
+
+Each reviewer round is normalized and deduplicated, then fingerprinted and compared with accepted prior states. The engine records `INITIAL`, `IMPROVING`, `STABLE`, `STAGNATING`, `REGRESSING`, `CYCLING`, or `CONVERGED`, plus resolved, persistent, and introduced finding fingerprints.
+
+No remediation-round maximum is used by this engine. Legacy `maxRemediationRounds` configuration is accepted for compatibility but ignored.
+
+## Regression rollback
+
+Immediately before a remediation the Harness snapshots the current modified paths. If the remediation breaks deterministic validation or increases review debt, only paths affected by that remediation are restored to the checkpoint. The implementation state that existed before the rejected remediation remains intact. The Harness never uses a blanket `git reset --hard` or `git clean` for this rollback.
+
+## Automatic escalation
+
+The default escalation ladder is:
+
+1. `normal` — current implementation worker / workhorse.
+2. `quality` — focused `quality-implementer` / workhorse.
+3. `senior` — `senior-implementer` / `@brain`.
+4. `diagnosis` — read-only `oracle` / `@brain`.
+5. `replan` — read-only `planner` / `@brain`, producing an advisory implementation plan without changing the sealed contract.
+
+Critical findings can enter at the senior stage immediately. Improving quality de-escalates toward cheaper workers; stagnation, regression, and cycles escalate. After replanning, remediation resumes at the stronger implementation tier. Projects can replace stage agent names and model aliases in `project.yaml`.
+
+## Autonomous replanning
+
+Replanning is implementation-only. The TaskContract, SDD artifacts and executable acceptance remain sealed normative truth. The planner receives the current quality state and normalized findings and emits the normal structured planner output. The result is persisted under the findings directory and supplied to subsequent remediation agents as advisory strategy.
+
+## Exception detection and human-on-exception
+
+Review findings can explicitly classify exceptional conditions. The Harness also recognizes canonical exception categories. Routine implementation defects and `SYSTEM_FAILURE` remain autonomous. Human intervention is reserved for:
+
+- `SPEC_CONTRADICTION` — authoritative requirements cannot simultaneously be satisfied.
+- `REQUIRES_PRODUCT_DECISION` — the repository and sealed artifacts do not contain the required product/business decision.
+- `BLOCKED_EXTERNAL` — required credentials, account permissions, secrets, or external resources are unavailable.
+
+Run records persist `finalState`, `humanRequired`, remediation count, final DebtScore/DebtPoints, severity counts, convergence state and reviewer/diagnostic session count.
+
+## Default configuration
+
+```yaml
+workflow:
+  reviews:
+    quality:
+      severityPoints:
+        critical: 300
+        high: 75
+        medium: 24
+        low: 3
+        note: 1
+    convergence:
+      minimumDebtPointImprovement: 3
+      stagnationWindow: 2
+      cycleDetection: true
+      regressionDetection: true
+    finalQualityGate:
+      maxBySeverity:
+        critical: 0
+        high: 0
+        medium: 0
+        low: 3
+      maxDebtPoints: 9
+    escalation:
+      criticalStartStage: 2
+      replanResumeStage: 2
+      stages:
+        - { name: normal, action: remediate }
+        - { name: quality, action: remediate, agent: quality-implementer }
+        - { name: senior, action: remediate, agent: senior-implementer, model: "@brain" }
+        - { name: diagnosis, action: diagnose, agent: oracle, model: "@brain" }
+        - { name: replan, action: replan, agent: planner, model: "@brain" }
+```