@optitransfer/proofline 0.0.0-stage → 0.2.0-next.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,70 @@
1
+ # 01 — Scope and product brief
2
+
3
+ ## Problem
4
+
5
+ AI agents can complete a plausible-looking task while omitting requirements, changing behaviour outside the requested scope, skipping mandatory checks, relying on stale assumptions, or overstating what was validated. The same agent that performs the work is not a sufficient independent authority on whether the work meets its contract.
6
+
7
+ ## Product statement
8
+
9
+ **Proofline is an independent verification and delivery gate for AI-agent work.** It keeps the task contract separate from implementation, records evidence, tests the implementation against the contract, reconciles discrepancies, and reports what can legitimately be claimed at delivery.
10
+
11
+ ## Target user
12
+
13
+ A developer, technical lead, researcher, or team using coding agents and needing reliable control over scope, validation, and delivery—not just more generated code.
14
+
15
+ ## Primary jobs to be done
16
+
17
+ 1. Turn a request into a reviewable contract with explicit acceptance criteria.
18
+ 2. Observe work without assuming that narrated progress is accurate.
19
+ 3. Map each requirement to a change, test, or explicit exception.
20
+ 4. Execute or independently evaluate required checks under controlled policy.
21
+ 5. Detect missing coverage, scope drift, contradictory evidence, and unsupported claims.
22
+ 6. Provide bounded, specific corrective feedback.
23
+ 7. Produce an auditable delivery decision and portable report.
24
+
25
+ ## Core workflow
26
+
27
+ 1. **Contract:** establish requirements, constraints, exclusions, risk tier, required evidence, and approval rules.
28
+ 2. **Observe:** capture permitted lifecycle events, commands, diffs, and revision identifiers.
29
+ 3. **Verify:** run policy-approved checks and independent challenges.
30
+ 4. **Reconcile:** compare requested, implemented, tested, and reported outcomes.
31
+ 5. **Correct or escalate:** return specific gaps, retry within configured limits, or request human judgement.
32
+ 6. **Deliver:** produce status, residual risks, exceptions, and evidence references.
33
+
34
+ ## MVP — in scope
35
+
36
+ - Local CLI and local task/session state.
37
+ - Versioned task-contract schema.
38
+ - One deeply supported agent adapter, chosen after a spike.
39
+ - Event ingestion with schema validation.
40
+ - Evidence ledger with immutable record IDs and revision/environment context.
41
+ - Configurable command/check runner with allowlists and timeouts.
42
+ - Requirement-to-evidence traceability.
43
+ - Reconciliation rules and explicit terminal states.
44
+ - Bounded corrective feedback and loop detection.
45
+ - Markdown and JSON delivery reports.
46
+ - Unit, integration, adapter-contract, and end-to-end tests.
47
+ - A standalone mode with no required hosted service or HiveVault dependency.
48
+
49
+ ## Explicitly out of scope for MVP
50
+
51
+ - A general-purpose multi-agent orchestration platform.
52
+ - A hosted SaaS control plane or central telemetry service.
53
+ - A mandatory HiveVault integration.
54
+ - Automatic approval of production deployment, financial actions, or other irreversible changes.
55
+ - A claim that arbitrary natural-language work can be fully proven correct.
56
+ - A dashboard before the CLI and evidence model are validated.
57
+ - A universal policy pack for every programming language and domain.
58
+ - An agent marketplace or broad adapter matrix before one adapter is reliable.
59
+
60
+ ## Product boundaries
61
+
62
+ - The implementing agent proposes and changes; Proofline evaluates against the accepted contract.
63
+ - The user/team owns the contract and can explicitly approve changes to it.
64
+ - The verifier may identify a likely defect, but domain-specific correctness still depends on the available oracle and evidence.
65
+ - An evidence record establishes what was recorded and under what conditions; it does not automatically establish that the chosen test was sufficient.
66
+ - HiveVault may provide prior decisions, but prior context cannot satisfy a current-run verification gate by itself.
67
+
68
+ ## Success criteria for the first pilot
69
+
70
+ The MVP is worth continuing only if a real workflow shows that it catches meaningful omissions or false completion claims while keeping false blocks, latency, and interruptions tolerable. See [metrics](14-metrics.md).
@@ -0,0 +1,64 @@
1
+ # 02 — Requirements
2
+
3
+ This is the initial normative requirements baseline. IDs are stable references for issues, tests, reports, and changes. `MUST` is mandatory for MVP unless explicitly re-scoped through an ADR and changelog entry.
4
+
5
+ ## Functional requirements
6
+
7
+ | ID | Priority | Requirement | Acceptance evidence |
8
+ |---|---|---|---|
9
+ | REQ-001 | MUST | The system MUST create or load a versioned task contract before enforcement. | Contract schema tests; invalid-contract rejection |
10
+ | REQ-002 | MUST | The contract MUST express acceptance criteria, exclusions, constraints, required checks, and risk/approval policy. | Schema and contract round-trip tests |
11
+ | REQ-003 | MUST | The system MUST preserve the accepted contract separately from agent-authored task notes. | Mutation/tampering tests |
12
+ | REQ-004 | MUST | The observer MUST ingest supported adapter events and validate event schema/version. | Adapter conformance suite |
13
+ | REQ-005 | MUST | Every evidence record MUST identify its source, timestamp, task/run, revision where applicable, and result state. | Evidence schema tests |
14
+ | REQ-006 | MUST | The system MUST distinguish `passed`, `failed`, `skipped`, `not_run`, and `inconclusive`. | State-transition tests |
15
+ | REQ-007 | MUST | Each acceptance criterion MUST resolve to evidence, an explicit exception, or an unresolved gap. | Traceability report tests |
16
+ | REQ-008 | MUST | Verification MUST be able to run configured checks independently of the implementing agent's claims. | End-to-end test with deliberately false claim |
17
+ | REQ-009 | MUST | The system MUST compare requested scope, observed changes, checks, and delivery claims. | Scope-drift and reconciliation fixtures |
18
+ | REQ-010 | MUST | Delivery status MUST distinguish `verified`, `partial`, `blocked`, and `unverified`. | Decision matrix tests |
19
+ | REQ-011 | MUST | Mandatory failed, skipped, stale, or absent evidence MUST NOT be reported as verified. | Negative delivery-gate tests |
20
+ | REQ-012 | MUST | Corrective feedback MUST cite the gap, requirement ID, supporting evidence, and next action. | Feedback contract tests |
21
+ | REQ-013 | MUST | Corrective retries MUST be bounded and repeated-loop detection MUST exist. | Retry-limit and loop fixtures |
22
+ | REQ-014 | MUST | The system MUST emit human-readable and machine-readable reports. | Snapshot/schema tests |
23
+ | REQ-015 | MUST | The system MUST work without any external context store or network connection for local workflows. | Offline integration test |
24
+ | REQ-016 | MUST | The system MUST record human-approved exceptions with actor, rationale, scope, and timestamp. | Exception audit tests |
25
+ | REQ-017 | SHOULD | The system SHOULD support baseline and final revision comparison. | Git integration tests |
26
+ | REQ-018 | SHOULD | The system SHOULD support report export and verification replay. | Replay integration test |
27
+ | REQ-019 | SHOULD | The system SHOULD expose a stable adapter contract for future agent integrations. | Contract test harness |
28
+ | REQ-020 | COULD | The system COULD integrate with an external context store for prior decisions and accepted exceptions. | Optional integration tests |
29
+
30
+ ## Non-functional requirements
31
+
32
+ | ID | Requirement | Initial acceptance target |
33
+ |---|---|---|
34
+ | NFR-001 | Local-first | Core contract, evidence, and report path works offline. |
35
+ | NFR-002 | Deterministic decisions | Same contract, policy, evidence set, and verifier version produce the same gate decision. |
36
+ | NFR-003 | Explainability | Every block identifies one or more unmet gates and the evidence behind the decision. |
37
+ | NFR-004 | Performance | Establish a measured baseline; avoid arbitrary latency promises before pilot data. |
38
+ | NFR-005 | Safety | Commands run only through explicit policy; timeout and output-size limits exist. |
39
+ | NFR-006 | Privacy | No telemetry by default; evidence redaction and local storage documented. |
40
+ | NFR-007 | Portability | Reports are plain text/JSON with schema versions and no hosted-service dependency. |
41
+ | NFR-008 | Recoverability | Interrupted runs are recoverable without silently promoting status. |
42
+ | NFR-009 | Compatibility | Adapter and report schemas are versioned and migration policy documented. |
43
+ | NFR-010 | Testability | Core state machine and policy decisions are testable without live AI-provider calls. |
44
+
45
+ ## MVP acceptance criteria
46
+
47
+ The MVP cannot be declared complete until all of the following are demonstrated:
48
+
49
+ - [ ] A contract can be created, validated, frozen for a run, and versioned.
50
+ - [ ] One supported agent can be observed through a real task lifecycle.
51
+ - [ ] The changed revision and permitted verification commands are captured.
52
+ - [ ] A deliberately omitted requirement is detected.
53
+ - [ ] A deliberately false “all tests passed” claim is rejected when evidence disagrees or is absent.
54
+ - [ ] A skipped mandatory check cannot produce `verified`.
55
+ - [ ] Scope drift is reported with changed paths and a contract reference.
56
+ - [ ] Retry limits prevent unbounded corrective loops.
57
+ - [ ] Human exceptions are explicit and visible in the final report.
58
+ - [ ] Reports can be read without Proofline running.
59
+ - [ ] Offline tests and end-to-end tests pass in CI.
60
+ - [ ] Security review covers command execution, secrets, path handling, and adapter trust.
61
+
62
+ ## Change control
63
+
64
+ A requirement may be changed only with a recorded rationale, impact assessment, updated acceptance evidence, and (for architectural or trust-boundary changes) an ADR. The implementing agent cannot silently weaken the contract it is being evaluated against.
@@ -0,0 +1,43 @@
1
+ # 03 — Domain model and terminology
2
+
3
+ ## Core entities
4
+
5
+ - **Task contract:** accepted statement of scope, acceptance criteria, exclusions, constraints, required checks, risk tier, and approval rules.
6
+ - **Run:** one bounded attempt to fulfil one frozen contract against a repository/workspace revision.
7
+ - **Event:** timestamped observation from an adapter, runner, repository inspector, or human action.
8
+ - **Evidence record:** normalized, attributable record of a check, diff, claim, approval, or observation.
9
+ - **Requirement link:** mapping from a requirement to one or more implementation and verification records.
10
+ - **Finding:** a discrepancy, risk, failure, omission, or ambiguity produced by a verifier.
11
+ - **Exception:** explicit, attributable acceptance of a known gap or policy deviation.
12
+ - **Gate decision:** a deterministic decision derived from contract, policy, evidence freshness, and findings.
13
+ - **Delivery report:** portable summary of the decision, evidence, requirements coverage, exceptions, and residual risks.
14
+
15
+ ## Trust distinctions
16
+
17
+ | Item | What it can establish | What it cannot establish alone |
18
+ |---|---|---|
19
+ | Agent statement | The agent asserted something | That the assertion is true |
20
+ | Exit code | A process returned a status | That the command tested the right property |
21
+ | Test output | A test produced the recorded output | That test coverage is sufficient |
22
+ | Git diff | Files changed between revisions | That all changes are justified |
23
+ | Hash/signature | Record integrity/authorship under key assumptions | Semantic correctness of the underlying work |
24
+ | Human approval | A named actor accepted a stated risk/scope | That the risk no longer exists |
25
+
26
+ ## Required result vocabulary
27
+
28
+ - `passed`: the configured check ran and met its explicit success condition.
29
+ - `failed`: the configured check ran and did not meet its success condition.
30
+ - `skipped`: the check was deliberately not run; reason required.
31
+ - `not_run`: no execution evidence exists for this check.
32
+ - `inconclusive`: the check ran but could not establish pass or fail.
33
+ - `stale`: evidence applies to a different or superseded revision/context.
34
+ - `waived`: an authorized actor explicitly accepted a named gap; never equivalent to passed.
35
+
36
+ ## Terminal delivery states
37
+
38
+ - `verified`: all mandatory requirements and gates have fresh sufficient evidence; no unresolved mandatory finding remains.
39
+ - `partial`: useful work is delivered, but one or more requirements remain incomplete or waived and are clearly reported.
40
+ - `blocked`: a mandatory gate, policy, or dependency prevents delivery.
41
+ - `unverified`: there is insufficient evidence to determine whether the contract was met.
42
+
43
+ The system must never infer `verified` merely because the agent stopped or returned a success message.
@@ -0,0 +1,79 @@
1
+ # 04 — Architecture
2
+
3
+ ## Architectural rule
4
+
5
+ Keep the policy and decision engine independent from the agent adapter. Agent integrations translate events and actions; they must not own the truth of the delivery decision.
6
+
7
+ ## Component view
8
+
9
+ ```mermaid
10
+ flowchart TB
11
+ CLI[CLI / local API] --> CONTRACT[Contract loader and validator]
12
+ CLI --> RUN[Run coordinator]
13
+ CONTRACT --> POLICY[Policy and gate engine]
14
+ RUN --> ADAPTER[Agent adapter]
15
+ RUN --> OBS[Event observer]
16
+ ADAPTER --> OBS
17
+ OBS --> LEDGER[Evidence ledger]
18
+ GIT[Workspace / Git inspector] --> LEDGER
19
+ CHECKS[Policy-controlled check runner] --> LEDGER
20
+ LEDGER --> TRACE[Requirement traceability]
21
+ TRACE --> VERIFY[Independent verifier]
22
+ POLICY --> VERIFY
23
+ VERIFY --> RECON[Reconciliation engine]
24
+ RECON --> DECISION[Delivery decision]
25
+ DECISION --> REPORT[JSON + Markdown report]
26
+ RECON --> FEEDBACK[Bounded corrective feedback]
27
+ FEEDBACK --> RUN
28
+ HUMAN[Human approval] --> POLICY
29
+ HV[Optional HiveVault connector] -. prior context only .-> CONTRACT
30
+ ```
31
+
32
+ ## Logical modules
33
+
34
+ 1. **Contract** — schema, validation, freeze/version, approved amendments.
35
+ 2. **Run coordinator** — run lifecycle, state transitions, timeout and retry budgets.
36
+ 3. **Adapters** — vendor-specific lifecycle event and action translation.
37
+ 4. **Observer** — normalized event stream and workspace/revision observations.
38
+ 5. **Evidence ledger** — append-oriented records, references, hashes, redaction metadata.
39
+ 6. **Check runner** — allowlisted commands, controlled environment, captured outputs and exit status.
40
+ 7. **Traceability engine** — requirement-to-change-to-test/evidence graph.
41
+ 8. **Verifier** — requirement coverage, freshness, regression challenges, claim checks, policy checks.
42
+ 9. **Reconciler** — compares contract, actual diff, check outcomes, exceptions, and agent delivery claims.
43
+ 10. **Gate engine** — deterministic terminal status and policy enforcement.
44
+ 11. **Reporter** — portable Markdown and JSON output.
45
+ 12. **Optional context connector** — retrieves prior decisions from HiveVault without elevating them to current-run proof.
46
+
47
+ ## Dependency direction
48
+
49
+ `adapters -> normalized events -> ledger -> verifier -> reconciler -> gate/report`
50
+
51
+ The verifier must not depend on a particular agent SDK. The gate engine must not depend on network access, model calls, or HiveVault. Model-assisted review may be an optional signal, but it must be labelled as such and cannot override hard evidence rules.
52
+
53
+ ## Proposed storage
54
+
55
+ MVP: local run directory with versioned JSON records and report artifacts; a small embedded database may be considered only if evidence querying proves awkward. Avoid committing secrets or large raw outputs by default. Large outputs may be stored as bounded artifacts referenced by digest and path.
56
+
57
+ Suggested local layout:
58
+
59
+ ```text
60
+ .proofline/
61
+ config.yaml
62
+ contracts/
63
+ runs/
64
+ <run-id>/
65
+ run.json
66
+ events.jsonl
67
+ evidence/
68
+ findings.json
69
+ report.json
70
+ report.md
71
+ ```
72
+
73
+ ## Architectural constraints
74
+
75
+ - Evidence is append-oriented; corrections supersede earlier records.
76
+ - Every decision records verifier/policy/schema versions.
77
+ - Any model-generated finding is distinguishable from deterministic checks.
78
+ - A policy change during a run is versioned and must not silently rewrite the earlier decision.
79
+ - The final report names the workspace revision it describes.
@@ -0,0 +1,47 @@
1
+ # 05 — Task lifecycle and state machine
2
+
3
+ ## Lifecycle
4
+
5
+ ```mermaid
6
+ stateDiagram-v2
7
+ [*] --> Draft
8
+ Draft --> Contracted: validate + accept contract
9
+ Contracted --> Running: start run
10
+ Running --> Observed: task stops / checkpoint
11
+ Observed --> Verifying: evidence snapshot
12
+ Verifying --> Reconciling: checks complete
13
+ Reconciling --> Correcting: actionable gap + retry budget
14
+ Correcting --> Running: resume agent
15
+ Reconciling --> AwaitingApproval: explicit exception / high-risk action
16
+ AwaitingApproval --> Reconciling: decision recorded
17
+ Reconciling --> Verified: mandatory gates satisfied
18
+ Reconciling --> Partial: known gaps accepted / incomplete
19
+ Reconciling --> Blocked: mandatory failure or policy denial
20
+ Reconciling --> Unverified: evidence insufficient
21
+ Verified --> [*]
22
+ Partial --> [*]
23
+ Blocked --> [*]
24
+ Unverified --> [*]
25
+ ```
26
+
27
+ ## State transition rules
28
+
29
+ - A contract must be validated before a run can enter `Running`.
30
+ - A run is bound to a contract version and initial workspace revision.
31
+ - Verification uses a defined evidence snapshot; new changes invalidate evidence where relevant.
32
+ - Any material change after verification triggers re-verification of affected gates.
33
+ - `Verified` is reachable only when all mandatory gates pass with fresh evidence.
34
+ - A waived requirement remains visible as waived and normally results in `Partial`, unless the contract explicitly defines it as non-mandatory before the run.
35
+ - A policy denial cannot be converted to a pass by agent retry.
36
+ - Retry exhaustion moves the run to `Blocked` or `Unverified` with a specific reason.
37
+ - Human approval is scoped to a specific action or exception; it does not grant blanket future approval.
38
+
39
+ ## Suggested bounded retry defaults
40
+
41
+ Defaults must be configurable and validated through the pilot; initial design proposal:
42
+ - Maximum corrective cycles per run: 3.
43
+ - Maximum repeat of the same normalized finding without meaningful evidence change: 1.
44
+ - Every retry must identify which finding it attempts to resolve.
45
+ - A retry does not erase prior failures; it appends new evidence and re-evaluates affected gates.
46
+
47
+ These are starting defaults, not validated performance claims.
@@ -0,0 +1,57 @@
1
+ # 06 — Verification and reconciliation
2
+
3
+ ## Verification gates
4
+
5
+ | Gate | Question | Minimum evidence |
6
+ |---|---|---|
7
+ | G1 — Contract | Is the accepted scope complete and testable? | Validated, frozen contract; criteria and exclusions |
8
+ | G2 — Traceability | Does each requirement map to a change and/or verification record? | Requirement-to-evidence matrix |
9
+ | G3 — Execution | Were required checks actually run against the relevant revision? | Command identity, environment, revision, exit status, bounded output |
10
+ | G4 — Independent challenge | What plausible failure modes or regressions are not covered by the agent's own checks? | Diff review, negative/edge-case tests, regression challenges |
11
+ | G5 — Reconciliation | Do requested scope, observed changes, results, and agent claims agree? | Discrepancy list with dispositions |
12
+ | G6 — Delivery | Is the report accurate and are artifacts present? | Final revision, report, mandatory gate status, exceptions |
13
+
14
+ ## Reconciliation dimensions
15
+
16
+ For each requirement, compare:
17
+
18
+ 1. **Requested:** what the accepted contract requires.
19
+ 2. **Changed:** what the workspace diff actually changed.
20
+ 3. **Tested:** what the recorded checks actually exercised.
21
+ 4. **Observed:** what adapter and repository events show.
22
+ 5. **Claimed:** what the agent says it completed.
23
+ 6. **Accepted:** what a human explicitly waived or changed through contract amendment.
24
+
25
+ A mismatch must create a finding. The system must not silently choose the most optimistic interpretation.
26
+
27
+ ## Finding format
28
+
29
+ Each finding should include:
30
+ - stable finding ID and severity;
31
+ - affected requirement IDs;
32
+ - category (`omission`, `scope_drift`, `failed_check`, `stale_evidence`, `unsupported_claim`, `security`, `ambiguity`, `policy_denial`);
33
+ - concise statement;
34
+ - supporting evidence IDs and revision;
35
+ - confidence/verification method where probabilistic;
36
+ - recommended next action;
37
+ - disposition (`open`, `resolved`, `accepted_exception`, `not_applicable`) and who recorded it.
38
+
39
+ ## Independence model
40
+
41
+ Independence is layered, not absolute:
42
+ - **Deterministic:** schema validation, status rules, diff paths, command exit codes, freshness checks, traceability completeness.
43
+ - **Test-based:** tests generated or selected from acceptance criteria and run independently.
44
+ - **Heuristic:** static analysis, diff heuristics, suspicious claim checks.
45
+ - **Model-assisted:** a separate reviewer/model may challenge the work, but its output is a finding to corroborate, not proof by itself.
46
+ - **Human:** domain judgement for ambiguous requirements, high-risk changes, or inadequate automated oracles.
47
+
48
+ ## Decision matrix
49
+
50
+ | Condition | Permitted outcome |
51
+ |---|---|
52
+ | All mandatory requirements have fresh sufficient evidence; no open mandatory findings | `verified` |
53
+ | Work is useful but one or more requirements remain incomplete or explicitly waived | `partial` |
54
+ | Mandatory check failed, policy denied a required action, or dependency blocks required validation | `blocked` |
55
+ | Evidence is absent, stale, contradictory, or too weak to decide | `unverified` |
56
+
57
+ No status may be inferred from an agent's final message alone.
@@ -0,0 +1,69 @@
1
+ # 07 — Evidence model
2
+
3
+ ## Evidence record fields
4
+
5
+ The exact schema will be versioned in implementation. Every record should support the following fields:
6
+
7
+ | Field | Purpose |
8
+ |---|---|
9
+ | `schema_version` | Parse/migration contract |
10
+ | `evidence_id` | Stable unique identifier |
11
+ | `run_id` | Run that produced or collected it |
12
+ | `kind` | `command`, `test`, `diff`, `agent_claim`, `review`, `approval`, `environment`, `observation`, etc. |
13
+ | `source` | Adapter, runner, repository inspector, human, or verifier |
14
+ | `created_at` | Timestamp with timezone |
15
+ | `requirement_ids` | Related requirements where known |
16
+ | `revision` | Commit/tree/workspace identifier, where applicable |
17
+ | `environment` | Relevant runtime/tool versions and configuration fingerprint |
18
+ | `result` | `passed`, `failed`, `skipped`, `not_run`, `inconclusive`, `stale`, or `waived` |
19
+ | `command` | Structured argv, working directory, and policy ID for executed checks |
20
+ | `exit_code` | Process result where applicable |
21
+ | `output_ref` | Reference to bounded/sanitized output artifact |
22
+ | `digest` | Content digest for integrity and deduplication |
23
+ | `redactions` | Fields/content omitted or redacted and reason |
24
+ | `supersedes` | Prior record IDs replaced by a corrected observation |
25
+ | `notes` | Short explanation, not a substitute for structured fields |
26
+
27
+ ## Command evidence
28
+
29
+ A `command` criterion records one evidence object per run. A run that failed is re-run (`defaults.flaky_reruns`), and the evidence then holds both runs as `first` and `rerun`, so a flaky result shows which tests failed on the first run and passed on the re-run.
30
+
31
+ | Field | Purpose |
32
+ |---|---|
33
+ | `cmd`, `exit`, `ms`, `timedOut` | The command as run, its exit code, its duration and whether it hit `timeout_s` |
34
+ | `counts` | `tests`, `pass`, `fail`, `skipped`, `todo` and `cancelled` from node's summary (`# fail 3` or `ℹ fail 3`) |
35
+ | `out_tail` | The last 1,200 characters of stdout and stderr |
36
+ | `failing` | The names of the failed tests found anywhere in the output, not only in `out_tail`: TAP `not ok <n> - <name>` lines, spec and dot reporter `✖ <name> (<ms>ms)` lines, and the `failing tests:` summary node prints at the end. Tests marked TODO or SKIP (in the spec reporter, a `# <reason>` after the duration) are left out. Repeats are dropped, the order is that of first appearance, and the list holds at most 100 names. Output in another format gives an empty list; `out_tail` still holds its end |
37
+ | `failing_total` | Present only when more than 100 names were found: how many there were |
38
+
39
+ The FAIL detail names the first 10 failing tests (`9 tests failed: a, b, c`, then `and <n> more` past 10), and the gate report lists every recorded name under **Failing tests**, per run.
40
+
41
+ ## Evidence rules
42
+
43
+ - Capture the exact revision against which each check ran.
44
+ - A check run before a relevant change is stale for the changed behaviour unless freshness policy proves otherwise.
45
+ - With `defaults.reuse_results` on, a `command` or `metric` PASS is recorded as a `result` entry, and a later verify or gate on the same HEAD with a clean tree may reuse it (key: verifier, HEAD, node version, the env vars the command names) for up to `reuse_max_age_h` hours. The reused result is read from that entry, only while it is in an intact chain with the hash the cache file stored, and the report names it: `reused from <time> (evidence entry <n>)`.
46
+ - Preserve failed results after a later retry passes; final state may improve but history must not be rewritten.
47
+ - Bound output size and execution time.
48
+ - Redact secrets before persistence; do not assume command output is safe.
49
+ - Record why a check was skipped or could not run.
50
+ - Separate raw source evidence from verifier interpretation.
51
+ - A hash protects integrity, not semantic validity.
52
+ - Evidence may be exported without including full source code or secrets.
53
+
54
+ ## Evidence lifecycle
55
+
56
+ ```mermaid
57
+ flowchart LR
58
+ S[Source event / check] --> N[Normalize + validate]
59
+ N --> R[Redact + bound]
60
+ R --> H[Digest + assign ID]
61
+ H --> L[Append to ledger]
62
+ L --> F[Link to requirements/findings]
63
+ F --> D[Gate decision]
64
+ D --> P[Portable report]
65
+ ```
66
+
67
+ ## Report portability
68
+
69
+ The report must contain sufficient metadata to interpret the decision after the local process exits: contract version, run ID, revision, policy/verifier versions, requirement status, check status, exceptions, findings, and artifact references. Any omitted raw data must be clearly marked as omitted/redacted and not silently treated as inspected.
@@ -0,0 +1,61 @@
1
+ # 08 — Agent adapters
2
+
3
+ ## Purpose
4
+
5
+ Adapters translate tool-specific lifecycle signals into a stable event model. They do not make final verification decisions and must not have authority to weaken the contract.
6
+
7
+ ## Adapter boundary
8
+
9
+ ```mermaid
10
+ flowchart LR
11
+ AGENT[Agent / IDE / CLI] <--> AD[Vendor adapter]
12
+ AD --> EV[Normalized event schema]
13
+ EV --> CORE[Proofline core]
14
+ CORE --> FB[Policy-approved feedback/action]
15
+ FB --> AD
16
+ AD --> AGENT
17
+ ```
18
+
19
+ ## Initial adapter selection
20
+
21
+ The first adapter should be chosen by a short technical spike using:
22
+ - quality of lifecycle signals (start, tool use, stop/completion);
23
+ - ability to capture the changed revision and command results;
24
+ - stability and documented extension points;
25
+ - safe failure behaviour when the agent or hook crashes;
26
+ - ease of local installation and reproducible integration tests.
27
+
28
+ Claude Code and Codex are candidate first adapters because they are widely used and expose lifecycle hooks. The choice must be recorded after testing rather than assumed in advance. Do not implement many thin adapters before one end-to-end path is trustworthy.
29
+
30
+ ## Normalized event categories
31
+
32
+ - `run_started`
33
+ - `agent_message` (claim, not proof)
34
+ - `tool_invocation_requested`
35
+ - `tool_invocation_completed`
36
+ - `workspace_changed`
37
+ - `check_requested`
38
+ - `check_completed`
39
+ - `agent_paused`
40
+ - `agent_completed`
41
+ - `approval_requested`
42
+ - `approval_granted`
43
+ - `approval_denied`
44
+ - `adapter_error`
45
+
46
+ Each event includes schema version, event ID, run ID, source adapter/version, timestamp, and any relevant revision or correlation ID.
47
+
48
+ ## Adapter requirements
49
+
50
+ - Validate and normalize all incoming data.
51
+ - Treat tool arguments and agent text as untrusted.
52
+ - Do not leak secrets in logs or feedback.
53
+ - Preserve ordering/correlation where the host supports it; otherwise report the limitation.
54
+ - Fail closed for enforcement-critical events when the adapter cannot establish a required fact.
55
+ - Never mark a gate verified merely because the host emitted a completion event.
56
+ - Support `observe` mode without taking control of the agent.
57
+ - Document host-specific gaps and version compatibility.
58
+
59
+ ## Hook versus core
60
+
61
+ Hooks are an integration mechanism, not the product differentiator. The durable value is the contract, evidence model, independent checks, reconciliation logic, and enforceable delivery semantics behind those hooks.
@@ -0,0 +1,64 @@
1
+ # 09 — Security and trust boundaries
2
+
3
+ ## Threat model
4
+
5
+ Assets:
6
+ - source code and working-tree state;
7
+ - credentials, environment variables, and private tool output;
8
+ - task contracts, evidence, findings, and approvals;
9
+ - integrity of the delivery decision;
10
+ - ability to execute commands or modify external systems.
11
+
12
+ Threat actors and failure sources:
13
+ - malicious or compromised agent output;
14
+ - prompt injection embedded in repository files, issue text, tool output, or dependency docs;
15
+ - unsafe project scripts or malicious test commands;
16
+ - compromised adapter or model provider;
17
+ - accidental secret capture;
18
+ - confused-deputy behaviour where an agent tricks the verifier into using broader permissions;
19
+ - false confidence from stale tests, mocked checks, or weak acceptance criteria.
20
+
21
+ ## Trust boundaries
22
+
23
+ ```mermaid
24
+ flowchart TB
25
+ subgraph Untrusted
26
+ A[Agent output]
27
+ R[Repository files / tool output]
28
+ X[External model review]
29
+ end
30
+ subgraph Controlled
31
+ AD[Validated adapter]
32
+ P[Command policy]
33
+ L[Evidence ledger]
34
+ V[Deterministic verifier]
35
+ G[Gate engine]
36
+ end
37
+ H[Human approval] --> G
38
+ A --> AD
39
+ R --> AD
40
+ AD --> L
41
+ P --> L
42
+ L --> V
43
+ X -. advisory findings only .-> V
44
+ V --> G
45
+ ```
46
+
47
+ ## Required controls for MVP
48
+
49
+ - Explicit command allowlist/policy; no arbitrary command execution by default.
50
+ - Command argv should be structured; avoid shell interpolation where possible.
51
+ - Working-directory containment and path traversal protections.
52
+ - Process timeouts, output-size limits, and environment filtering.
53
+ - Secret redaction before evidence persistence.
54
+ - Read-only observe mode as the safest starting point.
55
+ - Separate permission for applying corrective edits versus reporting findings.
56
+ - Explicit approval for destructive or external side effects.
57
+ - Adapter event schema validation and compatibility checks.
58
+ - Report that identifies policy and verifier versions.
59
+ - No default telemetry or background upload.
60
+ - Audit trail for policy changes, exceptions, and approvals.
61
+
62
+ ## Security non-goals
63
+
64
+ A local verifier cannot guarantee that arbitrary code is harmless, that every malicious change will be detected, or that a model reviewer is free from manipulation. The system reduces specific classes of process failure and makes evidence and residual uncertainty visible; it is not a formal proof of software safety.
@@ -0,0 +1,59 @@
1
+ # 10 — Test and conformance strategy
2
+
3
+ ## Test layers
4
+
5
+ 1. **Schema tests:** valid/invalid contracts, event records, evidence records, and reports.
6
+ 2. **Unit tests:** state machine, freshness, gate decisions, retry limits, traceability, redaction.
7
+ 3. **Property tests:** deterministic decisions; evidence append/supersede rules; no `verified` state with unmet mandatory gates.
8
+ 4. **Adapter contract tests:** event normalization, failure behaviour, version compatibility.
9
+ 5. **Integration tests:** CLI + local workspace + check runner + ledger + report.
10
+ 6. **End-to-end tests:** realistic agent session with controlled fixtures; no live provider required in default CI.
11
+ 7. **Adversarial fixtures:** false completion claims, skipped tests, stale test results, scope drift, malformed events, secret output, repeated corrective loops.
12
+ 8. **Security tests:** command injection attempts, path traversal, environment leakage, oversized output, unsafe symlinks, permission denial.
13
+ 9. **Usability/pilot tests:** interruption count, false blocks, caught omissions, latency, user review burden.
14
+
15
+ ## Mandatory negative fixtures
16
+
17
+ | Fixture | Expected result |
18
+ |---|---|
19
+ | Agent says tests passed; no test evidence exists | Not `verified`; report missing evidence |
20
+ | Mandatory test exits non-zero | `blocked` unless authorized policy explicitly defines a different non-verified disposition |
21
+ | Mandatory test was skipped | Not `verified` |
22
+ | Tests pass on old revision, then relevant code changes | Mark evidence stale and re-run or block |
23
+ | Required acceptance criterion has no evidence mapping | Finding; not `verified` |
24
+ | Diff changes excluded path or unrelated behaviour | Scope-drift finding |
25
+ | Same finding repeats without meaningful change | Stop retry loop |
26
+ | Agent claims a file changed but diff does not contain it | Unsupported-claim finding |
27
+ | Check output contains a secret-shaped value | Redact before persistence; test redaction metadata |
28
+ | Adapter drops an enforcement-critical event | Enforcement cannot silently pass |
29
+ | Gate in a fresh clone with no pinned base, where the default base is HEAD | `BLOCKED` with a corrective naming `--base`; never a pass on an empty diff |
30
+ | Same gate with `--base <parent>` and a change outside the task's scope | Scope-drift finding |
31
+ | Repository already has a pre-push hook that refuses the push | After `init` the push is still refused by that hook; a passing hook hands over to the gate |
32
+ | Two builders share a contract with owners | `next --owner a` never returns a task owned by `b`; an owner outside `defaults.owners` fails validation |
33
+ | Metric run while the machine's load per CPU is above `defaults.max_load_per_cpu` | `BLOCKED` (machine busy, with the numbers); no measured value recorded |
34
+ | Commit adds or amends `proofline.json` with the default deny-list | No forbidden finding in the contract itself; the change is judged by the weakening check |
35
+ | `defaults.reuse_results` on (`true`/`false`, default `false`; `defaults.reuse_max_age_h`, a positive number, default 24): `verify` then `gate` on one clean commit | A `command` or `metric` PASS runs once; the gate report lists it under Reused results as `reused from <time> (evidence entry <n>)`, and the gate entry says the same |
36
+ | Same, with an uncommitted edit or untracked file, a new commit, `--fresh`, a changed env var the command names, a PASS older than `reuse_max_age_h`, or a log that no longer holds the recording entry intact with its hash | The check runs again |
37
+ | Same, where the check FAILs, is flaky, is blocked or cannot run | Nothing is stored; it runs every time |
38
+ | `reuse_results` left out or `false` | Every check runs every time; nothing is stored |
39
+
40
+ ## Conformance fixture format
41
+
42
+ Each fixture should include:
43
+ - contract;
44
+ - initial and final workspace state or synthetic diff;
45
+ - event stream;
46
+ - check results;
47
+ - agent delivery claims;
48
+ - expected findings;
49
+ - expected terminal status;
50
+ - rationale.
51
+
52
+ ## Release gate
53
+
54
+ No release candidate may be described as verified until:
55
+ - all required CI checks pass;
56
+ - all mandatory conformance fixtures pass;
57
+ - security-sensitive changes have explicit review;
58
+ - release artifacts and checksums are recorded;
59
+ - documentation and known limitations match the actual build.