codex-workflow-v2 2.0.0-beta.1 → 2.0.0-beta.11

package/docs/problem-briefs/04-task-dependency-and-structural-replacement-integrity.md

@@ -0,0 +1,566 @@
+---
+brief_id: WF2-P04
+problem: "Milestone Task execution is not guarded by explicit canonical dependencies"
+status: approved
+owner: null
+created_at: 2026-08-23
+approved_at: 2026-08-24
+implemented_at: null
+revision: 5
+---
+
+# Brief — P04-A Task Dependency Integrity
+
+## 0. Evidence and scope
+
+The normal Task dependency gap is **confirmed**.
+
+Current Milestone membership represents:
+
+```text
+taskId
+disposition
+reason
+```
+
+It has no canonical Task dependency edges. `startTask` checks that the Milestone is active and the
+Task membership is required, but it does not check predecessor completion. Consequently, user or
+Coordinator intent such as “Required Tasks execute sequentially” is not enforced by Core.
+
+Current beta replacement already atomically updates Discovery, old Task, one replacement Task,
+Milestone plan/state, Knowledge Map when eligible, and writer-lock state. It is a 1→1 operation and
+does not preserve completed Steps or rewire downstream Tasks.
+
+This brief implements **P04-A only**:
+
+```text
+canonical Task dependencies
+-> graph validation
+-> runnable computation
+-> guarded routing, claim, lease acquisition, and direct start
+```
+
+It does not implement structural replacement. Until separate replacement authority exists, the
+current 1→1 route always fails closed; even an empty historical edge set cannot authorize the new
+Task boundary or topology.
+
+Old active-M3 recovery, historical replacement IDs, and old T07/T08/T09/T10 topology are excluded
+from mandatory materialization evidence. The implementation is validated from a clean pre-Milestone
+repository and fresh Discovery.
+
+## 1. Problem
+
+Milestone execution permission is currently derived from Task-local status and required membership.
+Membership order or Coordinator convention may influence which Task is selected, but neither is
+canonical dependency authority.
+
+Required rule:
+
+> A Task may start only when all explicit canonical predecessor Tasks are satisfied.
+
+Ordinal and membership position remain display, history, and deterministic tie-break information.
+They cannot create or satisfy a dependency.
+
+## 2. Required invariants
+
+### INV-1 — Dependencies are explicit
+
+A Task dependency exists only as a canonical Task-to-Task edge supplied by current
+Discovery/Milestone decomposition authority and committed in the Milestone Plan.
+
+### INV-2 — Runnable depends on predecessors
+
+```text
+TASK_RUNNABLE =
+  task_execution_eligible
+  AND all_required_predecessors_satisfied
+```
+
+### INV-3 — Every execution entry point uses the same predicate
+
+Core applies the predecessor predicate to repository navigation, scheduling, Task routing, handoff
+preparation, `task claim`, writer-lease acquisition, and direct `task start`. Authorization does not
+make a dependency-blocked Task runnable. A caller cannot bypass the graph by invoking a mutation
+directly.
+
+### INV-4 — Ordinal is not dependency semantics
+
+Membership position, display number, Task ID age, title, filename, and Coordinator prose cannot
+create, remove, or satisfy an edge.
+
+### INV-5 — Independent Tasks remain independent
+
+Workflow does not add edges merely to preserve a previous sequential scheduler policy. Scheduler
+policy chooses only after the complete runnable set is computed.
+
+### INV-6 — Decomposition owns topology; Core validates it
+
+Existing Discovery/Milestone planning supplies explicit provider/consumer Task mappings. P04 binds
+those mappings to concrete Task IDs and validates their structure. It does not require P01 to be
+implemented first.
+
+If deferred P01-B is later approved and implemented, it strengthens semantic admission by
+validating prerequisite, provider, output, and ownership facts before P04 receives the concrete
+edge. P01-A does not provide those facts. P04 does not duplicate that semantic model, and Core
+never infers a provider from prose, ordinals, titles, or paths.
+
+### INV-7 — Existing Milestone authority binds the graph
+
+Task dependencies are part of the Milestone Plan and membership authority. Existing
+`milestone.revision`, `membershipRevision`, and `planHash` invalidate stale authorization and role
+context when the relevant dependency view changes.
+
+### INV-8 — Task and Step dependencies are separate
+
+P02 Step dependencies do not automatically create Task edges. Task edges do not modify Step Plans.
+
+### INV-9 — Unknown dependency state is not permission
+
+New Milestone Plans explicitly declare an array for every membership, including `[]`. Missing
+dependency data in legacy active state cannot be interpreted as proof that the Task is independent.
+
+### INV-10 — Dependency mutation is atomic
+
+Memberships, dependency arrays, Milestone revisions, Plan hash, and authorization supersession are
+committed through the required Milestone Plan/scope-change transaction. P04-A must add this boundary
+to initial Plan commit where it is currently absent. No partial graph is observable after rejection
+or recovery.
+
+## 3. P04-A solution
+
+### A. Canonical dependency representation
+
+Extend Milestone membership or use an equivalent normalized representation:
+
+```text
+MilestoneMembership {
+  taskId
+  disposition
+  reason
+  dependsOnTaskIds[]
+}
+```
+
+The direction is exact:
+
+```text
+consumer.dependsOnTaskIds = [provider]
+```
+
+Every new canonical membership contains the field. An empty array means known independent under the
+approved Milestone Plan; absence does not mean the same thing.
+
+No separate dependency registry, Contract Registry, graph credential, or per-edge lifecycle is
+introduced.
+
+### B. Graph validation
+
+Before Milestone Plan commit or authorization:
+
+- every membership declares `dependsOnTaskIds`, including an explicit empty array;
+- every dependency target exists;
+- every target belongs to the same Milestone;
+- duplicate edges are rejected before hashing;
+- each valid dependency array is sorted by exact Task ID before Plan hashing and persistence;
+- self-edges and cycles are rejected;
+- a required consumer cannot depend on a waived or cancelled provider;
+- independent Tasks receive no inferred edges;
+- the canonical arrays are included in the Milestone Plan hash;
+- ambiguous semantic provider selection returns to decomposition authority; Scheduler does not
+  guess.
+
+P04 validates the supplied graph. Before P01-B or equivalent semantic authority exists, semantic
+correctness remains the responsibility of the existing decomposition/Plan review boundary rather
+than a claim made mechanically by Core.
+
+### C. Runnable predicate
+
+For each active required Task:
+
+```text
+predecessor satisfied =
+  predecessor membership is required
+  AND predecessor Task status is merged
+```
+
+The initial implementation uses `merged` because current Milestone validation already treats merged
+required Tasks as completed. No alternative terminal status is invented.
+
+Routing computes the complete runnable set first. Existing sequential scheduling may choose one
+Task from that set, using ordinal only as a deterministic tie-break. If two independent Tasks are
+runnable, sequential execution remains permitted but no artificial edge is recorded.
+
+### D. Guards on navigation, handoff, claim, and start
+
+The same predicate governs:
+
+- repository `next` and Task-specific `next` routing;
+- scheduler selection;
+- C1 handoff preparation;
+- C1 `task claim` and writer-lease acquisition;
+- direct `task start`;
+- any equivalent direct mutation that begins execution.
+
+A prepared or approved Task becomes non-runnable if its relevant Milestone dependency authority
+changes. A prepared handoff is bound to the current Milestone revision, membership revision, and
+Plan hash; a change makes it unclaimable. Both `task claim` and `task start` re-read current
+Milestone state and cannot rely only on an older routing result or credential. A dependency-blocked
+claim must not acquire or retain a writer lease.
+
+For a Milestone-linked Task, the canonical handoff event and its credential projection add exactly:
+
+```text
+dependencyBinding {
+  milestoneRevision
+  membershipRevision
+  milestonePlanHash
+}
+```
+
+The existing top-level `milestoneId` identifies the Milestone and is not duplicated inside the
+binding. The claim token remains bound to the handoff event as today. Claim validates the stored
+binding against current Milestone authority before writer-lease acquisition. Standalone Tasks use
+`dependencyBinding: null`. No dependency data or credential is copied into chat evidence.
+
+Execution authorization may exist before predecessors merge, but it does not bypass dependencies.
+Existing base-HEAD and authorization freshness rules remain independently applicable.
+
+### E. Freshness and atomic Milestone binding
+
+Reuse existing authority:
+
+```text
+milestone revision
+membership revision
+Milestone plan hash
+Task revision where already required
+```
+
+Do not add `dependencyGraphRevision` unless implementation proves that these bindings cannot safely
+identify dependency changes.
+
+Initial assembly, Plan amendment, membership evolution, and any other operation that changes
+dependencies must validate the complete candidate graph before writes.
+
+Current scope-change apply already has a Milestone transaction/recovery boundary. Current initial
+`setMilestonePlan` does not: it stages the Plan artifact, writes Milestone state, and publishes the
+artifact as separate operations. P04-A must move that initial transition onto the existing
+transaction/journal mechanism, or an equivalent atomic store primitive, before dependency data can
+become execution authority.
+
+Both initial Plan commit and later scope change commit, as one recoverable logical change:
+
+```text
+memberships and dependency arrays
+membershipRevision and Milestone revision
+Milestone Plan artifact and planHash
+execution-authorization supersession
+```
+
+A validation failure, injected write failure, process interruption, or recovery cannot leave
+membership state, dependency authority, Plan artifact, and Plan hash at different revisions. The
+initial empty Milestone posture must remain intact when initial Plan commit fails.
+
+### F. Legacy-state policy
+
+Historical stored memberships may lack `dependsOnTaskIds`.
+
+Compatibility rules:
+
+- accepted or cancelled terminal Milestones remain readable; their missing dependency data is
+  projected as legacy/unknown and is never reconstructed from ordinal;
+- terminal historical state does not become execution authority;
+- a nonterminal legacy Milestone with unstarted required Tasks cannot authorize, hand off, claim,
+  or start those Tasks until an explicit Milestone Plan amendment declares the complete dependency
+  arrays;
+- the amendment uses the existing scope-change, Plan-hash, revision, and reauthorization rules;
+- fresh Milestones and every newly written Plan require explicit arrays;
+- absence is never silently persisted again as canonical `[]`.
+
+The exact compatibility decoder may represent the legacy distinction internally without adding a
+new durable authority object.
+
+### G. Read-only diagnostics and context projection
+
+Dependency blocking must be observable. Repository `next`, Task-specific `next`, and `milestone
+progress` expose a read-only projection sufficient to report:
+
+```text
+taskId
+dependsOnTaskIds
+unsatisfiedPredecessorTaskIds
+predecessor Task status
+dependency-blocked reason
+```
+
+When no Task is runnable because of dependencies, navigation returns the exact blocking
+predecessors rather than an unexplained absence of work or an unrelated Task.
+
+P04 owns the canonical edges and status projection. P03, if later approved, may include only the
+relevant projection in role context; it cannot recompute or edit the graph.
+
+### H. Temporary fail-closed replacement boundary
+
+P04-A does not implement structural replacement, but it must define how the existing 1→1
+`replacement-materialize` transition behaves after dependencies become canonical.
+
+Before any replacement writes, Core may compute incident edges for diagnostics:
+
+```text
+incoming prerequisites = oldTask.dependsOnTaskIds
+downstream consumers = memberships whose dependsOnTaskIds contain oldTaskId
+```
+
+Temporary rule:
+
+- every `replacement-materialize` attempt is rejected before mutation with an explicit
+  `structural-replacement-required` diagnostic, including legacy/unknown dependency state and a
+  canonical empty edge set;
+- no dependency is copied, removed, or rewired automatically;
+- no new lifecycle stage or replacement credential is introduced;
+- rejection leaves Discovery, old Task, replacement Task, Milestone, Knowledge Map, lock state,
+  revisions, Plan artifact, and authorization unchanged.
+
+An isolated historical Task is not an exception. `split-required` proves that its boundary changes;
+an empty historical edge set does not prove that the replacement has no prerequisites or consumers.
+Allowing the new membership to inherit `[]` would silently make a new semantic claim. This is
+intentionally a safe stop until a later structural-replacement brief supplies a complete
+decomposition-authorized candidate membership and topology.
+
+### I. Validation
+
+#### I.1 Contract tests
+
+- missing dependency array on a new Plan is rejected;
+- explicit empty array is accepted as known independent;
+- missing target and dependency outside the Milestone are rejected;
+- duplicate, self, and cyclic edges cannot commit;
+- dependency-array input order is normalized deterministically after duplicate rejection;
+- ordinal without an edge creates no dependency;
+- required consumer depending on waived/cancelled provider fails closed;
+- Step dependencies remain unchanged.
+
+#### I.2 Guard tests
+
+- unsatisfied predecessor blocks repository navigation and direct start;
+- Task-specific navigation cannot select a dependency-blocked Task;
+- handoff preparation is blocked until predecessors merge;
+- all predecessors merged permits routing and start;
+- authorization alone does not bypass the predicate;
+- dependency change invalidates stale routing/context and supersedes authorization as applicable;
+- dependency change makes an older prepared handoff unclaimable;
+- direct claim of a dependency-blocked Task is rejected without acquiring a writer lease;
+- two independent Tasks are both runnable before scheduler tie-break.
+
+#### I.3 Legacy tests
+
+- terminal legacy Milestone without arrays remains readable and non-executable;
+- ordinal is not used to reconstruct legacy edges;
+- active legacy Milestone with unstarted Tasks fails closed;
+- explicit Plan amendment produces canonical arrays and requires fresh authorization;
+- newly written state never omits the field.
+
+#### I.4 Replacement-boundary tests
+
+- an isolated historical Task still blocks replacement before writes;
+- an incoming predecessor reports the incident edge and blocks replacement before writes;
+- a downstream consumer reports the incident edge and blocks replacement before writes;
+- combined incoming and downstream edges remain unchanged after rejection;
+- no Task, Discovery, Milestone, Knowledge, lock, artifact, sidecar, or authorization partial write
+  survives rejection or injected failure.
+
+#### I.5 Fresh synthetic E2E
+
+Use a new Milestone assembled from a clean repository and fresh Discovery:
+
+```text
+Contracts
+-> Exact root
+-> Deterministic formalization
+-> Extraction adapter
+-> Evidence ledger
+-> Delivery runtime
+-> Acceptance host
+```
+
+Expected:
+
+- only Tasks with satisfied explicit predecessors are initially runnable;
+- completion exposes exactly the next semantically declared consumer;
+- a direct start of a blocked Task is rejected;
+- no edge or result depends on Task IDs from a historical Milestone;
+- repeated replay produces the same graph, diagnostics, and runnable set.
+
+This graph is synthetic validation evidence, not a claim that production Signal semantics or
+deployment topology are decided.
+
+#### I.6 Independent and fan-in controls
+
+For:
+
+```text
+A -> C
+B -> C
+```
+
+A and B are independently runnable; C waits for both. Sequential scheduling remains permitted but
+does not add A→B or B→A.
+
+#### I.7 Property and transaction tests
+
+For generated Milestone graphs verify:
+
+- no cycle or dangling edge can commit;
+- every runnable Task has only satisfied predecessors;
+- adding an unrelated completed Task does not change another branch's runnable state;
+- ordinal permutations do not change dependency permission;
+- the same normalized Plan with different dependency-array input order yields the same Plan hash
+  and runnable set;
+- rejected and fault-injected scope-change transactions preserve the complete previous graph and
+  authority;
+- rejected, interrupted, and recovered initial Plan commits preserve either the complete initial
+  empty posture or the complete new Plan, never a state/artifact split.
+
+#### I.8 Successful-Milestone controls
+
+Replay completed non-replacement Milestones:
+
+- Tasks with explicit empty arrays remain runnable under existing policy;
+- valid sequential dependencies can be represented explicitly;
+- no new user-visible lifecycle stage is introduced;
+- ambiguous historical order is reported as unknown rather than reconstructed.
+
+### J. Deferred structural replacement integrity
+
+Not part of P04-A:
+
+- 1→N split materialization;
+- requirement and acceptance redistribution;
+- dependency rewiring from a superseded Task to replacements;
+- downstream provider rebinding and fan-in evolution;
+- transfer or adoption of completed Step commits;
+- replacement topology inference;
+- crash testing of a multi-Task graph rewrite.
+
+Confirmed residual facts:
+
+- current 1→1 replacement is already atomic and should be extended rather than replaced;
+- automatic transfer of completed Step commits is not currently defined;
+- safe future adoption requires exact commit, Step-definition, postcondition, base, and verification
+  evidence rather than silently copying completed state;
+- replacement topology must come from decomposition authority;
+- after P04-A, every replacement safely stops until this authority exists.
+
+A separate P05 is preferred because structural replacement also owns scope redistribution and
+completed-work adoption, not only graph mutation. If kept under P04, it must be a separately
+materialized P04-B with independent acceptance criteria and replay.
+
+## 4. Dependencies and conflicts
+
+- P04-A can be implemented before P01 using explicit mappings from existing Discovery/Milestone
+  planning authority.
+- Deferred P01-B, if approved, owns semantic prerequisite, output, ownership, and
+  ambiguous-provider admission. P01-A does not. Neither owns concrete graph storage or runnable
+  computation.
+- P02 owns Step dependencies independently.
+- P03 may project P04 facts only after P04 authority exists.
+- Current 1→1 replacement is blocked under the temporary rule in section H, including when the
+  historical Task has no incident edges.
+- P05 or P04-B owns dependency-bearing replacement, 1→N topology, residual scope, and completed-work
+  adoption.
+- Existing `membershipRevision`, Milestone revision, and Plan hash are preferred over a new graph
+  revision.
+
+## 5. Design approval gate
+
+P04-A is the recommended first implementation brief. It may move from `hypothesis` to `approved`
+only when a design review confirms that:
+
+1. `dependsOnTaskIds[]` has one canonical direction and duplicate edges are rejected.
+2. The predecessor predicate is shared by navigation, handoff preparation, claim, writer-lease
+   acquisition, and start.
+3. Handoff freshness binds to existing Milestone revision, membership revision, and Plan hash.
+4. Initial `setMilestonePlan` and later scope changes have one explicit atomic/recovery strategy.
+5. All replacement attempts fail before writes until P05/P04-B supplies complete topology
+   authority.
+6. Legacy absence remains distinguishable from canonical `[]`.
+7. The declared tests cover failure, recovery, independent Tasks, fan-in, and successful controls.
+
+No implementation result is required for this design approval; the tests below are the
+`approved -> implemented` gate.
+
+## 6. Implementation acceptance
+
+P04-A may move from `approved` to `implemented` only when:
+
+1. Every new canonical Milestone membership stores an explicit dependency array.
+2. Missing, external, duplicate, self, and cyclic edges cannot commit.
+3. Runnable computation uses dependencies rather than ordinal.
+4. Navigation, handoff preparation, `task claim`, writer-lease acquisition, and direct `task start`
+   enforce the same predicate.
+5. Independent Tasks receive no artificial edges.
+6. Existing decomposition authority can supply edges without requiring P01 implementation.
+7. Ambiguous provider selection returns to decomposition authority rather than Scheduler inference.
+8. Existing Milestone revisions and hashes invalidate stale dependency views and authorization.
+9. Initial Plan commit and later scope changes commit graph and Milestone authority atomically and
+   recover without state/artifact divergence.
+10. Legacy terminal state remains readable but cannot become execution authority.
+11. Active legacy state fails closed until an explicit dependency Plan is approved.
+12. `next` and `milestone progress` identify exact unsatisfied predecessors.
+13. Every replacement attempt fails before writes, including for an isolated historical Task.
+14. P01 semantic facts are not duplicated by a Contract Registry.
+15. P02 Step dependencies remain unchanged.
+16. Fresh synthetic replay is deterministic.
+17. Successful non-replacement Milestones receive no systematic false blockers.
+18. No new user-visible lifecycle stage, dependency credential, parallel scheduler, or parent/child
+    Task lifecycle is introduced.
+19. Multi-Task replacement, rewiring, scope redistribution, and completed-Step adoption remain out
+    of P04-A.
+
+## 7. Non-goals
+
+- structural replacement or completed-Step adoption;
+- automatic dependency inference from text, filenames, titles, or ordinal;
+- semantic prerequisite/output registry;
+- Step reslicing;
+- parallel execution;
+- parent/child Tasks;
+- Context Graph;
+- replacement-specific lifecycle or credential;
+- reconstruction of historical dependencies.
+
+## 8. Decision log
+
+- 2026-08-23 — Revision 1 created as `hypothesis` combining Task dependency and structural
+  replacement concerns.
+- 2026-08-24 — Source inspection confirmed missing normal Task dependency authority.
+- 2026-08-24 — Existing atomic 1→1 replacement recognized; the original claim that replacement is
+  generally non-atomic was rejected.
+- 2026-08-24 — Revision 2 limited implementation to normal Task dependency and runnable semantics;
+  1→N replacement, completed-Step adoption, and downstream rewiring were deferred.
+- 2026-08-24 — Mandatory old-M3 topology removed; fresh synthetic Task graph made the approval
+  replay.
+- 2026-08-24 — Existing Milestone revision, membership revision, and Plan hash preferred over a new
+  dependency graph authority.
+- 2026-08-24 — Revision 3 structured P04 as independently materializable P04-A, removed the hard
+  dependency on P01, added legacy-state policy, atomic binding, read-only diagnostics, and a
+  fail-closed boundary for dependency-bearing 1→1 replacement.
+- 2026-08-24 — Revision 4 rejected duplicate edges, added an exact claim/lease freshness guard,
+  required initial Plan commit to join the transaction/recovery boundary, and blocked even isolated
+  replacement because an old empty edge set cannot authorize new topology.
+- 2026-08-24 — P04-A designated the recommended first implementation brief; design approval and
+  implementation acceptance were separated.
+- 2026-08-24 — Design review accepted: existing Milestone transaction machinery is the required
+  initial-Plan strategy, and the temporary fail-closed replacement boundary is accepted.
+- 2026-08-24 — Status moved from `hypothesis` to `approved`; implementation has not started at this
+  decision boundary.
+- 2026-08-24 — Revision 5 implementation candidate completed on
+  `codex/p04-task-dependencies`: canonical dependency arrays and DAG validation, shared runnable
+  guards, dependency-bound C1 handoff/claim/start, two-target initial Plan transaction/recovery,
+  legacy read/fail-closed policy, diagnostics, and unconditional pre-write structural-replacement
+  shutdown are present. Typecheck, plugin/schema validation, focused fault/replay tests, and the
+  full 306/306 regression suite passed. Status remains `approved` until the candidate is
+  merged/installed and accepted under the registry's implementation gate.
+- 2026-08-24 — The exact final combined P04-A/P01-A suite passed 319/319. The packaged handshake now
+  advertises dependency DAG, atomic initial Plan, and structural-replacement-disabled capabilities
+  instead of the unavailable legacy replacement capability. Status remains `approved`.

package/docs/problem-briefs/BRIEF-TEMPLATE.md

@@ -0,0 +1,56 @@
+---
+brief_id: WF2-PXX
+problem: "TBD"
+status: hypothesis
+owner: null
+created_at: 2026-08-23
+approved_at: null
+implemented_at: null
+revision: 1
+---
+
+# Brief — TBD
+
+## 1. What we solve
+
+TBD
+
+## 2. How we solve it
+
+TBD
+
+## 3. Why this approach
+
+TBD
+
+## 4. Validation plan
+
+TBD
+
+## Design approval gate
+
+Define the problem/root-cause evidence, closed design decisions, dependency/conflict resolution, and
+pre-implementation replay required to move this brief from `hypothesis` to `approved`. Do not make
+tests that require the proposed code a circular prerequisite for design approval.
+
+## Implementation acceptance
+
+Define the contract, E2E, regression, compatibility, and fault-injection evidence required to move
+the approved brief to `implemented` after the code is merged/installed.
+
+## Dependencies and conflicts
+
+TBD
+
+## Recommended go/no-go
+
+Record `GO`, conditional `GO`, or `NO-GO`, the exact unresolved gate, and the required implementation
+order. This recommendation does not change frontmatter status.
+
+## Non-goals
+
+TBD
+
+## Decision log
+
+- 2026-08-23 — Brief created with status `hypothesis`.