dsh-compound-engineering 0.0.0-stage → 0.1.0

package/skills/ce-optimize/references/optimize-spec-schema.yaml

@@ -0,0 +1,570 @@
+# Optimization Spec Schema
+# This is the canonical schema for optimization spec files created by users
+# to configure a /ce-optimize run. The orchestrating agent validates specs
+# against this schema before proceeding.
+#
+# Usage: Create a YAML file matching this schema and pass it to /ce-optimize.
+# The agent reads this spec, validates required fields, and uses it to
+# configure the entire optimization run.
+
+# ============================================================================
+# REQUIRED FIELDS
+# ============================================================================
+
+required_fields:
+
+  name:
+    type: string
+    pattern: "^[a-z0-9]+(?:-[a-z0-9]+)*$"
+    description: "Unique identifier for this optimization run (lowercase kebab-case, safe for git refs and worktree paths)"
+    example: "improve-issue-clustering"
+
+  description:
+    type: string
+    description: "Human-readable description of the optimization goal"
+    example: "Improve coherence and coverage of issue/PR clusters"
+
+  metric:
+    type: object
+    description: "Three-tier metric configuration"
+    required_children:
+
+      primary:
+        type: object
+        description: "The metric the loop optimizes against"
+        required_children:
+
+          type:
+            type: enum
+            values:
+              - hard    # scalar metric from measurement command (e.g., build time, test pass rate)
+              - judge   # LLM-as-judge quality score from sampled outputs
+            description: "Whether the primary metric comes from the measurement command directly or from LLM-as-judge evaluation"
+
+          name:
+            type: string
+            description: "Metric name: must match a key in the measurement command's JSON output (for hard type) or a scoring field (for judge type)"
+            example: "cluster_coherence"
+
+          direction:
+            type: enum
+            values:
+              - maximize
+              - minimize
+            description: "Whether higher or lower is better"
+
+        optional_children:
+
+          baseline:
+            type: number
+            default: null
+            description: "Filled automatically during Phase 1 baseline measurement. Do not set manually."
+
+          target:
+            type: number
+            default: null
+            description: "Optional target value. Loop stops when this is reached. When metric.objectives is set, target_reached instead requires every required objective that declares a target."
+            example: 4.2
+
+      degenerate_gates:
+        type: array
+        description: "Fast boolean checks that reject obviously broken solutions before expensive evaluation. Run first, before the primary metric or judge."
+        required: true
+        items:
+          type: object
+          required_children:
+            name:
+              type: string
+              description: "Metric name: must match a key in the measurement command's JSON output"
+            check:
+              type: string
+              description: "Comparison operator and threshold. Supported operators: >=, <=, >, <, ==, !="
+              example: "<= 0.10"
+          optional_children:
+            description:
+              type: string
+              description: "Human-readable explanation of what this gate catches"
+
+    optional_children:
+
+      objectives:
+        type: array
+        default: []
+        description: >
+          Additional required or secondary hard targets. The primary is always
+          a required comparison, even when this list omits it. An experiment is
+          eligible when it improves at least one required objective beyond the
+          comparison threshold and does not violate any other required
+          objective. Completion via target_reached requires every declared
+          required target. A spec without metric.objectives keeps single-primary
+          acceptance: the primary is the only required objective.
+        items:
+          type: object
+          required_children:
+            name:
+              type: string
+              description: "Metric name: must match a key in the measurement command's JSON output"
+            direction:
+              type: enum
+              values:
+                - maximize
+                - minimize
+          optional_children:
+            role:
+              type: enum
+              values:
+                - required
+                - secondary
+              default: required
+              description: "required objectives participate in eligibility and completion; secondary objectives are logged only"
+            type:
+              type: enum
+              values:
+                - hard
+              default: hard
+              description: "Always hard. Additional objectives come from the measurement command."
+            target:
+              type: number
+              default: null
+              description: "Declared success value for this objective. target_reached requires every required objective that sets this."
+            max_regression:
+              type: object
+              description: "Optional explicit violation bound. When set, violation uses this bound rather than the comparison threshold, including when the bound is looser. When unset, a comparison-threshold regression is a violation."
+              required_children:
+                type:
+                  type: enum
+                  values:
+                    - relative
+                    - absolute
+                value:
+                  type: number
+
+      diagnostics:
+        type: array
+        default: []
+        description: "Metrics logged for understanding but never gated on. Useful for understanding WHY a primary metric changed."
+        items:
+          type: object
+          required_children:
+            name:
+              type: string
+              description: "Metric name: must match a key in the measurement command's JSON output"
+
+      judge:
+        type: object
+        description: "LLM-as-judge configuration. Required when metric.primary.type is 'judge'. Ignored when type is 'hard'."
+        required_when: "metric.primary.type == 'judge'"
+        required_children:
+          rubric:
+            type: string
+            description: "Multi-line rubric text sent to the judge model. Must instruct the judge to return JSON."
+            example: |
+              Rate this cluster 1-5:
+              - 5: All items clearly about the same issue/feature
+              - 4: Strong theme, minor outliers
+              - 3: Related but covers 2-3 sub-topics
+              - 2: Weak connection
+              - 1: Unrelated items grouped together
+          scoring:
+            type: object
+            required_children:
+              primary:
+                type: string
+                description: "Field name from judge JSON output to use as the primary optimization target"
+                example: "mean_score"
+            optional_children:
+              secondary:
+                type: array
+                default: []
+                description: "Additional scoring fields to log (not optimized against)"
+        optional_children:
+          model:
+            type: enum
+            values:
+              - cheap    # the host's fastest, lowest-cost tier
+              - strong   # the host's most capable tier
+            default: cheap
+            description: "Capability tier for the judge. The active harness resolves the tier to a concrete model; the spec never names one. Legacy values haiku and sonnet read as cheap and strong."
+          confirmation_seed:
+            type: integer
+            default: null
+            description: "Seed for a second, held-out judge sample scored only before a keep and at final confirmation. Must differ from sample_seed. A different seed alone does not make the sample held out: draw it only from items of that same output the selection sample did not draw, so the two share no item. An output with no item left outside the selection sample cannot be confirmed this way, so its candidate is not kept; a spec whose outputs are that small needs measurement.holdout.command instead. This is the judge-type holdout when measurement.holdout is unset; see the holdout validation rules."
+          calibration:
+            type: object
+            default: null
+            description: "Optional evidence that the judge scores the way a person would, checked in Phase 1 when labels are set. Without it the approval gate states that the judge is unchecked. Required (labels or an explicit waiver) only when the run waits between ticks through a wake after turn end."
+            optional_children:
+              labels:
+                type: string
+                default: null
+                description: "Path to a human-labeled sample: items with the user's rubric score and, when given, the reason. Phase 1 writes it for the user during the guided check (default <state-root>/judge-labels.yaml); a sample kept in the repo is reusable across runs. Around ten items shows a pattern; 20-50 makes the agreement figure dependable."
+              min_agreement:
+                type: number
+                default: 0.8
+                description: "Fraction of labeled items on which the judge's score must agree with the human label for the harness to be accepted."
+              waived:
+                type: boolean
+                default: false
+                description: "True only when the user explicitly waived calibration for a run that would otherwise require it. Record who waived and when in the experiment log's harness_validation."
+          sample_size:
+            type: integer
+            default: 10
+            description: "Total number of output items to sample for judge evaluation per experiment"
+          stratification:
+            type: array
+            default: null
+            description: "Stratified sampling buckets. If null, uses uniform random sampling."
+            items:
+              type: object
+              required_children:
+                bucket:
+                  type: string
+                  description: "Bucket name for this stratum"
+                count:
+                  type: integer
+                  description: "Number of items to sample from this bucket"
+          singleton_sample:
+            type: integer
+            default: 0
+            description: "Number of singleton items to sample for false-negative evaluation"
+          singleton_rubric:
+            type: string
+            default: null
+            description: "Rubric for evaluating sampled singletons. Required if singleton_sample > 0."
+          sample_seed:
+            type: integer
+            default: 42
+            description: "Fixed seed for reproducible sampling across experiments"
+          batch_size:
+            type: integer
+            default: 5
+            description: "Number of samples per judge sub-agent batch. Controls parallelism vs overhead."
+          minimum_improvement:
+            type: number
+            default: 0.3
+            description: "Minimum judge score improvement required to accept an experiment as 'better'. Accounts for sample-composition variance when output structure changes between experiments. Distinct from measurement.stability.noise_threshold which handles run-to-run flakiness."
+          max_total_cost_usd:
+            type: number
+            default: 5
+            description: "Do not start a judge pass whose estimated cost would take cumulative judge spend past this cap. Counts judge scoring only; stopping.max_total_cost_usd caps the whole run. This is a first-run safety default; raise it only after the rubric and harness are trustworthy. Set to null only with explicit user approval."
+
+  measurement:
+    type: object
+    description: "How to run the measurement harness"
+    required_children:
+      command:
+        type: string
+        description: "Shell command that runs the evaluation and outputs JSON to stdout. The JSON must contain keys matching all gate names and diagnostic names."
+        example: "python evaluate.py"
+    optional_children:
+      timeout_seconds:
+        type: integer
+        default: 600
+        description: "Maximum seconds for the measurement command to run before being killed"
+      output_format:
+        type: enum
+        values:
+          - json
+        default: json
+        description: "Format of the measurement command's stdout. Currently only JSON is supported."
+      working_directory:
+        type: string
+        default: "."
+        description: "Working directory for the measurement command, relative to the repo root"
+      holdout:
+        type: object
+        default: null
+        description: >
+          A held-out confirmation set the loop never selects or generates
+          hypotheses from. Scored only before a keep and at final
+          confirmation. When configured, a keep requires the holdout comparison
+          to agree with the selection comparison (decide.mjs takes the second
+          snapshot pair as `holdout` and returns `next_measurement: holdout`
+          when it is still missing). Required when metric.primary.type is
+          'judge' and when the run waits between ticks through a wake after
+          turn end; otherwise optional, and its absence is stated in the
+          Phase 1 approval presentation.
+        optional_children:
+          command:
+            type: string
+            default: null
+            description: "Shell command that runs the harness on the held-out set and prints the same JSON shape as measurement.command. For a judge primary, metric.judge.confirmation_seed may stand in for this command."
+      per_case:
+        type: boolean
+        default: false
+        description: "When true, the measurement JSON also carries a `cases` object mapping a stable case id to a boolean pass/fail (1/0 accepted). decide.mjs then reports `regressions`: cases the reference passed and the candidate fails. Report only; it does not change the decision."
+      stability:
+        type: object
+        default: { mode: "stable" }
+        description: "How to handle metric variance across runs"
+        required_children:
+          mode:
+            type: enum
+            values:
+              - stable   # run once, trust the result
+              - repeat   # run N times, aggregate
+              - ladder   # smoke, one paired exploratory sample, adaptive extras, full protocol only before keep
+            default: stable
+        optional_children:
+          repeat_count:
+            type: integer
+            default: 5
+            description: "Number of times to run the harness when mode is 'repeat', and the confirmation sample count when mode is 'ladder'"
+          aggregation:
+            type: enum
+            values:
+              - median
+              - mean
+              - min
+              - max
+            default: median
+            description: "How to combine repeated measurements into a single value"
+          noise_threshold:
+            type: number
+            default: 0.02
+            description: "Absolute improvement that must be exceeded when comparison.method is absolute (the default). Applied to hard metrics only."
+          comparison:
+            type: object
+            default: { method: "absolute" }
+            description: "How to decide keep / revert / inconclusive. Unset method preserves the legacy absolute noise_threshold rule."
+            required_children:
+              method:
+                type: enum
+                values:
+                  - absolute
+                  - relative
+                  - paired
+                default: absolute
+                description: "absolute uses noise_threshold; relative uses relative_threshold; paired uses sample ranges / paired diffs and may return inconclusive"
+            optional_children:
+              relative_threshold:
+                type: number
+                default: 0.05
+                description: "Fraction of the current-best value that counts as a real change when method is relative or paired"
+          ladder:
+            type: object
+            description: "Cost-aware measurement stages. Used when mode is ladder; ignored otherwise."
+            optional_children:
+              smoke_command:
+                type: string
+                default: null
+                description: "Cheap correctness check. Failure is degenerate and skips timed measurement."
+              exploratory_pairs:
+                type: integer
+                default: 1
+                description: "Paired exploratory samples before deciding whether to spend more. Must be a positive integer."
+              confirmation_repeats:
+                type: integer
+                default: null
+                description: "Full protocol sample count before keep. Defaults to repeat_count. Must be a positive integer at least as large as exploratory_pairs."
+              futility:
+                type: object
+                description: "Predeclared abort bound for a clearly noncompetitive live run"
+                optional_children:
+                  worse_factor:
+                    type: number
+                    default: 1.2
+                    description: "Censor after the first sample if the candidate is this multiple worse than the current best (minimize: candidate >= best * factor)"
+                  after_elapsed_seconds:
+                    type: number
+                    default: null
+                    description: "Censor a still-noncompetitive live run once elapsed time reaches this bound. When set, must be a positive number."
+
+  scope:
+    type: object
+    description: "What the experiment agent is allowed to modify"
+    required_children:
+      mutable:
+        type: array
+        description: "Files and directories the agent MAY modify during experiments"
+        items:
+          type: string
+          description: "File path or directory (relative to repo root). Directories match all files within."
+        example:
+          - "src/clustering/"
+          - "src/preprocessing/"
+          - "config/clustering.yaml"
+      immutable:
+        type: array
+        description: "Files and directories the agent MUST NOT modify. The measurement harness should always be listed here."
+        items:
+          type: string
+        example:
+          - "evaluate.py"
+          - "tests/fixtures/"
+          - "data/"
+
+# ============================================================================
+# OPTIONAL FIELDS
+# ============================================================================
+
+optional_fields:
+
+  execution:
+    type: object
+    default: { mode: "parallel", backend: "worktree", max_concurrent: 4 }
+    description: "How experiments are executed"
+    optional_children:
+      mode:
+        type: enum
+        values:
+          - parallel  # run experiments simultaneously (default)
+          - serial    # run one at a time
+        default: parallel
+      backend:
+        type: enum
+        values:
+          - worktree  # git worktrees for isolation (default)
+          - codex     # Codex sandboxes for isolation
+          - remote    # detached workers with their own checkout; results arrive as a pushed ref or store file
+        default: worktree
+        description: "Where experiments run. remote needs a detached-worker capability (SKILL.md Execution Surface); without one the run falls back to worktree. remote workers measure paired on their own machine, so it requires a relative or paired comparison."
+      max_concurrent:
+        type: integer
+        default: 4
+        minimum: 1
+        description: "Maximum experiments to run in parallel. Capped at 6 for worktree backend. 8+ only valid for Codex or remote backends. For remote this caps dispatched workers, not local worktrees."
+      max_experiments_per_tick:
+        type: integer
+        default: null
+        minimum: 1
+        description: "Optional cap on experiments dispatched in one tick (one batch) of the loop. Bounds how much a single wake or session does before it checkpoints. Unset means max_concurrent alone bounds the batch."
+      codex_security:
+        type: enum
+        values:
+          - full-auto                                # --full-auto (workspace write)
+          - yolo                                     # --dangerously-bypass-approvals-and-sandbox
+        default: null
+        description: "Codex security posture. If null, user is asked once per session."
+
+  parallel:
+    type: object
+    default: {}
+    description: "Parallelism configuration discovered or set during Phase 1"
+    optional_children:
+      port_strategy:
+        type: enum
+        values:
+          - parameterized  # use env var for port
+          - none           # no port parameterization needed
+        default: null
+        description: "If null, auto-detected during Phase 1 parallelism probe"
+      port_env_var:
+        type: string
+        default: null
+        description: "Environment variable name for port parameterization (e.g., EVAL_PORT)"
+      port_base:
+        type: integer
+        default: null
+        description: "Base port number. Each experiment gets port_base + experiment_index."
+      shared_files:
+        type: array
+        default: []
+        description: "Files that must be copied into each experiment worktree (e.g., SQLite databases)"
+        items:
+          type: string
+      exclusive_resources:
+        type: array
+        default: []
+        description: "Resources requiring exclusive access (e.g., 'gpu'). If non-empty, forces serial mode."
+        items:
+          type: string
+
+  dependencies:
+    type: object
+    default: { approved: [] }
+    description: "Dependency management for experiments"
+    optional_children:
+      approved:
+        type: array
+        default: []
+        description: "Pre-approved new dependencies that experiments may add"
+        items:
+          type: string
+
+  constraints:
+    type: array
+    default: []
+    description: "Free-text constraints that experiment agents must follow"
+    items:
+      type: string
+    example:
+      - "Do not change the output format of clusters"
+      - "Preserve backward compatibility with existing cluster consumers"
+
+  stopping:
+    type: object
+    default: { max_iterations: 100, max_hours: 8, max_wall_hours: 72, plateau_iterations: 10, target_reached: true }
+    description: "When the optimization loop should stop. Any criterion can trigger a stop."
+    optional_children:
+      max_iterations:
+        type: integer
+        default: 100
+        description: "Stop after this many total experiments"
+      max_hours:
+        type: number
+        default: 8
+        description: "Stop after this many active hours: time spent inside loop ticks since Phase 3 started, summed across ticks. Time spent waiting between ticks does not count. In one uninterrupted session this equals wall-clock time."
+      max_wall_hours:
+        type: number
+        default: 72
+        description: "Calendar backstop: stop once this many hours have passed since Phase 3 started, whether or not the run was active. No wake or resume extends it. Must be >= max_hours."
+      plateau_iterations:
+        type: integer
+        default: 10
+        description: "Stop if no improvement for this many consecutive experiments"
+      target_reached:
+        type: boolean
+        default: true
+        description: "Stop when the primary metric reaches the target value (if set)"
+      max_total_cost_usd:
+        type: number
+        default: null
+        description: "Do not start a dispatch whose estimated cost would take model spend for the whole run (experiment workers, judges, and research subagents) past this cap. Counted from host-reported usage where available and estimated otherwise."
+
+  max_runner_up_merges_per_batch:
+    type: integer
+    default: 1
+    description: "Maximum number of file-disjoint runner-up experiments to attempt merging per batch after keeping the best experiment"
+
+# ============================================================================
+# VALIDATION RULES
+# ============================================================================
+
+validation_rules:
+  - "All required fields must be present"
+  - "name must be lowercase kebab-case (`^[a-z0-9]+(?:-[a-z0-9]+)*$`)"
+  - "metric.primary.type must be 'hard' or 'judge'"
+  - "If metric.primary.type is 'judge', metric.judge must be present with rubric and scoring"
+  - "metric.degenerate_gates must have at least one entry"
+  - "measurement.command must be a non-empty string"
+  - "scope.mutable must have at least one entry"
+  - "scope.immutable must have at least one entry"
+  - "Gate check operators must be one of: >=, <=, >, <, ==, !="
+  - "execution.max_concurrent must be >= 1"
+  - "execution.max_concurrent must not exceed 6 when execution.backend is 'worktree'"
+  - "execution.backend must be one of: worktree, codex, remote"
+  - "If execution.backend is 'remote', measurement.stability.comparison.method must be 'paired' or 'relative'; absolute numbers from different machines are not comparable"
+  - "If parallel.exclusive_resources is non-empty, execution.mode should be 'serial'"
+  - "If metric.judge.singleton_sample > 0, metric.judge.singleton_rubric must be present"
+  - "If metric.primary.type is 'judge' and metric.judge.max_total_cost_usd is null, the user should explicitly approve uncapped spend"
+  - "stopping must have at least one non-default criterion or use defaults"
+  - "stopping.max_total_cost_usd, when set, must be a positive number"
+  - "If metric.objectives is set, each additional objective is a hard metric whose name is unique and matches a measurement JSON key"
+  - "The primary is always a required comparison, including when metric.objectives omits it"
+  - "If measurement.stability.mode is 'ladder', comparison.method should be relative or paired unless the user explicitly keeps absolute"
+  - "A spec without metric.objectives keeps single-primary acceptance"
+  - "comparison.method, when set, must be one of: absolute, relative, paired"
+  - "If measurement.stability.mode is 'ladder', exploratory_pairs and confirmation_repeats (or repeat_count) must be positive integers, and confirmation must be at least exploratory_pairs"
+  - "futility.after_elapsed_seconds, when set, must be a positive number"
+  - "stopping.max_wall_hours must be a positive number and at least stopping.max_hours"
+  - "execution.max_experiments_per_tick, when set, must be a positive integer"
+  - "If the run waits between ticks through a wake after turn end and metric.primary.type is 'judge', metric.judge.max_total_cost_usd must be set; unattended spend is never uncapped"
+  - "metric.judge.model, when set, must be cheap or strong (legacy haiku and sonnet read as cheap and strong)"
+  - "A holdout is configured when measurement.holdout.command is set, or when metric.primary.type is 'judge' and metric.judge.confirmation_seed is set and differs from sample_seed"
+  - "If metric.primary.type is 'judge', a holdout must be configured"
+  - "If the run waits between ticks through a wake after turn end, a holdout must be configured; an unattended keep is never confirmed on the selection sample alone"
+  - "When a holdout is configured, a keep requires the holdout comparison to agree with the selection comparison; the holdout is never used to select or generate hypotheses"
+  - "When metric.judge.calibration.labels is set, the harness is accepted only when the judge meets min_agreement against them"
+  - "If metric.primary.type is 'judge' and the run waits between ticks through a wake after turn end, calibration labels must meet min_agreement or metric.judge.calibration.waived must be true with the user's explicit waiver; an attended judge run may proceed uncalibrated, stated at approval"
+  - "If measurement.per_case is true, the measurement JSON must carry a `cases` object"

package/skills/ce-optimize/references/persistence.md

@@ -0,0 +1,93 @@
+# Persistence: the rules, the checkpoints, and resume
+
+Read this before Phase 0 and follow it for the whole run. The body states the invariant and names the six checkpoints; this file carries the rules that implement them, the state root, the checkpoint table, the file layout, the wait record, and the resume procedure.
+
+### Core Rules
+
+1. **Write each experiment result to disk IMMEDIATELY after measurement.** Not after the batch, not after evaluation, IMMEDIATELY. Append the experiment entry to the experiment log file the moment its metrics are known, before evaluating the next experiment. This is the #1 crash-safety rule.
+
+2. **VERIFY every critical write.** After writing the experiment log, read the file back and confirm the entry is present. This catches silent write failures. Do not proceed to the next experiment until verification passes.
+
+3. **Re-read from disk at every phase boundary and before every decision.** Never trust in-memory state across phase transitions, batch boundaries, or after any operation that might have taken significant time. Re-read the experiment log and strategy digest from disk.
+
+4. **One experiment, one log entry.** Append a new experiment entry on its first measurement. Later ladder samples for that same experiment update that entry's metrics and outcome in place so a crash can resume the ladder without losing samples or duplicating the hypothesis. Distinct `comparisons` pairings accumulate on that same entry; in-place updates must not replace a previously persisted distinct pairing. Never rewrite a different experiment's samples or gate values. Outcome, `best`, and `hypothesis_backlog` are also updated in place at batch evaluation (CP-4). Do not rebuild the file from memory.
+
+5. **Per-experiment result markers for crash recovery.** Each experiment writes a `result.yaml` marker in its worktree immediately after measurement. On resume, scan for these markers to recover experiments that were measured but not yet logged.
+
+6. **Write the strategy digest after every batch, before generating new hypotheses.** The agent reads the digest (not its memory) when deciding what to try next.
+
+7. **Never present results to the user without writing them to disk first.** The order is: measure -> write to disk -> verify -> THEN show the user. Not the reverse.
+
+### The State Root
+
+`<state-root>` is the directory that holds the run's ledger. Resolve it once, in Phase 0, by the body's Execution Surface rule: when the harness names a durable state root, `<state-root>` is `<that location>/ce-optimize/<spec-name>/`, placed where that location's own conventions keep working state rather than user-facing documents; otherwise it is `.context/compound-engineering/ce-optimize/<spec-name>/` in the repo checkout, which is gitignored and survives a local resume only on this machine. A run's root is wherever its log already is: never move a ledger mid-run, and a resume that finds the log at one root uses that root even when the other is now available. Give every subagent and worker the resolved path, not the rule.
+
+Examples of what the three capabilities look like on some harnesses. This table is not a tool list; the body's rule decides.
+
+| Harness | Durable state root | Wake after turn end | Detached worker |
+|---|---|---|---|
+| A coordinator harness with a persistent agent store (for example Cursor Projects' Agent Store) | The store path named in your context | Event subscriptions or a timer subscription that re-invokes the agent | A cloud-agent launch that returns a receipt and lands its work as a pushed branch or a store file |
+| Grok (CLI/TUI) | None named; `.context/` | `scheduler_create --durable` | None; `worktree` |
+| Claude Code, Codex, Cursor CLI sessions | None named; `.context/` | Whatever scheduling or wake tool the session lists; when it lists none, cron running the resume invocation is the user's escalation | None; `worktree` (or `codex` when the spec says so) |
+
+When the harness shows the user a status surface for this run, write a one-line run status there at CP-4 and at each stop, in addition to the log.
+
+### Mandatory Disk Checkpoints
+
+These are non-negotiable write-then-verify steps. At each checkpoint, the agent MUST write the specified file and then read it back to confirm the write succeeded.
+
+| Checkpoint | File Written | Phase |
+|---|---|---|
+| CP-0: Spec saved | `spec.yaml` | Phase 0, after user approval |
+| CP-1: Baseline recorded | `experiment-log.yaml` (initial with baseline) | Phase 1, after baseline measurement |
+| Approval recorded | `experiment-log.yaml` (`approval` section) | Phase 1.7, the moment the user approves |
+| CP-2: Hypothesis backlog saved | `experiment-log.yaml` (hypothesis_backlog section) | Phase 2, after hypothesis generation |
+| CP-3: Each experiment result | `experiment-log.yaml` (append on first measurement; update that entry on later samples) | Phase 3.3, immediately after each measurement |
+| CP-4: Batch summary | `experiment-log.yaml` (outcomes + best + `run_state`) + `strategy-digest.md` | Phase 3.5, after batch evaluation |
+| CP-5: Final summary | `experiment-log.yaml` (final state) | Phase 4, at wrap-up |
+
+**Format of a verification step:**
+1. Write the file using the native file-write tool
+2. Read the file back using the native file-read tool
+3. Confirm the expected content is present
+4. If verification fails, retry the write. If it fails twice, alert the user.
+
+### File Locations (all under `<state-root>`)
+
+Anything the branch's readers need durably must be exported to a tracked path; the ledger does not travel with the branch from either root.
+
+| File | Purpose | Written When |
+|------|---------|-------------|
+| `spec.yaml` | Optimization spec (fixed once the Phase 1 approval gate is cleared) | Phase 0 (CP-0) |
+| `experiment-log.yaml` | Full history of all experiments, the approval record, and `run_state` | Initialized at CP-1, appended at first CP-3, updated on later samples and at CP-4 |
+| `strategy-digest.md` | Compressed learnings for hypothesis generation | Written at CP-4 after each batch |
+| `source-digest.md` | What experiment workers need to know about the target's material and current approach | Phase 2.1; rewritten only when a keep changes what it describes |
+| `judge-cache.yaml` | Per-run judge results keyed by item content hash, so identical output is scored once (`references/experiment-log-schema.yaml`, JUDGE CACHE) | Phase 3.3, as each judge batch lands |
+| `<worktree>/result.yaml` | Per-experiment crash-recovery marker | Immediately after measurement, before CP-3 |
+
+### The Approval Record
+
+The Phase 1 approval is a user decision the log records so that a resume, including an unattended wake, does not re-ask it. Write `approval` the moment the user approves, before Phase 2 starts, with the fields the log schema names: the time, the SHA-256 of the saved `spec.yaml` bytes, and the caps in force (`stopping.*`, `metric.judge.max_total_cost_usd`, `execution.max_concurrent`). The record is valid while the spec digest and every recorded cap match the spec on disk. A record that is absent or no longer matches means the Phase 1 gate is presented again; the answer is a new record.
+
+The baseline and every logged experiment were measured under the spec that was approved, so what changed decides what still stands. When the change leaves how those measurements were produced untouched (a cap, or a holdout or calibration added to a spec that lacked one), the measurements stand and approving the new spec continues the run. When anything that produced them differs (the metric, the measurement command, the scope, the comparison), the measurements no longer describe this spec: while the log holds no hypothesis backlog and no experiments, go back through Phase 1 so the baseline matches the new spec; once it holds either, the spec is fixed for the run, so restore the approved spec or take Fresh start (Phase 0.4). Say which of the two applies when presenting the gate.
+
+### The Wait Record and Ticks
+
+Phase 3 runs as ticks (`references/loop.md`). Before ending a turn with work outstanding, verify the ledger with `run_state.status: waiting` and each pending receipt's result location and on-arrival action in `run_state.pending_waits`.
+
+Automatic resumption requires registered wakes for pending results and a timer wake at the wall deadline, even if no worker ever returns. Record the deadline timer in `pending_waits` too. The deadline is `phase3_started_at` plus `stopping.max_wall_hours`; a resume never resets it. If either result delivery or the deadline wake cannot be secured, continue an attached wait only while this session can enforce that deadline. Otherwise hand back the verified checkpoint, tell the user monitoring is paused, and print the resume invocation. Record unavailable wakes as such; do not claim automatic monitoring. Never fake a wait with a foreground sleep or an unmanaged detached process.
+
+Write and verify `run_state` whenever a wait is registered or cleared and at CP-4. It also carries `active_seconds` (time spent inside ticks), `phase3_started_at` (the calendar anchor), the completed `tick` count, and `status` (`running`, `waiting`, `blocked`, `final`). Use `blocked` when progress needs an external change other than an outstanding result or a manual resume; report the blocker instead of spinning.
+
+**User-runnable resume syntax.** When this reference tells you to print or copy a resume invocation, default to `/ce-optimize <state-root>/spec.yaml`. Use `$ce-optimize <state-root>/spec.yaml` only when the active harness is Codex or explicitly documents dollar-prefixed skill invocation. Render only the invocation as inline code and output one form only.
+
+### On Resume
+
+A wake, a scheduler fire, or a manual re-run re-enters here. When Phase 0.4 detects an existing run:
+1. Read the experiment log from `<state-root>`. It is the ground truth. Check the wall deadline before resuming work or re-registering a wait. If it has passed, persist any already-arrived results and go to Phase 4 without waiting for missing results or integrating candidates
+2. Scan worktree directories for `result.yaml` markers not yet in the log
+3. Recover any measured-but-unlogged experiments. The recovered first CP-3 entry copies `opportunity` from the hypothesis backlog as of dispatch; `result.yaml` holds metrics only, so a missing forecast stays unrecorded rather than being reconstructed from the result
+4. Collect every `run_state.pending_waits` item whose result has arrived, by the shape recorded for it, and persist each at CP-3. An item whose result has not arrived stays pending; an item whose wake is gone follows The Wait Record and Ticks above, including its manual-resume fallback
+5. Continue as the SKILL.md body's resume rule directs. Skip the work the log proves finished, and re-enter any gate whose record is absent or no longer matches
+
+---