ace-sim 0.13.0

data/docs/handbook.md

@@ -0,0 +1,24 @@
+---
+doc-type: user
+title: ace-sim Handbook
+purpose: Package-specific docs and workflow catalog
+ace-docs:
+  last-updated: 2026-03-22
+  last-checked: 2026-03-22
+---
+
+# ace-sim Handbook
+
+## Skills
+
+- [`as-sim-run`](../handbook/skills/as-sim-run/SKILL.md)
+
+## Workflows
+
+- [`sim/run`](../handbook/workflow-instructions/sim/run.wf.md)
+
+## Related Docs
+
+- [Getting Started](getting-started.md)
+- [Usage Reference](usage.md)
+- Command help: `ace-sim --help`

data/docs/usage.md

@@ -0,0 +1,127 @@
+---
+doc-type: reference
+title: ace-sim Usage Reference
+purpose: Complete CLI reference for simulation runs and command behavior
+ace-docs:
+  last-updated: 2026-03-22
+  last-checked: 2026-03-22
+---
+
+# ace-sim Usage Reference
+
+`ace-sim` runs provider simulations through configurable steps and presets.
+
+## Command Overview
+
+- `ace-sim` — entrypoint
+- `ace-sim run` — run a simulation preset with one or more source files
+- `ace-sim --help` — print command list and examples
+
+## `ace-sim --help`
+
+Shows command examples and the `run` subcommand.
+
+## `ace-sim run`
+
+### Syntax
+
+```bash
+ace-sim run [OPTIONS]
+```
+
+### Purpose
+
+Execute a preset-driven simulation with source files, provider list, and optional final synthesis.
+
+### Options
+
+| Option | Type | Default / Source | Description |
+|---|---|---|---|
+| `--preset` | string | config `sim.default_preset` or `validate-idea` | Preset name from `.ace/sim/presets/*.yml|yaml` |
+| `--source` | array | preset `source` | One or more source files (repeatable, supports globs) |
+| `--steps` | string | preset steps | Comma-separated step names (override preset step list) |
+| `--provider` | array | preset providers | Provider:model values (`--provider` may repeat) |
+| `--repeat` | integer | sim default repeat (or `1`) | Run each provider this many times |
+| `--synthesis-workflow` | string | preset / config | Workflow/file ref for final synthesis |
+| `--synthesis-provider` | string | preset / config provider | Provider for final suggestions generation |
+| `--dry-run` | flag | preset / false | Prepare and preview without mutating providers |
+| `--writeback` | flag | preset / false | Write final revised source back to source when set |
+| `--quiet`, `-q` | flag | false | Suppress non-essential status output |
+| `--verbose`, `-v` | flag | false | Print extended diagnostics |
+| `--help`, `-h` | flag | false | Show command help |
+
+## Preset configuration model
+
+- Presets are resolved by name.
+- Preset loading precedence for files is:
+  - gem defaults (`.ace-defaults/sim/presets`)
+  - user presets (`~/.ace/sim/presets`)
+  - project presets (`.ace/sim/presets`)
+
+- File extensions: `.yml` and `.yaml`.
+
+If a preset is missing but known, fallback behavior is an empty preset with default steps and the system-level defaults for provider/repeat behavior.
+
+## Step config resolution
+
+Each requested step is resolved in this order:
+
+1. `.ace/sim/steps/<step>.md`
+2. `~/.ace/sim/steps/<step>.md`
+3. `.ace-defaults/sim/steps/<step>.md`
+
+Run fails with `Missing step config` if a required step file is not found.
+
+## Synthesis and precedence
+
+- If `--synthesis-provider` is passed, that provider is used for final synthesis.
+- If not passed, synthesis defaults use: preset `synthesis_provider`, then global config `sim.synthesis_provider`.
+- `--synthesis-provider` requires `--synthesis-workflow` to be set.
+- `--dry-run` is a non-mutating preview and cannot be combined with `--writeback`.
+
+## Artifacts
+
+Run output lives under `.ace-local/sim/simulations/<run-id>/`.
+
+Run root:
+
+- `session.yml` — simulation session metadata
+- `synthesis.yml` — final synthesis status and summaries
+- `input.md` — bundled source content used for provider execution
+- `input.bundle.md` — source bundle manifest generated before execution
+
+Per chain (`<provider>-<iteration>`):
+
+- `NN-step/input.md` — effective input for that step
+- `NN-step/user.bundle.md` — step bundle for LLM prompt
+- `NN-step/user.prompt.md` — resolved prompt file
+- `NN-step/output.md` — provider output for that step
+
+Final directory:
+
+- `final/input.md` — combined chain outputs
+- `final/user.bundle.md` — final synthesis bundle
+- `final/user.prompt.md` — final synthesis prompt
+- `final/output.sequence.md` — raw LLM output sequence
+- `final/suggestions.report.md` — parsed suggestions block
+- `final/source.original.md` — original source snapshot
+- `final/source.revised.md` — revised source output (if synthesis enabled)
+
+## Behavior notes
+
+- Steps run sequentially within each chain — each step's output becomes the next step's input, so later steps build on earlier reasoning.
+- `draft`, `plan`, `work` are common defaults; custom step order is supported via `--steps`.
+- After all chains complete, the synthesis stage gathers feedback from every stage to propose improvements and produce a revised source artifact.
+- Synthesis is optional; enable via preset or explicit `--synthesis-workflow`.
+- `--dry-run` does not perform provider calls.
+
+## Troubleshooting
+
+- `Unknown preset`:
+  - verify preset exists under one of `.ace-defaults/sim/presets`, `~/.ace/sim/presets`, or `.ace/sim/presets`
+- `Missing step config`:
+  - verify step bundles exist in step search path above
+- `--source is required`:
+  - provide source files directly or via preset defaults
+- `synthesis_provider requires synthesis_workflow`:
+  - include both flags together when overriding synthesis provider

data/exe/ace-sim

@@ -0,0 +1,15 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "bundler/setup"
+require "ace/sim"
+
+args = ARGV.empty? ? ["--help"] : ARGV
+
+begin
+  exit_code = Ace::Sim::CLI.start(args)
+  exit(exit_code) if exit_code.is_a?(Integer) && exit_code.nonzero?
+rescue Ace::Support::Cli::Error => e
+  warn e.message
+  exit(e.exit_code)
+end

data/handbook/skills/as-sim-run/SKILL.md

@@ -0,0 +1,29 @@
+---
+name: as-sim-run
+description: Run scenario simulation with provider comparison
+# bundle: wfi://sim/run
+# agent: general-purpose
+user-invocable: true
+allowed-tools:
+  - Bash(ace-sim:*)
+  - Bash(ace-bundle:*)
+  - Read
+  - TodoWrite
+argument-hint: "[--preset NAME] [--source PATH]"
+last_modified: 2026-02-28
+source: ace-sim
+integration:
+  targets:
+    - claude
+    - codex
+    - gemini
+    - opencode
+    - pi
+  providers: {}
+skill:
+  kind: workflow
+  execution:
+    workflow: wfi://sim/run
+---
+
+Load and run `ace-bundle wfi://sim/run` in the current project, then follow the loaded workflow as the source of truth and execute it end-to-end instead of only summarizing it.

data/handbook/workflow-instructions/sim/run.wf.md

@@ -0,0 +1,155 @@
+---
+doc-type: workflow
+title: Simulation Run Workflow
+purpose: Run ace-sim simulation with preset, source, and provider configuration
+ace-docs:
+  last-updated: 2026-02-28
+  last-checked: 2026-03-21
+---
+
+# Simulation Run Workflow
+
+## Goal
+
+Run a simulation against a source document using a preset, then review the synthesis results (suggestions report and revised source).
+
+## Arguments
+
+- `--preset`: Preset name (optional). Available presets:
+  - `validate-task` — steps: plan, work (synthesis: `wfi://task/review`)
+  - `validate-idea` — steps: draft, plan, work (synthesis: `wfi://idea/review`)
+- `--source`: Source markdown file path (required unless preset defines a default source)
+- `--provider`: Provider:model override (repeatable). Examples: `google:flash-preview`, `google:pro-preview`
+- `--synthesis-provider`: Provider:model for final synthesis. Example: `claude:haiku`
+- `--dry-run`: Preview what would run without executing
+
+## Instructions
+
+### Step 1: Resolve Source
+
+If the source is a task number or multiple files, use `ace-bundle` to merge into a single file:
+
+```bash
+# Single task
+ace-bundle task:148 --output /tmp/sim-source.md
+
+# Task spec with usage context (recommended for validate-task)
+ace-sim run --preset validate-task --source "path/to/task.s.md,path/to/ux/usage.md"
+
+# Already a single file — use directly
+# --source path/to/spec.md
+```
+
+If `--source` points to an existing file, use it directly.
+
+If the source is a task with usage documentation (`ux/usage.md`), include both spec and usage files to give the simulation behavioral acceptance context.
+
+### Step 2: Run Simulation
+
+Execute `ace-sim run` with the resolved source and any overrides.
+
+```bash
+# With preset (defaults)
+ace-sim run --preset validate-task --source path/to/source.md
+
+# With provider override
+ace-sim run --preset validate-task --source path/to/source.md --provider google:flash-preview
+
+# With synthesis provider override
+ace-sim run --preset validate-task --source path/to/source.md --synthesis-provider google:pro-preview
+
+# Dry run — preview without executing
+ace-sim run --preset validate-task --source path/to/source.md --dry-run
+```
+
+**Important for Claude Code**: Run with 10-minute timeout (600000ms) and wait for completion inline (not background). Simulations may take several minutes depending on step count and provider.
+
+#### Execution Guard (Mandatory)
+
+- Completion is defined by **process exit** (success or failure), not by partial output.
+- Do **not** treat temporary silence/no new output as completion.
+- Do **not** run any Step 3+ commands until Step 2 process exit is confirmed.
+- If 10-minute timeout (600000ms) is reached, report timeout and last observed output, then stop dependent steps.
+
+Wait for the simulation process to exit. Note the **Run Dir** path from the output.
+
+### Step 3: Review Results
+
+After successful completion, read the synthesis output files from the run directory:
+
+```bash
+# Read the suggestions report
+Read: <run-dir>/final/suggestions.report.md
+
+# Read the revised source document
+Read: <run-dir>/final/source.revised.md
+```
+
+The run directory structure:
+```
+<run-dir>/
+  chains/          # Individual chain execution outputs
+  final/
+    suggestions.report.md   # Synthesis report with findings
+    source.revised.md       # Revised source incorporating suggestions
+```
+
+### Step 4: Apply Validated Changes
+
+After reviewing `source.revised.md`, apply the validated refinements back to the
+original source files. The simulation output is only useful if it feeds back into
+the actual specs.
+
+**When the source was a single file** — apply diffs directly using the Edit tool,
+comparing `source.revised.md` against the original.
+
+**When the source was bundled from multiple files** (e.g. merged task specs):
+- Compare `source.revised.md` section by section against each original file
+- Apply each relevant change to its correct source file using the Edit tool
+- Common changes to propagate: status promotions, spec refinements, new error
+  messages, updated success criteria, clarified edge cases
+
+**Skip or flag for human decision:**
+- Changes marked `[PENDING DECISION]` in the revised source
+- Structural additions that don't have a clear home in the originals
+- Questions from the suggestions report — surface these to the user instead
+
+Do not commit yet — let the commit workflow handle that after review.
+
+### Step 5: Present Summary
+
+Summarize the simulation findings to the user:
+
+1. **Run metadata** — preset, providers, step count, run directory
+2. **Key findings** — top suggestions from the suggestions report
+3. **Changes applied** — which original files were updated and what changed
+4. **Pending decisions** — any `[PENDING DECISION]` items needing human input
+5. **Recommended actions** — next steps (commit, re-run with different providers, etc.)
+
+## Quick Reference
+
+```bash
+# List available presets
+ace-sim run --help
+
+# Validate a task spec
+ace-sim run --preset validate-task --source path/to/task-spec.md
+
+# Validate an idea
+ace-sim run --preset validate-idea --source path/to/idea.md
+
+# Override providers
+ace-sim run --preset validate-task --source spec.md --provider google:flash-preview --provider google:pro-preview
+
+# Dry run
+ace-sim run --preset validate-task --source spec.md --dry-run
+```
+
+## Success Criteria
+
+- [ ] Simulation completed without errors
+- [ ] Synthesis report reviewed (`final/suggestions.report.md`)
+- [ ] Revised source reviewed (`final/source.revised.md`)
+- [ ] Validated changes applied back to original source files
+- [ ] Pending decisions surfaced to user
+- [ ] Key findings summarized to user

data/lib/ace/sim/cli/commands/run.rb

@@ -0,0 +1,139 @@
+# frozen_string_literal: true
+
+module Ace
+  module Sim
+    module CLI
+      module Commands
+        class Run < Ace::Support::Cli::Command
+          include Ace::Support::Cli::Base
+
+          desc "Run preset simulation"
+
+          option :preset, type: :string, desc: "Preset name (configured in .ace/sim/presets/*.yml)"
+          option :source, type: :array, desc: "Source file(s) - repeatable, supports globs"
+          option :steps, type: :string, desc: "Comma-separated step names"
+          option :provider, type: :array, desc: "Provider:model (repeatable, e.g. --provider codex:mini)"
+          option :repeat, type: :integer, desc: "Repeat count for each provider"
+          option :synthesis_workflow, type: :string, desc: "Workflow/file reference for final synthesis (e.g. wfi://task/review, wfi://idea/review)"
+          option :synthesis_provider, type: :string, desc: "Provider:model for final suggestions synthesis"
+          option :dry_run, type: :boolean, desc: "Enable non-mutating run"
+          option :writeback, type: :boolean, desc: "Enable writeback"
+          option :quiet, aliases: ["-q"], type: :boolean, default: false, desc: "Suppress non-essential output"
+          option :verbose, aliases: ["-v"], type: :boolean, default: false, desc: "Verbose output"
+
+          def initialize(runner: nil)
+            super()
+            @runner = runner
+          end
+
+          def call(**options)
+            session = build_session(options)
+            result = runner.run(session)
+
+            unless options[:quiet]
+              puts "Run ID: #{result[:run_id]}"
+              puts "Run Dir: #{result[:run_dir]}"
+              puts "Status: #{result[:status]}"
+            end
+
+            return if result[:success]
+
+            raise Ace::Support::Cli::Error.new(result[:error] || "Simulation failed")
+          end
+
+          private
+
+          def build_session(options)
+            preset_name = pick_value(options[:preset], Ace::Sim.default_preset_name)
+            preset_data = Ace::Sim.load_preset(preset_name)
+            if preset_data.nil?
+              raise Ace::Support::Cli::Error.new("Unknown preset '#{preset_name}'. Known presets: #{Ace::Sim.preset_names.join(", ")}")
+            end
+
+            if !options[:synthesis_provider].to_s.strip.empty? && options[:synthesis_workflow].nil?
+              raise Ace::Support::Cli::Error.new("synthesis_provider requires synthesis_workflow")
+            end
+
+            sources = options[:source] || Array(preset_data["source"])
+            raise Ace::Support::Cli::Error.new("--source is required") if sources.empty?
+
+            steps = if options[:steps] && !options[:steps].to_s.strip.empty?
+              parse_steps(options[:steps])
+            else
+              Ace::Sim.normalize_list(preset_data["steps"] || Ace::Sim.get("sim", "default_steps"))
+            end
+            raise Ace::Support::Cli::Error.new("At least one step is required") if steps.empty?
+
+            providers = if options[:provider].nil?
+              Ace::Sim.normalize_list(preset_data["provider"] || preset_data["providers"] || Ace::Sim.get("sim", "default_providers"))
+            else
+              Ace::Sim.normalize_list(options[:provider])
+            end
+            raise Ace::Support::Cli::Error.new("At least one --provider is required") if providers.empty?
+
+            repeat = pick_value(options[:repeat], preset_data["repeat"], Ace::Sim.get("sim", "default_repeat") || 1)
+            repeat = Integer(repeat)
+            raise Ace::Support::Cli::Error.new("--repeat must be >= 1") if repeat < 1
+
+            synthesis_workflow = pick_value(
+              options[:synthesis_workflow],
+              preset_data["synthesis_workflow"],
+              Ace::Sim.get("sim", "synthesis_workflow")
+            ).to_s.strip
+            synthesis_provider = pick_value(
+              options[:synthesis_provider],
+              preset_data["synthesis_provider"],
+              Ace::Sim.get("sim", "synthesis_provider")
+            ).to_s.strip
+
+            dry_run = pick_value(options[:dry_run], preset_data["dry_run"], false)
+            writeback = pick_value(options[:writeback], preset_data["writeback"], Ace::Sim.get("sim", "writeback") || false)
+
+            step_bundles = resolve_step_bundles(steps)
+
+            Models::SimulationSession.new(
+              preset: preset_name,
+              source: sources,
+              steps: steps,
+              providers: providers,
+              repeat: repeat,
+              dry_run: dry_run,
+              writeback: writeback,
+              verbose: options[:verbose],
+              step_bundles: step_bundles,
+              synthesis_workflow: synthesis_workflow,
+              synthesis_provider: synthesis_provider
+            )
+          rescue ArgumentError => e
+            raise Ace::Support::Cli::Error.new(e.message)
+          rescue Ace::Sim::ValidationError => e
+            raise Ace::Support::Cli::Error.new(e.message)
+          end
+
+          def parse_steps(raw_steps)
+            raw_steps.to_s.split(",").map(&:strip).reject(&:empty?)
+          end
+
+          def resolve_step_bundles(steps)
+            steps.each_with_object({}) do |step, configs|
+              config_path = Ace::Sim.step_bundle_path(step)
+              if config_path.nil?
+                raise Ace::Support::Cli::Error.new("Missing step config for '#{step}' in .ace/sim/steps or defaults")
+              end
+
+              configs[step] = config_path
+            end
+          end
+
+          def pick_value(*values)
+            values.find { |value| !value.nil? }
+          end
+
+          def runner
+            @runner ||= Organisms::SimulationRunner.new
+          end
+        end
+      end
+    end
+  end
+end

data/lib/ace/sim/cli.rb

@@ -0,0 +1,48 @@
+# frozen_string_literal: true
+
+require "ace/support/cli"
+require "ace/core"
+require_relative "cli/commands/run"
+
+module Ace
+  module Sim
+    module CLI
+      extend Ace::Support::Cli::RegistryDsl
+
+      PROGRAM_NAME = "ace-sim"
+
+      REGISTERED_COMMANDS = [
+        ["run", "Run a preset simulation"]
+      ].freeze
+
+      HELP_EXAMPLES = [
+        "ace-sim run --preset validate-idea --source path/to/source.md --provider codex:mini --dry-run",
+        "ace-sim run --preset validate-idea --source path/to/source.md --provider codex:mini --provider google:gflash --repeat 2",
+        "ace-sim run --preset validate-idea --source path/to/source.md --provider glite --synthesis-workflow wfi://task/review --synthesis-provider claude:haiku"
+      ].freeze
+
+      def self.start(args)
+        Ace::Support::Cli::Runner.new(self).call(args: args)
+      end
+
+      register "run", Commands::Run
+
+      version_cmd = Ace::Support::Cli::VersionCommand.build(
+        gem_name: "ace-sim",
+        version: Ace::Sim::VERSION
+      )
+      register "version", version_cmd
+      register "--version", version_cmd
+
+      help_cmd = Ace::Support::Cli::HelpCommand.build(
+        program_name: PROGRAM_NAME,
+        version: Ace::Sim::VERSION,
+        commands: REGISTERED_COMMANDS,
+        examples: HELP_EXAMPLES
+      )
+      register "help", help_cmd
+      register "--help", help_cmd
+      register "-h", help_cmd
+    end
+  end
+end

data/lib/ace/sim/models/simulation_session.rb

@@ -0,0 +1,85 @@
+# frozen_string_literal: true
+
+module Ace
+  module Sim
+    module Models
+      class SimulationSession
+        attr_reader :preset, :source, :steps, :providers, :repeat, :dry_run, :writeback, :run_id, :verbose,
+          :step_bundles, :synthesis_workflow, :synthesis_provider
+
+        def initialize(preset:, source:, steps:, providers:, repeat:, dry_run:, writeback:, verbose: false,
+          run_id: nil, step_bundles: {}, synthesis_workflow: nil, synthesis_provider: nil)
+          @preset = preset.to_s.strip
+          @source = Array(source).map(&:to_s).map(&:strip).reject(&:empty?)
+          @steps = Ace::Sim.normalize_list(steps)
+          @providers = Ace::Sim.normalize_list(providers)
+          @repeat = Integer(repeat)
+          @dry_run = !!dry_run
+          @writeback = !!writeback
+          @verbose = !!verbose
+          @step_bundles = stringify_step_bundles(step_bundles)
+          @synthesis_workflow = synthesis_workflow.to_s.strip
+          @synthesis_provider = synthesis_provider.to_s.strip
+          @run_id = run_id || Ace::Sim.next_run_id
+
+          validate!
+        end
+
+        def dry_run?
+          dry_run
+        end
+
+        def regenerate_run_id!
+          @run_id = Ace::Sim.next_run_id
+        end
+
+        def bundle_path_for(step)
+          step_bundles[step.to_s]
+        end
+
+        def synthesis_enabled?
+          !synthesis_workflow.empty?
+        end
+
+        def to_h
+          {
+            "run_id" => run_id,
+            "preset" => preset,
+            "source" => source,
+            "steps" => steps,
+            "providers" => providers,
+            "repeat" => repeat,
+            "dry_run" => dry_run,
+            "writeback" => writeback,
+            "synthesis_workflow" => synthesis_workflow,
+            "synthesis_provider" => synthesis_provider
+          }
+        end
+
+        private
+
+        def validate!
+          raise Ace::Sim::ValidationError, "source cannot be empty" if source.empty?
+          raise Ace::Sim::ValidationError, "steps cannot be empty" if steps.empty?
+          raise Ace::Sim::ValidationError, "providers cannot be empty" if providers.empty?
+          raise Ace::Sim::ValidationError, "repeat must be >= 1" if repeat < 1
+          raise Ace::Sim::ValidationError, "writeback cannot be enabled with --dry-run" if dry_run && writeback
+          if !synthesis_provider.empty? && synthesis_workflow.empty?
+            raise Ace::Sim::ValidationError, "synthesis_provider requires synthesis_workflow"
+          end
+
+          missing_bundles = steps.reject { |step| step_bundles.key?(step) && !step_bundles[step].to_s.strip.empty? }
+          return if missing_bundles.empty?
+
+          raise Ace::Sim::ValidationError, "Missing step configs for: #{missing_bundles.join(", ")}"
+        end
+
+        def stringify_step_bundles(raw)
+          raw.to_h.each_with_object({}) do |(step, path), acc|
+            acc[step.to_s] = path.to_s
+          end
+        end
+      end
+    end
+  end
+end