ace-sim 0.13.5 → 0.15.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 5857b6df4652d928775dc14c9dfefecc14b0f11bc419ac5a62c0684eaf868763
-  data.tar.gz: fe904143cb4898209650db83b64a682026074b9eb262d456acb0553c1e02a824
+  metadata.gz: eb9d75c8a26df59765eebf95ca07c9fba58befa9274cc63c11eb1ee628277a6e
+  data.tar.gz: fc567292fc23144ffe35858f7b50e92ee6bd4b67f10b0854d004dbe57dfb54a8
 SHA512:
-  metadata.gz: 619c3d0c94f268ff196f1c387cf99f9d58460c2e56b6cef4fc36c2e1d859dd0adc0d123e59af4bf590ca51709ceb11d8239cd8f9bc1ce14c1b465ea074134d31
-  data.tar.gz: 5f9adeebe2966da5f13b9877dc0e678d5ebbc09d7465cd005a78be3fbb84e09497c5769439bbd2482fee92883beb8fb34030255c5026ea2073b0569719e36446
+  metadata.gz: c60fad0c77420f786fad969382cdfee65cfe898953e1f0be3e55b3d531f7c650c27f596a1f28a625314228c61b9c0e6b58abb202b5cc434170a226d0c73c22af
+  data.tar.gz: 8cdedfcf8eb463a3e6b4580c21b3ab98543652b68cdd5fcfa3c85e1041a7c61bfd38f7c928cee22fa4cbe9a2c431e9699630cc0813d38371c4a66ed40a6974d7

data/CHANGELOG.md

@@ -4,6 +4,53 @@ All notable changes to this project will be documented in this file.
 
 ## [Unreleased]
 
+## [0.15.5] - 2026-04-23
+
+### Technical
+- Declared retained preset-contract metadata captures explicitly in the runner so `session.yml` and `synthesis.yml` checks follow the exact artifact-contract model enforced by `ace-test-runner-e2e`.
+
+## [0.15.4] - 2026-04-16
+
+### Fixed
+- Implemented real `--dry-run` semantics so dry runs record session and synthesis metadata without executing provider chains or final synthesis, and updated retained task-preset E2E verification to accept the actual run-directory artifacts as the canonical proof.
+
+## [0.15.3] - 2026-04-16
+
+### Fixed
+- Updated `TS-SIM-002` dry-run verification to accept both supported public synthesis failure messages when the run metadata records a failed final stage.
+
+## [0.15.2] - 2026-04-16
+
+### Fixed
+- Relaxed `TS-SIM-001` default-preset verification to accept successful public run artifacts without hard-requiring `session.yml`, while preserving the stricter metadata checks on constrained external-failure paths.
+
+## [0.15.1] - 2026-04-16
+
+### Fixed
+- Softened retained smoke E2E contracts so provider-constrained failure paths are judged from recorded run metadata and user-visible outcomes instead of requiring full chain artifact completion.
+
+## [0.15.0] - 2026-04-15
+
+### Changed
+- Rewrote E2E coverage to public-surface contracts by narrowing `TS-SIM-001` to retained high-value journeys, removing placeholder-based artifact assumptions, and adding `TS-SIM-002` for dry-run public contract validation.
+- Updated package usage documentation to clarify dry-run verification expectations and final synthesis failure-path recording semantics.
+
+## [0.14.2] - 2026-04-13
+### Fixed
+- **ace-sim v0.14.2**: Improved run-id uniqueness by extending run-id generation and collision handling so `SimulationRunner` retries more times before failing when run directories already exist.
+
+## [0.14.1] - 2026-04-13
+
+### Changed
+- Completed the batch i05 migration follow-through for this package and aligned it with the restarted `fast` / `feat` / `e2e` verification model.
+
+### Technical
+- Included in the coordinated assignment-driven patch release for batch i05 package updates.
+
+
+### Changed
+- Migrated deterministic package tests to `test/fast`, retained workflow-value E2E scenario coverage under `test/e2e`, added an E2E decision record, and aligned package docs and scenario artifact contracts with the restarted `fast` / `feat` / `e2e` model.
+
 ## [0.13.5] - 2026-03-31
 
 ### Changed

data/README.md

@@ -24,7 +24,7 @@
 
 1. Select a simulation preset (`validate-idea` or `validate-task`) and provide a source file, with context assembled by [ace-bundle](../ace-bundle).
 2. The simulation engine runs each step (draft, plan, work) sequentially — each step's output feeds as input into the next, building on prior reasoning through [ace-llm](../ace-llm).
-3. After all steps complete, a final synthesis stage gathers feedback from every stage to propose improvements, surface questions, and produce a revised source artifact — feeding better specs back into [ace-task](../ace-taskflow) or sharper ideas into [ace-idea](../ace-idea).
+3. After all steps complete, a final synthesis stage gathers feedback from every stage to propose improvements, surface questions, and produce a revised source artifact — feeding better specs back into [ace-task](../ace-task) or sharper ideas into [ace-idea](../ace-idea).
 
 ## Use Cases
 
@@ -36,5 +36,14 @@
 
 **Synthesize recommendations from simulation runs** - enable `--synthesis-workflow` and `--synthesis-provider` to produce actionable suggestions, then feed results into [ace-review](../ace-review) for follow-up review.
 
+## Testing Model
+
+`ace-sim` follows the restarted `fast` / `feat` / `e2e` contract:
+
+- `ace-test ace-sim` - default deterministic fast loop (`test/fast`).
+- `ace-test ace-sim feat` - deterministic feature-contract lane (only when `test/feat` exists).
+- `ace-test ace-sim all` - run all deterministic lanes.
+- `ace-test-e2e ace-sim` - scenario workflow validation under `test/e2e`.
+
 ---
 [Getting Started](docs/getting-started.md) | [Usage Guide](docs/usage.md) | [Handbook - Skills, Agents, Templates](docs/handbook.md) | Part of [ACE](https://github.com/cs3b/ace)

data/docs/usage.md

@@ -78,6 +78,7 @@ Run fails with `Missing step config` if a required step file is not found.
 - If not passed, synthesis defaults use: preset `synthesis_provider`, then global config `sim.synthesis_provider`.
 - `--synthesis-provider` requires `--synthesis-workflow` to be set.
 - `--dry-run` is a non-mutating preview and cannot be combined with `--writeback`.
+- `synthesis.yml` always records the final-stage outcome. External provider failures are recorded as `final_stage.status: failed`; callers should treat this as run evidence, not synthesize placeholder artifacts.
 
 ## Artifacts
 
@@ -113,7 +114,16 @@ Final directory:
 - `draft`, `plan`, `work` are common defaults; custom step order is supported via `--steps`.
 - After all chains complete, the synthesis stage gathers feedback from every stage to propose improvements and produce a revised source artifact.
 - Synthesis is optional; enable via preset or explicit `--synthesis-workflow`.
-- `--dry-run` does not perform provider calls.
+- `--dry-run` does not perform provider calls and should be verified via recorded run metadata/artifacts instead of provider output files.
+
+## Test Commands
+
+`ace-sim` uses the `fast` / `feat` / `e2e` testing model:
+
+- `ace-test ace-sim` - default deterministic fast suite
+- `ace-test ace-sim feat` - deterministic feature-contract suite (when present)
+- `ace-test ace-sim all` - aggregate deterministic suites
+- `ace-test-e2e ace-sim` - scenario workflow suite under `test/e2e`
 
 ## Troubleshooting
 

data/lib/ace/sim/molecules/synthesis_builder.rb

@@ -29,6 +29,7 @@ module Ace
 
         def overall_status(chains, final_stage: nil)
           return "failed" if final_stage && final_stage["status"] == "failed"
+          return "ok" if final_stage && final_stage["status"] == "skipped" && chains.empty?
           return "failed" if chains.empty?
           return "failed" if chains.all? { |chain| chain["status"] == "failed" }
           return "partial" if chains.any? { |chain| chain["status"] == "failed" }

data/lib/ace/sim/organisms/simulation_runner.rb

@@ -1,5 +1,7 @@
 # frozen_string_literal: true
 
+require "fileutils"
+
 module Ace
   module Sim
     module Organisms
@@ -25,22 +27,12 @@ module Ace
             output_path: File.join(run_dir, "input.md")
           )
 
-          chains = []
-          session.providers.each do |provider|
-            1.upto(session.repeat) do |iteration|
-              chains << run_chain(
-                session: session,
-                run_dir: run_dir,
-                provider: provider,
-                iteration: iteration,
-                bundled_input_path: bundled_input_path
-              )
-            end
-          end
+          chains = session.dry_run? ? [] : run_chains(session: session, run_dir: run_dir, bundled_input_path: bundled_input_path)
 
-          final_stage = nil
-          if session.synthesis_enabled?
-            final_stage = final_synthesis_executor.execute(
+          final_stage = if session.dry_run?
+            skipped_final_stage(run_dir: run_dir, bundled_input_path: bundled_input_path)
+          elsif session.synthesis_enabled?
+            final_synthesis_executor.execute(
               run_dir: run_dir,
               session: session,
               chains: chains,
@@ -106,6 +98,22 @@ module Ace
         attr_reader :session_store, :stage_executor, :synthesis_builder, :final_synthesis_executor,
           :source_bundler
 
+        def run_chains(session:, run_dir:, bundled_input_path:)
+          chains = []
+          session.providers.each do |provider|
+            1.upto(session.repeat) do |iteration|
+              chains << run_chain(
+                session: session,
+                run_dir: run_dir,
+                provider: provider,
+                iteration: iteration,
+                bundled_input_path: bundled_input_path
+              )
+            end
+          end
+          chains
+        end
+
         def run_chain(session:, run_dir:, provider:, iteration:, bundled_input_path:)
           current_input_path = bundled_input_path
           step_results = []
@@ -153,13 +161,37 @@ module Ace
             run_dir = session_store.prepare_run(session.run_id)
             [session.run_id, run_dir]
           rescue Molecules::SessionStore::RunDirectoryExistsError
-            raise Ace::Sim::ValidationError, "Could not allocate unique run id" if attempts >= 5
+            raise Ace::Sim::ValidationError, "Could not allocate unique run id" if attempts >= 20
 
             session.regenerate_run_id!
             retry
           end
         end
 
+        def skipped_final_stage(run_dir:, bundled_input_path:)
+          final_dir = session_store.final_dir(run_dir)
+          source_original_path = File.join(final_dir, "source.original.md")
+          input_path = File.join(final_dir, "input.md")
+
+          FileUtils.mkdir_p(final_dir)
+          FileUtils.cp(bundled_input_path, source_original_path)
+          File.write(
+            input_path,
+            <<~MD
+              # ace-sim dry-run final stage
+
+              Dry-run skipped provider chain execution and final synthesis.
+            MD
+          )
+
+          {
+            "status" => "skipped",
+            "reason" => "dry-run",
+            "source_original_path" => source_original_path,
+            "input_path" => input_path
+          }
+        end
+
         def failure_reason(chains, final_stage)
           return "Final synthesis failed" if final_stage && final_stage["status"] == "failed"
           return "All chains failed" if chains.all? { |chain| chain["status"] == "failed" }

data/lib/ace/sim/version.rb

@@ -2,6 +2,6 @@
 
 module Ace
   module Sim
-    VERSION = '0.13.5'
+    VERSION = '0.15.5'
   end
 end

data/lib/ace/sim.rb

@@ -106,12 +106,16 @@ module Ace
     end
 
     def next_run_id
-      Ace::B36ts.now
-    rescue
       timestamp = Time.now.utc.to_i.to_s(36)
-      entropy = rand(36**2).to_s(36).rjust(2, "0")
-      "#{timestamp}#{entropy}"
+      "#{Ace::B36ts.now}-#{timestamp}-#{random_run_suffix}"
+    rescue
+      "#{Time.now.utc.to_i.to_s(36)}-#{random_run_suffix}"
+    end
+
+    def random_run_suffix
+      rand(36**5).to_s(36).rjust(5, "0")
     end
+    private_class_method :random_run_suffix
 
     def config_resolver
       @config_resolver ||= Ace::Support::Config.create(gem_path: gem_root)

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: ace-sim
 version: !ruby/object:Gem::Version
-  version: 0.13.5
+  version: 0.15.5
 platform: ruby
 authors:
 - Michal Czyz
 bindir: exe
 cert_chain: []
-date: 2026-04-01 00:00:00.000000000 Z
+date: 2026-04-26 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: ace-support-config