ace-test 0.6.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: b1e7e2715b9ce0c40caefcd4352c9d55d4fe48cce56a16608ce2a83422e1c2f1
+  data.tar.gz: 2fd404ee96520dceea07be4d826edf3a7dcc460fe9b41d86053ed9e4ef3d071f
+SHA512:
+  metadata.gz: 9c8cd343fceb8f3676d91caa02187408197d8d024681fb463e250ca3d3d0cc5e67a6dddb81c2a1bf51f93851654b0e0839a3e8967975d1e8e6b28664b142e746
+  data.tar.gz: 36541ccd8b7a4316db513c36bd56f07ebcea2f819284970fb85e780a6e31b35e837257653312423342147329dee28324284ad9f3bce06cf0ee84a85ab23c2bd8

data/.ace-defaults/nav/protocols/agent-sources/ace-test.yml

@@ -0,0 +1,19 @@
+---
+# Agent Sources Protocol Configuration for ace-test gem
+# This enables agent discovery from the installed ace-test gem
+
+name: ace-test
+type: gem
+description: Testing agents from ace-test gem
+priority: 10
+
+# Configuration for agent discovery within the gem
+config:
+  # Relative path within the gem
+  relative_path: handbook/agents
+
+  # Pattern for finding agent files
+  pattern: "*.ag.md"
+
+  # Enable discovery
+  enabled: true

data/.ace-defaults/nav/protocols/guide-sources/ace-test.yml

@@ -0,0 +1,19 @@
+---
+# Guide Sources Protocol Configuration for ace-test gem
+# This enables guide discovery from the installed ace-test gem
+
+name: ace-test
+type: gem
+description: Testing guides from ace-test gem
+priority: 10
+
+# Configuration for guide discovery within the gem
+config:
+  # Relative path within the gem
+  relative_path: handbook/guides
+
+  # Pattern for finding guide files
+  pattern: "*.g.md"
+
+  # Enable discovery
+  enabled: true

data/.ace-defaults/nav/protocols/tmpl-sources/ace-test.yml

@@ -0,0 +1,11 @@
+---
+# Template Sources Protocol Configuration for ace-test gem
+name: ace-test
+type: gem
+description: Testing templates from ace-test gem
+priority: 10
+
+config:
+  relative_path: handbook/templates
+  pattern: "*.template.md"
+  enabled: true

data/.ace-defaults/nav/protocols/wfi-sources/ace-test.yml

@@ -0,0 +1,19 @@
+---
+# WFI Sources Protocol Configuration for ace-test gem
+# This enables workflow discovery from the installed ace-test gem
+
+name: ace-test
+type: gem
+description: Testing workflow instructions from ace-test gem
+priority: 10
+
+# Configuration for workflow discovery within the gem
+config:
+  # Relative path within the gem (default: handbook/workflow-instructions)
+  relative_path: handbook/workflow-instructions
+
+  # Pattern for finding workflow files (default: *.wf.md)
+  pattern: "*.wf.md"
+
+  # Enable discovery
+  enabled: true

data/CHANGELOG.md

@@ -0,0 +1,169 @@
+# Changelog
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.6.0] - 2026-03-24
+
+### Changed
+- Refreshed README tagline to emphasize knowledge-base role and fast, reliable tests.
+- Fixed verify-suite use-case to reference `ace-test-runner` instead of stale `ace-test` CLI link.
+
+## [0.5.2] - 2026-03-23
+
+### Changed
+- Refreshed README structure to match the current package layout pattern, including quick links, use-case framing, and standardized section flow.
+
+## [0.5.1] - 2026-03-22
+
+### Technical
+- Removed obsolete `docs/demo` recording assets (`ace-test-getting-started.tape` and generated GIF) from the non-CLI documentation package.
+
+## [0.5.0] - 2026-03-22
+
+### Changed
+- Rewrote README as concise landing page (~57 lines, down from 233)
+- Updated gemspec summary and description to match new tagline
+
+### Added
+- New `docs/getting-started.md` tutorial-style guide
+- New `docs/usage.md` with full CLI reference
+- New `docs/handbook.md` with skills and workflows catalog
+- Demo VHS tape and GIF in `docs/demo/`
+
+## [0.4.6] - 2026-03-21
+
+### Fixed
+- Restored the historical release entries removed during the documentation sweep so the package changelog again preserves prior release traceability.
+
+## [0.4.5] - 2026-03-21
+
+### Added
+- Added `as-test-improve-coverage` as the canonical workflow skill for coverage gap planning.
+- Added `test/improve-coverage.wf.md` workflow instruction to execute coverage planning from the test domain.
+- Updated package README and handbook references to point to `wfi://test/improve-coverage`.
+
+### Changed
+- Removed task-domain ownership of coverage planning workflow artifacts and migrated those responsibilities to `ace-test`.
+
+## [0.4.4] - 2026-03-13
+
+### Technical
+- Updated canonical test planning skills for direct workflow execution in this project.
+
+## [0.4.3] - 2026-03-13
+
+### Changed
+- Updated canonical test-planning and suite-health skills to explicitly run bundled workflows in the current project and execute them end-to-end.
+
+## [0.4.2] - 2026-03-13
+
+### Changed
+- Removed the Codex-specific delegated execution metadata from the canonical `as-test-verify-suite` skill so provider projections now inherit the canonical skill body unchanged.
+
+## [0.4.1] - 2026-03-12
+
+### Changed
+- Updated README and workflow guidance to use direct `ace-bundle` workflow loading instead of legacy slash-command references.
+
+## [0.4.0] - 2026-03-12
+
+### Added
+- Added Codex-specific delegated execution metadata to the canonical `as-test-verify-suite` skill so the generated Codex skill runs in fork context on `gpt-5.3-codex-spark`.
+
+## [0.3.0] - 2026-03-10
+
+### Added
+- Added canonical handbook-owned test planning, optimization, suite verification, performance-audit, and test-review skills.
+- Added package workflows for `wfi://test/performance-audit` and `wfi://test/review` to replace inline legacy skill bodies.
+
+## [0.2.2] - 2026-03-04
+
+### Changed
+- E2E sandbox checklist template now uses `.ace-local/test-e2e`.
+
+## [0.2.1] - 2026-02-24
+
+### Changed
+- Strengthen `test/analyze-failures` output contract with autonomous fix decisions, concrete candidate file targets, and explicit no-touch boundaries.
+- Update `test/fix` to consume autonomous analysis decisions directly and proceed without user clarification for normal targeting/scope choices.
+
+## [0.2.0] - 2026-02-24
+
+### Added
+- Add `test/analyze-failures` workflow to classify failing tests before applying fixes.
+
+### Changed
+- Rewrite `test/fix` as an execution-only workflow with a hard gate to prior analysis output.
+- Update testing guide references to use analyze-first then fix workflow sequencing.
+
+## [0.1.5] - 2026-02-22
+
+### Changed
+- Migrate skill naming and invocation references to hyphenated `ace-*` format (no underscores).
+
+## [0.1.4] - 2026-02-20
+
+### Technical
+- Update stale wfi://work-on-task references to wfi://task/work
+
+## [0.1.3] - 2026-02-19
+
+### Technical
+- Namespace workflow instructions into test/ subdirectory with updated wfi:// URIs
+- Update skill name references to use namespaced ace:test-action format
+
+## [0.1.2] - 2026-02-18
+
+### Changed
+- Remove all MT-format references from guides, workflows, and templates — TS-format is now the only documented E2E test format
+- Update E2E test location references from `test/e2e/*.mt.md` to `test/e2e/TS-*/` across testing-philosophy, testing-strategy, test-layer-decision, test-review-checklist, and test-responsibility-map guides
+- Update create-test-cases, optimize-tests, and plan-tests workflows to use TS-format examples and directory creation instead of MT single-file patterns
+- Update e2e-sandbox-checklist, test-responsibility-map, and test-review-checklist templates to use `TS-` prefix and TS-format paths
+
+## [0.1.1] - 2026-02-03
+
+### Added
+- Step 3b "Implementation Subprocess Detection" in verify-test-suite workflow
+- Explicit subprocess source file search in molecules checklist
+- Test base class check (Section 6) in test-review-checklist guide
+
+### Changed
+- Molecules checklist now requires searching SOURCE files for subprocess patterns
+- Renumbered E2E section to 7 in test-review-checklist guide
+
+## [0.1.0] - 2026-01-22
+
+### Added
+- Initial release of ace-test gem as pure workflow package
+- Testing guides extracted from docs/testing-patterns.md:
+  - `quick-reference.g.md` - TL;DR testing patterns
+  - `testing-philosophy.g.md` - Testing pyramid and IO isolation
+  - `test-organization.g.md` - Flat structure and naming conventions
+  - `mocking-patterns.g.md` - Git, HTTP, subprocess, ENV mocking
+  - `test-performance.g.md` - Performance targets and optimization
+  - `testable-code-patterns.g.md` - Status codes, exceptions, exit handling
+- Migrated workflow instructions from ace-taskflow:
+  - `create-test-cases.wf.md` - Generate structured test cases
+  - `fix-tests.wf.md` - Diagnose and fix failing tests
+- Copied guides from ace-test-runner:
+  - `testing.g.md` - General testing guidelines
+  - `testing-tdd-cycle.g.md` - TDD implementation cycle
+  - `test-driven-development-cycle/` subdirectory (7 files)
+  - `testing/` subdirectory (7 files)
+- Copied guide from ace-docs:
+  - `embedded-testing-guide.g.md` - Embedding tests in workflows
+- 3 new testing agents:
+  - `test.ag.md` - Run tests with smart defaults
+  - `mock.ag.md` - Generate mock helpers
+  - `profile-tests.ag.md` - Profile slow tests
+- Protocol registration for guide://, wfi://, agent://, and tmpl:// discovery
+- Complete gem structure following ACE patterns
+
+### Technical Details
+- Pure workflow package with no Ruby runtime dependencies
+- Auto-discovery via ace-nav through handbook/ directories
+- Protocol registration enables ace-nav to discover resources from installed gem

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Michal Czyz
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,40 @@
+<div align="center">
+  <h1> ACE - Test </h1>
+
+  Testing knowledge base for ACE — guides, patterns, and workflows for fast, reliable tests.
+
+  <img src="https://raw.githubusercontent.com/cs3b/ace/main/docs/brand/AgenticCodingEnvironment.Logo.XS.jpg" alt="ACE Logo" width="480">
+  <br><br>
+
+  <a href="https://rubygems.org/gems/ace-test"><img alt="Gem Version" src="https://img.shields.io/gem/v/ace-test.svg" /></a>
+  <a href="https://www.ruby-lang.org"><img alt="Ruby" src="https://img.shields.io/badge/Ruby-3.2+-CC342D?logo=ruby" /></a>
+  <a href="https://opensource.org/licenses/MIT"><img alt="License: MIT" src="https://img.shields.io/badge/License-MIT-blue.svg" /></a>
+
+</div>
+
+> Works with: Claude Code, Codex CLI, OpenCode, Gemini CLI, pi-agent, and more.
+
+[Getting Started](docs/getting-started.md) | [Usage Guide](docs/usage.md) | [Handbook - Skills, Agents, Templates](docs/handbook.md)
+
+`ace-test` is the handbook and workflow package for testing in ACE. It centralizes strategy guides, workflow instructions, and skill entry points for agents and developers. Use [ace-test-runner](../ace-test-runner) when you need to execute tests; use `ace-test` when you need to plan, review, or improve them.
+
+## How It Works
+
+1. Load test planning and strategy content through protocol URLs (`guide://`, `wfi://`, `agent://`) via [ace-bundle](../ace-bundle).
+2. Invoke agent skills like `/as-test-plan` or `/as-test-fix` to drive structured test workflows with full context.
+3. Apply review, coverage, and optimization guidance to improve suite quality iteratively.
+
+## Use Cases
+
+**Plan test coverage before implementation starts** - use `/as-test-plan` and `/as-test-create-cases` to map expected behavior, ownership, and verification checkpoints before writing code.
+
+**Repair failing tests with a repeatable workflow** - use `/as-test-fix` to follow structured failure triage and fix loops that isolate regressions quickly and restore green runs with evidence.
+
+**Improve suite quality over time** - use `/as-test-improve-coverage` and `/as-test-performance-audit` to identify weak spots, prioritize upgrades, and keep the fast loop healthy alongside [ace-test-runner](../ace-test-runner) execution.
+
+**Standardize review quality for test code** - use `/as-test-review` for dedicated test review workflows covering mock quality, layering fit, and maintainability.
+
+**Verify full-suite health** - use `/as-test-verify-suite` to confirm that the complete test suite passes after changes, coordinating with [`ace-test-runner`](../ace-test-runner) for execution.
+
+---
+[Getting Started](docs/getting-started.md) | [Usage Guide](docs/usage.md) | [Handbook - Skills, Agents, Templates](docs/handbook.md) | Part of [ACE](https://github.com/cs3b/ace)

data/Rakefile

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rake/testtask"
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << "test" << "lib"
+  t.test_files = FileList["test/**/*_test.rb"]
+end
+
+task spec: :test
+task default: :test

data/handbook/agents/mock.ag.md

@@ -0,0 +1,164 @@
+---
+name: mock
+description: Generate mock helpers and stubbing patterns for tests
+expected_params:
+  required:
+  - target: 'What to mock - class name, module, or dependency type (git, http, subprocess, env)'
+  optional:
+  - pattern: 'Specific pattern (protected-method, composite-helper, webmock, mock-git-repo)'
+  - context: 'Test file or class for context-aware generation'
+last_modified: '2026-01-22'
+type: agent
+source: ace-test
+---
+
+You are a test mocking specialist who helps generate proper mock helpers and stubbing patterns.
+
+## Core Responsibilities
+
+Your primary role is to help create effective test mocks:
+- Generate mock helpers following ACE patterns
+- Convert real I/O operations to stubbed versions
+- Create composite helpers to reduce nesting
+- Identify zombie mocks and suggest fixes
+
+## Mocking Patterns by Type
+
+### ENV Dependencies
+Use the protected method pattern:
+
+```ruby
+# Production code
+class MyClass
+  def find_config
+    config_path = env_config_path || default_path
+    # ...
+  end
+
+  protected
+
+  def env_config_path
+    ENV['CONFIG_PATH']
+  end
+end
+
+# Test helper
+def test_with_custom_config
+  obj = MyClass.new
+  obj.stub :env_config_path, "/custom/path" do
+    result = obj.find_config
+    assert_equal expected, result
+  end
+end
+```
+
+### Subprocess Calls (Open3)
+```ruby
+def with_stubbed_subprocess(stdout: "", stderr: "", success: true)
+  mock_status = Object.new
+  mock_status.define_singleton_method(:success?) { success }
+  mock_status.define_singleton_method(:exitstatus) { success ? 0 : 1 }
+
+  Open3.stub :capture3, [stdout, stderr, mock_status] do
+    yield
+  end
+end
+```
+
+### HTTP Calls (WebMock)
+```ruby
+def stub_github_api_success
+  stub_request(:get, "https://api.github.com/repos/owner/repo")
+    .to_return(status: 200, body: { "id" => 123 }.to_json)
+end
+
+def stub_api_error(status: 401)
+  stub_request(:any, /api\.example\.com/)
+    .to_return(status: status, body: { "error" => "Unauthorized" }.to_json)
+end
+```
+
+### Git Operations (MockGitRepo)
+```ruby
+def with_mocked_git_repo
+  repo = Ace::TestSupport::Fixtures::GitMocks::MockGitRepo.new
+  begin
+    yield repo
+  ensure
+    repo.cleanup
+  end
+end
+
+# Usage
+def test_file_processing
+  with_mocked_git_repo do |repo|
+    repo.add_file("config.yml", "key: value")
+    result = MyProcessor.new(repo.path).process
+    assert result.success?
+  end
+end
+```
+
+### DiffOrchestrator (Git Diffs)
+```ruby
+def with_empty_git_diff
+  empty_result = Ace::Git::Models::DiffResult.empty
+  Ace::Git::Organisms::DiffOrchestrator.stub(:generate, empty_result) do
+    yield
+  end
+end
+
+def with_mock_diff(content:, files: [])
+  mock_result = Ace::Git::Models::DiffResult.new(
+    content: content,
+    stats: { additions: 1, deletions: 0, files: files.size },
+    files: files
+  )
+  Ace::Git::Organisms::DiffOrchestrator.stub(:generate, mock_result) do
+    yield
+  end
+end
+```
+
+### Sleep (Retry Logic)
+```ruby
+def with_stubbed_sleep
+  Kernel.stub :sleep, nil do
+    yield
+  end
+end
+```
+
+## Composite Helpers
+
+When multiple stubs are needed together:
+
+```ruby
+def with_mock_repo_load(branch: "main", task_pattern: nil)
+  branch_info = build_mock_branch_info(name: branch, task_pattern: task_pattern)
+  mock_config = build_mock_config
+  mock_diff = Ace::Git::Models::DiffResult.empty
+
+  Ace::Config.stub :create, mock_config do
+    Ace::Git::Molecules::BranchInfo.stub :fetch, branch_info do
+      Ace::Git::Organisms::DiffOrchestrator.stub :generate, mock_diff do
+        yield
+      end
+    end
+  end
+end
+```
+
+## Related Guides
+
+- [Mocking Patterns](guide://mocking-patterns) - Full pattern reference
+- [Test Performance](guide://test-performance) - Why proper mocking matters
+- [Testable Code Patterns](guide://testable-code-patterns) - Designing for testability
+
+## Response Format
+
+When generating mocks:
+1. Show the helper code with clear naming
+2. Include usage example in a test
+3. Explain what's being stubbed and why
+4. Note any thread-safety considerations

data/handbook/agents/profile-tests.ag.md

@@ -0,0 +1,132 @@
+---
+name: profile-tests
+description: Profile slow tests and identify performance bottlenecks
+expected_params:
+  required: []
+  optional:
+  - count: 'Number of slowest tests to show (default: 10)'
+  - target: 'Test target - file, directory, or group'
+  - threshold: 'Only show tests exceeding this time (e.g., 100ms)'
+last_modified: '2026-01-22'
+type: agent
+source: ace-test
+---
+
+You are a test performance specialist who helps identify and fix slow tests.
+
+## Core Responsibilities
+
+Your primary role is to analyze test performance and suggest optimizations:
+- Profile test suites to find slow tests
+- Identify common performance issues (zombie mocks, real I/O)
+- Recommend specific fixes based on test patterns
+- Guide optimization efforts based on layer thresholds
+
+## Primary Tool: ace-test --profile
+
+```bash
+# Profile 10 slowest tests
+ace-test --profile 10
+
+# Profile slowest tests in a specific group
+ace-test atoms --profile 20
+
+# Profile a specific test file
+ace-test test/molecules/some_test.rb --profile 5
+```
+
+## Performance Thresholds
+
+Tests exceeding these limits need investigation:
+
+| Test Layer | Target Time | Hard Limit | Common Issues |
+|------------|-------------|------------|---------------|
+| Unit (atoms) | <10ms | 50ms | Real git ops, subprocess spawns |
+| Unit (molecules) | <50ms | 100ms | Unstubbed dependencies |
+| Unit (organisms) | <100ms | 200ms | Missing composite helpers |
+| Integration | <500ms | 1s | Too many real operations |
+| E2E | <2s | 5s | Should be rare - ONE per file |
+
+## Common Performance Issues
+
+### 1. Zombie Mocks
+**Symptom**: Tests pass but are slow; mock setup doesn't match code.
+**Detection**: Profile shows unit tests taking >100ms.
+**Fix**: Update stubs to target actual code paths.
+
+```ruby
+# WRONG - Stubs method no longer in code path
+ChangeDetector.stub :execute_git_command, "" do
+  # Tests pass but run REAL git operations!
+end
+
+# CORRECT - Stubs actual method
+Ace::Git::Organisms::DiffOrchestrator.stub :generate, empty_result do
+  # Fast, properly mocked
+end
+```
+
+### 2. Real Git Operations
+**Symptom**: Tests in atoms/molecules taking 150-200ms.
+**Detection**: `git init` or `git commit` in test output.
+**Fix**: Use MockGitRepo instead.
+
+### 3. Subprocess Spawning
+**Symptom**: Tests taking ~150ms each.
+**Detection**: `Open3.capture3` in code path.
+**Fix**: Stub Open3 or use API tests.
+
+### 4. Sleep in Retry Logic
+**Symptom**: Tests taking seconds.
+**Detection**: `sleep` calls in production code.
+**Fix**: Stub `Kernel.sleep`.
+
+### 5. Deep Nesting
+**Symptom**: Test setup overhead adds up.
+**Detection**: 6-7 levels of stub nesting.
+**Fix**: Use composite helpers.
+
+## Investigation Process
+
+1. **Profile First**
+   ```bash
+   ace-test --profile 20
+   ```
+
+2. **Categorize by Layer**
+   - Unit tests >50ms: Likely zombie mocks or real I/O
+   - Integration >500ms: May need optimization
+   - E2E >5s: Check if multiple E2Es exist (should be ONE)
+
+3. **Check Mock Targets**
+   - Read the slow test file
+   - Trace the code path being tested
+   - Verify stubs match actual method calls
+
+4. **Apply Fixes**
+   - Update zombie mocks
+   - Add composite helpers
+   - Convert E2E to mocked tests
+
+## Real Example: ace-docs Optimization
+
+**Before**: 14 seconds (zombie mocks)
+**After**: 1.5 seconds (89% improvement)
+
+**Root Cause**: Tests stubbed `execute_git_command` but code used `DiffOrchestrator.generate`.
+
+**Fix**: Update all tests to use `with_empty_git_diff` helper.
+
+## Related Guides
+
+- [Test Performance](guide://test-performance) - Full optimization guide
+- [Mocking Patterns](guide://mocking-patterns) - Proper stubbing patterns
+- [Quick Reference](guide://quick-reference) - Performance targets summary
+
+## Response Format
+
+When analyzing results:
+1. List tests exceeding thresholds by category
+2. Identify patterns (same file, similar names, same layer)
+3. Suggest specific fixes with code examples
+4. Estimate improvement potential