smartest 0.1.0 → 0.2.0.alpha2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7b82e9b950c32ec5f3a65ca3a99eb123e1befcacedd8f6960cc8ace06a750a10
-  data.tar.gz: 9ad2694c4ac24b45fc848b3d9d1f4ec0470c5a1895a75f057ee50f4fa091236d
+  metadata.gz: e0953c11bf61f80c4a310701920e5b6a22727149ed135d86678b3e49b25c8a92
+  data.tar.gz: d51839a9cb597da3d63a15aa803661946ceb3ae2a6982317ea108797f4203380
 SHA512:
-  metadata.gz: 22900d08d05c65b6f3ec4fba2a694dd93521d3942592c49d5066c911b475a8e64847e45e25e068e7424129afe2d0756319b1e559e75996e616245d057f8b852d
-  data.tar.gz: f97daa9190ccfa9698c6a0c7c4655e3fd4a3f3df888333dd4fef954d8aa4332239e2d037481b01b56bf09b8d8e56c49c6b874b78fd8f506dcfc0f77605fe5320
+  metadata.gz: 2e3be2ebdc5b116e6f16e396d4befda198c0063bb4da6e7f487266642d1edd8083bcc4dce9b37746969fdcbd9916424068f043344cc732edf4738f849fb87d9c
+  data.tar.gz: 0c299af8b8dd05fac80ec1eefaeade8f747f8d2776e87ba6cb684ff84c619be8a21dfd3e33c99fa43d05ee0d40aabc361f86ab1e33af696a5a9ef4c9ee78054d

data/DEVELOPMENT.md

@@ -102,7 +102,8 @@ Responsibilities:
 
 - test registry
 - fixture class registry
-- hook registry, if hooks are implemented later
+- matcher registry
+- `around_suite` hook registry
 
 Example shape:
 
@@ -125,10 +126,14 @@ Required methods:
 
 ```ruby
 test(name, **metadata, &block)
-use_fixture(klass)
-use_matcher(matcher_module)
+around_suite(&block)
+around_test(&block)
 ```
 
+`use_fixture(klass)` and `use_matcher(matcher_module)` are not top-level DSL
+methods. They are available only from hook execution contexts: `around_suite`
+and `around_test`.
+
 Possible later methods:
 
 ```ruby
@@ -219,6 +224,7 @@ Responsibilities:
 - supports inheritance
 - exposes `cleanup` to fixture blocks
 - optionally delegates helper methods to `ExecutionContext`
+- does not delegate `skip` or `pending` to fixture blocks
 
 Fixture definitions should not execute at declaration time.
 
@@ -263,7 +269,10 @@ Responsibilities:
 Example:
 
 ```ruby
-use_fixture AppFixture
+around_suite do |suite|
+  use_fixture AppFixture
+  suite.run
+end
 ```
 
 should register `AppFixture`.
@@ -298,6 +307,7 @@ Responsibilities:
 
 - include expectation methods
 - include matchers
+- expose `skip` and `pending` to test bodies
 - expose helper methods
 - optionally provide integration helpers later
 
@@ -314,21 +324,29 @@ Runs tests.
 Responsibilities:
 
 - iterate over registered test cases
+- run registered `around_suite` hooks around the suite body
 - create a lazy suite-scoped `FixtureSet`
 - create a fresh `ExecutionContext` per test
 - create a fresh `FixtureSet` per test
 - resolve test keyword fixtures
 - run test body
 - run cleanup in `ensure`
+- track skipped and pending test state
 - run suite fixture cleanup after all tests
 - produce `TestResult`
 - notify reporter
 
+`around_suite` hooks receive a run target and must call `suite.run` exactly once.
+Registered hooks compose in registration order, so the first hook is the
+outermost wrapper. If a hook raises or does not call `suite.run`, the runner
+reports a suite failure and exits with status `1`.
+
 Pseudo-code:
 
 ```ruby
 def run_one(test_case)
-  context = ExecutionContext.new
+  run_state = TestRunState.new
+  context = ExecutionContext.new(run_state: run_state)
   fixture_set = FixtureSet.new(
     @fixture_classes,
     context: context,
@@ -339,9 +357,19 @@ def run_one(test_case)
 
   context.instance_exec(**kwargs, &test_case.block)
 
-  TestResult.passed(test_case)
+  if run_state.pending?
+    TestResult.failed(test_case, PendingPassedError.new(run_state.pending_reason))
+  else
+    TestResult.passed(test_case)
+  end
+rescue Skipped => skipped
+  TestResult.skipped(test_case, skipped.reason)
 rescue Exception => error
-  TestResult.failed(test_case, error)
+  if run_state.pending?
+    TestResult.pending(test_case, run_state.pending_reason)
+  else
+    TestResult.failed(test_case, error)
+  end
 ensure
   fixture_set&.run_cleanups
 end
@@ -356,13 +384,15 @@ Fields:
 - test case
 - status
 - error
+- reason
 - duration
 
 Statuses:
 
 - passed
 - failed
-- skipped, later
+- skipped
+- pending
 
 ### `Smartest::Reporter`
 
@@ -371,7 +401,7 @@ Console reporter for MVP.
 Responsibilities:
 
 - print run start
-- print per-test pass/fail
+- print per-test pass/fail/skip/pending
 - print failure details
 - print summary
 - return appropriate exit status through runner
@@ -551,6 +581,8 @@ module Smartest
   class CircularFixtureDependencyError < Error; end
   class InvalidFixtureParameterError < Error; end
   class AssertionFailed < Error; end
+  class Skipped < Error; end
+  class PendingPassedError < AssertionFailed; end
 end
 ```
 
@@ -577,7 +609,7 @@ A practical approach:
 
 - `Smartest::Fixture`
 - `fixture :name do ... end`
-- `use_fixture`
+- `use_fixture` from a hook context
 - test block keyword fixture resolution
 
 ### Phase 3: Fixture dependencies
@@ -620,6 +652,30 @@ A practical approach:
 - generate a `smartest/test_helper.rb` that loads `smartest/fixtures/**/*.rb`
 - exit code 0 on success, 1 on failure
 
+### Phase 7: Suite hooks
+
+- `around_suite do |suite| ... end`
+- run hooks around the full suite body
+- include suite fixture cleanup inside the wrapped body
+- report hook failures as suite failures
+
+### Phase 8: Test hooks
+
+- `around_test do |test| ... end`
+- snapshot file-local hooks when each test is registered
+- run hooks around fixture setup, test body, and fixture cleanup
+- expose `use_fixture` and `use_matcher` only inside hook contexts
+- make `around_test` registered from `around_suite` suite-wide
+
+### Phase 9: Skipped and pending tests
+
+- `skip "reason"` inside test bodies and `around_test` hooks
+- `pending "reason"` inside test bodies and `around_test` hooks
+- skipped tests do not fail the run
+- pending tests pass only when the remaining execution fails
+- pending tests fail with `Smartest::PendingPassedError` when they pass
+- `skip` and `pending` are not `test` metadata
+
 ## MVP API rules
 
 Supported:
@@ -629,6 +685,19 @@ test("name") do
 end
 ```
 
+```ruby
+test("name") do
+  skip "reason" if runtime_condition?
+end
+```
+
+```ruby
+test("name") do
+  pending "reason" if expected_to_fail?
+  expect(actual).to eq(expected)
+end
+```
+
 ```ruby
 test("name") do |user:|
 end
@@ -648,6 +717,19 @@ end
 cleanup { ... }
 ```
 
+```ruby
+around_suite do |suite|
+  suite.run
+end
+```
+
+```ruby
+around_test do |test|
+  pending "reason" if expected_to_fail?
+  test.run
+end
+```
+
 Not supported in MVP:
 
 ```ruby
@@ -665,6 +747,16 @@ fixture :client, with: [:server] do |server|
 end
 ```
 
+```ruby
+test("name", skip: true) do
+end
+```
+
+```ruby
+test("name", pending: true) do
+end
+```
+
 ```ruby
 resource :server do |use|
 end

data/README.md

@@ -143,6 +143,34 @@ end
 
 This makes fixture usage explicit and avoids relying on positional argument order.
 
+## Skipping and pending tests
+
+Use `skip` at the start of a test when the rest of the body should not run:
+
+```ruby
+test("PDF export") do |browser:|
+  skip "firefox is not supported" if browser.firefox?
+
+  export_pdf(browser)
+end
+```
+
+Use `pending` when the test should continue running but is expected to fail. If
+the test passes after `pending`, Smartest fails it so the stale pending marker is
+removed.
+
+```ruby
+test("PDF export") do |browser:|
+  pending "Not supported by WebDriver BiDi yet" if browser.bidi?
+
+  export_pdf(browser)
+end
+```
+
+`skip` and `pending` are available in test bodies and `around_test` hooks, but
+not as `test` metadata or fixture APIs. See
+[Skipping Tests](documentation/docs/skipping-tests.md).
+
 ## Expectations
 
 Smartest uses an expectation style:
@@ -173,9 +201,9 @@ be_nil
 raise_error(ErrorClass)
 ```
 
-Custom matcher modules can be registered with `use_matcher`. The generated
-scaffold includes a `PredicateMatcher` custom matcher for `be_<predicate>` calls.
-See [Matchers](documentation/docs/matchers.md).
+Custom matcher modules can be registered from `around_suite` or `around_test`
+with `use_matcher`. The generated scaffold includes a `PredicateMatcher` custom
+matcher for `be_<predicate>` calls. See [Matchers](documentation/docs/matchers.md).
 
 ## Fixtures
 
@@ -192,10 +220,13 @@ class AppFixture < Smartest::Fixture
 end
 ```
 
-Register fixture classes from `smartest/test_helper.rb`:
+Register fixture classes from `around_suite` in `smartest/test_helper.rb`:
 
 ```ruby
-use_fixture AppFixture
+around_suite do |suite|
+  use_fixture AppFixture
+  suite.run
+end
 ```
 
 Tests request fixtures by keyword:
@@ -274,6 +305,100 @@ Suite fixtures are lazy: setup runs the first time a test requests the fixture,
 and cleanup runs once after all tests finish. Test-scoped fixtures can depend on
 suite fixtures, but suite fixtures cannot depend on test-scoped fixtures.
 
+## Suite hooks
+
+Use `around_suite` when the full test run must execute inside another block:
+
+```ruby
+around_suite do |suite|
+  Async do
+    suite.run
+  end
+end
+```
+
+The hook receives a run target and must call `suite.run` exactly once. The block
+wraps every test, test-scoped fixture setup and cleanup, suite fixture setup, and
+suite fixture cleanup.
+
+Fixture and matcher registrations made before `suite.run` are applied to that
+run:
+
+```ruby
+around_suite do |suite|
+  use_fixture GlobalFixture
+  suite.run
+end
+```
+
+Multiple `around_suite` hooks run in registration order. The first hook is the
+outermost wrapper:
+
+```ruby
+around_suite do |suite|
+  with_outer_resource { suite.run }
+end
+
+around_suite do |suite|
+  with_inner_resource { suite.run }
+end
+```
+
+If an `around_suite` hook raises or does not call `suite.run`, Smartest reports a
+suite failure and exits with status `1`.
+
+## Test hooks
+
+Use `around_test` when each test needs to run inside another block:
+
+```ruby
+around_test do |test|
+  SomeAutoCloseResource.new do
+    test.run
+  end
+end
+```
+
+The hook receives a run target and must call `test.run` exactly once. It wraps
+fixture setup, the test body, and fixture cleanup.
+
+`around_test` is file-scoped when it is written directly in a test file. Smartest
+copies the current file's `around_test` hooks when each `test` is registered, so
+hooks apply to tests defined later in the same file.
+
+Define `around_test` inside `around_suite` when the hook should apply to the
+whole run:
+
+```ruby
+around_suite do |suite|
+  around_test do |test|
+    with_some_resource do
+      test.run
+    end
+  end
+
+  suite.run
+end
+```
+
+`around_test` can also register fixture classes or matcher modules for that test
+run:
+
+```ruby
+around_test do |test|
+  use_fixture LocalFixture
+  use_matcher LocalMatcher
+  test.run
+end
+```
+
+Fixture classes registered from `around_test` must define only test-scoped
+fixtures. If a class defines `suite_fixture`, register it from `around_suite`
+instead so its cache and cleanup belong to the suite lifecycle.
+
+`use_fixture` and `use_matcher` are only available inside `around_suite` or
+`around_test` blocks. They are not top-level DSL methods.
+
 ## Fixtures with teardown
 
 Not every fixture needs teardown. For fixtures that do, use `cleanup`.
@@ -355,7 +480,10 @@ end
 
 ```ruby
 # smartest/test_helper.rb
-use_fixture WebFixture
+around_suite do |suite|
+  use_fixture WebFixture
+  suite.run
+end
 ```
 
 ```ruby
@@ -391,41 +519,34 @@ server cleanup
 
 ## Registering fixture classes
 
-Use `use_fixture` from `smartest/test_helper.rb`:
+Use `use_fixture` inside `around_suite` from `smartest/test_helper.rb`:
 
 ```ruby
-use_fixture AppFixture
+around_suite do |suite|
+  use_fixture AppFixture
+  suite.run
+end
 ```
 
 Multiple fixture classes can be registered:
 
 ```ruby
-use_fixture UserFixture
-use_fixture WebFixture
-use_fixture ApiFixture
+around_suite do |suite|
+  use_fixture UserFixture
+  use_fixture WebFixture
+  use_fixture ApiFixture
+  suite.run
+end
 ```
 
 Fixture names must be unique across registered fixture classes.
 
 If two fixture classes define the same fixture name, Smartest raises an error.
 
-## Hooks
+## Suite hooks and fixture cleanup
 
-Smartest may support simple hooks:
-
-```ruby
-before do
-  DatabaseCleaner.start
-end
-
-after do
-  DatabaseCleaner.clean
-end
-```
-
-Hooks are separate from fixture cleanup.
-
-Use fixture cleanup for resource-specific teardown:
+Suite hooks are separate from fixture cleanup. Use fixture cleanup for
+resource-specific teardown:
 
 ```ruby
 fixture :server do
@@ -435,11 +556,11 @@ fixture :server do
 end
 ```
 
-Use hooks for broad test-level behavior:
+Use `around_suite` for broad suite-level execution context:
 
 ```ruby
-before do
-  reset_global_state
+around_suite do |suite|
+  Async { suite.run }
 end
 ```
 
@@ -469,13 +590,16 @@ Dir[File.join(__dir__, "matchers", "**", "*.rb")].sort.each do |matcher_file|
   require matcher_file
 end
 
-use_fixture WebFixture
-use_matcher PredicateMatcher
+around_suite do |suite|
+  use_fixture WebFixture
+  use_matcher PredicateMatcher
+  suite.run
+end
 ```
 
 The generated helper loads Ruby files under `smartest/fixtures/` and
-`smartest/matchers/` in sorted order. Register fixture classes and matcher
-modules from the helper with `use_fixture` and `use_matcher`.
+`smartest/matchers/` in sorted order. Register suite-wide fixture classes and
+matcher modules from `around_suite` with `use_fixture` and `use_matcher`.
 
 Example:
 
@@ -550,6 +674,9 @@ The intended MVP includes:
 - keyword-argument fixture injection
 - fixture dependencies through keyword arguments
 - fixture cleanup
+- suite hooks with `around_suite`
+- test hooks with `around_test`
+- skipped and pending tests through `skip` and `pending`
 - `expect(...).to eq(...)`
 - console reporter
 - CLI runner