constable-rails 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +88 -0
- data/LICENSE.txt +21 -0
- data/README.md +515 -0
- data/exe/constable +7 -0
- data/lib/constable/case.rb +336 -0
- data/lib/constable/cli.rb +475 -0
- data/lib/constable/cold_case/minitest.rb +342 -0
- data/lib/constable/cold_case/rspec.rb +334 -0
- data/lib/constable/cold_case.rb +280 -0
- data/lib/constable/config.rb +125 -0
- data/lib/constable/coverage.rb +951 -0
- data/lib/constable/diff.rb +212 -0
- data/lib/constable/dsl.rb +833 -0
- data/lib/constable/identity.rb +121 -0
- data/lib/constable/importer/modernizer.rb +860 -0
- data/lib/constable/importer/reopener.rb +468 -0
- data/lib/constable/importer.rb +51 -0
- data/lib/constable/investigation.rb +67 -0
- data/lib/constable/isolation.rb +171 -0
- data/lib/constable/jail.rb +399 -0
- data/lib/constable/log_router.rb +197 -0
- data/lib/constable/matchers.rb +834 -0
- data/lib/constable/order_audit.rb +130 -0
- data/lib/constable/rails_support.rb +213 -0
- data/lib/constable/railtie.rb +36 -0
- data/lib/constable/registry.rb +57 -0
- data/lib/constable/reporter.rb +625 -0
- data/lib/constable/result.rb +149 -0
- data/lib/constable/runner.rb +697 -0
- data/lib/constable/selection.rb +205 -0
- data/lib/constable/storage/adapter.rb +91 -0
- data/lib/constable/storage/mysql_adapter.rb +125 -0
- data/lib/constable/storage/postgres_adapter.rb +125 -0
- data/lib/constable/storage/sqlite_adapter.rb +84 -0
- data/lib/constable/storage.rb +847 -0
- data/lib/constable/version.rb +5 -0
- data/lib/constable/warrants.rb +290 -0
- data/lib/constable-rails.rb +16 -0
- data/lib/constable.rb +151 -0
- data/lib/generators/constable/base.rb +99 -0
- data/lib/generators/constable/channel/channel_generator.rb +20 -0
- data/lib/generators/constable/channel/templates/channel_case.rb.tt +29 -0
- data/lib/generators/constable/controller/controller_generator.rb +25 -0
- data/lib/generators/constable/controller/templates/controller_case.rb.tt +32 -0
- data/lib/generators/constable/generator/generator_generator.rb +31 -0
- data/lib/generators/constable/generator/templates/generator_case.rb.tt +28 -0
- data/lib/generators/constable/helper/helper_generator.rb +23 -0
- data/lib/generators/constable/helper/templates/helper_case.rb.tt +19 -0
- data/lib/generators/constable/import_generator.rb +137 -0
- data/lib/generators/constable/install_generator.rb +188 -0
- data/lib/generators/constable/integration/integration_generator.rb +27 -0
- data/lib/generators/constable/integration/templates/request_case.rb.tt +22 -0
- data/lib/generators/constable/job/job_generator.rb +20 -0
- data/lib/generators/constable/job/templates/job_case.rb.tt +33 -0
- data/lib/generators/constable/mailbox/mailbox_generator.rb +20 -0
- data/lib/generators/constable/mailbox/templates/mailbox_case.rb.tt +26 -0
- data/lib/generators/constable/mailer/mailer_generator.rb +32 -0
- data/lib/generators/constable/mailer/templates/mailer_case.rb.tt +34 -0
- data/lib/generators/constable/mailer/templates/preview.rb.tt +14 -0
- data/lib/generators/constable/model/model_generator.rb +31 -0
- data/lib/generators/constable/model/templates/model_case.rb.tt +37 -0
- data/lib/generators/constable/resource/resource_generator.rb +27 -0
- data/lib/generators/constable/scaffold/scaffold_generator.rb +42 -0
- data/lib/generators/constable/scaffold/templates/api_controller_case.rb.tt +54 -0
- data/lib/generators/constable/scaffold/templates/controller_case.rb.tt +70 -0
- data/lib/generators/constable/scaffold/templates/system_case.rb.tt +53 -0
- data/lib/generators/constable/system/system_generator.rb +20 -0
- data/lib/generators/constable/system/templates/system_case.rb.tt +18 -0
- data/lib/generators/constable/templates/authenticatable.rb.tt +31 -0
- data/lib/generators/constable/templates/case_helper.rb.tt +179 -0
- data/lib/generators/constable/templates/config.yml.tt +67 -0
- data/lib/generators/constable/templates/example_case.rb.tt +56 -0
- data/lib/generators/constable/templates/matchers.rb.tt +36 -0
- data/lib/generators/constable/templates/rubocop.yml.tt +12 -0
- metadata +209 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: 3a0630c0ca6e000d33e21515a284616186d1939531ba5f5e9c5c61882d6dc9ec
|
|
4
|
+
data.tar.gz: 10fcc4a75763bec06db7049bd3815214e1320ecdb4f3ad2780ef0497b8f8af6b
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 9a823bc22286ffaff8ab4c29f4f357ba40af37e423a9c6d1cab82382edf5d19c0ca0fae9796c0171f256bab6bdef93c9e2f14a65dd9a57e6e49e2cede204e959
|
|
7
|
+
data.tar.gz: 0c043327dc97ace117a1dae5919c9b066df22059deafa5a422066e96b3abafc6f67df7e33f911f68a8eb314b3cca6e7bab0150c7d91c3d6b26f92482baee8b0e
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project are documented here. This project adheres to
|
|
4
|
+
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
5
|
+
|
|
6
|
+
## [Unreleased]
|
|
7
|
+
|
|
8
|
+
## [0.1.0]
|
|
9
|
+
|
|
10
|
+
Initial release.
|
|
11
|
+
|
|
12
|
+
### The case file
|
|
13
|
+
|
|
14
|
+
- `Constable::Case` with the full case-file DSL: `investigate`, `witness`, `briefing`,
|
|
15
|
+
`docket`, and the `tier` macro.
|
|
16
|
+
- `investigate` is a registration DSL, not a method definition — every investigation runs
|
|
17
|
+
in its own fresh instance, so no two tests can reach each other.
|
|
18
|
+
- `witness` memoizes per test, never per process.
|
|
19
|
+
- No `before(:all)` equivalent exists, deliberately.
|
|
20
|
+
- Shared behavior across files is a plain Ruby module you `include` — no shared-examples DSL.
|
|
21
|
+
|
|
22
|
+
### Rails
|
|
23
|
+
|
|
24
|
+
- A railtie registers Constable as the app's generator test framework, so
|
|
25
|
+
`rails generate scaffold` writes cases rather than Minitest files. Every generator Rails
|
|
26
|
+
hooks is covered, and no fixtures are generated — a `witness` replaces them.
|
|
27
|
+
- `Constable::RailsSupport::Integration` and `::System` give the tier base classes the
|
|
28
|
+
request stack and Capybara respectively.
|
|
29
|
+
- `Constable::Case` answers to minitest's `setup`/`teardown` macros and lifecycle hooks, so
|
|
30
|
+
the Rails ecosystem's testing modules compose with it. `briefing` remains the primary API.
|
|
31
|
+
|
|
32
|
+
### Adoption
|
|
33
|
+
|
|
34
|
+
- `Constable::ColdCase::RSpec` and `Constable::ColdCase::Minitest` run an existing spec or
|
|
35
|
+
test file completely untouched, through its own real engine, feeding results into
|
|
36
|
+
Constable's reporting, flake history and CI gate alongside native cases.
|
|
37
|
+
- Zero-file-change adoption via `cold_cases:` globs in `.constable/config.yml`.
|
|
38
|
+
- `constable import --from=rspec|minitest` (reopen, verbatim) and the opt-in
|
|
39
|
+
`constable modernize PATH` AST rewrite into the native DSL.
|
|
40
|
+
- `rails generate constable:install` writes `test/case_helper.rb`, `test/support/`,
|
|
41
|
+
`.constable/config.yml`, and the optional `:cold_case` Gemfile group.
|
|
42
|
+
|
|
43
|
+
### Strictness
|
|
44
|
+
|
|
45
|
+
- `unsafe { }` escape hatch — always warns, never silent, with `file:line` and the reason.
|
|
46
|
+
- `rubocop-constable` companion gem: `NoSleep`, `NoUnfrozenTime`, `NoNetworkWithoutStub`,
|
|
47
|
+
`NoSharedMutableState`, `NoConditionalAssertions`, `NoRetryHelpers`, `UnsafeBlockVisibility`
|
|
48
|
+
— scoped to native cases only; cold cases are exempt by design.
|
|
49
|
+
|
|
50
|
+
### Anti-flake
|
|
51
|
+
|
|
52
|
+
- Random order every run with a printed, replayable seed.
|
|
53
|
+
- Order-dependency detection in CI.
|
|
54
|
+
- State-leak check after each native investigation.
|
|
55
|
+
- Flake history keyed by a **content hash of the investigate block**, so renaming a class,
|
|
56
|
+
rewording a description or moving a file carries history over untouched.
|
|
57
|
+
- Rename detection with `auto_relink` and `constable history relink OLD NEW`.
|
|
58
|
+
- Jail, parole and parole-violation tracking for legacy red suites.
|
|
59
|
+
- Warrants — automatic flaky detection that answers "is this failure even real."
|
|
60
|
+
|
|
61
|
+
### Speed
|
|
62
|
+
|
|
63
|
+
- Boot tiers via base classes, with path-based inference as a fallback.
|
|
64
|
+
- Parallel workers, load-balanced by a cached per-test duration index.
|
|
65
|
+
- Git-diff-based local test selection; `--full` for everything.
|
|
66
|
+
|
|
67
|
+
### Watching the suite
|
|
68
|
+
|
|
69
|
+
- `constable watchlist` — jailed, paroled and warranted tests in one view.
|
|
70
|
+
- `constable status` — the trend view: how much of the suite is still running as cold
|
|
71
|
+
cases and whether that number is moving, the recent runs, and the ten slowest tests
|
|
72
|
+
historically.
|
|
73
|
+
|
|
74
|
+
### Output
|
|
75
|
+
|
|
76
|
+
- Live glyph stream, then a summary that leads with what is most urgent: parole
|
|
77
|
+
violations, failures, warnings, slowest.
|
|
78
|
+
- Failures carry their own context, point at the `investigate` line rather than framework
|
|
79
|
+
internals, and print a ready-to-paste rerun command with the seed.
|
|
80
|
+
- stdout is results only; `Rails.logger` and SQL go to `log/test.log`, streamed with `--verbose`.
|
|
81
|
+
|
|
82
|
+
### Coverage
|
|
83
|
+
|
|
84
|
+
- Diff-based coverage gate — only lines changed in the current diff are held to the
|
|
85
|
+
threshold. `constable beat` for the full picture, `--html` for a browsable report.
|
|
86
|
+
|
|
87
|
+
[Unreleased]: https://github.com/Ray-Hughes/constable/compare/v0.1.0...HEAD
|
|
88
|
+
[0.1.0]: https://github.com/Ray-Hughes/constable/releases/tag/v0.1.0
|
data/LICENSE.txt
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
The MIT License (MIT)
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Ray Hughes
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,515 @@
|
|
|
1
|
+
<div align="center">
|
|
2
|
+
|
|
3
|
+
<img src="https://raw.githubusercontent.com/Ray-Hughes/constable/main/docs/assets/logo.png" alt="Constable" width="200">
|
|
4
|
+
|
|
5
|
+
# Constable
|
|
6
|
+
|
|
7
|
+
**A strict Rails testing framework where fast and non-flaky are structural, not disciplinary.**
|
|
8
|
+
|
|
9
|
+
[](https://rubygems.org/gems/constable-rails)
|
|
10
|
+
[](https://github.com/Ray-Hughes/constable/actions/workflows/ci.yml)
|
|
11
|
+
[](https://www.ruby-lang.org)
|
|
12
|
+
[](https://rubyonrails.org)
|
|
13
|
+
[](LICENSE.txt)
|
|
14
|
+
|
|
15
|
+
[Install](#installation) · [Quick start](#quick-start) · [Documentation](#documentation) · [Contributing](#contributing)
|
|
16
|
+
|
|
17
|
+
</div>
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
```ruby
|
|
22
|
+
class UsersController::CreatesUserCase < IntegrationCase
|
|
23
|
+
witness(:valid_params) { { user: { email: "a@b.com", password: "secret123" } } }
|
|
24
|
+
|
|
25
|
+
briefing { stub_network! }
|
|
26
|
+
|
|
27
|
+
investigate "creates a user with valid params" do
|
|
28
|
+
freeze_time
|
|
29
|
+
|
|
30
|
+
post users_path, params: valid_params
|
|
31
|
+
|
|
32
|
+
attest(response).to be_created
|
|
33
|
+
attest(User).to exist(email: "a@b.com")
|
|
34
|
+
end
|
|
35
|
+
end
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Most suites are fast and reliable because a team keeps them that way by hand. Constable
|
|
39
|
+
makes it structural instead — isolation you cannot opt out of, nondeterminism caught by a
|
|
40
|
+
linter instead of by CI, and an adoption path that never asks you to rewrite anything.
|
|
41
|
+
|
|
42
|
+
> **Installed as `constable-rails`.** The name `constable` was claimed on RubyGems in 2011
|
|
43
|
+
> by an unrelated, long-abandoned gem. That is the only thing the suffix affects —
|
|
44
|
+
> everything you actually type is `constable`: the module, the CLI, the config directory,
|
|
45
|
+
> the generators.
|
|
46
|
+
|
|
47
|
+
## Table of contents
|
|
48
|
+
|
|
49
|
+
- [Why](#why)
|
|
50
|
+
- [Requirements](#requirements)
|
|
51
|
+
- [Installation](#installation)
|
|
52
|
+
- [Quick start](#quick-start)
|
|
53
|
+
- [Documentation](#documentation)
|
|
54
|
+
- [The DSL](#the-dsl)
|
|
55
|
+
- [Tiers](#tiers-are-base-classes-not-magic)
|
|
56
|
+
- [Matchers](#matchers)
|
|
57
|
+
- [Shared behavior](#shared-behavior-is-just-ruby)
|
|
58
|
+
- [Rails generators](#rails-generators)
|
|
59
|
+
- [Adopting an existing suite](#adopting-an-existing-suite)
|
|
60
|
+
- [Escape hatches](#escape-hatches-always-visible)
|
|
61
|
+
- [The linter](#the-linter)
|
|
62
|
+
- [Jail, parole and warrants](#jail-parole-and-warrants)
|
|
63
|
+
- [Identity survives renames](#identity-survives-renames)
|
|
64
|
+
- [Command reference](#command-reference)
|
|
65
|
+
- [Output](#output)
|
|
66
|
+
- [The blotter](#the-blotter)
|
|
67
|
+
- [Configuration](#configuration)
|
|
68
|
+
- [Contributing](#contributing)
|
|
69
|
+
- [Reporting a problem](#reporting-a-problem)
|
|
70
|
+
- [License](#license)
|
|
71
|
+
|
|
72
|
+
## Why
|
|
73
|
+
|
|
74
|
+
1. **Isolation is non-negotiable in native code.** No class-level shared state, no
|
|
75
|
+
`before(:all)` equivalent. Every native test gets a clean transaction and a clean
|
|
76
|
+
object graph.
|
|
77
|
+
2. **Nondeterminism is caught by the linter, not discovered in CI.** Bare `sleep`,
|
|
78
|
+
unfrozen `Time.now`, and unstubbed network calls are lint errors before they are flakes.
|
|
79
|
+
3. **Adoption never requires a rewrite.** A whole existing RSpec or Minitest file runs
|
|
80
|
+
completely untouched from day one. Strictness applies to new code — it is not a
|
|
81
|
+
precondition for installing the gem.
|
|
82
|
+
4. **Every escape hatch is visible.** An `unsafe` block, a cold case, a jailed test — none
|
|
83
|
+
are ever silent. They are reported every run until someone deals with them.
|
|
84
|
+
5. **Fast is the default, not an opt-in.** Boot tiers, parallel workers and git-diff test
|
|
85
|
+
selection all ship in the base gem.
|
|
86
|
+
|
|
87
|
+
## Requirements
|
|
88
|
+
|
|
89
|
+
| | Minimum | Notes |
|
|
90
|
+
|---|---|---|
|
|
91
|
+
| **Ruby** | **3.1** | Parallel workers use `fork`, so they are unavailable on Windows and JRuby; those platforms fall back to serial automatically. |
|
|
92
|
+
| **Rails** | **7.0** | Tested against 7.1 and 8.1. |
|
|
93
|
+
|
|
94
|
+
Constable pulls in five gems, all of them small and already present in most Rails apps:
|
|
95
|
+
|
|
96
|
+
| Gem | Version | What needs it |
|
|
97
|
+
|---|---|---|
|
|
98
|
+
| `activesupport` | `>= 7.0` | `freeze_time` / `travel_to` delegate to it when it's there |
|
|
99
|
+
| `railties` | `>= 7.0` | the generators and the railtie that registers Constable as your test framework |
|
|
100
|
+
| `thor` | `>= 1.2` | the `constable` CLI |
|
|
101
|
+
| `sqlite3` | `>= 1.6` | the blotter — flake history, the jail docket, warrants |
|
|
102
|
+
| `parser` | `>= 3.1` | the AST rewrite behind `constable modernize` |
|
|
103
|
+
|
|
104
|
+
Your app's own database is untouched by any of this: the blotter is a separate SQLite file
|
|
105
|
+
Constable owns. See [The blotter](#the-blotter).
|
|
106
|
+
|
|
107
|
+
Nothing else is required. These are all optional, and only if you want the feature:
|
|
108
|
+
|
|
109
|
+
| Optional | For |
|
|
110
|
+
|---|---|
|
|
111
|
+
| `rubocop-constable` | the linter — the seven cops that catch nondeterminism at edit time |
|
|
112
|
+
| `rspec-rails` / `minitest` | cold cases, if you are adopting an existing suite |
|
|
113
|
+
| `capybara` + a driver | the `:system` tier |
|
|
114
|
+
| `pg` / `mysql2` | pointing the blotter at Postgres or MySQL instead of SQLite |
|
|
115
|
+
|
|
116
|
+
## Installation
|
|
117
|
+
|
|
118
|
+
```ruby
|
|
119
|
+
# Gemfile
|
|
120
|
+
group :development, :test do
|
|
121
|
+
gem "constable-rails"
|
|
122
|
+
gem "rubocop-constable", require: false
|
|
123
|
+
end
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
```console
|
|
127
|
+
$ bundle install
|
|
128
|
+
$ rails generate constable:install
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
That writes `test/case_helper.rb`, `test/support/`, `.constable/config.yml`, a `.rubocop.yml`
|
|
132
|
+
snippet, and a worked example case so `constable test` does something immediately.
|
|
133
|
+
|
|
134
|
+
## Quick start
|
|
135
|
+
|
|
136
|
+
```console
|
|
137
|
+
$ constable test # only what your current git diff touches
|
|
138
|
+
$ constable test --full # everything. this is what CI runs
|
|
139
|
+
$ constable test path/to/case.rb:12
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
## Documentation
|
|
143
|
+
|
|
144
|
+
### The DSL
|
|
145
|
+
|
|
146
|
+
The vocabulary is the API, not decoration.
|
|
147
|
+
|
|
148
|
+
| Constable | Replaces | Notes |
|
|
149
|
+
|---|---|---|
|
|
150
|
+
| `Constable::Case` | `describe` / `TestCase` | One file, roughly one subject under test |
|
|
151
|
+
| `investigate "..." do` | `it` / `def test_` | A plain string — punctuation and interpolation are fine |
|
|
152
|
+
| `witness(:name) { }` | `let` | Memoized **per test**, never per process |
|
|
153
|
+
| `briefing do ... end` | `before` / `setup` | Runs before every investigation. There is no `before(:all)` |
|
|
154
|
+
| `docket "..." do ... end` | nested `describe` | Grouping that introduces no shared state |
|
|
155
|
+
| `attest(x).to matcher` | `expect(x).to` | Sugar over `assert_*` primitives, which are always available too |
|
|
156
|
+
|
|
157
|
+
`investigate` is a **registration DSL, not a method definition.** Each block runs in its own
|
|
158
|
+
fresh instance, fully isolated from every other one.
|
|
159
|
+
|
|
160
|
+
```ruby
|
|
161
|
+
class UsersController::CreatesUserCase < IntegrationCase
|
|
162
|
+
docket "as an admin" do
|
|
163
|
+
briefing { sign_in(:admin) }
|
|
164
|
+
|
|
165
|
+
investigate "creates a user with valid params" do
|
|
166
|
+
post users_path, params: valid_params
|
|
167
|
+
attest(response).to be_created
|
|
168
|
+
end
|
|
169
|
+
end
|
|
170
|
+
|
|
171
|
+
docket "as a guest" do
|
|
172
|
+
investigate "is redirected to sign in" do
|
|
173
|
+
post users_path, params: valid_params
|
|
174
|
+
attest(response).to redirect_to(sign_in_path)
|
|
175
|
+
end
|
|
176
|
+
end
|
|
177
|
+
end
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
### Tiers are base classes, not magic
|
|
181
|
+
|
|
182
|
+
```ruby
|
|
183
|
+
# test/case_helper.rb
|
|
184
|
+
class UnitCase < Constable::Case
|
|
185
|
+
tier :unit
|
|
186
|
+
end
|
|
187
|
+
|
|
188
|
+
class IntegrationCase < Constable::Case
|
|
189
|
+
include Constable::RailsSupport::Integration
|
|
190
|
+
tier :integration
|
|
191
|
+
end
|
|
192
|
+
|
|
193
|
+
class SystemCase < Constable::Case
|
|
194
|
+
include Constable::RailsSupport::System if defined?(Capybara)
|
|
195
|
+
tier :system
|
|
196
|
+
end
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
Subclass whichever fits. Path-based inference (`test/cases/models/**` → `:unit`) still works
|
|
200
|
+
as a fallback, but ordinary inheritance is the recommended pattern — nothing to infer.
|
|
201
|
+
|
|
202
|
+
`RailsSupport::Integration` is what gives a case `get`/`post`, `response` and your app's URL
|
|
203
|
+
helpers; `RailsSupport::System` gives it Capybara and `driven_by`. `UnitCase` gets neither,
|
|
204
|
+
deliberately — that is the tier that boots without them. The installer writes all three.
|
|
205
|
+
|
|
206
|
+
Rails' testing modules expect minitest's lifecycle, so `Constable::Case` also answers to the
|
|
207
|
+
`setup` and `teardown` class macros. `setup` is an exact synonym for `briefing` and exists so
|
|
208
|
+
those modules compose — **`briefing` is still the way to write setup.**
|
|
209
|
+
|
|
210
|
+
### Matchers
|
|
211
|
+
|
|
212
|
+
```ruby
|
|
213
|
+
# test/support/matchers.rb
|
|
214
|
+
Constable::Matchers.define(:be_created) { |response| response.status == 201 }
|
|
215
|
+
Constable::Matchers.define(:exist) { |model_class, attrs| model_class.exists?(attrs) }
|
|
216
|
+
```
|
|
217
|
+
|
|
218
|
+
Built in: `eq`, `eql`, `include`, `match`, `raise_error`, `have_attributes`, `exist`,
|
|
219
|
+
`be_created`, `redirect_to`, `have_http_status`, `change`, plus `be_a`, `be_nil`, `be_empty`,
|
|
220
|
+
`be_truthy`, `be_falsey` and a `be_*` / `have_*` predicate fallback. Plain `assert_*` and
|
|
221
|
+
`refute_*` primitives are always available alongside `attest`.
|
|
222
|
+
|
|
223
|
+
### Shared behavior is just Ruby
|
|
224
|
+
|
|
225
|
+
There is deliberately no shared-examples mechanism. Reuse across files is a module:
|
|
226
|
+
|
|
227
|
+
```ruby
|
|
228
|
+
# test/support/authenticatable.rb
|
|
229
|
+
module Authenticatable
|
|
230
|
+
def sign_in(user)
|
|
231
|
+
post session_path, params: { email: user.email, password: "password" }
|
|
232
|
+
end
|
|
233
|
+
end
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
`include Authenticatable` in any case. Ruby's own composition tools are more flexible than a
|
|
237
|
+
parallel DSL that does the same job.
|
|
238
|
+
|
|
239
|
+
### Rails generators
|
|
240
|
+
|
|
241
|
+
`rails generate` asks whatever is registered as the app's test framework what a test file
|
|
242
|
+
looks like. Constable registers itself, so scaffolds produce cases rather than Minitest
|
|
243
|
+
files for a framework you replaced.
|
|
244
|
+
|
|
245
|
+
```console
|
|
246
|
+
$ rails generate scaffold Post title:string
|
|
247
|
+
create test/cases/controllers/posts_controller_case.rb
|
|
248
|
+
create test/cases/system/posts_case.rb
|
|
249
|
+
```
|
|
250
|
+
|
|
251
|
+
Every generator Rails hooks is covered — `model`, `controller`, `scaffold`, `integration_test`,
|
|
252
|
+
`system_test`, `mailer`, `job`, `helper`, `channel`, `mailbox`, `generator`, `resource` — each
|
|
253
|
+
writing a case that subclasses the right tier base class.
|
|
254
|
+
|
|
255
|
+
No fixtures are generated, deliberately: a `witness` builds exactly what one investigation
|
|
256
|
+
needs and throws it away with it. A factory gem registered as your `fixture_replacement`
|
|
257
|
+
still gets its turn.
|
|
258
|
+
|
|
259
|
+
### Adopting an existing suite
|
|
260
|
+
|
|
261
|
+
Nothing gets rewritten. **Cold cases** run your original file through its own real engine —
|
|
262
|
+
RSpec or Minitest — and feed pass/fail/timing into Constable's reporting, flake history and
|
|
263
|
+
CI gate alongside native cases.
|
|
264
|
+
|
|
265
|
+
**One line changes.** The file body is untouched:
|
|
266
|
+
|
|
267
|
+
```ruby
|
|
268
|
+
class LegacyUsersSpec < Constable::ColdCase::RSpec
|
|
269
|
+
describe UsersController do
|
|
270
|
+
it "creates a user" do
|
|
271
|
+
post users_path, params: valid_params
|
|
272
|
+
expect(response).to have_http_status(:created)
|
|
273
|
+
end
|
|
274
|
+
end
|
|
275
|
+
end
|
|
276
|
+
```
|
|
277
|
+
|
|
278
|
+
**Or nothing changes at all** — match the path in config:
|
|
279
|
+
|
|
280
|
+
```yaml
|
|
281
|
+
cold_cases:
|
|
282
|
+
- spec/controllers/**/*_spec.rb
|
|
283
|
+
```
|
|
284
|
+
|
|
285
|
+
```console
|
|
286
|
+
$ constable import --from=rspec # reopen everything, verbatim
|
|
287
|
+
$ constable modernize spec/controllers/users_controller_spec.rb --alongside
|
|
288
|
+
```
|
|
289
|
+
|
|
290
|
+
`modernize` converts `describe`/`it` → `Constable::Case`/`investigate`, `let` → `witness`,
|
|
291
|
+
`before` → `briefing`, `expect` → `attest`, and `def test_foo` → `investigate "foo"`. It
|
|
292
|
+
**flags `before(:all)` and `let!` rather than converting them** — those need a human decision —
|
|
293
|
+
and leaves custom matchers and `shared_examples` alone, logging everything to
|
|
294
|
+
`constable_modernize_report.md`. It writes nothing unless you ask it to.
|
|
295
|
+
|
|
296
|
+
Native and cold cases run side by side in one `constable test`. No big-bang cutover.
|
|
297
|
+
|
|
298
|
+
### Escape hatches, always visible
|
|
299
|
+
|
|
300
|
+
```ruby
|
|
301
|
+
investigate "times out after thirty seconds" do
|
|
302
|
+
unsafe { sleep(0.1) } # testing an actual timeout path, not a code smell
|
|
303
|
+
attest(subject).to have_timed_out
|
|
304
|
+
end
|
|
305
|
+
```
|
|
306
|
+
|
|
307
|
+
Every `unsafe` emits a warning with its `file:line` and that adjacent comment as the reason.
|
|
308
|
+
One warning per cold-case *file*, one per `unsafe` occurrence. Warnings never fail the build
|
|
309
|
+
by default — `fail_on_warnings: true` opts CI into enforcing a downward trend — but they are
|
|
310
|
+
never silent either.
|
|
311
|
+
|
|
312
|
+
### The linter
|
|
313
|
+
|
|
314
|
+
`rubocop-constable` is scoped to native cases only; cold cases are exempt by design.
|
|
315
|
+
|
|
316
|
+
| Cop | Catches |
|
|
317
|
+
|---|---|
|
|
318
|
+
| `NoSleep` | bare `sleep` outside `unsafe` |
|
|
319
|
+
| `NoUnfrozenTime` | `Time.now` / `Date.today` / `Time.current` outside `freeze_time`/`travel_to` |
|
|
320
|
+
| `NoNetworkWithoutStub` | HTTP calls without `stub_network!` |
|
|
321
|
+
| `NoSharedMutableState` | class variables and globals mutated across investigations |
|
|
322
|
+
| `NoConditionalAssertions` | `if`/`else` branching around assertions |
|
|
323
|
+
| `NoRetryHelpers` | any retry/eventually pattern |
|
|
324
|
+
| `UnsafeBlockVisibility` | an `unsafe` block with no comment explaining why |
|
|
325
|
+
|
|
326
|
+
### Jail, parole and warrants
|
|
327
|
+
|
|
328
|
+
A large red legacy suite has an on-ramp. Run once in jail mode for a clean baseline, then
|
|
329
|
+
work the docket down.
|
|
330
|
+
|
|
331
|
+
```console
|
|
332
|
+
$ constable test --jail # failures get jailed instead of failing the build
|
|
333
|
+
$ constable jail # the docket: reason, file:line, date jailed
|
|
334
|
+
$ constable jail run # re-run jailed tests sequentially
|
|
335
|
+
$ constable jail parole PATH:LINE
|
|
336
|
+
$ constable jail release PATH:LINE
|
|
337
|
+
```
|
|
338
|
+
|
|
339
|
+
Jailing isn't hiding — it swaps "blocks the build" for "tracked and skipped." Jailed tests are
|
|
340
|
+
always their own summary category, never folded into passed, and their `briefing`/`witness`
|
|
341
|
+
setup still runs so setup rot surfaces immediately.
|
|
342
|
+
|
|
343
|
+
**Parole** is "probably fixed, not fully trusted yet." A paroled test runs normally but is
|
|
344
|
+
watched: one failure is an immediate violation straight back to jail, and `parole_period`
|
|
345
|
+
consecutive clean runs (default 10) auto-releases it. `jail run` never auto-releases on a
|
|
346
|
+
pass — a single green run doesn't prove anything.
|
|
347
|
+
|
|
348
|
+
**Warrants** answer a different question — not "does this block the build" but "is this
|
|
349
|
+
failure even real." With warrants on, a failing test is rerun in isolation `warrant_retries`
|
|
350
|
+
times (default 5). Fails every retry, it's a genuine failure. Passes even once, it's flaky
|
|
351
|
+
rather than broken: a warrant is written to the blotter, never to your source, and the result
|
|
352
|
+
stops blocking the build while staying loudly visible.
|
|
353
|
+
|
|
354
|
+
```console
|
|
355
|
+
$ constable warrants
|
|
356
|
+
$ constable warrants release PATH:LINE
|
|
357
|
+
$ constable watchlist # everything under supervision: jailed, paroled, warranted
|
|
358
|
+
$ constable status # trend: native-vs-cold %, recent runs, slowest historically
|
|
359
|
+
```
|
|
360
|
+
|
|
361
|
+
### Identity survives renames
|
|
362
|
+
|
|
363
|
+
Each test's key is a **content hash of its `investigate` block body**, whitespace-normalized.
|
|
364
|
+
Class name, description and file are stored alongside purely as a display label.
|
|
365
|
+
|
|
366
|
+
- Rename the class, reword the description, move the file → hash untouched, history carries over.
|
|
367
|
+
- Change what the test actually *does* → hash changes, history starts fresh. Correct, not a limitation.
|
|
368
|
+
- Renamed *and* tweaked in one commit? Constable notices an old test vanishing as a similar
|
|
369
|
+
new one appears and suggests `constable history relink OLD NEW`. Set `auto_relink: true` to
|
|
370
|
+
confirm high-confidence matches automatically.
|
|
371
|
+
|
|
372
|
+
### Command reference
|
|
373
|
+
|
|
374
|
+
| Command | Runs |
|
|
375
|
+
|---|---|
|
|
376
|
+
| `constable test` | Everything, git-diff-scoped locally |
|
|
377
|
+
| `constable test --full` | The whole suite. CI always uses this |
|
|
378
|
+
| `constable test PATH[:LINE]` | One file, or one investigation at that line |
|
|
379
|
+
| `constable test --unsafe` | Cold cases only |
|
|
380
|
+
| `constable test --jail` | The full run, in jail mode |
|
|
381
|
+
| `constable jail [run\|parole\|release]` | The docket |
|
|
382
|
+
| `constable warrants [release]` | Outstanding warrants |
|
|
383
|
+
| `constable watchlist` | Everything under supervision right now |
|
|
384
|
+
| `constable status` | How the suite is doing over time |
|
|
385
|
+
| `constable beat [--html]` | Coverage: overall %, per-file, the unpatrolled list |
|
|
386
|
+
| `constable history relink OLD NEW` | Carry history across a real body change |
|
|
387
|
+
| `constable import --from=rspec` | Adopt an existing suite as cold cases |
|
|
388
|
+
| `constable modernize PATH` | Opt-in AST rewrite into the native DSL |
|
|
389
|
+
|
|
390
|
+
Flags: `--full --unsafe --jail --warrants --coverage --seed N --workers N --verbose --tier T --no-color`.
|
|
391
|
+
|
|
392
|
+
Order is randomized every run for native cases, with the seed printed and replayable via
|
|
393
|
+
`--seed`. Cold cases keep their own engine's order. Workers run in parallel by default,
|
|
394
|
+
load-balanced by a cached per-test duration index.
|
|
395
|
+
|
|
396
|
+
### Output
|
|
397
|
+
|
|
398
|
+
stdout is reserved for results. `Rails.logger`, SQL and request/response logging go to
|
|
399
|
+
`log/test.log`; `--verbose` streams it back for active debugging.
|
|
400
|
+
|
|
401
|
+
```
|
|
402
|
+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
403
|
+
CONSTABLE 482 tests · 3 cases · 12.4s
|
|
404
|
+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
405
|
+
✓ 478 passed ✗ 2 failed ⛓ 2 jailed (1 parole violation) ◑ 1 on parole ⚖ 1 warrant issued ⚠ 3 warnings ◐ 92% covered
|
|
406
|
+
|
|
407
|
+
FAILURES
|
|
408
|
+
────────
|
|
409
|
+
✗ SessionsCase
|
|
410
|
+
"expires after inactivity"
|
|
411
|
+
spec/cases/sessions_case.rb:12
|
|
412
|
+
|
|
413
|
+
Expected response to be :created, got :unprocessable_entity
|
|
414
|
+
|
|
415
|
+
Response body:
|
|
416
|
+
{ "errors": ["Email has already been taken"] }
|
|
417
|
+
|
|
418
|
+
Rerun just this test:
|
|
419
|
+
constable test spec/cases/sessions_case.rb:12 --seed 8841
|
|
420
|
+
```
|
|
421
|
+
|
|
422
|
+
Sections print worst-to-least-urgent: parole violations, failures, warnings, slowest.
|
|
423
|
+
Failures carry their own context and point at your `investigate` line, not at framework
|
|
424
|
+
internals.
|
|
425
|
+
|
|
426
|
+
### The blotter
|
|
427
|
+
|
|
428
|
+
Flake history, the jail docket and warrants live in a store Constable owns entirely — by
|
|
429
|
+
default a self-contained `.constable/constable.sqlite3` in WAL mode. Never your app's
|
|
430
|
+
database: native cases roll back their transaction and would roll this data back with it,
|
|
431
|
+
`:unit`-tier runs skip booting the DB stack for speed, and the workload is a handful of
|
|
432
|
+
tables that doesn't need a client-server database.
|
|
433
|
+
|
|
434
|
+
Teams who need one queryable store across many CI machines can point it elsewhere — always a
|
|
435
|
+
separate connection from the app's own:
|
|
436
|
+
|
|
437
|
+
```yaml
|
|
438
|
+
storage:
|
|
439
|
+
adapter: postgres
|
|
440
|
+
url: postgres://user:pass@host/constable_metadata
|
|
441
|
+
```
|
|
442
|
+
|
|
443
|
+
### Configuration
|
|
444
|
+
|
|
445
|
+
```yaml
|
|
446
|
+
# .constable/config.yml
|
|
447
|
+
cold_cases:
|
|
448
|
+
- spec/controllers/**/*_spec.rb
|
|
449
|
+
|
|
450
|
+
storage:
|
|
451
|
+
adapter: sqlite # sqlite (default) | postgres | mysql
|
|
452
|
+
path: .constable/constable.sqlite3
|
|
453
|
+
|
|
454
|
+
warrants: false # opt-in flaky detector
|
|
455
|
+
warrant_retries: 5
|
|
456
|
+
auto_relink: false
|
|
457
|
+
|
|
458
|
+
parole_period: 10 # consecutive clean runs to auto-release
|
|
459
|
+
|
|
460
|
+
coverage: false
|
|
461
|
+
coverage_threshold: 90 # diff-based — only lines changed in the current diff
|
|
462
|
+
coverage_html: false
|
|
463
|
+
|
|
464
|
+
fail_on_warnings: false
|
|
465
|
+
parallel_workers: auto
|
|
466
|
+
|
|
467
|
+
tiers: # fallback inference; base classes are primary
|
|
468
|
+
unit: "test/cases/models/**/*"
|
|
469
|
+
integration: "test/cases/controllers/**/*"
|
|
470
|
+
system: "test/cases/system/**/*"
|
|
471
|
+
```
|
|
472
|
+
|
|
473
|
+
## Contributing
|
|
474
|
+
|
|
475
|
+
Bug reports and pull requests are welcome at
|
|
476
|
+
<https://github.com/Ray-Hughes/constable>.
|
|
477
|
+
|
|
478
|
+
```console
|
|
479
|
+
$ git clone git@github.com:Ray-Hughes/constable.git
|
|
480
|
+
$ cd constable
|
|
481
|
+
$ bin/setup
|
|
482
|
+
$ bundle exec rake test # the framework's own suite
|
|
483
|
+
$ bundle exec rake cops # the RuboCop extension's suite
|
|
484
|
+
$ bundle exec rubocop # lint
|
|
485
|
+
```
|
|
486
|
+
|
|
487
|
+
The repo holds two gems: `constable-rails` at the root, and `rubocop-constable` in its own
|
|
488
|
+
directory with its own gemspec and suite. `docs/ARCHITECTURE.md` is the interface contract
|
|
489
|
+
between components and is worth reading before a substantial change; `docs/SPEC.md` is the
|
|
490
|
+
product spec.
|
|
491
|
+
|
|
492
|
+
A few house rules, so a change lands cleanly:
|
|
493
|
+
|
|
494
|
+
- **Constable's own suite is Minitest**, not Constable — it cannot test itself before it
|
|
495
|
+
works. Add tests under `test/unit/` or `test/integration/`.
|
|
496
|
+
- **New behavior needs a test that would fail without it.** Several of the nastiest bugs in
|
|
497
|
+
this gem were invisible to unit tests and only appeared when the real binary ran against a
|
|
498
|
+
real Rails app; an integration test is often the honest one.
|
|
499
|
+
- **Keep `rake test` and `rubocop` green.** CI runs both on Ruby 3.1, 3.2 and 3.3.
|
|
500
|
+
- Comments explain *why*, not *what*.
|
|
501
|
+
|
|
502
|
+
## Reporting a problem
|
|
503
|
+
|
|
504
|
+
Please open a [GitHub issue](https://github.com/Ray-Hughes/constable/issues). Include:
|
|
505
|
+
|
|
506
|
+
- what you ran, and the full summary block it printed
|
|
507
|
+
- the seed, so the order is replayable (`constable test --seed N`)
|
|
508
|
+
- your Ruby and Rails versions, and whether the case is native or a cold case
|
|
509
|
+
|
|
510
|
+
If a test behaves differently alone than in a full run, say so explicitly — that is an
|
|
511
|
+
order-dependency bug and Constable has machinery specifically for it.
|
|
512
|
+
|
|
513
|
+
## License
|
|
514
|
+
|
|
515
|
+
[MIT](LICENSE.txt).
|