rails_proof 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +54 -0
- data/MIT-LICENSE +20 -0
- data/README.md +1210 -0
- data/Rakefile +3 -0
- data/lib/generators/rails_proof/test/USAGE +8 -0
- data/lib/generators/rails_proof/test/test_generator.rb +542 -0
- data/lib/rails_proof/ai_concern_filter.rb +142 -0
- data/lib/rails_proof/ai_test_executor.rb +263 -0
- data/lib/rails_proof/ai_test_identity.rb +261 -0
- data/lib/rails_proof/ai_test_planner.rb +118 -0
- data/lib/rails_proof/ai_test_validator.rb +65 -0
- data/lib/rails_proof/ai_test_writer.rb +51 -0
- data/lib/rails_proof/controller_inspector.rb +44 -0
- data/lib/rails_proof/controller_test_coverage_plan.rb +79 -0
- data/lib/rails_proof/controller_test_plan.rb +28 -0
- data/lib/rails_proof/controller_test_writer.rb +66 -0
- data/lib/rails_proof/model_inspector.rb +50 -0
- data/lib/rails_proof/model_test_plan.rb +61 -0
- data/lib/rails_proof/open_ai_client.rb +189 -0
- data/lib/rails_proof/railtie.rb +4 -0
- data/lib/rails_proof/review_store.rb +241 -0
- data/lib/rails_proof/target_discovery.rb +145 -0
- data/lib/rails_proof/test_coverage_plan.rb +84 -0
- data/lib/rails_proof/test_inspector.rb +54 -0
- data/lib/rails_proof/test_runner.rb +136 -0
- data/lib/rails_proof/test_writer.rb +87 -0
- data/lib/rails_proof/version.rb +3 -0
- data/lib/rails_proof.rb +13 -0
- data/lib/tasks/rails_proof_tasks.rake +4 -0
- metadata +96 -0
data/README.md
ADDED
|
@@ -0,0 +1,1210 @@
|
|
|
1
|
+
# RailsProof
|
|
2
|
+
|
|
3
|
+
> **You write the Rails app. We write the tests.**
|
|
4
|
+
|
|
5
|
+
RailsProof analyzes a Rails application, discovers behavior that should be tested, writes ordinary Minitest tests, and verifies generated tests against the application before deciding what to keep.
|
|
6
|
+
|
|
7
|
+
It combines two approaches:
|
|
8
|
+
|
|
9
|
+
1. **Deterministic Rails inspection** for behavior RailsProof can identify with confidence.
|
|
10
|
+
2. **AI-assisted analysis** for application-specific behavior that a fixed rule engine cannot reasonably anticipate.
|
|
11
|
+
|
|
12
|
+
RailsProof is being developed for modern Rails applications, with current development targeting **Rails 8.1+** and **Ruby 4.0+**.
|
|
13
|
+
|
|
14
|
+
> **Status: Early development**
|
|
15
|
+
>
|
|
16
|
+
> RailsProof is under active development ahead of its first public release. Its behavior and public API may change. Use version control when trying it on real applications.
|
|
17
|
+
|
|
18
|
+
Need help, found something weird, or just want to talk about RailsProof?
|
|
19
|
+
|
|
20
|
+
**Visit us at https://support.oakharborventures.com**
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## What RailsProof Does
|
|
25
|
+
|
|
26
|
+
Given a model like:
|
|
27
|
+
|
|
28
|
+
```ruby
|
|
29
|
+
class Post < ApplicationRecord
|
|
30
|
+
belongs_to :user
|
|
31
|
+
|
|
32
|
+
validates :title, presence: true
|
|
33
|
+
|
|
34
|
+
def title_matches?(query)
|
|
35
|
+
return false if query.blank?
|
|
36
|
+
|
|
37
|
+
title.to_s.downcase.include?(query.to_s.downcase)
|
|
38
|
+
end
|
|
39
|
+
end
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
RailsProof can deterministically identify:
|
|
43
|
+
|
|
44
|
+
```text
|
|
45
|
+
belongs_to :user
|
|
46
|
+
validates presence of title
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
It can then use AI to analyze custom application behavior and discover useful coverage such as:
|
|
50
|
+
|
|
51
|
+
```text
|
|
52
|
+
title_matches? returns false for blank queries
|
|
53
|
+
title_matches? performs a case insensitive substring match
|
|
54
|
+
title_matches? returns false when the query is absent from the title
|
|
55
|
+
title_matches? handles a nil title
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
RailsProof generates candidate Minitest code, validates it, inserts one candidate at a time, and runs the actual Rails test suite against that candidate.
|
|
59
|
+
|
|
60
|
+
A candidate that passes can become permanent coverage.
|
|
61
|
+
|
|
62
|
+
A candidate that fails is **not automatically assumed to be wrong**.
|
|
63
|
+
|
|
64
|
+
That distinction matters.
|
|
65
|
+
|
|
66
|
+
A failing generated test may mean:
|
|
67
|
+
|
|
68
|
+
- the generated test is incorrect
|
|
69
|
+
- the application contains a bug
|
|
70
|
+
- the test exposed a contract mismatch
|
|
71
|
+
- the application and candidate simply disagree about intended behavior
|
|
72
|
+
|
|
73
|
+
RailsProof preserves that distinction instead of silently throwing useful evidence away.
|
|
74
|
+
|
|
75
|
+
---
|
|
76
|
+
|
|
77
|
+
## Quick Start
|
|
78
|
+
|
|
79
|
+
Run RailsProof against the entire supported application:
|
|
80
|
+
|
|
81
|
+
```bash
|
|
82
|
+
rails generate rails_proof:test
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
Inspect a specific model:
|
|
86
|
+
|
|
87
|
+
```bash
|
|
88
|
+
rails generate rails_proof:test app/models/post.rb
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
Inspect all models:
|
|
92
|
+
|
|
93
|
+
```bash
|
|
94
|
+
rails generate rails_proof:test app/models
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
Inspect a controller:
|
|
98
|
+
|
|
99
|
+
```bash
|
|
100
|
+
rails generate rails_proof:test app/controllers/posts_controller.rb
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
Or inspect all controllers:
|
|
104
|
+
|
|
105
|
+
```bash
|
|
106
|
+
rails generate rails_proof:test app/controllers
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
There is no separate AI mode and no `--ai` flag.
|
|
110
|
+
|
|
111
|
+
AI analysis is part of the normal RailsProof workflow.
|
|
112
|
+
|
|
113
|
+
---
|
|
114
|
+
|
|
115
|
+
## How It Works
|
|
116
|
+
|
|
117
|
+
RailsProof deliberately does not hand unrestricted control of your test files to an AI model.
|
|
118
|
+
|
|
119
|
+
The workflow looks like this:
|
|
120
|
+
|
|
121
|
+
```text
|
|
122
|
+
Rails application
|
|
123
|
+
|
|
|
124
|
+
v
|
|
125
|
+
Target discovery
|
|
126
|
+
|
|
|
127
|
+
v
|
|
128
|
+
Rails runtime/source inspection
|
|
129
|
+
|
|
|
130
|
+
v
|
|
131
|
+
Deterministic test planning
|
|
132
|
+
|
|
|
133
|
+
v
|
|
134
|
+
Existing test analysis
|
|
135
|
+
|
|
|
136
|
+
v
|
|
137
|
+
AI behavior analysis
|
|
138
|
+
|
|
|
139
|
+
v
|
|
140
|
+
Deterministic AI deduplication
|
|
141
|
+
|
|
|
142
|
+
v
|
|
143
|
+
Candidate Minitest generation
|
|
144
|
+
|
|
|
145
|
+
v
|
|
146
|
+
Syntax and structure validation
|
|
147
|
+
|
|
|
148
|
+
v
|
|
149
|
+
Insert one candidate
|
|
150
|
+
|
|
|
151
|
+
v
|
|
152
|
+
Run the actual test
|
|
153
|
+
|
|
|
154
|
+
+---- PASS -----------------> KEPT
|
|
155
|
+
|
|
|
156
|
+
+---- FAIL -----------------> restore live test file
|
|
157
|
+
|
|
|
158
|
+
v
|
|
159
|
+
NEEDS REVIEW
|
|
160
|
+
|
|
|
161
|
+
v
|
|
162
|
+
persist review evidence
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
Malformed or unusable generated tests are:
|
|
166
|
+
|
|
167
|
+
```text
|
|
168
|
+
REJECTED
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
Tests already covered, duplicated in the same AI response, or already represented by an unresolved review finding are:
|
|
172
|
+
|
|
173
|
+
```text
|
|
174
|
+
SKIPPED
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
RailsProof owns the file mutation, validation, execution, rollback, review storage, and deduplication process.
|
|
178
|
+
|
|
179
|
+
AI supplies analysis and candidate tests.
|
|
180
|
+
|
|
181
|
+
The Rails test runner determines whether a candidate agrees with the application as it currently exists.
|
|
182
|
+
|
|
183
|
+
The developer remains responsible for deciding whether a disagreement represents an application bug or an incorrect expectation.
|
|
184
|
+
|
|
185
|
+
---
|
|
186
|
+
|
|
187
|
+
## Result States
|
|
188
|
+
|
|
189
|
+
AI-generated candidate tests currently have four possible outcomes.
|
|
190
|
+
|
|
191
|
+
### KEPT
|
|
192
|
+
|
|
193
|
+
```text
|
|
194
|
+
KEPT: title_matches? handles a nil title
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
The candidate was structurally valid and passed against the application.
|
|
198
|
+
|
|
199
|
+
RailsProof leaves the test in the live test suite.
|
|
200
|
+
|
|
201
|
+
A passing test proves that the candidate expectation is compatible with the current implementation. It does not, by itself, prove that the implementation is correct.
|
|
202
|
+
|
|
203
|
+
### NEEDS REVIEW
|
|
204
|
+
|
|
205
|
+
```text
|
|
206
|
+
NEEDS REVIEW: title_matches_exactly? rejects partial title matches
|
|
207
|
+
candidate test failed against application
|
|
208
|
+
Review saved: .rails_proof/review/...
|
|
209
|
+
```
|
|
210
|
+
|
|
211
|
+
The candidate was structurally valid but failed against the application.
|
|
212
|
+
|
|
213
|
+
RailsProof:
|
|
214
|
+
|
|
215
|
+
1. restores the live test file to its previous state
|
|
216
|
+
2. preserves the candidate test
|
|
217
|
+
3. preserves the reason it was generated
|
|
218
|
+
4. preserves the test-run failure output
|
|
219
|
+
5. records the finding for human review
|
|
220
|
+
|
|
221
|
+
A failing valid test is evidence of a disagreement, not proof that the generated test is bad.
|
|
222
|
+
|
|
223
|
+
### REJECTED
|
|
224
|
+
|
|
225
|
+
```text
|
|
226
|
+
REJECTED: malformed generated test
|
|
227
|
+
test code is not valid Ruby
|
|
228
|
+
```
|
|
229
|
+
|
|
230
|
+
The generated candidate itself is unusable.
|
|
231
|
+
|
|
232
|
+
Examples include:
|
|
233
|
+
|
|
234
|
+
- invalid Ruby
|
|
235
|
+
- missing or multiple test declarations
|
|
236
|
+
- class declarations inside the candidate
|
|
237
|
+
- `require` statements
|
|
238
|
+
- Markdown code fences
|
|
239
|
+
- other structurally invalid generated output
|
|
240
|
+
|
|
241
|
+
Rejected tests are not preserved as application-bug findings because RailsProof could not establish that they were valid candidate tests.
|
|
242
|
+
|
|
243
|
+
### SKIPPED
|
|
244
|
+
|
|
245
|
+
```text
|
|
246
|
+
SKIPPED: title_matches_exactly? requires the whole title to match
|
|
247
|
+
already awaiting human review
|
|
248
|
+
```
|
|
249
|
+
|
|
250
|
+
RailsProof determined that executing the candidate would add no useful new information.
|
|
251
|
+
|
|
252
|
+
Current skip reasons include:
|
|
253
|
+
|
|
254
|
+
```text
|
|
255
|
+
already exists in test suite
|
|
256
|
+
already awaiting human review
|
|
257
|
+
duplicate AI suggestion
|
|
258
|
+
```
|
|
259
|
+
|
|
260
|
+
---
|
|
261
|
+
|
|
262
|
+
## Contract Checks
|
|
263
|
+
|
|
264
|
+
RailsProof does not assume that the current implementation is automatically the intended contract.
|
|
265
|
+
|
|
266
|
+
This is especially important for AI-assisted testing.
|
|
267
|
+
|
|
268
|
+
Consider:
|
|
269
|
+
|
|
270
|
+
```ruby
|
|
271
|
+
def title_matches_exactly?(query)
|
|
272
|
+
return false if query.blank?
|
|
273
|
+
|
|
274
|
+
title.to_s.downcase.include?(query.to_s.downcase)
|
|
275
|
+
end
|
|
276
|
+
```
|
|
277
|
+
|
|
278
|
+
The method is named:
|
|
279
|
+
|
|
280
|
+
```text
|
|
281
|
+
title_matches_exactly?
|
|
282
|
+
```
|
|
283
|
+
|
|
284
|
+
but its implementation performs a substring match with:
|
|
285
|
+
|
|
286
|
+
```ruby
|
|
287
|
+
include?
|
|
288
|
+
```
|
|
289
|
+
|
|
290
|
+
A test generator that blindly treats implementation as specification might generate tests proving that partial matches succeed and permanently encode the bug into the test suite.
|
|
291
|
+
|
|
292
|
+
RailsProof's AI protocol distinguishes two kinds of suggestions.
|
|
293
|
+
|
|
294
|
+
### coverage
|
|
295
|
+
|
|
296
|
+
A `coverage` suggestion tests behavior where the implementation and apparent contract agree.
|
|
297
|
+
|
|
298
|
+
For example:
|
|
299
|
+
|
|
300
|
+
```text
|
|
301
|
+
title_matches? returns false for blank queries
|
|
302
|
+
```
|
|
303
|
+
|
|
304
|
+
### contract_check
|
|
305
|
+
|
|
306
|
+
A `contract_check` is used when strong evidence in the source suggests that the apparent public contract and implementation disagree.
|
|
307
|
+
|
|
308
|
+
For example:
|
|
309
|
+
|
|
310
|
+
```text
|
|
311
|
+
title_matches_exactly? rejects partial title matches
|
|
312
|
+
```
|
|
313
|
+
|
|
314
|
+
RailsProof may generate:
|
|
315
|
+
|
|
316
|
+
```ruby
|
|
317
|
+
test "title_matches_exactly? rejects partial title matches" do
|
|
318
|
+
post = Post.new(title: "Learning Rails")
|
|
319
|
+
|
|
320
|
+
assert_not post.title_matches_exactly?("Rails")
|
|
321
|
+
end
|
|
322
|
+
```
|
|
323
|
+
|
|
324
|
+
If the implementation uses `include?`, that test fails.
|
|
325
|
+
|
|
326
|
+
RailsProof does not delete the evidence and does not leave the failing test in the live suite.
|
|
327
|
+
|
|
328
|
+
It produces:
|
|
329
|
+
|
|
330
|
+
```text
|
|
331
|
+
NEEDS REVIEW
|
|
332
|
+
```
|
|
333
|
+
|
|
334
|
+
That gives the developer a concrete test, a reason, and the actual failure output to evaluate.
|
|
335
|
+
|
|
336
|
+
---
|
|
337
|
+
|
|
338
|
+
## Human Review
|
|
339
|
+
|
|
340
|
+
RailsProof treats failing valid generated tests differently from malformed generated tests.
|
|
341
|
+
|
|
342
|
+
That distinction is intentional.
|
|
343
|
+
|
|
344
|
+
Suppose RailsProof finds:
|
|
345
|
+
|
|
346
|
+
```text
|
|
347
|
+
Method: title_matches_exactly?
|
|
348
|
+
Implementation: include?
|
|
349
|
+
```
|
|
350
|
+
|
|
351
|
+
and generates a test requiring a full-title match.
|
|
352
|
+
|
|
353
|
+
If the test fails, RailsProof cannot safely conclude:
|
|
354
|
+
|
|
355
|
+
```text
|
|
356
|
+
the AI was wrong
|
|
357
|
+
```
|
|
358
|
+
|
|
359
|
+
It also cannot safely conclude:
|
|
360
|
+
|
|
361
|
+
```text
|
|
362
|
+
the application is wrong
|
|
363
|
+
```
|
|
364
|
+
|
|
365
|
+
Instead it records:
|
|
366
|
+
|
|
367
|
+
```text
|
|
368
|
+
application behavior and candidate expectation disagree
|
|
369
|
+
```
|
|
370
|
+
|
|
371
|
+
and asks for human judgment.
|
|
372
|
+
|
|
373
|
+
Review findings are currently stored beneath:
|
|
374
|
+
|
|
375
|
+
```text
|
|
376
|
+
.rails_proof/review/
|
|
377
|
+
```
|
|
378
|
+
|
|
379
|
+
A review record contains information such as:
|
|
380
|
+
|
|
381
|
+
```text
|
|
382
|
+
status
|
|
383
|
+
created_at
|
|
384
|
+
last_seen_at
|
|
385
|
+
occurrences
|
|
386
|
+
target_path
|
|
387
|
+
target_fingerprint
|
|
388
|
+
test_file_path
|
|
389
|
+
test_class_name
|
|
390
|
+
kind
|
|
391
|
+
name
|
|
392
|
+
reason
|
|
393
|
+
test_code
|
|
394
|
+
test_fingerprint
|
|
395
|
+
test_output
|
|
396
|
+
```
|
|
397
|
+
|
|
398
|
+
The review workflow is still evolving, but the underlying evidence is already preserved rather than discarded.
|
|
399
|
+
|
|
400
|
+
---
|
|
401
|
+
|
|
402
|
+
## Review Deduplication and Convergence
|
|
403
|
+
|
|
404
|
+
AI output is nondeterministic.
|
|
405
|
+
|
|
406
|
+
The same underlying behavior might be described differently on separate runs:
|
|
407
|
+
|
|
408
|
+
```text
|
|
409
|
+
title_matches_exactly? requires a case-insensitive full-title match
|
|
410
|
+
```
|
|
411
|
+
|
|
412
|
+
then:
|
|
413
|
+
|
|
414
|
+
```text
|
|
415
|
+
title_matches_exactly? requires the entire title to match
|
|
416
|
+
```
|
|
417
|
+
|
|
418
|
+
then:
|
|
419
|
+
|
|
420
|
+
```text
|
|
421
|
+
title_matches_exactly? rejects partial title matches
|
|
422
|
+
```
|
|
423
|
+
|
|
424
|
+
RailsProof does not rely only on those human-readable names to determine identity.
|
|
425
|
+
|
|
426
|
+
Its deterministic deduplication layer examines candidate behavior so differently worded suggestions can still be recognized as the same underlying finding.
|
|
427
|
+
|
|
428
|
+
That means repeated runs against unchanged source converge instead of endlessly creating duplicate tests or duplicate review findings.
|
|
429
|
+
|
|
430
|
+
For an unresolved finding:
|
|
431
|
+
|
|
432
|
+
```text
|
|
433
|
+
first run
|
|
434
|
+
-> NEEDS REVIEW
|
|
435
|
+
-> review evidence stored
|
|
436
|
+
|
|
437
|
+
later run against unchanged source
|
|
438
|
+
-> same underlying behavior recognized
|
|
439
|
+
-> SKIPPED
|
|
440
|
+
-> already awaiting human review
|
|
441
|
+
```
|
|
442
|
+
|
|
443
|
+
RailsProof also fingerprints the target source associated with a review finding.
|
|
444
|
+
|
|
445
|
+
If the application source changes, an old review finding no longer automatically suppresses new testing.
|
|
446
|
+
|
|
447
|
+
```text
|
|
448
|
+
unchanged source + same finding
|
|
449
|
+
-> SKIPPED
|
|
450
|
+
|
|
451
|
+
changed source
|
|
452
|
+
-> eligible for analysis and execution again
|
|
453
|
+
```
|
|
454
|
+
|
|
455
|
+
This allows a natural development loop:
|
|
456
|
+
|
|
457
|
+
```text
|
|
458
|
+
RailsProof discovers disagreement
|
|
459
|
+
|
|
|
460
|
+
v
|
|
461
|
+
NEEDS REVIEW
|
|
462
|
+
|
|
|
463
|
+
v
|
|
464
|
+
developer evaluates finding
|
|
465
|
+
|
|
|
466
|
+
v
|
|
467
|
+
application code changes
|
|
468
|
+
|
|
|
469
|
+
v
|
|
470
|
+
source fingerprint changes
|
|
471
|
+
|
|
|
472
|
+
v
|
|
473
|
+
RailsProof analyzes behavior again
|
|
474
|
+
|
|
|
475
|
+
v
|
|
476
|
+
candidate passes
|
|
477
|
+
|
|
|
478
|
+
v
|
|
479
|
+
KEPT
|
|
480
|
+
```
|
|
481
|
+
|
|
482
|
+
---
|
|
483
|
+
|
|
484
|
+
## Setup-Sensitive Deduplication
|
|
485
|
+
|
|
486
|
+
Two tests can contain the same assertion without testing the same behavior.
|
|
487
|
+
|
|
488
|
+
For example:
|
|
489
|
+
|
|
490
|
+
```ruby
|
|
491
|
+
post = Post.new(title: "Learning Rails")
|
|
492
|
+
|
|
493
|
+
assert_not post.title_matches_exactly?("Rails")
|
|
494
|
+
```
|
|
495
|
+
|
|
496
|
+
and:
|
|
497
|
+
|
|
498
|
+
```ruby
|
|
499
|
+
post = Post.new(title: nil)
|
|
500
|
+
|
|
501
|
+
assert_not post.title_matches_exactly?("Rails")
|
|
502
|
+
```
|
|
503
|
+
|
|
504
|
+
have the same assertion text but different setup and therefore test different behavior.
|
|
505
|
+
|
|
506
|
+
RailsProof's AI deduplication accounts for setup context when comparing meaningful candidate behavior.
|
|
507
|
+
|
|
508
|
+
This prevents an important class of false-positive deduplication where legitimate coverage could otherwise be silently discarded.
|
|
509
|
+
|
|
510
|
+
RailsProof therefore aims to be conservative about skipping tests:
|
|
511
|
+
|
|
512
|
+
```text
|
|
513
|
+
same assertion + different setup
|
|
514
|
+
!= same behavior
|
|
515
|
+
```
|
|
516
|
+
|
|
517
|
+
---
|
|
518
|
+
|
|
519
|
+
## Example
|
|
520
|
+
|
|
521
|
+
A normal successful RailsProof run can look like:
|
|
522
|
+
|
|
523
|
+
```text
|
|
524
|
+
RailsProof targets: 1
|
|
525
|
+
RailsProof inspection
|
|
526
|
+
Model file: app/models/post.rb
|
|
527
|
+
Model class: Post
|
|
528
|
+
Test file: test/models/post_test.rb
|
|
529
|
+
Test status: exists
|
|
530
|
+
Test cases: 7
|
|
531
|
+
|
|
532
|
+
Source associations: 1
|
|
533
|
+
belongs_to :user
|
|
534
|
+
|
|
535
|
+
Source validations: 1
|
|
536
|
+
validates :title, presence: true
|
|
537
|
+
|
|
538
|
+
Runtime inspection: available
|
|
539
|
+
Table: posts
|
|
540
|
+
|
|
541
|
+
Suggested tests: 2
|
|
542
|
+
belongs_to :user
|
|
543
|
+
validates presence of title
|
|
544
|
+
|
|
545
|
+
Coverage:
|
|
546
|
+
Covered: 2
|
|
547
|
+
Missing: 0
|
|
548
|
+
|
|
549
|
+
AI suggested tests: 3
|
|
550
|
+
title_matches? returns false for blank queries
|
|
551
|
+
title_matches? performs a case insensitive substring match
|
|
552
|
+
title_matches? handles a nil title
|
|
553
|
+
|
|
554
|
+
AI test results: 3
|
|
555
|
+
KEPT: title_matches? returns false for blank queries
|
|
556
|
+
KEPT: title_matches? performs a case insensitive substring match
|
|
557
|
+
KEPT: title_matches? handles a nil title
|
|
558
|
+
```
|
|
559
|
+
|
|
560
|
+
A run that discovers a possible application bug can look like:
|
|
561
|
+
|
|
562
|
+
```text
|
|
563
|
+
AI suggested tests: 1
|
|
564
|
+
title_matches_exactly? rejects partial title matches
|
|
565
|
+
Reason: The method name strongly indicates equality semantics,
|
|
566
|
+
while the implementation performs substring matching.
|
|
567
|
+
|
|
568
|
+
AI test results: 1
|
|
569
|
+
NEEDS REVIEW: title_matches_exactly? rejects partial title matches
|
|
570
|
+
candidate test failed against application
|
|
571
|
+
Review saved: .rails_proof/review/...
|
|
572
|
+
```
|
|
573
|
+
|
|
574
|
+
Running RailsProof again without changing the application can then produce:
|
|
575
|
+
|
|
576
|
+
```text
|
|
577
|
+
AI test results: 1
|
|
578
|
+
SKIPPED: title_matches_exactly? requires the whole title to match
|
|
579
|
+
already awaiting human review
|
|
580
|
+
```
|
|
581
|
+
|
|
582
|
+
even though the AI described the finding differently.
|
|
583
|
+
|
|
584
|
+
---
|
|
585
|
+
|
|
586
|
+
## Deterministic Analysis
|
|
587
|
+
|
|
588
|
+
RailsProof first handles the parts of a Rails application that can be understood without AI.
|
|
589
|
+
|
|
590
|
+
### Models
|
|
591
|
+
|
|
592
|
+
Current deterministic model inspection includes:
|
|
593
|
+
|
|
594
|
+
- Active Record columns
|
|
595
|
+
- associations
|
|
596
|
+
- presence validations
|
|
597
|
+
- existing Minitest tests
|
|
598
|
+
- missing deterministic coverage
|
|
599
|
+
|
|
600
|
+
For example:
|
|
601
|
+
|
|
602
|
+
```ruby
|
|
603
|
+
class Post < ApplicationRecord
|
|
604
|
+
belongs_to :user
|
|
605
|
+
|
|
606
|
+
validates :title, presence: true
|
|
607
|
+
end
|
|
608
|
+
```
|
|
609
|
+
|
|
610
|
+
can produce:
|
|
611
|
+
|
|
612
|
+
```ruby
|
|
613
|
+
test "belongs to user" do
|
|
614
|
+
association = Post.reflect_on_association(:user)
|
|
615
|
+
|
|
616
|
+
assert_not_nil association
|
|
617
|
+
assert_equal :belongs_to, association.macro
|
|
618
|
+
end
|
|
619
|
+
|
|
620
|
+
test "validates presence of title" do
|
|
621
|
+
record = Post.new(title: nil)
|
|
622
|
+
|
|
623
|
+
record.validate
|
|
624
|
+
|
|
625
|
+
assert record.errors.of_kind?(:title, :blank)
|
|
626
|
+
end
|
|
627
|
+
```
|
|
628
|
+
|
|
629
|
+
RailsProof also avoids separately generating the implicit presence validation Rails adds for a required `belongs_to` association.
|
|
630
|
+
|
|
631
|
+
### Controllers
|
|
632
|
+
|
|
633
|
+
RailsProof currently understands controller actions and their matching routes.
|
|
634
|
+
|
|
635
|
+
For example:
|
|
636
|
+
|
|
637
|
+
```ruby
|
|
638
|
+
class PostsController < ApplicationController
|
|
639
|
+
def index
|
|
640
|
+
end
|
|
641
|
+
|
|
642
|
+
def show
|
|
643
|
+
end
|
|
644
|
+
end
|
|
645
|
+
```
|
|
646
|
+
|
|
647
|
+
with matching routes can produce:
|
|
648
|
+
|
|
649
|
+
```ruby
|
|
650
|
+
test "should get index" do
|
|
651
|
+
get posts_index_url
|
|
652
|
+
|
|
653
|
+
assert_response :success
|
|
654
|
+
end
|
|
655
|
+
|
|
656
|
+
test "should get show" do
|
|
657
|
+
get posts_show_url
|
|
658
|
+
|
|
659
|
+
assert_response :success
|
|
660
|
+
end
|
|
661
|
+
```
|
|
662
|
+
|
|
663
|
+
Existing Rails-generated controller tests are recognized so RailsProof does not needlessly recreate them.
|
|
664
|
+
|
|
665
|
+
---
|
|
666
|
+
|
|
667
|
+
## Existing Tests
|
|
668
|
+
|
|
669
|
+
RailsProof is designed to work with an existing test suite rather than assume it is starting from scratch.
|
|
670
|
+
|
|
671
|
+
It currently recognizes:
|
|
672
|
+
|
|
673
|
+
- Rails-style `test "..." do` declarations
|
|
674
|
+
- method-style `def test_...` declarations
|
|
675
|
+
|
|
676
|
+
Existing tests are used for both deterministic and AI-assisted analysis.
|
|
677
|
+
|
|
678
|
+
RailsProof can:
|
|
679
|
+
|
|
680
|
+
- compare deterministic concerns against existing tests
|
|
681
|
+
- insert only missing deterministic tests
|
|
682
|
+
- provide existing tests to AI as context
|
|
683
|
+
- avoid AI tests already represented in the live suite
|
|
684
|
+
- preserve passing generated tests across later candidate failures
|
|
685
|
+
- avoid repeated AI suggestions within the same run
|
|
686
|
+
- avoid repeatedly executing unresolved review findings
|
|
687
|
+
|
|
688
|
+
RailsProof is intended to be run repeatedly as an application evolves, not merely once when the project is created.
|
|
689
|
+
|
|
690
|
+
---
|
|
691
|
+
|
|
692
|
+
## AI Analysis
|
|
693
|
+
|
|
694
|
+
Fixed rules can only take automatic test generation so far.
|
|
695
|
+
|
|
696
|
+
Consider:
|
|
697
|
+
|
|
698
|
+
```ruby
|
|
699
|
+
def eligible_for_upgrade?
|
|
700
|
+
active? && account_age_days > 30 && !past_due?
|
|
701
|
+
end
|
|
702
|
+
```
|
|
703
|
+
|
|
704
|
+
or:
|
|
705
|
+
|
|
706
|
+
```ruby
|
|
707
|
+
def cancel!
|
|
708
|
+
transaction do
|
|
709
|
+
update!(status: :cancelled)
|
|
710
|
+
subscription&.cancel!
|
|
711
|
+
CancellationMailer.confirmation(self).deliver_later
|
|
712
|
+
end
|
|
713
|
+
end
|
|
714
|
+
```
|
|
715
|
+
|
|
716
|
+
There is no general Rails reflection API that can determine every meaningful behavioral test for application-specific code like this.
|
|
717
|
+
|
|
718
|
+
That is where RailsProof uses AI.
|
|
719
|
+
|
|
720
|
+
The AI receives structured context including:
|
|
721
|
+
|
|
722
|
+
- target type
|
|
723
|
+
- class name
|
|
724
|
+
- application source
|
|
725
|
+
- existing tests
|
|
726
|
+
- deterministic concerns already discovered by RailsProof
|
|
727
|
+
|
|
728
|
+
It is instructed to avoid duplicating behavior already covered by deterministic analysis or existing tests.
|
|
729
|
+
|
|
730
|
+
It is also instructed not to assume that current implementation behavior is necessarily the intended public contract.
|
|
731
|
+
|
|
732
|
+
The AI returns structured suggestions containing:
|
|
733
|
+
|
|
734
|
+
- suggestion kind
|
|
735
|
+
- test name
|
|
736
|
+
- reason
|
|
737
|
+
- candidate Minitest code
|
|
738
|
+
|
|
739
|
+
Supported suggestion kinds currently include:
|
|
740
|
+
|
|
741
|
+
```text
|
|
742
|
+
coverage
|
|
743
|
+
contract_check
|
|
744
|
+
```
|
|
745
|
+
|
|
746
|
+
RailsProof independently validates, deduplicates, writes, and executes the returned candidate code.
|
|
747
|
+
|
|
748
|
+
The AI does not directly control application test files.
|
|
749
|
+
|
|
750
|
+
---
|
|
751
|
+
|
|
752
|
+
## AI Safety
|
|
753
|
+
|
|
754
|
+
Generated code is not blindly accepted.
|
|
755
|
+
|
|
756
|
+
Before an AI-generated test can execute, RailsProof currently checks that the candidate:
|
|
757
|
+
|
|
758
|
+
- is nonblank
|
|
759
|
+
- parses as valid Ruby
|
|
760
|
+
- contains exactly one Minitest test declaration
|
|
761
|
+
- does not contain a class declaration
|
|
762
|
+
- does not contain a `require` statement
|
|
763
|
+
- does not contain Markdown code fences
|
|
764
|
+
|
|
765
|
+
Candidates are evaluated one at a time.
|
|
766
|
+
|
|
767
|
+
Before each candidate, RailsProof captures the previous state of the test file.
|
|
768
|
+
|
|
769
|
+
If the candidate passes, it may remain in the suite.
|
|
770
|
+
|
|
771
|
+
If it fails, the previous file state is restored.
|
|
772
|
+
|
|
773
|
+
That means:
|
|
774
|
+
|
|
775
|
+
```text
|
|
776
|
+
candidate 1 passes
|
|
777
|
+
-> kept
|
|
778
|
+
|
|
779
|
+
candidate 2 fails
|
|
780
|
+
-> rolled back
|
|
781
|
+
|
|
782
|
+
candidate 1 remains
|
|
783
|
+
```
|
|
784
|
+
|
|
785
|
+
A later failure does not discard earlier successful coverage.
|
|
786
|
+
|
|
787
|
+
---
|
|
788
|
+
|
|
789
|
+
## Why a Passing Generated Test Is Not Enough
|
|
790
|
+
|
|
791
|
+
A generated test can pass and still be a bad test.
|
|
792
|
+
|
|
793
|
+
For example, if RailsProof sees:
|
|
794
|
+
|
|
795
|
+
```ruby
|
|
796
|
+
def title_matches_exactly?(query)
|
|
797
|
+
title.include?(query)
|
|
798
|
+
end
|
|
799
|
+
```
|
|
800
|
+
|
|
801
|
+
and generates:
|
|
802
|
+
|
|
803
|
+
```ruby
|
|
804
|
+
assert post.title_matches_exactly?("Rails")
|
|
805
|
+
```
|
|
806
|
+
|
|
807
|
+
for a title of:
|
|
808
|
+
|
|
809
|
+
```text
|
|
810
|
+
Learning Rails
|
|
811
|
+
```
|
|
812
|
+
|
|
813
|
+
the test passes.
|
|
814
|
+
|
|
815
|
+
But it may simply be locking an implementation bug into the suite.
|
|
816
|
+
|
|
817
|
+
That is why RailsProof distinguishes normal coverage from contract checks and why application behavior is not treated as unquestionable specification.
|
|
818
|
+
|
|
819
|
+
The goal is not:
|
|
820
|
+
|
|
821
|
+
```text
|
|
822
|
+
generate tests that make the current code pass
|
|
823
|
+
```
|
|
824
|
+
|
|
825
|
+
The goal is:
|
|
826
|
+
|
|
827
|
+
```text
|
|
828
|
+
generate meaningful tests for the application's apparent contract
|
|
829
|
+
and surface disagreements that deserve human attention
|
|
830
|
+
```
|
|
831
|
+
|
|
832
|
+
---
|
|
833
|
+
|
|
834
|
+
## Installation
|
|
835
|
+
|
|
836
|
+
RailsProof is currently under active development and has not yet reached its first public release.
|
|
837
|
+
|
|
838
|
+
For development from a local checkout, add RailsProof to the application's `Gemfile`:
|
|
839
|
+
|
|
840
|
+
```ruby
|
|
841
|
+
gem "rails_proof", path: "../railsproof"
|
|
842
|
+
```
|
|
843
|
+
|
|
844
|
+
Then run:
|
|
845
|
+
|
|
846
|
+
```bash
|
|
847
|
+
bundle install
|
|
848
|
+
```
|
|
849
|
+
|
|
850
|
+
After the public release, installation will be:
|
|
851
|
+
|
|
852
|
+
```ruby
|
|
853
|
+
gem "rails_proof"
|
|
854
|
+
```
|
|
855
|
+
|
|
856
|
+
followed by:
|
|
857
|
+
|
|
858
|
+
```bash
|
|
859
|
+
bundle install
|
|
860
|
+
```
|
|
861
|
+
|
|
862
|
+
The planned first public release is **RailsProof 1.0.0**.
|
|
863
|
+
|
|
864
|
+
---
|
|
865
|
+
|
|
866
|
+
## AI Provider Setup
|
|
867
|
+
|
|
868
|
+
RailsProof's core AI interface is provider-agnostic.
|
|
869
|
+
|
|
870
|
+
The first implemented provider adapter uses OpenAI.
|
|
871
|
+
|
|
872
|
+
Applications using the OpenAI adapter currently need the OpenAI Ruby SDK available:
|
|
873
|
+
|
|
874
|
+
```ruby
|
|
875
|
+
gem "openai"
|
|
876
|
+
```
|
|
877
|
+
|
|
878
|
+
Then run:
|
|
879
|
+
|
|
880
|
+
```bash
|
|
881
|
+
bundle install
|
|
882
|
+
```
|
|
883
|
+
|
|
884
|
+
Set your API key through the environment:
|
|
885
|
+
|
|
886
|
+
```bash
|
|
887
|
+
export OPENAI_API_KEY="your-key-here"
|
|
888
|
+
```
|
|
889
|
+
|
|
890
|
+
Do not commit API keys to your repository.
|
|
891
|
+
|
|
892
|
+
RailsProof currently defaults to:
|
|
893
|
+
|
|
894
|
+
```text
|
|
895
|
+
gpt-5.6
|
|
896
|
+
```
|
|
897
|
+
|
|
898
|
+
The model can be overridden with:
|
|
899
|
+
|
|
900
|
+
```bash
|
|
901
|
+
export RAILSPROOF_OPENAI_MODEL="your-model"
|
|
902
|
+
```
|
|
903
|
+
|
|
904
|
+
AI usage is performed using your provider credentials and may incur charges from that provider.
|
|
905
|
+
|
|
906
|
+
RailsProof's own test suite does not require paid AI calls. Development tests use a fake AI client.
|
|
907
|
+
|
|
908
|
+
---
|
|
909
|
+
|
|
910
|
+
## Running RailsProof
|
|
911
|
+
|
|
912
|
+
### Entire supported application
|
|
913
|
+
|
|
914
|
+
```bash
|
|
915
|
+
rails generate rails_proof:test
|
|
916
|
+
```
|
|
917
|
+
|
|
918
|
+
### One model
|
|
919
|
+
|
|
920
|
+
```bash
|
|
921
|
+
rails generate rails_proof:test app/models/post.rb
|
|
922
|
+
```
|
|
923
|
+
|
|
924
|
+
### All models
|
|
925
|
+
|
|
926
|
+
```bash
|
|
927
|
+
rails generate rails_proof:test app/models
|
|
928
|
+
```
|
|
929
|
+
|
|
930
|
+
### One controller
|
|
931
|
+
|
|
932
|
+
```bash
|
|
933
|
+
rails generate rails_proof:test app/controllers/posts_controller.rb
|
|
934
|
+
```
|
|
935
|
+
|
|
936
|
+
### All controllers
|
|
937
|
+
|
|
938
|
+
```bash
|
|
939
|
+
rails generate rails_proof:test app/controllers
|
|
940
|
+
```
|
|
941
|
+
|
|
942
|
+
RailsProof currently discovers supported targets beneath:
|
|
943
|
+
|
|
944
|
+
```text
|
|
945
|
+
app/models
|
|
946
|
+
app/controllers
|
|
947
|
+
```
|
|
948
|
+
|
|
949
|
+
Base framework files and concern directories are ignored.
|
|
950
|
+
|
|
951
|
+
---
|
|
952
|
+
|
|
953
|
+
## Generated Tests
|
|
954
|
+
|
|
955
|
+
RailsProof generates ordinary Minitest tests.
|
|
956
|
+
|
|
957
|
+
A generated model test looks like normal Rails code:
|
|
958
|
+
|
|
959
|
+
```ruby
|
|
960
|
+
class PostTest < ActiveSupport::TestCase
|
|
961
|
+
end
|
|
962
|
+
```
|
|
963
|
+
|
|
964
|
+
Controller tests use:
|
|
965
|
+
|
|
966
|
+
```ruby
|
|
967
|
+
class PostsControllerTest < ActionDispatch::IntegrationTest
|
|
968
|
+
end
|
|
969
|
+
```
|
|
970
|
+
|
|
971
|
+
There is no RailsProof-specific runtime DSL inside generated tests.
|
|
972
|
+
|
|
973
|
+
After a test is kept, developers can:
|
|
974
|
+
|
|
975
|
+
- read it
|
|
976
|
+
- edit it
|
|
977
|
+
- move it
|
|
978
|
+
- run it directly
|
|
979
|
+
- maintain it like any other Rails test
|
|
980
|
+
|
|
981
|
+
The generated suite remains a normal Rails test suite.
|
|
982
|
+
|
|
983
|
+
---
|
|
984
|
+
|
|
985
|
+
## Why Minitest?
|
|
986
|
+
|
|
987
|
+
RailsProof currently targets the test framework Rails ships with by default:
|
|
988
|
+
|
|
989
|
+
**Minitest.**
|
|
990
|
+
|
|
991
|
+
The goal is not to create a RailsProof-specific testing framework.
|
|
992
|
+
|
|
993
|
+
The goal is to produce tests a Rails developer already understands.
|
|
994
|
+
|
|
995
|
+
That also keeps generated coverage useful even if RailsProof itself is later removed from the application.
|
|
996
|
+
|
|
997
|
+
---
|
|
998
|
+
|
|
999
|
+
## Current Limitations
|
|
1000
|
+
|
|
1001
|
+
RailsProof is still young.
|
|
1002
|
+
|
|
1003
|
+
Current limitations include:
|
|
1004
|
+
|
|
1005
|
+
- deterministic model analysis covers only a subset of Rails validations and behaviors
|
|
1006
|
+
- deterministic controller analysis currently focuses primarily on routed response behavior
|
|
1007
|
+
- AI behavior identity is intentionally conservative and will continue to evolve as more real-world cases are discovered
|
|
1008
|
+
- generated tests can still contain logically incorrect expectations
|
|
1009
|
+
- complex setup may require more application context than RailsProof currently supplies
|
|
1010
|
+
- the human review workflow currently persists findings but does not yet provide a complete review-management command
|
|
1011
|
+
- review findings do not yet have a full accepted/rejected/resolved lifecycle
|
|
1012
|
+
- jobs, mailers, services, channels, system tests, and other Rails components are not yet first-class targets
|
|
1013
|
+
- OpenAI is currently the first implemented AI provider adapter
|
|
1014
|
+
- configuration is still minimal
|
|
1015
|
+
- APIs and generated output may change as RailsProof evolves
|
|
1016
|
+
|
|
1017
|
+
Use Git and review generated changes.
|
|
1018
|
+
|
|
1019
|
+
---
|
|
1020
|
+
|
|
1021
|
+
## Roadmap
|
|
1022
|
+
|
|
1023
|
+
Near-term development includes:
|
|
1024
|
+
|
|
1025
|
+
- human review commands and workflow
|
|
1026
|
+
- review resolution states
|
|
1027
|
+
- retrying or accepting saved review candidates
|
|
1028
|
+
- richer existing-test analysis
|
|
1029
|
+
- additional model validations and associations
|
|
1030
|
+
- deeper controller behavior analysis
|
|
1031
|
+
- improved context gathering for application-specific code
|
|
1032
|
+
- generated-test failure analysis and repair
|
|
1033
|
+
- regression detection when previously passing generated tests begin failing
|
|
1034
|
+
- jobs
|
|
1035
|
+
- mailers
|
|
1036
|
+
- service objects
|
|
1037
|
+
- additional Rails components
|
|
1038
|
+
- additional AI providers
|
|
1039
|
+
- provider configuration
|
|
1040
|
+
- improved command output and reporting
|
|
1041
|
+
|
|
1042
|
+
Deterministic AI suggestion deduplication and source-aware convergence are already implemented.
|
|
1043
|
+
|
|
1044
|
+
The long-term goal is straightforward:
|
|
1045
|
+
|
|
1046
|
+
> Point RailsProof at a Rails application and let it continuously build and maintain meaningful test coverage as the application evolves.
|
|
1047
|
+
|
|
1048
|
+
---
|
|
1049
|
+
|
|
1050
|
+
## Development
|
|
1051
|
+
|
|
1052
|
+
Clone the repository and install dependencies:
|
|
1053
|
+
|
|
1054
|
+
```bash
|
|
1055
|
+
bundle install
|
|
1056
|
+
```
|
|
1057
|
+
|
|
1058
|
+
Run the RailsProof test suite:
|
|
1059
|
+
|
|
1060
|
+
```bash
|
|
1061
|
+
bin/test
|
|
1062
|
+
```
|
|
1063
|
+
|
|
1064
|
+
Individual tests can also be run directly:
|
|
1065
|
+
|
|
1066
|
+
```bash
|
|
1067
|
+
bin/test test/rails_proof/model_inspector_test.rb
|
|
1068
|
+
```
|
|
1069
|
+
|
|
1070
|
+
RailsProof includes a dummy Rails application under:
|
|
1071
|
+
|
|
1072
|
+
```text
|
|
1073
|
+
test/dummy
|
|
1074
|
+
```
|
|
1075
|
+
|
|
1076
|
+
which is used for integration testing against real Rails models, controllers, routes, generated tests, AI candidate execution, rollback, review persistence, and convergence behavior.
|
|
1077
|
+
|
|
1078
|
+
Dummy application tests can be run explicitly, for example:
|
|
1079
|
+
|
|
1080
|
+
```bash
|
|
1081
|
+
bin/test test/dummy/test/models/post_test.rb
|
|
1082
|
+
```
|
|
1083
|
+
|
|
1084
|
+
---
|
|
1085
|
+
|
|
1086
|
+
## Testing Philosophy
|
|
1087
|
+
|
|
1088
|
+
RailsProof itself is tested against both deterministic fixtures and deliberately tricky AI-generation scenarios.
|
|
1089
|
+
|
|
1090
|
+
Important behavior is tested at multiple layers:
|
|
1091
|
+
|
|
1092
|
+
```text
|
|
1093
|
+
unit behavior
|
|
1094
|
+
|
|
|
1095
|
+
v
|
|
1096
|
+
filter / executor behavior
|
|
1097
|
+
|
|
|
1098
|
+
v
|
|
1099
|
+
generator integration
|
|
1100
|
+
|
|
|
1101
|
+
v
|
|
1102
|
+
real dummy Rails application
|
|
1103
|
+
```
|
|
1104
|
+
|
|
1105
|
+
The dummy application is useful for deliberately introducing plausible bugs and verifying the complete RailsProof lifecycle.
|
|
1106
|
+
|
|
1107
|
+
For example:
|
|
1108
|
+
|
|
1109
|
+
```text
|
|
1110
|
+
introduce implementation bug
|
|
1111
|
+
|
|
|
1112
|
+
v
|
|
1113
|
+
run RailsProof
|
|
1114
|
+
|
|
|
1115
|
+
v
|
|
1116
|
+
contract check generated
|
|
1117
|
+
|
|
|
1118
|
+
v
|
|
1119
|
+
candidate fails
|
|
1120
|
+
|
|
|
1121
|
+
v
|
|
1122
|
+
NEEDS REVIEW
|
|
1123
|
+
|
|
|
1124
|
+
v
|
|
1125
|
+
run RailsProof again
|
|
1126
|
+
|
|
|
1127
|
+
v
|
|
1128
|
+
same finding recognized
|
|
1129
|
+
|
|
|
1130
|
+
v
|
|
1131
|
+
SKIPPED
|
|
1132
|
+
|
|
|
1133
|
+
v
|
|
1134
|
+
fix application
|
|
1135
|
+
|
|
|
1136
|
+
v
|
|
1137
|
+
source fingerprint changes
|
|
1138
|
+
|
|
|
1139
|
+
v
|
|
1140
|
+
RailsProof analyzes again
|
|
1141
|
+
|
|
|
1142
|
+
v
|
|
1143
|
+
passing coverage KEPT
|
|
1144
|
+
```
|
|
1145
|
+
|
|
1146
|
+
The intent is to test RailsProof against the messy cases produced by real nondeterministic AI output, not only idealized fixtures.
|
|
1147
|
+
|
|
1148
|
+
---
|
|
1149
|
+
|
|
1150
|
+
## Support
|
|
1151
|
+
|
|
1152
|
+
Questions about RailsProof? Something not working? Found behavior you aren't sure is a bug?
|
|
1153
|
+
|
|
1154
|
+
**Visit https://support.oakharborventures.com**
|
|
1155
|
+
|
|
1156
|
+
Ask a question, report a problem, tell us what RailsProof got wrong, or just come talk to us about what you're building.
|
|
1157
|
+
|
|
1158
|
+
We'd much rather hear from you than have you spend an afternoon fighting with RailsProof by yourself.
|
|
1159
|
+
|
|
1160
|
+
---
|
|
1161
|
+
|
|
1162
|
+
## Contributing
|
|
1163
|
+
|
|
1164
|
+
RailsProof is in active development, and bug reports, test cases, design discussion, and pull requests are welcome.
|
|
1165
|
+
|
|
1166
|
+
For changes to test generation or deduplication behavior, please include tests demonstrating both:
|
|
1167
|
+
|
|
1168
|
+
- behavior that RailsProof should recognize as equivalent
|
|
1169
|
+
- behavior that RailsProof must keep distinct
|
|
1170
|
+
|
|
1171
|
+
False-positive deduplication is especially important to avoid because silently skipping legitimate coverage is worse than generating an occasional duplicate.
|
|
1172
|
+
|
|
1173
|
+
Because RailsProof can modify application test files, changes involving generation, AI output, validation, execution, rollback, review persistence, or deduplication should be treated carefully and tested thoroughly.
|
|
1174
|
+
|
|
1175
|
+
---
|
|
1176
|
+
|
|
1177
|
+
## Philosophy
|
|
1178
|
+
|
|
1179
|
+
RailsProof is not intended to replace developers or make test design invisible.
|
|
1180
|
+
|
|
1181
|
+
It is intended to eliminate the repetitive part of keeping a Rails application tested while still producing ordinary, understandable test code.
|
|
1182
|
+
|
|
1183
|
+
Rails already knows a tremendous amount about an application.
|
|
1184
|
+
|
|
1185
|
+
AI can reason about behavior that Rails cannot describe through reflection alone.
|
|
1186
|
+
|
|
1187
|
+
The test runner can determine whether a generated expectation agrees with the current implementation.
|
|
1188
|
+
|
|
1189
|
+
Deterministic safeguards can prevent nondeterministic AI output from repeatedly creating the same work.
|
|
1190
|
+
|
|
1191
|
+
Human review can resolve the cases where implementation and apparent contract disagree.
|
|
1192
|
+
|
|
1193
|
+
RailsProof brings those pieces together.
|
|
1194
|
+
|
|
1195
|
+
The principle is simple:
|
|
1196
|
+
|
|
1197
|
+
```text
|
|
1198
|
+
Do not blindly trust the AI.
|
|
1199
|
+
Do not blindly trust the implementation.
|
|
1200
|
+
Do not throw away useful disagreements.
|
|
1201
|
+
Do not generate the same work forever.
|
|
1202
|
+
```
|
|
1203
|
+
|
|
1204
|
+
**You write the Rails app. We write the tests.**
|
|
1205
|
+
|
|
1206
|
+
---
|
|
1207
|
+
|
|
1208
|
+
## License
|
|
1209
|
+
|
|
1210
|
+
RailsProof is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
|