rails_proof 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md ADDED
@@ -0,0 +1,1210 @@
1
+ # RailsProof
2
+
3
+ > **You write the Rails app. We write the tests.**
4
+
5
+ RailsProof analyzes a Rails application, discovers behavior that should be tested, writes ordinary Minitest tests, and verifies generated tests against the application before deciding what to keep.
6
+
7
+ It combines two approaches:
8
+
9
+ 1. **Deterministic Rails inspection** for behavior RailsProof can identify with confidence.
10
+ 2. **AI-assisted analysis** for application-specific behavior that a fixed rule engine cannot reasonably anticipate.
11
+
12
+ RailsProof is being developed for modern Rails applications, with current development targeting **Rails 8.1+** and **Ruby 4.0+**.
13
+
14
+ > **Status: Early development**
15
+ >
16
+ > RailsProof is under active development ahead of its first public release. Its behavior and public API may change. Use version control when trying it on real applications.
17
+
18
+ Need help, found something weird, or just want to talk about RailsProof?
19
+
20
+ **Visit us at https://support.oakharborventures.com**
21
+
22
+ ---
23
+
24
+ ## What RailsProof Does
25
+
26
+ Given a model like:
27
+
28
+ ```ruby
29
+ class Post < ApplicationRecord
30
+ belongs_to :user
31
+
32
+ validates :title, presence: true
33
+
34
+ def title_matches?(query)
35
+ return false if query.blank?
36
+
37
+ title.to_s.downcase.include?(query.to_s.downcase)
38
+ end
39
+ end
40
+ ```
41
+
42
+ RailsProof can deterministically identify:
43
+
44
+ ```text
45
+ belongs_to :user
46
+ validates presence of title
47
+ ```
48
+
49
+ It can then use AI to analyze custom application behavior and discover useful coverage such as:
50
+
51
+ ```text
52
+ title_matches? returns false for blank queries
53
+ title_matches? performs a case insensitive substring match
54
+ title_matches? returns false when the query is absent from the title
55
+ title_matches? handles a nil title
56
+ ```
57
+
58
+ RailsProof generates candidate Minitest code, validates it, inserts one candidate at a time, and runs the actual Rails test suite against that candidate.
59
+
60
+ A candidate that passes can become permanent coverage.
61
+
62
+ A candidate that fails is **not automatically assumed to be wrong**.
63
+
64
+ That distinction matters.
65
+
66
+ A failing generated test may mean:
67
+
68
+ - the generated test is incorrect
69
+ - the application contains a bug
70
+ - the test exposed a contract mismatch
71
+ - the application and candidate simply disagree about intended behavior
72
+
73
+ RailsProof preserves that distinction instead of silently throwing useful evidence away.
74
+
75
+ ---
76
+
77
+ ## Quick Start
78
+
79
+ Run RailsProof against the entire supported application:
80
+
81
+ ```bash
82
+ rails generate rails_proof:test
83
+ ```
84
+
85
+ Inspect a specific model:
86
+
87
+ ```bash
88
+ rails generate rails_proof:test app/models/post.rb
89
+ ```
90
+
91
+ Inspect all models:
92
+
93
+ ```bash
94
+ rails generate rails_proof:test app/models
95
+ ```
96
+
97
+ Inspect a controller:
98
+
99
+ ```bash
100
+ rails generate rails_proof:test app/controllers/posts_controller.rb
101
+ ```
102
+
103
+ Or inspect all controllers:
104
+
105
+ ```bash
106
+ rails generate rails_proof:test app/controllers
107
+ ```
108
+
109
+ There is no separate AI mode and no `--ai` flag.
110
+
111
+ AI analysis is part of the normal RailsProof workflow.
112
+
113
+ ---
114
+
115
+ ## How It Works
116
+
117
+ RailsProof deliberately does not hand unrestricted control of your test files to an AI model.
118
+
119
+ The workflow looks like this:
120
+
121
+ ```text
122
+ Rails application
123
+ |
124
+ v
125
+ Target discovery
126
+ |
127
+ v
128
+ Rails runtime/source inspection
129
+ |
130
+ v
131
+ Deterministic test planning
132
+ |
133
+ v
134
+ Existing test analysis
135
+ |
136
+ v
137
+ AI behavior analysis
138
+ |
139
+ v
140
+ Deterministic AI deduplication
141
+ |
142
+ v
143
+ Candidate Minitest generation
144
+ |
145
+ v
146
+ Syntax and structure validation
147
+ |
148
+ v
149
+ Insert one candidate
150
+ |
151
+ v
152
+ Run the actual test
153
+ |
154
+ +---- PASS -----------------> KEPT
155
+ |
156
+ +---- FAIL -----------------> restore live test file
157
+ |
158
+ v
159
+ NEEDS REVIEW
160
+ |
161
+ v
162
+ persist review evidence
163
+ ```
164
+
165
+ Malformed or unusable generated tests are:
166
+
167
+ ```text
168
+ REJECTED
169
+ ```
170
+
171
+ Tests already covered, duplicated in the same AI response, or already represented by an unresolved review finding are:
172
+
173
+ ```text
174
+ SKIPPED
175
+ ```
176
+
177
+ RailsProof owns the file mutation, validation, execution, rollback, review storage, and deduplication process.
178
+
179
+ AI supplies analysis and candidate tests.
180
+
181
+ The Rails test runner determines whether a candidate agrees with the application as it currently exists.
182
+
183
+ The developer remains responsible for deciding whether a disagreement represents an application bug or an incorrect expectation.
184
+
185
+ ---
186
+
187
+ ## Result States
188
+
189
+ AI-generated candidate tests currently have four possible outcomes.
190
+
191
+ ### KEPT
192
+
193
+ ```text
194
+ KEPT: title_matches? handles a nil title
195
+ ```
196
+
197
+ The candidate was structurally valid and passed against the application.
198
+
199
+ RailsProof leaves the test in the live test suite.
200
+
201
+ A passing test proves that the candidate expectation is compatible with the current implementation. It does not, by itself, prove that the implementation is correct.
202
+
203
+ ### NEEDS REVIEW
204
+
205
+ ```text
206
+ NEEDS REVIEW: title_matches_exactly? rejects partial title matches
207
+ candidate test failed against application
208
+ Review saved: .rails_proof/review/...
209
+ ```
210
+
211
+ The candidate was structurally valid but failed against the application.
212
+
213
+ RailsProof:
214
+
215
+ 1. restores the live test file to its previous state
216
+ 2. preserves the candidate test
217
+ 3. preserves the reason it was generated
218
+ 4. preserves the test-run failure output
219
+ 5. records the finding for human review
220
+
221
+ A failing valid test is evidence of a disagreement, not proof that the generated test is bad.
222
+
223
+ ### REJECTED
224
+
225
+ ```text
226
+ REJECTED: malformed generated test
227
+ test code is not valid Ruby
228
+ ```
229
+
230
+ The generated candidate itself is unusable.
231
+
232
+ Examples include:
233
+
234
+ - invalid Ruby
235
+ - missing or multiple test declarations
236
+ - class declarations inside the candidate
237
+ - `require` statements
238
+ - Markdown code fences
239
+ - other structurally invalid generated output
240
+
241
+ Rejected tests are not preserved as application-bug findings because RailsProof could not establish that they were valid candidate tests.
242
+
243
+ ### SKIPPED
244
+
245
+ ```text
246
+ SKIPPED: title_matches_exactly? requires the whole title to match
247
+ already awaiting human review
248
+ ```
249
+
250
+ RailsProof determined that executing the candidate would add no useful new information.
251
+
252
+ Current skip reasons include:
253
+
254
+ ```text
255
+ already exists in test suite
256
+ already awaiting human review
257
+ duplicate AI suggestion
258
+ ```
259
+
260
+ ---
261
+
262
+ ## Contract Checks
263
+
264
+ RailsProof does not assume that the current implementation is automatically the intended contract.
265
+
266
+ This is especially important for AI-assisted testing.
267
+
268
+ Consider:
269
+
270
+ ```ruby
271
+ def title_matches_exactly?(query)
272
+ return false if query.blank?
273
+
274
+ title.to_s.downcase.include?(query.to_s.downcase)
275
+ end
276
+ ```
277
+
278
+ The method is named:
279
+
280
+ ```text
281
+ title_matches_exactly?
282
+ ```
283
+
284
+ but its implementation performs a substring match with:
285
+
286
+ ```ruby
287
+ include?
288
+ ```
289
+
290
+ A test generator that blindly treats implementation as specification might generate tests proving that partial matches succeed and permanently encode the bug into the test suite.
291
+
292
+ RailsProof's AI protocol distinguishes two kinds of suggestions.
293
+
294
+ ### coverage
295
+
296
+ A `coverage` suggestion tests behavior where the implementation and apparent contract agree.
297
+
298
+ For example:
299
+
300
+ ```text
301
+ title_matches? returns false for blank queries
302
+ ```
303
+
304
+ ### contract_check
305
+
306
+ A `contract_check` is used when strong evidence in the source suggests that the apparent public contract and implementation disagree.
307
+
308
+ For example:
309
+
310
+ ```text
311
+ title_matches_exactly? rejects partial title matches
312
+ ```
313
+
314
+ RailsProof may generate:
315
+
316
+ ```ruby
317
+ test "title_matches_exactly? rejects partial title matches" do
318
+ post = Post.new(title: "Learning Rails")
319
+
320
+ assert_not post.title_matches_exactly?("Rails")
321
+ end
322
+ ```
323
+
324
+ If the implementation uses `include?`, that test fails.
325
+
326
+ RailsProof does not delete the evidence and does not leave the failing test in the live suite.
327
+
328
+ It produces:
329
+
330
+ ```text
331
+ NEEDS REVIEW
332
+ ```
333
+
334
+ That gives the developer a concrete test, a reason, and the actual failure output to evaluate.
335
+
336
+ ---
337
+
338
+ ## Human Review
339
+
340
+ RailsProof treats failing valid generated tests differently from malformed generated tests.
341
+
342
+ That distinction is intentional.
343
+
344
+ Suppose RailsProof finds:
345
+
346
+ ```text
347
+ Method: title_matches_exactly?
348
+ Implementation: include?
349
+ ```
350
+
351
+ and generates a test requiring a full-title match.
352
+
353
+ If the test fails, RailsProof cannot safely conclude:
354
+
355
+ ```text
356
+ the AI was wrong
357
+ ```
358
+
359
+ It also cannot safely conclude:
360
+
361
+ ```text
362
+ the application is wrong
363
+ ```
364
+
365
+ Instead it records:
366
+
367
+ ```text
368
+ application behavior and candidate expectation disagree
369
+ ```
370
+
371
+ and asks for human judgment.
372
+
373
+ Review findings are currently stored beneath:
374
+
375
+ ```text
376
+ .rails_proof/review/
377
+ ```
378
+
379
+ A review record contains information such as:
380
+
381
+ ```text
382
+ status
383
+ created_at
384
+ last_seen_at
385
+ occurrences
386
+ target_path
387
+ target_fingerprint
388
+ test_file_path
389
+ test_class_name
390
+ kind
391
+ name
392
+ reason
393
+ test_code
394
+ test_fingerprint
395
+ test_output
396
+ ```
397
+
398
+ The review workflow is still evolving, but the underlying evidence is already preserved rather than discarded.
399
+
400
+ ---
401
+
402
+ ## Review Deduplication and Convergence
403
+
404
+ AI output is nondeterministic.
405
+
406
+ The same underlying behavior might be described differently on separate runs:
407
+
408
+ ```text
409
+ title_matches_exactly? requires a case-insensitive full-title match
410
+ ```
411
+
412
+ then:
413
+
414
+ ```text
415
+ title_matches_exactly? requires the entire title to match
416
+ ```
417
+
418
+ then:
419
+
420
+ ```text
421
+ title_matches_exactly? rejects partial title matches
422
+ ```
423
+
424
+ RailsProof does not rely only on those human-readable names to determine identity.
425
+
426
+ Its deterministic deduplication layer examines candidate behavior so differently worded suggestions can still be recognized as the same underlying finding.
427
+
428
+ That means repeated runs against unchanged source converge instead of endlessly creating duplicate tests or duplicate review findings.
429
+
430
+ For an unresolved finding:
431
+
432
+ ```text
433
+ first run
434
+ -> NEEDS REVIEW
435
+ -> review evidence stored
436
+
437
+ later run against unchanged source
438
+ -> same underlying behavior recognized
439
+ -> SKIPPED
440
+ -> already awaiting human review
441
+ ```
442
+
443
+ RailsProof also fingerprints the target source associated with a review finding.
444
+
445
+ If the application source changes, an old review finding no longer automatically suppresses new testing.
446
+
447
+ ```text
448
+ unchanged source + same finding
449
+ -> SKIPPED
450
+
451
+ changed source
452
+ -> eligible for analysis and execution again
453
+ ```
454
+
455
+ This allows a natural development loop:
456
+
457
+ ```text
458
+ RailsProof discovers disagreement
459
+ |
460
+ v
461
+ NEEDS REVIEW
462
+ |
463
+ v
464
+ developer evaluates finding
465
+ |
466
+ v
467
+ application code changes
468
+ |
469
+ v
470
+ source fingerprint changes
471
+ |
472
+ v
473
+ RailsProof analyzes behavior again
474
+ |
475
+ v
476
+ candidate passes
477
+ |
478
+ v
479
+ KEPT
480
+ ```
481
+
482
+ ---
483
+
484
+ ## Setup-Sensitive Deduplication
485
+
486
+ Two tests can contain the same assertion without testing the same behavior.
487
+
488
+ For example:
489
+
490
+ ```ruby
491
+ post = Post.new(title: "Learning Rails")
492
+
493
+ assert_not post.title_matches_exactly?("Rails")
494
+ ```
495
+
496
+ and:
497
+
498
+ ```ruby
499
+ post = Post.new(title: nil)
500
+
501
+ assert_not post.title_matches_exactly?("Rails")
502
+ ```
503
+
504
+ have the same assertion text but different setup and therefore test different behavior.
505
+
506
+ RailsProof's AI deduplication accounts for setup context when comparing meaningful candidate behavior.
507
+
508
+ This prevents an important class of false-positive deduplication where legitimate coverage could otherwise be silently discarded.
509
+
510
+ RailsProof therefore aims to be conservative about skipping tests:
511
+
512
+ ```text
513
+ same assertion + different setup
514
+ != same behavior
515
+ ```
516
+
517
+ ---
518
+
519
+ ## Example
520
+
521
+ A normal successful RailsProof run can look like:
522
+
523
+ ```text
524
+ RailsProof targets: 1
525
+ RailsProof inspection
526
+ Model file: app/models/post.rb
527
+ Model class: Post
528
+ Test file: test/models/post_test.rb
529
+ Test status: exists
530
+ Test cases: 7
531
+
532
+ Source associations: 1
533
+ belongs_to :user
534
+
535
+ Source validations: 1
536
+ validates :title, presence: true
537
+
538
+ Runtime inspection: available
539
+ Table: posts
540
+
541
+ Suggested tests: 2
542
+ belongs_to :user
543
+ validates presence of title
544
+
545
+ Coverage:
546
+ Covered: 2
547
+ Missing: 0
548
+
549
+ AI suggested tests: 3
550
+ title_matches? returns false for blank queries
551
+ title_matches? performs a case insensitive substring match
552
+ title_matches? handles a nil title
553
+
554
+ AI test results: 3
555
+ KEPT: title_matches? returns false for blank queries
556
+ KEPT: title_matches? performs a case insensitive substring match
557
+ KEPT: title_matches? handles a nil title
558
+ ```
559
+
560
+ A run that discovers a possible application bug can look like:
561
+
562
+ ```text
563
+ AI suggested tests: 1
564
+ title_matches_exactly? rejects partial title matches
565
+ Reason: The method name strongly indicates equality semantics,
566
+ while the implementation performs substring matching.
567
+
568
+ AI test results: 1
569
+ NEEDS REVIEW: title_matches_exactly? rejects partial title matches
570
+ candidate test failed against application
571
+ Review saved: .rails_proof/review/...
572
+ ```
573
+
574
+ Running RailsProof again without changing the application can then produce:
575
+
576
+ ```text
577
+ AI test results: 1
578
+ SKIPPED: title_matches_exactly? requires the whole title to match
579
+ already awaiting human review
580
+ ```
581
+
582
+ even though the AI described the finding differently.
583
+
584
+ ---
585
+
586
+ ## Deterministic Analysis
587
+
588
+ RailsProof first handles the parts of a Rails application that can be understood without AI.
589
+
590
+ ### Models
591
+
592
+ Current deterministic model inspection includes:
593
+
594
+ - Active Record columns
595
+ - associations
596
+ - presence validations
597
+ - existing Minitest tests
598
+ - missing deterministic coverage
599
+
600
+ For example:
601
+
602
+ ```ruby
603
+ class Post < ApplicationRecord
604
+ belongs_to :user
605
+
606
+ validates :title, presence: true
607
+ end
608
+ ```
609
+
610
+ can produce:
611
+
612
+ ```ruby
613
+ test "belongs to user" do
614
+ association = Post.reflect_on_association(:user)
615
+
616
+ assert_not_nil association
617
+ assert_equal :belongs_to, association.macro
618
+ end
619
+
620
+ test "validates presence of title" do
621
+ record = Post.new(title: nil)
622
+
623
+ record.validate
624
+
625
+ assert record.errors.of_kind?(:title, :blank)
626
+ end
627
+ ```
628
+
629
+ RailsProof also avoids separately generating the implicit presence validation Rails adds for a required `belongs_to` association.
630
+
631
+ ### Controllers
632
+
633
+ RailsProof currently understands controller actions and their matching routes.
634
+
635
+ For example:
636
+
637
+ ```ruby
638
+ class PostsController < ApplicationController
639
+ def index
640
+ end
641
+
642
+ def show
643
+ end
644
+ end
645
+ ```
646
+
647
+ with matching routes can produce:
648
+
649
+ ```ruby
650
+ test "should get index" do
651
+ get posts_index_url
652
+
653
+ assert_response :success
654
+ end
655
+
656
+ test "should get show" do
657
+ get posts_show_url
658
+
659
+ assert_response :success
660
+ end
661
+ ```
662
+
663
+ Existing Rails-generated controller tests are recognized so RailsProof does not needlessly recreate them.
664
+
665
+ ---
666
+
667
+ ## Existing Tests
668
+
669
+ RailsProof is designed to work with an existing test suite rather than assume it is starting from scratch.
670
+
671
+ It currently recognizes:
672
+
673
+ - Rails-style `test "..." do` declarations
674
+ - method-style `def test_...` declarations
675
+
676
+ Existing tests are used for both deterministic and AI-assisted analysis.
677
+
678
+ RailsProof can:
679
+
680
+ - compare deterministic concerns against existing tests
681
+ - insert only missing deterministic tests
682
+ - provide existing tests to AI as context
683
+ - avoid AI tests already represented in the live suite
684
+ - preserve passing generated tests across later candidate failures
685
+ - avoid repeated AI suggestions within the same run
686
+ - avoid repeatedly executing unresolved review findings
687
+
688
+ RailsProof is intended to be run repeatedly as an application evolves, not merely once when the project is created.
689
+
690
+ ---
691
+
692
+ ## AI Analysis
693
+
694
+ Fixed rules can only take automatic test generation so far.
695
+
696
+ Consider:
697
+
698
+ ```ruby
699
+ def eligible_for_upgrade?
700
+ active? && account_age_days > 30 && !past_due?
701
+ end
702
+ ```
703
+
704
+ or:
705
+
706
+ ```ruby
707
+ def cancel!
708
+ transaction do
709
+ update!(status: :cancelled)
710
+ subscription&.cancel!
711
+ CancellationMailer.confirmation(self).deliver_later
712
+ end
713
+ end
714
+ ```
715
+
716
+ There is no general Rails reflection API that can determine every meaningful behavioral test for application-specific code like this.
717
+
718
+ That is where RailsProof uses AI.
719
+
720
+ The AI receives structured context including:
721
+
722
+ - target type
723
+ - class name
724
+ - application source
725
+ - existing tests
726
+ - deterministic concerns already discovered by RailsProof
727
+
728
+ It is instructed to avoid duplicating behavior already covered by deterministic analysis or existing tests.
729
+
730
+ It is also instructed not to assume that current implementation behavior is necessarily the intended public contract.
731
+
732
+ The AI returns structured suggestions containing:
733
+
734
+ - suggestion kind
735
+ - test name
736
+ - reason
737
+ - candidate Minitest code
738
+
739
+ Supported suggestion kinds currently include:
740
+
741
+ ```text
742
+ coverage
743
+ contract_check
744
+ ```
745
+
746
+ RailsProof independently validates, deduplicates, writes, and executes the returned candidate code.
747
+
748
+ The AI does not directly control application test files.
749
+
750
+ ---
751
+
752
+ ## AI Safety
753
+
754
+ Generated code is not blindly accepted.
755
+
756
+ Before an AI-generated test can execute, RailsProof currently checks that the candidate:
757
+
758
+ - is nonblank
759
+ - parses as valid Ruby
760
+ - contains exactly one Minitest test declaration
761
+ - does not contain a class declaration
762
+ - does not contain a `require` statement
763
+ - does not contain Markdown code fences
764
+
765
+ Candidates are evaluated one at a time.
766
+
767
+ Before each candidate, RailsProof captures the previous state of the test file.
768
+
769
+ If the candidate passes, it may remain in the suite.
770
+
771
+ If it fails, the previous file state is restored.
772
+
773
+ That means:
774
+
775
+ ```text
776
+ candidate 1 passes
777
+ -> kept
778
+
779
+ candidate 2 fails
780
+ -> rolled back
781
+
782
+ candidate 1 remains
783
+ ```
784
+
785
+ A later failure does not discard earlier successful coverage.
786
+
787
+ ---
788
+
789
+ ## Why a Passing Generated Test Is Not Enough
790
+
791
+ A generated test can pass and still be a bad test.
792
+
793
+ For example, if RailsProof sees:
794
+
795
+ ```ruby
796
+ def title_matches_exactly?(query)
797
+ title.include?(query)
798
+ end
799
+ ```
800
+
801
+ and generates:
802
+
803
+ ```ruby
804
+ assert post.title_matches_exactly?("Rails")
805
+ ```
806
+
807
+ for a title of:
808
+
809
+ ```text
810
+ Learning Rails
811
+ ```
812
+
813
+ the test passes.
814
+
815
+ But it may simply be locking an implementation bug into the suite.
816
+
817
+ That is why RailsProof distinguishes normal coverage from contract checks and why application behavior is not treated as unquestionable specification.
818
+
819
+ The goal is not:
820
+
821
+ ```text
822
+ generate tests that make the current code pass
823
+ ```
824
+
825
+ The goal is:
826
+
827
+ ```text
828
+ generate meaningful tests for the application's apparent contract
829
+ and surface disagreements that deserve human attention
830
+ ```
831
+
832
+ ---
833
+
834
+ ## Installation
835
+
836
+ RailsProof is currently under active development and has not yet reached its first public release.
837
+
838
+ For development from a local checkout, add RailsProof to the application's `Gemfile`:
839
+
840
+ ```ruby
841
+ gem "rails_proof", path: "../railsproof"
842
+ ```
843
+
844
+ Then run:
845
+
846
+ ```bash
847
+ bundle install
848
+ ```
849
+
850
+ After the public release, installation will be:
851
+
852
+ ```ruby
853
+ gem "rails_proof"
854
+ ```
855
+
856
+ followed by:
857
+
858
+ ```bash
859
+ bundle install
860
+ ```
861
+
862
+ The planned first public release is **RailsProof 1.0.0**.
863
+
864
+ ---
865
+
866
+ ## AI Provider Setup
867
+
868
+ RailsProof's core AI interface is provider-agnostic.
869
+
870
+ The first implemented provider adapter uses OpenAI.
871
+
872
+ Applications using the OpenAI adapter currently need the OpenAI Ruby SDK available:
873
+
874
+ ```ruby
875
+ gem "openai"
876
+ ```
877
+
878
+ Then run:
879
+
880
+ ```bash
881
+ bundle install
882
+ ```
883
+
884
+ Set your API key through the environment:
885
+
886
+ ```bash
887
+ export OPENAI_API_KEY="your-key-here"
888
+ ```
889
+
890
+ Do not commit API keys to your repository.
891
+
892
+ RailsProof currently defaults to:
893
+
894
+ ```text
895
+ gpt-5.6
896
+ ```
897
+
898
+ The model can be overridden with:
899
+
900
+ ```bash
901
+ export RAILSPROOF_OPENAI_MODEL="your-model"
902
+ ```
903
+
904
+ AI usage is performed using your provider credentials and may incur charges from that provider.
905
+
906
+ RailsProof's own test suite does not require paid AI calls. Development tests use a fake AI client.
907
+
908
+ ---
909
+
910
+ ## Running RailsProof
911
+
912
+ ### Entire supported application
913
+
914
+ ```bash
915
+ rails generate rails_proof:test
916
+ ```
917
+
918
+ ### One model
919
+
920
+ ```bash
921
+ rails generate rails_proof:test app/models/post.rb
922
+ ```
923
+
924
+ ### All models
925
+
926
+ ```bash
927
+ rails generate rails_proof:test app/models
928
+ ```
929
+
930
+ ### One controller
931
+
932
+ ```bash
933
+ rails generate rails_proof:test app/controllers/posts_controller.rb
934
+ ```
935
+
936
+ ### All controllers
937
+
938
+ ```bash
939
+ rails generate rails_proof:test app/controllers
940
+ ```
941
+
942
+ RailsProof currently discovers supported targets beneath:
943
+
944
+ ```text
945
+ app/models
946
+ app/controllers
947
+ ```
948
+
949
+ Base framework files and concern directories are ignored.
950
+
951
+ ---
952
+
953
+ ## Generated Tests
954
+
955
+ RailsProof generates ordinary Minitest tests.
956
+
957
+ A generated model test looks like normal Rails code:
958
+
959
+ ```ruby
960
+ class PostTest < ActiveSupport::TestCase
961
+ end
962
+ ```
963
+
964
+ Controller tests use:
965
+
966
+ ```ruby
967
+ class PostsControllerTest < ActionDispatch::IntegrationTest
968
+ end
969
+ ```
970
+
971
+ There is no RailsProof-specific runtime DSL inside generated tests.
972
+
973
+ After a test is kept, developers can:
974
+
975
+ - read it
976
+ - edit it
977
+ - move it
978
+ - run it directly
979
+ - maintain it like any other Rails test
980
+
981
+ The generated suite remains a normal Rails test suite.
982
+
983
+ ---
984
+
985
+ ## Why Minitest?
986
+
987
+ RailsProof currently targets the test framework Rails ships with by default:
988
+
989
+ **Minitest.**
990
+
991
+ The goal is not to create a RailsProof-specific testing framework.
992
+
993
+ The goal is to produce tests a Rails developer already understands.
994
+
995
+ That also keeps generated coverage useful even if RailsProof itself is later removed from the application.
996
+
997
+ ---
998
+
999
+ ## Current Limitations
1000
+
1001
+ RailsProof is still young.
1002
+
1003
+ Current limitations include:
1004
+
1005
+ - deterministic model analysis covers only a subset of Rails validations and behaviors
1006
+ - deterministic controller analysis currently focuses primarily on routed response behavior
1007
+ - AI behavior identity is intentionally conservative and will continue to evolve as more real-world cases are discovered
1008
+ - generated tests can still contain logically incorrect expectations
1009
+ - complex setup may require more application context than RailsProof currently supplies
1010
+ - the human review workflow currently persists findings but does not yet provide a complete review-management command
1011
+ - review findings do not yet have a full accepted/rejected/resolved lifecycle
1012
+ - jobs, mailers, services, channels, system tests, and other Rails components are not yet first-class targets
1013
+ - OpenAI is currently the first implemented AI provider adapter
1014
+ - configuration is still minimal
1015
+ - APIs and generated output may change as RailsProof evolves
1016
+
1017
+ Use Git and review generated changes.
1018
+
1019
+ ---
1020
+
1021
+ ## Roadmap
1022
+
1023
+ Near-term development includes:
1024
+
1025
+ - human review commands and workflow
1026
+ - review resolution states
1027
+ - retrying or accepting saved review candidates
1028
+ - richer existing-test analysis
1029
+ - additional model validations and associations
1030
+ - deeper controller behavior analysis
1031
+ - improved context gathering for application-specific code
1032
+ - generated-test failure analysis and repair
1033
+ - regression detection when previously passing generated tests begin failing
1034
+ - jobs
1035
+ - mailers
1036
+ - service objects
1037
+ - additional Rails components
1038
+ - additional AI providers
1039
+ - provider configuration
1040
+ - improved command output and reporting
1041
+
1042
+ Deterministic AI suggestion deduplication and source-aware convergence are already implemented.
1043
+
1044
+ The long-term goal is straightforward:
1045
+
1046
+ > Point RailsProof at a Rails application and let it continuously build and maintain meaningful test coverage as the application evolves.
1047
+
1048
+ ---
1049
+
1050
+ ## Development
1051
+
1052
+ Clone the repository and install dependencies:
1053
+
1054
+ ```bash
1055
+ bundle install
1056
+ ```
1057
+
1058
+ Run the RailsProof test suite:
1059
+
1060
+ ```bash
1061
+ bin/test
1062
+ ```
1063
+
1064
+ Individual tests can also be run directly:
1065
+
1066
+ ```bash
1067
+ bin/test test/rails_proof/model_inspector_test.rb
1068
+ ```
1069
+
1070
+ RailsProof includes a dummy Rails application under:
1071
+
1072
+ ```text
1073
+ test/dummy
1074
+ ```
1075
+
1076
+ which is used for integration testing against real Rails models, controllers, routes, generated tests, AI candidate execution, rollback, review persistence, and convergence behavior.
1077
+
1078
+ Dummy application tests can be run explicitly, for example:
1079
+
1080
+ ```bash
1081
+ bin/test test/dummy/test/models/post_test.rb
1082
+ ```
1083
+
1084
+ ---
1085
+
1086
+ ## Testing Philosophy
1087
+
1088
+ RailsProof itself is tested against both deterministic fixtures and deliberately tricky AI-generation scenarios.
1089
+
1090
+ Important behavior is tested at multiple layers:
1091
+
1092
+ ```text
1093
+ unit behavior
1094
+ |
1095
+ v
1096
+ filter / executor behavior
1097
+ |
1098
+ v
1099
+ generator integration
1100
+ |
1101
+ v
1102
+ real dummy Rails application
1103
+ ```
1104
+
1105
+ The dummy application is useful for deliberately introducing plausible bugs and verifying the complete RailsProof lifecycle.
1106
+
1107
+ For example:
1108
+
1109
+ ```text
1110
+ introduce implementation bug
1111
+ |
1112
+ v
1113
+ run RailsProof
1114
+ |
1115
+ v
1116
+ contract check generated
1117
+ |
1118
+ v
1119
+ candidate fails
1120
+ |
1121
+ v
1122
+ NEEDS REVIEW
1123
+ |
1124
+ v
1125
+ run RailsProof again
1126
+ |
1127
+ v
1128
+ same finding recognized
1129
+ |
1130
+ v
1131
+ SKIPPED
1132
+ |
1133
+ v
1134
+ fix application
1135
+ |
1136
+ v
1137
+ source fingerprint changes
1138
+ |
1139
+ v
1140
+ RailsProof analyzes again
1141
+ |
1142
+ v
1143
+ passing coverage KEPT
1144
+ ```
1145
+
1146
+ The intent is to test RailsProof against the messy cases produced by real nondeterministic AI output, not only idealized fixtures.
1147
+
1148
+ ---
1149
+
1150
+ ## Support
1151
+
1152
+ Questions about RailsProof? Something not working? Found behavior you aren't sure is a bug?
1153
+
1154
+ **Visit https://support.oakharborventures.com**
1155
+
1156
+ Ask a question, report a problem, tell us what RailsProof got wrong, or just come talk to us about what you're building.
1157
+
1158
+ We'd much rather hear from you than have you spend an afternoon fighting with RailsProof by yourself.
1159
+
1160
+ ---
1161
+
1162
+ ## Contributing
1163
+
1164
+ RailsProof is in active development, and bug reports, test cases, design discussion, and pull requests are welcome.
1165
+
1166
+ For changes to test generation or deduplication behavior, please include tests demonstrating both:
1167
+
1168
+ - behavior that RailsProof should recognize as equivalent
1169
+ - behavior that RailsProof must keep distinct
1170
+
1171
+ False-positive deduplication is especially important to avoid because silently skipping legitimate coverage is worse than generating an occasional duplicate.
1172
+
1173
+ Because RailsProof can modify application test files, changes involving generation, AI output, validation, execution, rollback, review persistence, or deduplication should be treated carefully and tested thoroughly.
1174
+
1175
+ ---
1176
+
1177
+ ## Philosophy
1178
+
1179
+ RailsProof is not intended to replace developers or make test design invisible.
1180
+
1181
+ It is intended to eliminate the repetitive part of keeping a Rails application tested while still producing ordinary, understandable test code.
1182
+
1183
+ Rails already knows a tremendous amount about an application.
1184
+
1185
+ AI can reason about behavior that Rails cannot describe through reflection alone.
1186
+
1187
+ The test runner can determine whether a generated expectation agrees with the current implementation.
1188
+
1189
+ Deterministic safeguards can prevent nondeterministic AI output from repeatedly creating the same work.
1190
+
1191
+ Human review can resolve the cases where implementation and apparent contract disagree.
1192
+
1193
+ RailsProof brings those pieces together.
1194
+
1195
+ The principle is simple:
1196
+
1197
+ ```text
1198
+ Do not blindly trust the AI.
1199
+ Do not blindly trust the implementation.
1200
+ Do not throw away useful disagreements.
1201
+ Do not generate the same work forever.
1202
+ ```
1203
+
1204
+ **You write the Rails app. We write the tests.**
1205
+
1206
+ ---
1207
+
1208
+ ## License
1209
+
1210
+ RailsProof is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).