ndr_dev_support 5.9.0 → 6.0.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +27 -0
- data/README.md +17 -1
- data/code_safety.yml +120 -104
- data/gemfiles/Gemfile.rails52 +0 -1
- data/gemfiles/Gemfile.rails60 +1 -2
- data/gemfiles/Gemfile.rails61 +4 -0
- data/lib/minitest/rake_ci_plugin.rb +1 -1
- data/lib/ndr_dev_support/integration_testing/flakey_tests.rb +12 -3
- data/lib/ndr_dev_support/rubocop/executor.rb +2 -2
- data/lib/ndr_dev_support/rubocop/inject.rb +9 -1
- data/lib/ndr_dev_support/rubocop/range_finder.rb +0 -1
- data/lib/ndr_dev_support/rubocop/reporter.rb +9 -8
- data/lib/ndr_dev_support/version.rb +1 -1
- data/lib/tasks/audit_code.rake +209 -136
- data/ndr_dev_support.gemspec +4 -6
- metadata +8 -13
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 5d8ff07e3e56e4665b16792797daed209e8842a69ea341959b6a6631eb8ced02
|
|
4
|
+
data.tar.gz: fcac18ced590c4d02cbe4664572a741f6ff439637064c5213af2d30b6d8c5efa
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: b85718ff1c1076d682c42b75c2257d89e40f0bb1b654c622a15006565b326499476a6d50d82c90bd6b0fcd8b5c70557d534a58866851364e256606b22a134e4a
|
|
7
|
+
data.tar.gz: e6fd89a89b2cae3cff85ea645feac1d9e26b6610305af90c4276d1b967352927272a62452adac95f271c22bf33a1e9b718100617a0f756d484d3f2db88a21da5
|
data/CHANGELOG.md
CHANGED
|
@@ -1,6 +1,33 @@
|
|
|
1
1
|
## [Unreleased]
|
|
2
2
|
*no unreleased changes*
|
|
3
3
|
|
|
4
|
+
## 6.0.1 / 2021-07-09
|
|
5
|
+
### Fixed
|
|
6
|
+
* Fix ruby warnings in CI plugin
|
|
7
|
+
* Remove inconsistent trailing whitespace in code_safety.yml
|
|
8
|
+
|
|
9
|
+
## 6.0.0 / 2021-02-25
|
|
10
|
+
### Changed
|
|
11
|
+
* Linting: return a zero exit status when there are no lintable changes (#97)
|
|
12
|
+
|
|
13
|
+
## 5.10.2 / 2021-02-15
|
|
14
|
+
### Fixed
|
|
15
|
+
* Fixed an issue using `flakey_test` with `minitest` v5.11 onwards
|
|
16
|
+
* Allow use with Rails 6.1
|
|
17
|
+
|
|
18
|
+
## 5.10.1 / 2021-02-01
|
|
19
|
+
### Fixed
|
|
20
|
+
* Fixed an issue with binary files breaking certain code audit modes (#94)
|
|
21
|
+
|
|
22
|
+
## 5.10.0 / 2021-01-29
|
|
23
|
+
### Added
|
|
24
|
+
* Added `test_repeatedly` for integration test debugging (#85)
|
|
25
|
+
* Added `outfile` and `filter` options to code auditing (#93)
|
|
26
|
+
|
|
27
|
+
### Fixed
|
|
28
|
+
* Fixed excessive `parser/current` warnings (#91)
|
|
29
|
+
* Improved performance of interactive code auditing (#93)
|
|
30
|
+
|
|
4
31
|
## 5.9.0 / 2021-01-12
|
|
5
32
|
### Added
|
|
6
33
|
* Move to keep up with Rubocop releases
|
data/README.md
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
## NdrDevSupport [](https://codeclimate.com/github/PublicHealthEngland/ndr_dev_support/maintainability) [](https://codeclimate.com/github/PublicHealthEngland/ndr_dev_support/maintainability) [](https://github.com/publichealthengland/ndr_dev_support/actions?query=workflow%3Atest) [](https://badge.fury.io/rb/ndr_dev_support)
|
|
2
2
|
|
|
3
3
|
This is the Public Health England (PHE) National Disease Registers (NDR) Developer Support ruby gem,
|
|
4
4
|
providing:
|
|
@@ -156,6 +156,22 @@ end
|
|
|
156
156
|
|
|
157
157
|
If tests still fail, they'll fail as normal. If tests pass after flakey failure, they'll be flagged to the RakeCI server, and rendered in purple on Slack.
|
|
158
158
|
|
|
159
|
+
#### Repeating Flakey Tests
|
|
160
|
+
|
|
161
|
+
To aid with investigations into potentially-flakey tests, `ndr_dev_support` also provides the ability to run an integration test repeatedly (by default, 100 times):
|
|
162
|
+
|
|
163
|
+
```ruby
|
|
164
|
+
test_repeatedly 'thing that we think might fail' do
|
|
165
|
+
# something flakey
|
|
166
|
+
end
|
|
167
|
+
|
|
168
|
+
test_repeatedly 'thing that we think might fail very occassionally', times: 1000 do
|
|
169
|
+
# something slightly flakey
|
|
170
|
+
end
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
This may be faster to work with than repeatedly executing the entire test runner in a bash loop, for example.
|
|
174
|
+
|
|
159
175
|
### Deployment support
|
|
160
176
|
|
|
161
177
|
There are various capistrano plugins in the `ndr_dev_support/capistrano` directory - see each one for details.
|
data/code_safety.yml
CHANGED
|
@@ -1,323 +1,339 @@
|
|
|
1
1
|
---
|
|
2
2
|
file safety:
|
|
3
|
+
".github/CODEOWNERS":
|
|
4
|
+
comments:
|
|
5
|
+
reviewed_by: josh.pencheon
|
|
6
|
+
safe_revision: ae4a81930ad3ab1b858474bc73b714aeed0f83a8
|
|
7
|
+
".github/workflows/lint.yml":
|
|
8
|
+
comments:
|
|
9
|
+
reviewed_by: josh.pencheon
|
|
10
|
+
safe_revision: 0ce43640c417e174054f903fd82043948ebe8ccb
|
|
11
|
+
".github/workflows/test.yml":
|
|
12
|
+
comments:
|
|
13
|
+
reviewed_by: joshpencheon
|
|
14
|
+
safe_revision: 366ef1122fa4502c25cba9e508c565989545ef25
|
|
3
15
|
".gitignore":
|
|
4
|
-
comments:
|
|
16
|
+
comments:
|
|
5
17
|
reviewed_by: josh.pencheon
|
|
6
18
|
safe_revision: 38266fd047417ecb4d5e6de71746f40495fc9de9
|
|
7
19
|
".hound.yml":
|
|
8
|
-
comments:
|
|
20
|
+
comments:
|
|
9
21
|
reviewed_by: timgentry
|
|
10
22
|
safe_revision: c03d4ff0dd037a2798563d464d0daeca1189debe
|
|
11
23
|
".rubocop.yml":
|
|
12
|
-
comments:
|
|
24
|
+
comments:
|
|
13
25
|
reviewed_by: josh.pencheon
|
|
14
26
|
safe_revision: 6211cff0ce44645ed3752723b5b0ee65f24c66aa
|
|
15
|
-
".travis.yml":
|
|
16
|
-
comments:
|
|
17
|
-
reviewed_by: josh.pencheon
|
|
18
|
-
safe_revision: e5ac2d160975264f40bac7e871b61aa6b54f7af2
|
|
19
27
|
CHANGELOG.md:
|
|
20
|
-
comments:
|
|
21
|
-
reviewed_by:
|
|
22
|
-
safe_revision:
|
|
28
|
+
comments:
|
|
29
|
+
reviewed_by: brian.shand
|
|
30
|
+
safe_revision: d4d4ea4d2dcdf68819e75cbdac1009b145740b22
|
|
23
31
|
CODE_OF_CONDUCT.md:
|
|
24
|
-
comments:
|
|
32
|
+
comments:
|
|
25
33
|
reviewed_by: timgentry
|
|
26
34
|
safe_revision: c59a45986f8b6d087c8c21b1e889f31f7346da17
|
|
27
35
|
Gemfile:
|
|
28
|
-
comments:
|
|
36
|
+
comments:
|
|
29
37
|
reviewed_by: timgentry
|
|
30
38
|
safe_revision: c59a45986f8b6d087c8c21b1e889f31f7346da17
|
|
31
39
|
LICENSE.txt:
|
|
32
|
-
comments:
|
|
40
|
+
comments:
|
|
33
41
|
reviewed_by: timgentry
|
|
34
42
|
safe_revision: c59a45986f8b6d087c8c21b1e889f31f7346da17
|
|
35
43
|
README.md:
|
|
36
|
-
comments:
|
|
44
|
+
comments:
|
|
37
45
|
reviewed_by: josh.pencheon
|
|
38
|
-
safe_revision:
|
|
46
|
+
safe_revision: a18f35895baebb1b0971043b3b3f666bb4eb9b5c
|
|
39
47
|
Rakefile:
|
|
40
|
-
comments:
|
|
48
|
+
comments:
|
|
41
49
|
reviewed_by: josh.pencheon
|
|
42
50
|
safe_revision: a601dbd139132f50eb22b1c88d5eda14a465a7c2
|
|
43
51
|
bin/console:
|
|
44
|
-
comments:
|
|
52
|
+
comments:
|
|
45
53
|
reviewed_by: josh.pencheon
|
|
46
54
|
safe_revision: 3377bef3131bbe58acaa82083fcb27c9c6d1e2b0
|
|
47
55
|
bin/setup:
|
|
48
|
-
comments:
|
|
56
|
+
comments:
|
|
49
57
|
reviewed_by: timgentry
|
|
50
58
|
safe_revision: c59a45986f8b6d087c8c21b1e889f31f7346da17
|
|
51
59
|
config/rubocop/ndr.yml:
|
|
52
|
-
comments:
|
|
60
|
+
comments:
|
|
53
61
|
reviewed_by: josh.pencheon
|
|
54
62
|
safe_revision: 05476acacadfa0ad7404f3c37cf3c17c4e0ba11b
|
|
55
63
|
gemfiles/Gemfile.rails52:
|
|
56
|
-
comments:
|
|
57
|
-
reviewed_by:
|
|
58
|
-
safe_revision:
|
|
64
|
+
comments:
|
|
65
|
+
reviewed_by: joshpencheon
|
|
66
|
+
safe_revision: f25001ef74c44ab727eef5cb29cef9a54525d36f
|
|
59
67
|
gemfiles/Gemfile.rails60:
|
|
60
|
-
comments:
|
|
61
|
-
reviewed_by:
|
|
62
|
-
safe_revision:
|
|
68
|
+
comments:
|
|
69
|
+
reviewed_by: joshpencheon
|
|
70
|
+
safe_revision: f25001ef74c44ab727eef5cb29cef9a54525d36f
|
|
71
|
+
gemfiles/Gemfile.rails61:
|
|
72
|
+
comments:
|
|
73
|
+
reviewed_by: joshpencheon
|
|
74
|
+
safe_revision: f25001ef74c44ab727eef5cb29cef9a54525d36f
|
|
63
75
|
lib/minitest/rake_ci.rb:
|
|
64
|
-
comments:
|
|
76
|
+
comments:
|
|
65
77
|
reviewed_by: josh.pencheon
|
|
66
78
|
safe_revision: 26efe45f35f211e7cb7a04967c51c0b6d116321e
|
|
67
79
|
lib/minitest/rake_ci_plugin.rb:
|
|
68
|
-
comments:
|
|
69
|
-
reviewed_by:
|
|
70
|
-
safe_revision:
|
|
80
|
+
comments:
|
|
81
|
+
reviewed_by: brian.shand
|
|
82
|
+
safe_revision: 5cb6964bee8c7a7856193232f6b06653235a8309
|
|
71
83
|
lib/ndr_dev_support.rb:
|
|
72
|
-
comments:
|
|
84
|
+
comments:
|
|
73
85
|
reviewed_by: josh.pencheon
|
|
74
86
|
safe_revision: 6211cff0ce44645ed3752723b5b0ee65f24c66aa
|
|
75
87
|
lib/ndr_dev_support/capistrano/assets.rb:
|
|
76
|
-
comments:
|
|
88
|
+
comments:
|
|
77
89
|
reviewed_by: josh.pencheon
|
|
78
90
|
safe_revision: 74a4cd909afaf2ee8bcabdd252b2383b1474b6d4
|
|
79
91
|
lib/ndr_dev_support/capistrano/ndr_model.rb:
|
|
80
|
-
comments:
|
|
92
|
+
comments:
|
|
81
93
|
reviewed_by: josh.pencheon
|
|
82
94
|
safe_revision: 57271c07e44808d8809e1fa11227a8c2e91a14bf
|
|
83
95
|
lib/ndr_dev_support/capistrano/restart.rb:
|
|
84
|
-
comments:
|
|
96
|
+
comments:
|
|
85
97
|
reviewed_by: josh.pencheon
|
|
86
98
|
safe_revision: a2b7c20eb58572c213f77677b36aa2f7e6db747e
|
|
87
99
|
lib/ndr_dev_support/capistrano/revision_logger.rb:
|
|
88
|
-
comments:
|
|
100
|
+
comments:
|
|
89
101
|
reviewed_by: josh.pencheon
|
|
90
102
|
safe_revision: a2b7c20eb58572c213f77677b36aa2f7e6db747e
|
|
91
103
|
lib/ndr_dev_support/capistrano/ruby_version.rb:
|
|
92
|
-
comments:
|
|
104
|
+
comments:
|
|
93
105
|
reviewed_by: josh.pencheon
|
|
94
106
|
safe_revision: a2b7c20eb58572c213f77677b36aa2f7e6db747e
|
|
95
107
|
lib/ndr_dev_support/capistrano/standalone_gems.rb:
|
|
96
|
-
comments:
|
|
108
|
+
comments:
|
|
97
109
|
reviewed_by: josh.pencheon
|
|
98
110
|
safe_revision: eec7079a777c84f61b05c236df60195d4ddcf488
|
|
99
111
|
lib/ndr_dev_support/capistrano/svn_cache.rb:
|
|
100
|
-
comments:
|
|
112
|
+
comments:
|
|
101
113
|
reviewed_by: josh.pencheon
|
|
102
114
|
safe_revision: 2baa360b06290775a961a7573fe08edd18f8312e
|
|
103
115
|
lib/ndr_dev_support/capistrano/sysadmin_scripts.rb:
|
|
104
|
-
comments:
|
|
116
|
+
comments:
|
|
105
117
|
reviewed_by: josh.pencheon
|
|
106
118
|
safe_revision: a2b7c20eb58572c213f77677b36aa2f7e6db747e
|
|
107
119
|
lib/ndr_dev_support/daemon/ci_server.rb:
|
|
108
|
-
comments:
|
|
120
|
+
comments:
|
|
109
121
|
reviewed_by: josh.pencheon
|
|
110
122
|
safe_revision: 3fdf010a91bd9927ef34e3df66b8a4bbbd20315a
|
|
111
123
|
lib/ndr_dev_support/daemon/stoppable.rb:
|
|
112
|
-
comments:
|
|
124
|
+
comments:
|
|
113
125
|
reviewed_by: josh.pencheon
|
|
114
126
|
safe_revision: 07ba05e9dd59eeac187b5fe730ee653baa1a2732
|
|
115
127
|
lib/ndr_dev_support/integration_testing.rb:
|
|
116
|
-
comments:
|
|
128
|
+
comments:
|
|
117
129
|
reviewed_by: josh.pencheon
|
|
118
130
|
safe_revision: 5b427cb12907bc87e5e0cb754dcaa524fdbfb1b9
|
|
119
131
|
lib/ndr_dev_support/integration_testing/drivers/chrome.rb:
|
|
120
|
-
comments:
|
|
132
|
+
comments:
|
|
121
133
|
reviewed_by: josh.pencheon
|
|
122
134
|
safe_revision: f1a32b1f2d1851b87a883dbf8620aa0e921e436c
|
|
123
135
|
lib/ndr_dev_support/integration_testing/drivers/chrome_headless.rb:
|
|
124
|
-
comments:
|
|
136
|
+
comments:
|
|
125
137
|
reviewed_by: josh.pencheon
|
|
126
138
|
safe_revision: '0409b687d31202b2a4281ebfb7b0c330cc78d904'
|
|
127
139
|
lib/ndr_dev_support/integration_testing/drivers/firefox.rb:
|
|
128
|
-
comments:
|
|
140
|
+
comments:
|
|
129
141
|
reviewed_by: josh.pencheon
|
|
130
142
|
safe_revision: f1a32b1f2d1851b87a883dbf8620aa0e921e436c
|
|
131
143
|
lib/ndr_dev_support/integration_testing/drivers/poltergeist.rb:
|
|
132
|
-
comments:
|
|
144
|
+
comments:
|
|
133
145
|
reviewed_by: josh.pencheon
|
|
134
146
|
safe_revision: f1a32b1f2d1851b87a883dbf8620aa0e921e436c
|
|
135
147
|
lib/ndr_dev_support/integration_testing/drivers/switchable.rb:
|
|
136
|
-
comments:
|
|
148
|
+
comments:
|
|
137
149
|
reviewed_by: josh.pencheon
|
|
138
150
|
safe_revision: e9b74559d28e7520c0376d1bec4a2566aae14b05
|
|
139
151
|
lib/ndr_dev_support/integration_testing/dsl.rb:
|
|
140
|
-
comments:
|
|
152
|
+
comments:
|
|
141
153
|
reviewed_by: josh.pencheon
|
|
142
154
|
safe_revision: f1a32b1f2d1851b87a883dbf8620aa0e921e436c
|
|
143
155
|
lib/ndr_dev_support/integration_testing/flakey_tests.rb:
|
|
144
|
-
comments:
|
|
145
|
-
reviewed_by:
|
|
146
|
-
safe_revision:
|
|
156
|
+
comments:
|
|
157
|
+
reviewed_by: joshpencheon
|
|
158
|
+
safe_revision: 9fc4527cbc132218533cec8d696bd2aadd4a2291
|
|
147
159
|
lib/ndr_dev_support/rake_ci/brakeman_helper.rb:
|
|
148
|
-
comments:
|
|
160
|
+
comments:
|
|
149
161
|
reviewed_by: josh.pencheon
|
|
150
162
|
safe_revision: 053c9834ca5d402a1f8dc8d09257dc7075a5ec06
|
|
151
163
|
lib/ndr_dev_support/rake_ci/commit_cop.rb:
|
|
152
|
-
comments:
|
|
164
|
+
comments:
|
|
153
165
|
reviewed_by: josh.pencheon
|
|
154
166
|
safe_revision: '01682912c2fc526b0bc29e2836de0aea31c65b12'
|
|
155
167
|
lib/ndr_dev_support/rake_ci/commit_cop/concerns/deputisable.rb:
|
|
156
|
-
comments:
|
|
168
|
+
comments:
|
|
157
169
|
reviewed_by: josh.pencheon
|
|
158
170
|
safe_revision: a3f5b7d3709f7e340f2f7e81dab026a63b21e3ff
|
|
159
171
|
lib/ndr_dev_support/rake_ci/commit_cop/migration_without_structure_dump.rb:
|
|
160
|
-
comments:
|
|
172
|
+
comments:
|
|
161
173
|
reviewed_by: josh.pencheon
|
|
162
174
|
safe_revision: a3f5b7d3709f7e340f2f7e81dab026a63b21e3ff
|
|
163
175
|
lib/ndr_dev_support/rake_ci/commit_cop/missing_associated_test_file.rb:
|
|
164
|
-
comments:
|
|
176
|
+
comments:
|
|
165
177
|
reviewed_by: josh.pencheon
|
|
166
178
|
safe_revision: '01682912c2fc526b0bc29e2836de0aea31c65b12'
|
|
167
179
|
lib/ndr_dev_support/rake_ci/commit_cop/modified_migration.rb:
|
|
168
|
-
comments:
|
|
180
|
+
comments:
|
|
169
181
|
reviewed_by: josh.pencheon
|
|
170
182
|
safe_revision: a3f5b7d3709f7e340f2f7e81dab026a63b21e3ff
|
|
171
183
|
lib/ndr_dev_support/rake_ci/commit_cop/renamed_migration.rb:
|
|
172
|
-
comments:
|
|
184
|
+
comments:
|
|
173
185
|
reviewed_by: josh.pencheon
|
|
174
186
|
safe_revision: a3f5b7d3709f7e340f2f7e81dab026a63b21e3ff
|
|
175
187
|
lib/ndr_dev_support/rake_ci/concerns/commit_metadata_persistable.rb:
|
|
176
|
-
comments:
|
|
188
|
+
comments:
|
|
177
189
|
reviewed_by: josh.pencheon
|
|
178
190
|
safe_revision: fd9fc5dd275a2285549984688d1d3744a628fd73
|
|
179
191
|
lib/ndr_dev_support/rake_ci/redmine/ticket_resolver.rb:
|
|
180
|
-
comments:
|
|
192
|
+
comments:
|
|
181
193
|
reviewed_by: josh.pencheon
|
|
182
194
|
safe_revision: 788b8fc31952a6fb640101df96a370b21d269249
|
|
183
195
|
lib/ndr_dev_support/rake_ci/simple_cov_helper.rb:
|
|
184
|
-
comments:
|
|
196
|
+
comments:
|
|
185
197
|
reviewed_by: timgentry
|
|
186
198
|
safe_revision: 678403f777614189ab89aafa1cb1eafbcf48763c
|
|
187
199
|
lib/ndr_dev_support/rubocop/executor.rb:
|
|
188
|
-
comments:
|
|
200
|
+
comments:
|
|
189
201
|
reviewed_by: josh.pencheon
|
|
190
|
-
safe_revision:
|
|
202
|
+
safe_revision: 1d8a97dc5dd2e58bfd689d5e2c94e077a4ca0649
|
|
191
203
|
lib/ndr_dev_support/rubocop/inject.rb:
|
|
192
|
-
comments:
|
|
204
|
+
comments:
|
|
193
205
|
reviewed_by: josh.pencheon
|
|
194
|
-
safe_revision:
|
|
206
|
+
safe_revision: 1d8a97dc5dd2e58bfd689d5e2c94e077a4ca0649
|
|
195
207
|
lib/ndr_dev_support/rubocop/range_augmenter.rb:
|
|
196
|
-
comments:
|
|
208
|
+
comments:
|
|
197
209
|
reviewed_by: josh.pencheon
|
|
198
210
|
safe_revision: 136936265a5b79f0eca1bc7e58c607a2492e57a5
|
|
199
211
|
lib/ndr_dev_support/rubocop/range_finder.rb:
|
|
200
|
-
comments:
|
|
212
|
+
comments:
|
|
201
213
|
reviewed_by: josh.pencheon
|
|
202
|
-
safe_revision:
|
|
214
|
+
safe_revision: 1d8a97dc5dd2e58bfd689d5e2c94e077a4ca0649
|
|
203
215
|
lib/ndr_dev_support/rubocop/reporter.rb:
|
|
204
|
-
comments:
|
|
205
|
-
reviewed_by:
|
|
206
|
-
safe_revision:
|
|
216
|
+
comments:
|
|
217
|
+
reviewed_by: joshpencheon
|
|
218
|
+
safe_revision: 0c1489e51848d8e7a0903128ea934064c0ab254e
|
|
207
219
|
lib/ndr_dev_support/slack_message_publisher.rb:
|
|
208
|
-
comments:
|
|
220
|
+
comments:
|
|
209
221
|
reviewed_by: josh.pencheon
|
|
210
222
|
safe_revision: ad38e92c6e56b9d81fdab10681d8f2924eeadf5a
|
|
211
223
|
lib/ndr_dev_support/tasks.rb:
|
|
212
|
-
comments:
|
|
224
|
+
comments:
|
|
213
225
|
reviewed_by: josh.pencheon
|
|
214
226
|
safe_revision: 2154aa7f32e731933ff6091b8f42b2b014028a6a
|
|
215
227
|
lib/ndr_dev_support/version.rb:
|
|
216
|
-
comments:
|
|
217
|
-
reviewed_by:
|
|
218
|
-
safe_revision:
|
|
228
|
+
comments:
|
|
229
|
+
reviewed_by: brian.shand
|
|
230
|
+
safe_revision: d4d4ea4d2dcdf68819e75cbdac1009b145740b22
|
|
219
231
|
lib/tasks/audit_code.rake:
|
|
220
232
|
comments: Identical to the version reviewed by josh.pencheon when contained within
|
|
221
233
|
ndr_support
|
|
222
|
-
reviewed_by:
|
|
223
|
-
safe_revision:
|
|
234
|
+
reviewed_by: brian.shand
|
|
235
|
+
safe_revision: 0d1c9d3e7192ad9e6c979d8d772ca49fc91f93cb
|
|
224
236
|
lib/tasks/ci/brakeman.rake:
|
|
225
|
-
comments:
|
|
237
|
+
comments:
|
|
226
238
|
reviewed_by: josh.pencheon
|
|
227
239
|
safe_revision: fe7dfc7f1775e8b85ae6968e8f475b9e930addf7
|
|
228
240
|
lib/tasks/ci/bundle_audit.rake:
|
|
229
|
-
comments:
|
|
241
|
+
comments:
|
|
230
242
|
reviewed_by: josh.pencheon
|
|
231
243
|
safe_revision: ddcf7de0bf1f5fb17cc28e2460009e162ff8917c
|
|
232
244
|
lib/tasks/ci/commit_cop.rake:
|
|
233
|
-
comments:
|
|
245
|
+
comments:
|
|
234
246
|
reviewed_by: josh.pencheon
|
|
235
247
|
safe_revision: a3f5b7d3709f7e340f2f7e81dab026a63b21e3ff
|
|
236
248
|
lib/tasks/ci/dependencies.rake:
|
|
237
|
-
comments:
|
|
249
|
+
comments:
|
|
238
250
|
reviewed_by: josh.pencheon
|
|
239
251
|
safe_revision: 2f22aa6cc2f86b84e0bf2247e73bed205930db5a
|
|
240
252
|
lib/tasks/ci/housekeep.rake:
|
|
241
|
-
comments:
|
|
253
|
+
comments:
|
|
242
254
|
reviewed_by: timgentry
|
|
243
255
|
safe_revision: 3dfcdb6c76fbccc44a331986673d63282f87f2fa
|
|
244
256
|
lib/tasks/ci/linguist.rake:
|
|
245
|
-
comments:
|
|
257
|
+
comments:
|
|
246
258
|
reviewed_by: timgentry
|
|
247
259
|
safe_revision: 00cd6c2f71612a467ee88ad53c21e08f073871d5
|
|
248
260
|
lib/tasks/ci/minitest.rake:
|
|
249
|
-
comments:
|
|
261
|
+
comments:
|
|
250
262
|
reviewed_by: josh.pencheon
|
|
251
263
|
safe_revision: 12aad1254186ee9ce0fde263bf8dccf46cdfebe2
|
|
252
264
|
lib/tasks/ci/notes.rake:
|
|
253
|
-
comments:
|
|
265
|
+
comments:
|
|
254
266
|
reviewed_by: timgentry
|
|
255
267
|
safe_revision: f828113894a16581d0aa181504c799e661f8401d
|
|
256
268
|
lib/tasks/ci/prometheus.rake:
|
|
257
|
-
comments:
|
|
269
|
+
comments:
|
|
258
270
|
reviewed_by: josh.pencheon
|
|
259
271
|
safe_revision: 2d1b9d19acafebe6ca5f995bf15243abe2847eec
|
|
260
272
|
lib/tasks/ci/redmine.rake:
|
|
261
|
-
comments:
|
|
273
|
+
comments:
|
|
262
274
|
reviewed_by: josh.pencheon
|
|
263
275
|
safe_revision: 12aad1254186ee9ce0fde263bf8dccf46cdfebe2
|
|
264
276
|
lib/tasks/ci/rugged.rake:
|
|
265
|
-
comments:
|
|
277
|
+
comments:
|
|
266
278
|
reviewed_by: josh.pencheon
|
|
267
279
|
safe_revision: 15543b915771dfe2ed81b8b326bdd3ab2cb5b562
|
|
268
280
|
lib/tasks/ci/server.rake:
|
|
269
|
-
comments:
|
|
281
|
+
comments:
|
|
270
282
|
reviewed_by: josh.pencheon
|
|
271
283
|
safe_revision: fe8682a3300a322c5f6b9a17dce471f11fa5e307
|
|
272
284
|
lib/tasks/ci/simplecov.rake:
|
|
273
|
-
comments:
|
|
285
|
+
comments:
|
|
274
286
|
reviewed_by: josh.pencheon
|
|
275
287
|
safe_revision: 6240a84dc859af0623328f8667b93299c8eae257
|
|
276
288
|
lib/tasks/ci/slack.rake:
|
|
277
|
-
comments:
|
|
289
|
+
comments:
|
|
278
290
|
reviewed_by: josh.pencheon
|
|
279
291
|
safe_revision: 6cb22447594ebf450fc60deb90fd223c8067ff18
|
|
280
292
|
lib/tasks/ci/stats.rake:
|
|
281
|
-
comments:
|
|
293
|
+
comments:
|
|
282
294
|
reviewed_by: josh.pencheon
|
|
283
295
|
safe_revision: 137be2205f283d444286c47502387487bb1b7015
|
|
284
296
|
lib/tasks/rubocop.rake:
|
|
285
|
-
comments:
|
|
297
|
+
comments:
|
|
286
298
|
reviewed_by: josh.pencheon
|
|
287
299
|
safe_revision: 9f33cc3f6fed0c24cb215befa1beaa4bc03e81ef
|
|
288
300
|
ndr_dev_support.gemspec:
|
|
289
|
-
comments:
|
|
290
|
-
reviewed_by:
|
|
291
|
-
safe_revision:
|
|
301
|
+
comments:
|
|
302
|
+
reviewed_by: brian.shand
|
|
303
|
+
safe_revision: 5cb6964bee8c7a7856193232f6b06653235a8309
|
|
292
304
|
test/daemon/ci_server_test.rb:
|
|
293
|
-
comments:
|
|
305
|
+
comments:
|
|
294
306
|
reviewed_by: josh.pencheon
|
|
295
307
|
safe_revision: cdd176f2e90284e11899c5c6eea1032f9b785b74
|
|
308
|
+
test/integration_testing/flakey_tests_test.rb:
|
|
309
|
+
comments:
|
|
310
|
+
reviewed_by: joshpencheon
|
|
311
|
+
safe_revision: 9fc4527cbc132218533cec8d696bd2aadd4a2291
|
|
296
312
|
test/ndr_dev_support_test.rb:
|
|
297
|
-
comments:
|
|
313
|
+
comments:
|
|
298
314
|
reviewed_by: timgentry
|
|
299
315
|
safe_revision: 26a8907a7a8a6b0d836aa12ca0335fa40be12240
|
|
300
316
|
test/rake_ci/commit_cop/migration_without_structure_dump_test.rb:
|
|
301
|
-
comments:
|
|
317
|
+
comments:
|
|
302
318
|
reviewed_by: josh.pencheon
|
|
303
319
|
safe_revision: a3f5b7d3709f7e340f2f7e81dab026a63b21e3ff
|
|
304
320
|
test/rake_ci/commit_cop/missing_associated_test_file_test.rb:
|
|
305
|
-
comments:
|
|
321
|
+
comments:
|
|
306
322
|
reviewed_by: josh.pencheon
|
|
307
323
|
safe_revision: '01682912c2fc526b0bc29e2836de0aea31c65b12'
|
|
308
324
|
test/rake_ci/commit_cop/modified_migration_test.rb:
|
|
309
|
-
comments:
|
|
325
|
+
comments:
|
|
310
326
|
reviewed_by: josh.pencheon
|
|
311
327
|
safe_revision: a3f5b7d3709f7e340f2f7e81dab026a63b21e3ff
|
|
312
328
|
test/rake_ci/commit_cop/renamed_migration_test.rb:
|
|
313
|
-
comments:
|
|
329
|
+
comments:
|
|
314
330
|
reviewed_by: josh.pencheon
|
|
315
331
|
safe_revision: a3f5b7d3709f7e340f2f7e81dab026a63b21e3ff
|
|
316
332
|
test/rake_ci/redmine/ticket_resolver_test.rb:
|
|
317
|
-
comments:
|
|
333
|
+
comments:
|
|
318
334
|
reviewed_by: josh.pencheon
|
|
319
335
|
safe_revision: 12aad1254186ee9ce0fde263bf8dccf46cdfebe2
|
|
320
336
|
test/test_helper.rb:
|
|
321
|
-
comments:
|
|
337
|
+
comments:
|
|
322
338
|
reviewed_by: josh.pencheon
|
|
323
339
|
safe_revision: cdd176f2e90284e11899c5c6eea1032f9b785b74
|
data/gemfiles/Gemfile.rails52
CHANGED
data/gemfiles/Gemfile.rails60
CHANGED
|
@@ -15,6 +15,12 @@ module NdrDevSupport
|
|
|
15
15
|
self.attempts_per_test = attempts_per_test.merge(test_name.to_s => attempts)
|
|
16
16
|
end
|
|
17
17
|
end
|
|
18
|
+
|
|
19
|
+
def test_repeatedly(description, times: 100, &block)
|
|
20
|
+
(1..times).map do |n|
|
|
21
|
+
test("#{description} - #{n}/#{times}", &block)
|
|
22
|
+
end
|
|
23
|
+
end
|
|
18
24
|
end
|
|
19
25
|
|
|
20
26
|
def flakes
|
|
@@ -27,11 +33,12 @@ module NdrDevSupport
|
|
|
27
33
|
|
|
28
34
|
previous_failure = failures.last
|
|
29
35
|
failed_attempts = []
|
|
36
|
+
result = nil
|
|
30
37
|
|
|
31
38
|
loop do
|
|
32
39
|
break if attempts_remaining < 1
|
|
33
40
|
|
|
34
|
-
super
|
|
41
|
+
result = super
|
|
35
42
|
|
|
36
43
|
# No failure was added; we passed!
|
|
37
44
|
break if failures.last == previous_failure
|
|
@@ -46,10 +53,12 @@ module NdrDevSupport
|
|
|
46
53
|
# Attempts were only flakey if we eventually passed:
|
|
47
54
|
flakes.concat(failed_attempts) if failures.last == previous_failure
|
|
48
55
|
|
|
49
|
-
|
|
56
|
+
result
|
|
50
57
|
end
|
|
51
58
|
end
|
|
52
59
|
end
|
|
53
60
|
end
|
|
54
61
|
|
|
55
|
-
ActionDispatch
|
|
62
|
+
if defined?(ActionDispatch)
|
|
63
|
+
ActionDispatch::IntegrationTest.include(NdrDevSupport::IntegrationTesting::FlakeyTests)
|
|
64
|
+
end
|
|
@@ -10,7 +10,7 @@ module NdrDevSupport
|
|
|
10
10
|
class << self
|
|
11
11
|
# Use RuboCop to produce a list of all files that should be scanned.
|
|
12
12
|
def target_files
|
|
13
|
-
@target_files ||= `rubocop -L`.each_line.map(&:strip)
|
|
13
|
+
@target_files ||= `rubocop -L 2>/dev/null`.each_line.map(&:strip)
|
|
14
14
|
end
|
|
15
15
|
end
|
|
16
16
|
|
|
@@ -23,7 +23,7 @@ module NdrDevSupport
|
|
|
23
23
|
def offenses_by_file
|
|
24
24
|
return [] if @filenames.empty?
|
|
25
25
|
|
|
26
|
-
output = JSON.parse(`rubocop --format json #{escaped_paths.join(' ')}`)
|
|
26
|
+
output = JSON.parse(`rubocop --format json #{escaped_paths.join(' ')} 2>/dev/null`)
|
|
27
27
|
|
|
28
28
|
output['files'].each_with_object({}) do |file_output, result|
|
|
29
29
|
result[file_output['path']] = file_output['offenses']
|
|
@@ -22,15 +22,16 @@ module NdrDevSupport
|
|
|
22
22
|
# Prints out a report, and returns an appriopriate
|
|
23
23
|
# exit status for the rake task to terminate with.
|
|
24
24
|
def report
|
|
25
|
-
if @offenses.
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
print_offenses
|
|
29
|
-
return @offenses.values.all?(&:empty?)
|
|
30
|
-
else
|
|
31
|
-
puts Rainbow('No relevant changes found.').yellow
|
|
32
|
-
return false
|
|
25
|
+
if @offenses.none?
|
|
26
|
+
warn Rainbow('No relevant changes found.').yellow
|
|
27
|
+
return true
|
|
33
28
|
end
|
|
29
|
+
|
|
30
|
+
print_summary
|
|
31
|
+
puts
|
|
32
|
+
print_offenses
|
|
33
|
+
|
|
34
|
+
@offenses.values.all?(&:empty?)
|
|
34
35
|
end
|
|
35
36
|
|
|
36
37
|
private
|
data/lib/tasks/audit_code.rake
CHANGED
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
require 'csv'
|
|
1
2
|
require 'pathname'
|
|
2
3
|
require 'yaml'
|
|
3
4
|
|
|
@@ -10,9 +11,6 @@ SAFETY_FILE =
|
|
|
10
11
|
Pathname.new('code_safety.yml').expand_path
|
|
11
12
|
end
|
|
12
13
|
|
|
13
|
-
# Temporary overrides to only audit external access files
|
|
14
|
-
SAFETY_REPOS = [['/svn/era', '/svn/extra/era/external-access']]
|
|
15
|
-
|
|
16
14
|
# Returns the (Bundler aware) rake command
|
|
17
15
|
def rake_cmd
|
|
18
16
|
ENV['BUNDLE_BIN_PATH'] ? 'bundle exec rake' : 'rake'
|
|
@@ -32,8 +30,12 @@ def update_safety_file(file_safety)
|
|
|
32
30
|
File.open(SAFETY_FILE, 'w') do |file|
|
|
33
31
|
# Consistent file diffs, as ruby preserves Hash insertion order since v1.9
|
|
34
32
|
list = {}
|
|
35
|
-
list['file safety'] =
|
|
36
|
-
|
|
33
|
+
list['file safety'] = file_safety.sort.to_h
|
|
34
|
+
# Remove inconsistent whitespace which old libyaml version introduces at the ends of lines
|
|
35
|
+
# https://github.com/yaml/libyaml/issues/46
|
|
36
|
+
yaml = YAML.dump(list).gsub(/^( *[a-z_]*:) $/, '\1')
|
|
37
|
+
yaml = YAML.dump(list) unless YAML.safe_load(yaml) == list
|
|
38
|
+
file << yaml # Save changes before checking latest revisions
|
|
37
39
|
end
|
|
38
40
|
end
|
|
39
41
|
|
|
@@ -48,12 +50,11 @@ end
|
|
|
48
50
|
|
|
49
51
|
# Parameter max_print is number of entries to print before truncating output
|
|
50
52
|
# (negative value => print all)
|
|
51
|
-
def audit_code_safety(max_print = 20, ignore_new = false, show_diffs = false,
|
|
52
|
-
puts 'Running source code safety audit script.'
|
|
53
|
-
puts
|
|
54
|
-
|
|
53
|
+
def audit_code_safety(max_print = 20, ignore_new = false, show_diffs = false, usr = 'usr', interactive = false, outfile = nil, filter = nil)
|
|
55
54
|
max_print = 1_000_000 if max_print.negative?
|
|
56
55
|
show_diffs = true if interactive
|
|
56
|
+
ignore_new = true if filter
|
|
57
|
+
|
|
57
58
|
file_safety = load_file_safety
|
|
58
59
|
file_safety.each_value do |v|
|
|
59
60
|
rev = v['safe_revision']
|
|
@@ -63,76 +64,143 @@ def audit_code_safety(max_print = 20, ignore_new = false, show_diffs = false, sh
|
|
|
63
64
|
|
|
64
65
|
safety_repo = trunk_repo = get_trunk_repo
|
|
65
66
|
|
|
66
|
-
# TODO: below is broken for git-svn
|
|
67
|
-
# Is it needed?
|
|
68
|
-
|
|
69
|
-
SAFETY_REPOS.each do |suffix, alt|
|
|
70
|
-
# Temporarily override to only audit a different file list
|
|
71
|
-
if safety_repo.end_with?(suffix)
|
|
72
|
-
safety_repo = safety_repo[0...-suffix.length] + alt
|
|
73
|
-
break
|
|
74
|
-
end
|
|
75
|
-
end
|
|
76
|
-
|
|
77
67
|
if ignore_new
|
|
78
68
|
puts "Not checking for new files in #{safety_repo}"
|
|
79
69
|
else
|
|
80
70
|
puts "Checking for new files in #{safety_repo}"
|
|
81
|
-
|
|
82
|
-
# Ignore subdirectories, and exclude code_safety.yml by default.
|
|
83
|
-
new_files.delete_if { |f| f =~ /[\/\\]$/ || Pathname.new(f).expand_path == SAFETY_FILE }
|
|
84
|
-
new_files.each { |f| add_new_file_to_file_safety(file_safety, f) }
|
|
85
|
-
update_safety_file(file_safety) # Save changes before checking latest revisions
|
|
71
|
+
add_new_files(safety_repo, file_safety)
|
|
86
72
|
end
|
|
87
|
-
puts "Updating latest revisions for #{file_safety.size} files"
|
|
88
|
-
set_last_changed_revision(trunk_repo, file_safety, file_safety.keys)
|
|
89
|
-
puts "\nSummary:"
|
|
90
|
-
puts "Number of files originally in #{SAFETY_FILE}: #{orig_count}"
|
|
91
|
-
puts "Number of new files added: #{file_safety.size - orig_count}"
|
|
92
73
|
|
|
93
74
|
missing_files = file_safety.keys.reject { |path| File.file?(path) }
|
|
75
|
+
abort(<<~MESSAGE) if missing_files.any?
|
|
94
76
|
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
77
|
+
The following file(s) no longer exist in the repository:
|
|
78
|
+
#{missing_files.join("\n ")}
|
|
79
|
+
|
|
80
|
+
Please run `#{rake_cmd} audit:tidy_code_safety_file` to remove them.
|
|
81
|
+
MESSAGE
|
|
82
|
+
|
|
83
|
+
if filter
|
|
84
|
+
filter_file_safety(file_safety, filter)
|
|
85
|
+
puts "Applying provided file filter: #{file_safety.size} file(s) remain"
|
|
99
86
|
end
|
|
100
87
|
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
88
|
+
if interactive
|
|
89
|
+
run_review_wizard(trunk_repo, file_safety, usr)
|
|
90
|
+
|
|
91
|
+
# Post-wizard, reload the results and continue to print a summary:
|
|
92
|
+
file_safety = load_file_safety
|
|
93
|
+
filter_file_safety(file_safety, filter) if filter
|
|
94
|
+
|
|
95
|
+
max_print = 0
|
|
96
|
+
show_diffs = false
|
|
97
|
+
else
|
|
98
|
+
# Get updates for all files in one go:
|
|
99
|
+
puts "Updating latest revisions for #{file_safety.size} files"
|
|
100
|
+
set_last_changed_revisions(trunk_repo, file_safety)
|
|
108
101
|
end
|
|
109
|
-
|
|
110
|
-
|
|
102
|
+
|
|
103
|
+
print_summary(file_safety, usr, trunk_repo, max_print, show_diffs, orig_count, ignore_new, outfile, filter)
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
def filter_file_safety(file_safety, filter)
|
|
107
|
+
filtered_paths = CSV.read(filter, headers: true).map { |row| row.to_h.fetch('path') }
|
|
108
|
+
file_safety.select! { |fname, _entry| filtered_paths.include?(fname) }
|
|
109
|
+
end
|
|
110
|
+
|
|
111
|
+
def run_review_wizard(trunk_repo, file_safety, usr)
|
|
112
|
+
puts <<~INTRO
|
|
113
|
+
|
|
114
|
+
In interactive mode, you'll be presented with one diff at a time for review.
|
|
115
|
+
|
|
116
|
+
For each diff, you can enter a decision (if you can make one) along with
|
|
117
|
+
any comments you may have, before being taken to the next diff.
|
|
118
|
+
|
|
119
|
+
INTRO
|
|
120
|
+
|
|
121
|
+
file_safety.each_key do |fname|
|
|
122
|
+
set_last_changed_revision(trunk_repo, file_safety, fname)
|
|
123
|
+
next unless print_file_safety(file_safety, fname, false, true)
|
|
124
|
+
|
|
125
|
+
print_file_diffs(file_safety, trunk_repo, fname, usr, true)
|
|
126
|
+
end
|
|
127
|
+
end
|
|
128
|
+
|
|
129
|
+
def print_summary(file_safety, usr, trunk_repo, max_print, show_diffs, orig_count, ignore_new, outfile, filter)
|
|
130
|
+
puts "\nSummary:"
|
|
131
|
+
puts "Number of files originally in #{SAFETY_FILE}: #{orig_count}"
|
|
132
|
+
puts "Number of new files added: #{file_safety.size - orig_count}" unless ignore_new
|
|
133
|
+
|
|
134
|
+
# Now generate statistics:
|
|
135
|
+
unknown = file_safety.values.select { |x| unreviewed?(x) }
|
|
136
|
+
unsafe = file_safety.values.select { |x| stale_review?(x) }
|
|
137
|
+
puts "Number of#{' filtered' if filter} files with no safe version: #{unknown.size}"
|
|
138
|
+
puts "Number of#{' filtered' if filter} files which are no longer safe: #{unsafe.size}"
|
|
111
139
|
puts
|
|
112
140
|
printed = []
|
|
113
|
-
|
|
141
|
+
|
|
114
142
|
file_list =
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
file_safety.keys.sort
|
|
119
|
-
end
|
|
143
|
+
file_safety.
|
|
144
|
+
sort_by { |_k, v| v.nil? ? -100 : v['last_changed_rev'].to_i }.
|
|
145
|
+
map(&:first)
|
|
120
146
|
|
|
121
147
|
file_list.each do |f|
|
|
122
148
|
printed << f if print_file_safety(file_safety, f, false, printed.size >= max_print)
|
|
123
149
|
end
|
|
124
|
-
puts "... and #{printed.size - max_print} others" if printed.size > max_print
|
|
150
|
+
puts "... and #{printed.size - max_print} others" if printed.size > max_print && max_print > 0
|
|
125
151
|
if show_diffs
|
|
126
152
|
puts
|
|
127
153
|
printed.each do |f|
|
|
128
|
-
print_file_diffs(file_safety, trunk_repo, f, usr,
|
|
154
|
+
print_file_diffs(file_safety, trunk_repo, f, usr, false)
|
|
129
155
|
end
|
|
130
156
|
end
|
|
131
157
|
|
|
158
|
+
export_csv(trunk_repo, file_safety, printed, outfile) if outfile
|
|
159
|
+
|
|
132
160
|
# Returns `true` unless there are pending reviews:
|
|
133
161
|
unsafe.length.zero? && unknown.length.zero?
|
|
134
162
|
end
|
|
135
163
|
|
|
164
|
+
def export_csv(repo, file_safety, printed, outfile)
|
|
165
|
+
CSV.open(outfile, 'w') do |csv|
|
|
166
|
+
csv << %w[path last_changed_revision safe_revision diff_lines]
|
|
167
|
+
|
|
168
|
+
file_safety.each do |fname, entry|
|
|
169
|
+
# Only emit files needing review:
|
|
170
|
+
next unless printed.include?(fname)
|
|
171
|
+
|
|
172
|
+
last_changed_revision = entry['last_changed_rev']
|
|
173
|
+
safe_revision = entry['safe_revision']
|
|
174
|
+
|
|
175
|
+
_cmd, diffs = capture_file_diffs(repo, fname, safe_revision, last_changed_revision)
|
|
176
|
+
diff_lines = diffs.force_encoding('BINARY').split("\n").length
|
|
177
|
+
|
|
178
|
+
csv << [fname, last_changed_revision, safe_revision, diff_lines]
|
|
179
|
+
end
|
|
180
|
+
end
|
|
181
|
+
|
|
182
|
+
puts <<~MESSAGE
|
|
183
|
+
|
|
184
|
+
CSV output exported to: #{outfile}
|
|
185
|
+
|
|
186
|
+
MESSAGE
|
|
187
|
+
end
|
|
188
|
+
|
|
189
|
+
def stale_review?(entry)
|
|
190
|
+
return false if unreviewed?(entry)
|
|
191
|
+
|
|
192
|
+
# Could be comparing Git SHAs, or SVN revisions:
|
|
193
|
+
entry['last_changed_rev'] != entry['safe_revision']
|
|
194
|
+
end
|
|
195
|
+
|
|
196
|
+
def unreviewed?(entry)
|
|
197
|
+
entry['safe_revision'].nil?
|
|
198
|
+
end
|
|
199
|
+
|
|
200
|
+
def needs_review?(entry)
|
|
201
|
+
unreviewed?(entry) || stale_review?(entry)
|
|
202
|
+
end
|
|
203
|
+
|
|
136
204
|
# Print summary details of a file's known safety
|
|
137
205
|
# If not verbose, only prints details for unsafe files
|
|
138
206
|
# Returns true if anything printed (or would have been printed if silent),
|
|
@@ -210,14 +278,11 @@ def get_trunk_repo
|
|
|
210
278
|
case repository_type
|
|
211
279
|
when 'svn'
|
|
212
280
|
repo_info = %x[svn info]
|
|
213
|
-
puts 'svn case'
|
|
214
281
|
repo_info.split("\n").select { |x| x =~ /^URL: / }.collect { |x| x[5..-1] }.first
|
|
215
282
|
when 'git-svn'
|
|
216
|
-
puts 'git-svn case'
|
|
217
283
|
repo_info = %x[git svn info]
|
|
218
284
|
repo_info.split("\n").select { |x| x =~ /^URL: / }.collect { |x| x[5..-1] }.first
|
|
219
285
|
when 'git'
|
|
220
|
-
puts 'git case'
|
|
221
286
|
repo_info = %x[git remote -v]
|
|
222
287
|
repo_info.split("\n").first[7..-9]
|
|
223
288
|
else
|
|
@@ -225,69 +290,45 @@ def get_trunk_repo
|
|
|
225
290
|
end
|
|
226
291
|
end
|
|
227
292
|
|
|
228
|
-
def
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
293
|
+
def add_new_files(safety_repo, file_safety)
|
|
294
|
+
new_files =
|
|
295
|
+
case repository_type
|
|
296
|
+
when 'svn', 'git-svn'
|
|
297
|
+
%x[svn ls -R "#{safety_repo}"].split("\n")
|
|
298
|
+
when 'git'
|
|
299
|
+
%x[git ls-files].split("\n")
|
|
300
|
+
else
|
|
301
|
+
[]
|
|
302
|
+
end
|
|
235
303
|
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
end
|
|
304
|
+
# Ignore subdirectories, and exclude code_safety.yml by default.
|
|
305
|
+
new_files.delete_if { |f| f =~ /[\/\\]$/ || Pathname.new(f).expand_path == SAFETY_FILE }
|
|
306
|
+
|
|
307
|
+
# Save changes before checking latest revisions
|
|
308
|
+
new_files.each { |f| add_new_file_to_file_safety(file_safety, f) }
|
|
309
|
+
update_safety_file(file_safety)
|
|
243
310
|
end
|
|
244
311
|
|
|
245
312
|
# Fill in the latest changed revisions in a file safety map.
|
|
246
313
|
# (Don't write this data to the YAML file, as it is intrinsic to the SVN
|
|
247
314
|
# repository.)
|
|
248
|
-
def
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
info = %x[git log -n 1 -- #{f}].split("\n").first[7..-1]
|
|
256
|
-
if info.nil? || info.empty?
|
|
257
|
-
file_safety[f]['last_changed_rev'] = -1
|
|
258
|
-
else
|
|
259
|
-
file_safety[f]['last_changed_rev'] = info
|
|
260
|
-
end
|
|
261
|
-
# Show progress
|
|
262
|
-
print '.' if (i % dot_freq) == 0
|
|
263
|
-
end
|
|
264
|
-
puts
|
|
265
|
-
when 'git-svn', 'svn'
|
|
266
|
-
fnames = file_safety.keys if fnames.nil?
|
|
267
|
-
|
|
268
|
-
fnames.each_with_index do |f, i|
|
|
269
|
-
last_revision = get_last_changed_revision(repo, f)
|
|
270
|
-
if last_revision.nil? || last_revision.empty?
|
|
271
|
-
file_safety[f]['last_changed_rev'] = -1
|
|
272
|
-
else
|
|
273
|
-
file_safety[f]['last_changed_rev'] = last_revision
|
|
274
|
-
end
|
|
275
|
-
# Show progress
|
|
276
|
-
print '.' if (i % dot_freq) == 0
|
|
277
|
-
end
|
|
278
|
-
puts
|
|
279
|
-
# NOTE: Do we need the following for retries?
|
|
280
|
-
# if retries && result.size != fnames.size && fnames.size > 1
|
|
281
|
-
# # At least one invalid (deleted file --> subsequent arguments ignored)
|
|
282
|
-
# # Try each file individually
|
|
283
|
-
# # (It would probably be safe to continue from the extra_info.size argument)
|
|
284
|
-
# puts "Retrying (got #{result.size}, expected #{fnames.size})" if debug >= 2
|
|
285
|
-
# result = []
|
|
286
|
-
# fnames.each{ |f|
|
|
287
|
-
# result += svn_info_entries([f], repo, false, debug)
|
|
288
|
-
# }
|
|
289
|
-
# end
|
|
315
|
+
def set_last_changed_revisions(repo, file_safety)
|
|
316
|
+
fnames = file_safety.keys
|
|
317
|
+
dot_freq = (fnames.size / 40.0).ceil # Print up to 40 progress dots
|
|
318
|
+
|
|
319
|
+
fnames.each_with_index do |f, i|
|
|
320
|
+
set_last_changed_revision(repo, file_safety, f)
|
|
321
|
+
print '.' if (i % dot_freq) == 0
|
|
290
322
|
end
|
|
323
|
+
puts
|
|
324
|
+
end
|
|
325
|
+
|
|
326
|
+
# Fill in the latest changed revision for the given file in a file safety map.
|
|
327
|
+
def set_last_changed_revision(repo, file_safety, fname)
|
|
328
|
+
last_revision = get_last_changed_revision(repo, fname)
|
|
329
|
+
last_revision = -1 if last_revision.blank?
|
|
330
|
+
|
|
331
|
+
file_safety[fname]['last_changed_rev'] = last_revision
|
|
291
332
|
end
|
|
292
333
|
|
|
293
334
|
# Return the last changed revision
|
|
@@ -331,31 +372,44 @@ def root_revision
|
|
|
331
372
|
end
|
|
332
373
|
end
|
|
333
374
|
|
|
375
|
+
def capture_file_diffs(repo, fname, safe_revision, repolatest)
|
|
376
|
+
cmd =
|
|
377
|
+
case repository_type
|
|
378
|
+
when 'git'
|
|
379
|
+
cmd = ['git', '--no-pager', 'diff', '--color', '-b', "#{safe_revision}..#{repolatest}", fname]
|
|
380
|
+
when 'git-svn', 'svn'
|
|
381
|
+
cmd = ['svn', 'diff', '-r', "#{safe_revision.to_i}:#{repolatest.to_i}", '-x', '-b', "#{repo}/#{fname}"]
|
|
382
|
+
end
|
|
383
|
+
|
|
384
|
+
stdout_and_err_str, _status = Open3.capture2e(*cmd)
|
|
385
|
+
|
|
386
|
+
if stdout_and_err_str.start_with?('fatal: Invalid revision range ')
|
|
387
|
+
abort <<~ERROR
|
|
388
|
+
Error running:
|
|
389
|
+
#{cmd}
|
|
390
|
+
|
|
391
|
+
Invalid commit ID in code_safety.yml for #{fname}? (#{safe_revision})
|
|
392
|
+
ERROR
|
|
393
|
+
end
|
|
394
|
+
|
|
395
|
+
[cmd.join(' '), stdout_and_err_str]
|
|
396
|
+
end
|
|
397
|
+
|
|
334
398
|
def print_repo_file_diffs(repolatest, repo, fname, usr, safe_revision, interactive)
|
|
335
399
|
require 'open3'
|
|
336
400
|
require 'highline/import'
|
|
337
401
|
|
|
338
402
|
if interactive
|
|
339
|
-
ask("
|
|
403
|
+
ask("<%= color('Press Enter to continue ...', :yellow) %>")
|
|
340
404
|
system('clear')
|
|
341
405
|
system("printf '\033[3J'") # clear the scrollback
|
|
342
406
|
end
|
|
343
407
|
|
|
344
|
-
cmd =
|
|
345
|
-
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
|
|
349
|
-
cmd = ['svn', 'diff', '-r', "#{safe_revision.to_i}:#{repolatest.to_i}", '-x', '-b', "#{repo}/#{fname}"]
|
|
350
|
-
end
|
|
351
|
-
if cmd
|
|
352
|
-
puts(cmd.join(' '))
|
|
353
|
-
stdout_and_err_str, _status = Open3.capture2e(*cmd)
|
|
354
|
-
puts 'Invalid commit ID in code_safety.yml ' + safe_revision if stdout_and_err_str.start_with?('fatal: Invalid revision range ')
|
|
355
|
-
puts(stdout_and_err_str)
|
|
356
|
-
else
|
|
357
|
-
puts 'Unknown repo'
|
|
358
|
-
end
|
|
408
|
+
cmd, diffs = capture_file_diffs(repo, fname, safe_revision, repolatest)
|
|
409
|
+
puts cmd
|
|
410
|
+
puts
|
|
411
|
+
puts diffs
|
|
412
|
+
puts
|
|
359
413
|
|
|
360
414
|
if interactive
|
|
361
415
|
response = ask("Flag #{fname} changes safe? [Yes|No|Abort]: ") { |q| q.case = :down }
|
|
@@ -363,18 +417,19 @@ def print_repo_file_diffs(repolatest, repo, fname, usr, safe_revision, interacti
|
|
|
363
417
|
puts 'Flagging as safe...'
|
|
364
418
|
release = get_release(repolatest)
|
|
365
419
|
if usr.to_s.strip.empty?
|
|
366
|
-
usr = ask('File reviewed by:') do |q|
|
|
420
|
+
usr = ask('File reviewed by: ') do |q|
|
|
367
421
|
q.whitespace = :strip_and_collapse
|
|
368
|
-
q.validate = /\A[\w \-.]+\
|
|
422
|
+
q.validate = /\A[\w \-.]+\z/
|
|
369
423
|
end
|
|
370
424
|
end
|
|
371
|
-
comment = ask('Please write your comments (optional):')
|
|
425
|
+
comment = ask('Please write your comments (optional): ')
|
|
372
426
|
# use to_s to convert response from !ruby/string:HighLine::String to String
|
|
373
427
|
flag_file_as_safe(release, usr.to_s, comment.to_s, fname)
|
|
374
|
-
elsif
|
|
375
|
-
|
|
428
|
+
elsif response =~ /\Aa/i
|
|
429
|
+
say("\n<%= color('Aborted by user.', :red) %>")
|
|
430
|
+
abort
|
|
376
431
|
else
|
|
377
|
-
say("\n<%= color('
|
|
432
|
+
say("\n<%= color('Skipping review of #{fname}.', :magenta) %>")
|
|
378
433
|
end
|
|
379
434
|
else
|
|
380
435
|
puts 'To flag the changes to this file as safe, run:'
|
|
@@ -436,17 +491,35 @@ File #{SAFETY_FILE} lists the safety and revision information
|
|
|
436
491
|
of the era source code. This task updates the list, and [TODO] warns about
|
|
437
492
|
files which have changed since they were last verified as safe."
|
|
438
493
|
task(:code) do
|
|
439
|
-
puts
|
|
440
|
-
|
|
494
|
+
puts <<~USAGE
|
|
495
|
+
|
|
496
|
+
NDR's code auditing tool.
|
|
497
|
+
|
|
498
|
+
Usage:
|
|
499
|
+
|
|
500
|
+
#{rake_cmd} audit:code
|
|
501
|
+
[interactive=false|true] # use an interactive wizard to perform code review
|
|
502
|
+
[max_print=n] # limit the number of summary rows to n
|
|
503
|
+
[ignore_new=false|true] # don't add new files to code_safety.yml
|
|
504
|
+
[show_diffs=false|true] # display the full diff for each file
|
|
505
|
+
[reviewed_by=usr] # your name, to author to any reviews added
|
|
506
|
+
[filter=path/to/input.csv] # filter reviewing to only files specified in the filter file
|
|
507
|
+
[outfile=path/to/output.csv] # dump a summary of outstanding review to CSV
|
|
508
|
+
|
|
509
|
+
USAGE
|
|
510
|
+
|
|
511
|
+
puts "This is a #{repository_type} repository."
|
|
512
|
+
puts
|
|
441
513
|
|
|
442
514
|
ignore_new = (ENV['ignore_new'].to_s =~ /\Atrue\Z/i)
|
|
443
515
|
show_diffs = (ENV['show_diffs'].to_s =~ /\Atrue\Z/i)
|
|
444
|
-
show_in_priority = (ENV['show_in_priority'].to_s =~ /\Atrue\Z/i)
|
|
445
516
|
max_print = ENV['max_print'] =~ /\A-?[0-9][0-9]*\Z/ ? ENV['max_print'].to_i : 20
|
|
446
517
|
reviewer = ENV['reviewed_by']
|
|
447
518
|
interactive = (ENV['interactive'].to_s =~ /\Atrue\Z/i)
|
|
519
|
+
filter = ENV['filter']
|
|
520
|
+
outfile = ENV['outfile']
|
|
448
521
|
|
|
449
|
-
all_safe = audit_code_safety(max_print, ignore_new, show_diffs,
|
|
522
|
+
all_safe = audit_code_safety(max_print, ignore_new, show_diffs, reviewer, interactive, outfile, filter)
|
|
450
523
|
|
|
451
524
|
unless show_diffs || interactive
|
|
452
525
|
puts "To show file diffs, run: #{rake_cmd} audit:code max_print=-1 show_diffs=true"
|
data/ndr_dev_support.gemspec
CHANGED
|
@@ -12,9 +12,7 @@ Gem::Specification.new do |spec|
|
|
|
12
12
|
spec.homepage = 'https://github.com/PublicHealthEngland/ndr_dev_support'
|
|
13
13
|
spec.license = 'MIT'
|
|
14
14
|
|
|
15
|
-
spec.files = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
|
|
16
|
-
# SECURE BNS 2018-08-06: Minimise sharing of (public-key encrypted) slack secrets in .travis.yml
|
|
17
|
-
spec.files -= %w[.travis.yml] # Not needed in the gem
|
|
15
|
+
spec.files = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(\.github|test|spec|features)/}) }
|
|
18
16
|
spec.bindir = 'exe'
|
|
19
17
|
spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
|
|
20
18
|
spec.require_paths = ['lib']
|
|
@@ -37,14 +35,14 @@ Gem::Specification.new do |spec|
|
|
|
37
35
|
# Integration test dependencies:
|
|
38
36
|
spec.add_dependency 'capybara', '>= 3.34'
|
|
39
37
|
spec.add_dependency 'capybara-screenshot'
|
|
40
|
-
spec.add_dependency 'minitest', '~> 5.
|
|
38
|
+
spec.add_dependency 'minitest', '~> 5.11'
|
|
41
39
|
spec.add_dependency 'poltergeist', '>= 1.8.0'
|
|
42
40
|
spec.add_dependency 'selenium-webdriver'
|
|
43
41
|
spec.add_dependency 'show_me_the_cookies'
|
|
44
42
|
spec.add_dependency 'webdrivers', '>= 3.9'
|
|
45
43
|
|
|
46
44
|
# CI server dependencies:
|
|
47
|
-
spec.add_dependency 'activesupport', '<
|
|
45
|
+
spec.add_dependency 'activesupport', '< 7'
|
|
48
46
|
spec.add_dependency 'brakeman', '>= 4.7.1'
|
|
49
47
|
spec.add_dependency 'bundler-audit'
|
|
50
48
|
spec.add_dependency 'github-linguist'
|
|
@@ -58,5 +56,5 @@ Gem::Specification.new do |spec|
|
|
|
58
56
|
|
|
59
57
|
spec.add_development_dependency 'bundler'
|
|
60
58
|
spec.add_development_dependency 'mocha'
|
|
61
|
-
spec.add_development_dependency 'rake', '
|
|
59
|
+
spec.add_development_dependency 'rake', '>= 12.3.3'
|
|
62
60
|
end
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: ndr_dev_support
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version:
|
|
4
|
+
version: 6.0.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- NCRS Development Team
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: exe
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2021-
|
|
11
|
+
date: 2021-07-09 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: pry
|
|
@@ -156,14 +156,14 @@ dependencies:
|
|
|
156
156
|
requirements:
|
|
157
157
|
- - "~>"
|
|
158
158
|
- !ruby/object:Gem::Version
|
|
159
|
-
version: '5.
|
|
159
|
+
version: '5.11'
|
|
160
160
|
type: :runtime
|
|
161
161
|
prerelease: false
|
|
162
162
|
version_requirements: !ruby/object:Gem::Requirement
|
|
163
163
|
requirements:
|
|
164
164
|
- - "~>"
|
|
165
165
|
- !ruby/object:Gem::Version
|
|
166
|
-
version: '5.
|
|
166
|
+
version: '5.11'
|
|
167
167
|
- !ruby/object:Gem::Dependency
|
|
168
168
|
name: poltergeist
|
|
169
169
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -226,14 +226,14 @@ dependencies:
|
|
|
226
226
|
requirements:
|
|
227
227
|
- - "<"
|
|
228
228
|
- !ruby/object:Gem::Version
|
|
229
|
-
version: '
|
|
229
|
+
version: '7'
|
|
230
230
|
type: :runtime
|
|
231
231
|
prerelease: false
|
|
232
232
|
version_requirements: !ruby/object:Gem::Requirement
|
|
233
233
|
requirements:
|
|
234
234
|
- - "<"
|
|
235
235
|
- !ruby/object:Gem::Version
|
|
236
|
-
version: '
|
|
236
|
+
version: '7'
|
|
237
237
|
- !ruby/object:Gem::Dependency
|
|
238
238
|
name: brakeman
|
|
239
239
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -378,9 +378,6 @@ dependencies:
|
|
|
378
378
|
name: rake
|
|
379
379
|
requirement: !ruby/object:Gem::Requirement
|
|
380
380
|
requirements:
|
|
381
|
-
- - "~>"
|
|
382
|
-
- !ruby/object:Gem::Version
|
|
383
|
-
version: '12.3'
|
|
384
381
|
- - ">="
|
|
385
382
|
- !ruby/object:Gem::Version
|
|
386
383
|
version: 12.3.3
|
|
@@ -388,9 +385,6 @@ dependencies:
|
|
|
388
385
|
prerelease: false
|
|
389
386
|
version_requirements: !ruby/object:Gem::Requirement
|
|
390
387
|
requirements:
|
|
391
|
-
- - "~>"
|
|
392
|
-
- !ruby/object:Gem::Version
|
|
393
|
-
version: '12.3'
|
|
394
388
|
- - ">="
|
|
395
389
|
- !ruby/object:Gem::Version
|
|
396
390
|
version: 12.3.3
|
|
@@ -415,6 +409,7 @@ files:
|
|
|
415
409
|
- config/rubocop/ndr.yml
|
|
416
410
|
- gemfiles/Gemfile.rails52
|
|
417
411
|
- gemfiles/Gemfile.rails60
|
|
412
|
+
- gemfiles/Gemfile.rails61
|
|
418
413
|
- lib/minitest/rake_ci.rb
|
|
419
414
|
- lib/minitest/rake_ci_plugin.rb
|
|
420
415
|
- lib/ndr_dev_support.rb
|
|
@@ -491,7 +486,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
491
486
|
- !ruby/object:Gem::Version
|
|
492
487
|
version: '0'
|
|
493
488
|
requirements: []
|
|
494
|
-
rubygems_version: 3.
|
|
489
|
+
rubygems_version: 3.1.6
|
|
495
490
|
signing_key:
|
|
496
491
|
specification_version: 4
|
|
497
492
|
summary: NDR Developer Support library
|