cassette_guard 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: a2a4d74a3f348d0ea84c8f97ae762dd031aa8e9612304e0de391a4f4fa10ba10
4
+ data.tar.gz: '09b1ae8190fd00e7a5f2f068e0a569559c4070b2ffe1a6579eeafdc8a59face3'
5
+ SHA512:
6
+ metadata.gz: 1e7563ad2acb8e38e78868ce4bfe06d5d1e7e9510b1e068adf420befae3a428eea0502e3976dd539770ad4797aadfc42c699080f38fc4b56445a431466bd02ca
7
+ data.tar.gz: 064cad727b758d8376cb3ff85b341724b3d7ea5525cb26369ad5dedcb1b335c83ec2903afac76a5221bce474c419d9289c53c2ae2f22b1881b7294e055462d95
data/CHANGELOG.md ADDED
@@ -0,0 +1,15 @@
1
+ # Changelog
2
+
3
+ ## 0.1.0
4
+
5
+ First release.
6
+
7
+ - `audit`: finds credentials, personal data, oversized cassettes and bodies, old recordings and unused
8
+ cassettes in Ruby VCR and Python vcrpy cassettes, YAML or JSON. Repeated values are grouped into one
9
+ finding, and secrets shared by several cassettes are listed together.
10
+ - `scrub`: replaces secrets and personal data with deterministic, format-preserving fakes, and writes each
11
+ cassette back the way its recorder does, so only the scrubbed values change.
12
+ - `trim`: replaces large binary bodies and base64 fields with a short stub.
13
+ - `check` and `baseline`: fail CI only on new findings.
14
+ - Text, JSON and SARIF reports.
15
+ - Usage trackers for RSpec (`CassetteGuard::Usage.track!`) and pytest (`python/cassette_guard_usage.py`).
data/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Opara Ifeanyi Christian
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,454 @@
1
+ # cassette_guard
2
+
3
+ Your VCR cassettes are full of real API traffic: bearer tokens, session cookies, customers' names,
4
+ emails, passport numbers, and 5 MB PDFs nobody looks at. `cassette_guard` finds them, scrubs them
5
+ without breaking your tests, and keeps new ones out in CI.
6
+
7
+ It reads and writes cassettes from Ruby's [VCR](https://github.com/vcr/vcr) and Python's
8
+ [vcrpy](https://github.com/kevin1024/vcrpy) (used by pytest-recording and pytest-vcr), YAML or JSON.
9
+
10
+ ```
11
+ $ cassette_guard audit # the demo cassettes in examples/, with max_body_size_kb: 4
12
+ cassettes/auth/login.yml 793 B
13
+ SECRET #0 request.body form.password key:password Tr… (17 chars)
14
+ PII #0 request.body form.username email ma… (22 chars)
15
+ SECRET #0 response.body $.access_token key:access_token ey… (116 chars)
16
+ SECRET #0 response.headers.Set-Cookie cookie:session_id 8f… (16 chars)
17
+
18
+ cassettes/booking/create_order.yml 8 KB
19
+ PII #0 request.body $.passengers[0].birth_date key:birth_date 19… (10 chars)
20
+ PII #0 request.body $.passengers[0].email key:email ma… (22 chars)
21
+ SECRET #0 request.headers.Authorization header:authorization ey… (116 chars)
22
+ SIZE #0 response.body 7 KB (application/json)
23
+ PII #0 response.body $.payment.iban key:iban DE… (22 chars)
24
+ …
25
+
26
+ cassettes/search/zurich_paris.yml 522 B
27
+ AGE recorded 2019-03-02 (2,819 days ago)
28
+
29
+ 3 cassettes · 20 findings (4 secret, 14 pii, 1 size, 1 age)
30
+ ```
31
+
32
+ Each finding names the interaction (`#0` is the first request/response pair in the cassette), the part of
33
+ it (`request.uri`, `request.headers.<name>`, `request.body`, and the same for `response`) and, for bodies,
34
+ the field: `$.passengers[0].email` for JSON, `form.password` for form data, `xml <Surname>` or
35
+ `xml @passportNumber` for XML. Values are never printed in full, and never written to the baseline file.
36
+
37
+ The same value at the same place in several interactions is one finding: `#0-3,5 request.body form.password …
38
+ ×5` is one password sent with five requests, and the summary counts findings, then the places they occur
39
+ (`… · 1 finding (1 secret) at 5 places`). A secret found in three or more cassettes is listed again under
40
+ **Secrets shared by several cassettes**, since it is one credential to rotate, however many files hold it.
41
+
42
+ > **Scrubbing does not remove anything from git history.** A token or address that was ever committed in a
43
+ > cassette is still in every earlier commit, and in every clone and fork of the repository. Treat any real
44
+ > credential `cassette_guard` finds as leaked: rotate it first, then scrub. If personal data has to be gone
45
+ > for good, rewrite the history as well (for example with `git filter-repo`).
46
+
47
+ ## Install
48
+
49
+ In a Ruby project, add it to your Gemfile and run it with `bundle exec cassette_guard`:
50
+
51
+ ```ruby
52
+ # Gemfile
53
+ group :development, :test do
54
+ gem "cassette_guard", require: false
55
+ end
56
+ ```
57
+
58
+ Or install it on its own, which puts the `cassette_guard` command on your `PATH`:
59
+
60
+ ```bash
61
+ gem install cassette_guard
62
+ ```
63
+
64
+ No runtime dependencies. Ruby 3.3+.
65
+
66
+ To run the latest unreleased code, point your Gemfile at the repository instead:
67
+ `gem "cassette_guard", git: "https://gitlab.com/chrisgeeq/cassette_guard.git", require: false`.
68
+
69
+ ### Python projects
70
+
71
+ The tool itself is Ruby, so install Ruby 3.3+ and the gem; your project needs nothing else:
72
+
73
+ ```bash
74
+ gem install cassette_guard
75
+ ```
76
+
77
+ ```bash
78
+ cassette_guard audit tests/cassettes
79
+ ```
80
+
81
+ The pytest usage tracker is not part of the gem: copy
82
+ [`python/cassette_guard_usage.py`](https://gitlab.com/chrisgeeq/cassette_guard/-/blob/main/python/cassette_guard_usage.py)
83
+ from the repository (see [Unused cassettes](#unused-cassettes)).
84
+
85
+ `tests/cassettes`, `tests/fixtures/cassettes` and the other usual pytest locations are found without
86
+ configuration. Cassettes need a `.yaml`, `.yml` or `.json` extension.
87
+
88
+ ## First run
89
+
90
+ 1. **See what is there.** Nothing is changed.
91
+
92
+ ```bash
93
+ cassette_guard audit
94
+ ```
95
+
96
+ 2. **Rotate every real credential the audit found** (see the warning above). Scrubbing hides a secret
97
+ from now on; it does not un-leak it.
98
+
99
+ 3. **Preview the scrub.** It prints every replacement it would make and writes nothing.
100
+
101
+ ```bash
102
+ cassette_guard scrub --dry-run
103
+ ```
104
+
105
+ 4. **Scrub**, with the team's private salt (see [The salt](#the-salt)):
106
+
107
+ ```bash
108
+ CASSETTE_GUARD_SALT="$CASSETTE_GUARD_SALT" cassette_guard scrub
109
+ ```
110
+
111
+ 5. **Run your tests**, then review the diff of the cassettes before committing. If a test fails, see
112
+ [When a test fails after scrubbing](#when-a-test-fails-after-scrubbing).
113
+
114
+ 6. **Accept what is left** (findings you have decided to keep, old cassettes you will re-record later) and
115
+ commit `.cassette_guard_baseline.yml`:
116
+
117
+ ```bash
118
+ cassette_guard baseline
119
+ ```
120
+
121
+ 7. **Add `cassette_guard check` to CI** (see [In CI](#in-ci)), so new findings fail the build.
122
+
123
+ ## Commands
124
+
125
+ | Command | What it does |
126
+ |---|---|
127
+ | `cassette_guard audit` | Report everything not in the baseline. Changes nothing. Default command. |
128
+ | `cassette_guard scrub` | Replace secrets and personal data with deterministic fakes. |
129
+ | `cassette_guard trim` | Replace large binary bodies and large base64 fields with a short stub. Opt-in. |
130
+ | `cassette_guard check` | For CI: report findings missing from the baseline, and exit 1 if there are any. |
131
+ | `cassette_guard baseline` | Accept every current finding, so you can adopt `check` today and fix the backlog later. |
132
+ | `cassette_guard version` | Print the version. |
133
+ | `cassette_guard help` | Print the commands and options. |
134
+
135
+ Pass directories, files or glob patterns to scan; they replace `cassettes:` from the configuration. With
136
+ neither, `cassette_guard` looks in `spec/fixtures/vcr_cassettes`, `spec/vcr_cassettes`, `spec/cassettes`,
137
+ `spec/fixtures/cassettes`, `test/vcr_cassettes`, `test/cassettes`, `test/fixtures/vcr_cassettes`,
138
+ `tests/cassettes`, `tests/fixtures/cassettes`, `tests/fixtures/vcr_cassettes` and `tests/vcr_cassettes`.
139
+ Files that are not cassettes are skipped and listed at the end of the report.
140
+
141
+ ### Options
142
+
143
+ | Option | Applies to | What it does |
144
+ |---|---|---|
145
+ | `-c`, `--config FILE` | all | Configuration file. Default: `.cassette_guard.yml` if it exists. |
146
+ | `-f`, `--format FORMAT` | `audit`, `check` | `text` (default), `json` or `sarif`. |
147
+ | `-o`, `--output FILE` | `audit`, `check` | Write the report to `FILE` instead of standard output. |
148
+ | `--all` | `audit` | Include findings that are in the baseline. |
149
+ | `--dry-run` | `scrub`, `trim` | Show what would change, write nothing. |
150
+ | `--only-responses` | `scrub` | Leave recorded requests untouched. |
151
+ | `--min-size KB` | `trim` | Smallest body or field to trim, in KB, at least 1. Default 100. |
152
+ | `--usage-dir DIR` | `audit`, `check`, `baseline` | Where the usage trackers wrote their files. Default `tmp/cassette_guard`. |
153
+ | `--baseline FILE` | `audit`, `check`, `baseline`, `scrub` | Baseline file. Default `.cassette_guard_baseline.yml`. |
154
+ | `-h`, `--help` | all | Print the commands and options. |
155
+
156
+ ### Exit codes
157
+
158
+ | Code | When |
159
+ |---|---|
160
+ | `0` | `audit`, `baseline`, `trim`, `version` and `help` when they ran. `check` with no findings outside the baseline. `scrub` when every value was scrubbed, or there was nothing to scrub. |
161
+ | `1` | `check` found findings that are not in the baseline. `scrub` could not replace some values, or could not write a cassette; both are listed at the end of its output. |
162
+ | `2` | Bad options or configuration: an unknown option or format, a `--config` file that does not exist, `trim_min_size_kb` under 1, or no cassettes found. |
163
+
164
+ `trim` keeps going when one cassette cannot be read or written: it prints `Skipped <path>: <reason>` and
165
+ still exits 0.
166
+
167
+ ### JSON output
168
+
169
+ `--format json` writes one object:
170
+
171
+ ```json
172
+ {
173
+ "version": "0.1.0",
174
+ "cassettes": 3,
175
+ "usage_checked": false,
176
+ "hidden": 0,
177
+ "occurrences": 20,
178
+ "findings": [
179
+ {
180
+ "rule": "pii",
181
+ "kind": "key:email",
182
+ "cassette": "cassettes/booking/create_order.yml",
183
+ "location": "request.body $.passengers[0].email",
184
+ "detail": "key:email ma… (22 chars)",
185
+ "fingerprint": "c4f1d1a0e9b2…",
186
+ "interactions": [0],
187
+ "occurrences": 1
188
+ }
189
+ ],
190
+ "skipped": [{ "path": "cassettes/notes.yml", "reason": "not a VCR cassette" }]
191
+ }
192
+ ```
193
+
194
+ `rule` is one of `secret`, `pii`, `size`, `age` or `unused`. Each entry is one finding; `interactions` lists
195
+ every interaction it occurs in and `occurrences` how many places that is, while the top-level `occurrences`
196
+ is the total. `interactions` is empty, and `location` left out, for findings about a whole cassette.
197
+ `hidden` counts findings left out because they are in the baseline, and
198
+ `usage_checked` says whether unused cassettes were checked at all. `--format sarif` writes SARIF 2.1.0 for
199
+ GitHub code scanning (see [In CI](#in-ci)).
200
+
201
+ ## What it looks for
202
+
203
+ Bodies are decoded first (base64, gzip) and parsed as JSON, XML/SOAP or form data, so every finding
204
+ names its exact field.
205
+
206
+ - **Secrets**: `Authorization`, `X-Api-Key` and similar headers; session cookies in `Cookie` and `Set-Cookie`;
207
+ values under keys like `password`, `access_token`, `client_secret`, `signature`; passwords in URIs; and JWTs,
208
+ AWS keys, Stripe live keys, GitHub and Slack tokens, Google API keys and private keys anywhere.
209
+ A cookie counts when its name says it authenticates someone (`JSESSIONID`, `session_id`, `auth_token`,
210
+ `remember_me`: names containing `sess`, `sid`, `token`, `auth`, `jwt`, `login`, `remember`, `secret`,
211
+ `refresh`, `access`, `identity`, `saml` or `oauth`, but not CSRF/XSRF tokens) and its value is 8 characters or
212
+ more. Other cookies (analytics ids, load-balancer stickiness, correlation ids, preferences) are only
213
+ reported if they contain a recognizable token such as a JWT. Add your own with `secret_cookies:`.
214
+ - **Personal data**: values under keys like `email`, `first_name`, `lastName`, `birth_date`, `passport_number`,
215
+ `phone`, `iban`, `card_number` (camelCase, snake_case and SOAP element names alike). Without a key, only
216
+ values that validate count: an email address, an IBAN with a correct checksum, a card number that passes Luhn.
217
+ - **Size**: cassettes over 1 MB, bodies over 256 KB.
218
+ - **Age**: cassettes recorded more than a year ago (from `recorded_at`, or the response `Date` header for vcrpy).
219
+ - **Unused**: cassettes that no test loaded (see [Unused cassettes](#unused-cassettes)).
220
+
221
+ Placeholders such as `<API_KEY>`, `[FILTERED]` or `***`, addresses at `example.com`, and well-known test
222
+ card numbers are ignored.
223
+
224
+ ## Scrubbing without breaking tests
225
+
226
+ ```
227
+ $ CASSETTE_GUARD_SALT=… cassette_guard scrub
228
+ cassettes/booking/create_order.yml
229
+ #0 response.body $.passengers[0].birth_date key:birth_date 19… (10 chars) → 1946-01-01
230
+ #0 response.body $.passengers[0].email key:email ma… (22 chars) → user-2a1129bd@example.com
231
+ #0 response.body $.passengers[0].first_name key:first_name Ma… (5 chars) → Daywv
232
+ #0 response.body $.payment.iban key:iban DE… (22 chars) → DE52345454508016988861
233
+ …
234
+ ```
235
+
236
+ - **Same input, same fake.** Fakes come from a keyed hash of the original, so a token issued in one
237
+ response and sent back in the next request, or a booking reference shared by two cassettes, still matches.
238
+ - **Same shape.** Names keep their length and case, dates stay valid dates in the same format, IBANs and
239
+ card numbers still validate, emails become `user-…@example.com`, secrets become `scrubbed-…`.
240
+ - **Only where it belongs.** Short values like a first name are replaced only next to their key, so a
241
+ passenger called Paris leaves the station called Paris alone. Long, distinctive values (tokens, emails,
242
+ document numbers) are replaced in every encoding they appear in: raw, JSON-escaped, URL-encoded, XML-escaped.
243
+ - **Small diffs.** Cassettes are written back exactly as their recorder writes them: Psych for VCR, and for
244
+ vcrpy PyYAML's output byte for byte, including its `\xFC`-escaped non-ASCII strings and line folding (or
245
+ `json.dumps(indent=4)` for JSON cassettes). Gzip, base64 and `!!binary` bodies are re-encoded the way they
246
+ were stored, and `Content-Length` is updated when it described the old body.
247
+ - **Idempotent.** The fakes are recorded in `.cassette_guard_baseline.yml`, so the next run leaves them alone.
248
+ Commit that file.
249
+
250
+ Old vcrpy cassettes holding `!!python/...` tags are reported but never rewritten; re-record them first.
251
+
252
+ ### The salt
253
+
254
+ Fakes are derived from the originals with a keyed hash, and the key is the salt. With the built-in salt,
255
+ anyone could recover short values such as birth dates by hashing every candidate, so `scrub` warns when it
256
+ uses it.
257
+
258
+ - Use **one** private salt for the whole team and CI. Fakes made with a different salt do not match, so a
259
+ value scrubbed on one machine would no longer match the same value scrubbed later on another.
260
+ - Keep it where your other secrets live: a CI secret, the team's password manager. Pass it as
261
+ `CASSETTE_GUARD_SALT`, which takes precedence over `salt:` in the configuration.
262
+ - Do not put it in `.cassette_guard.yml`: that file is committed, and a published salt is as good as none.
263
+
264
+ ### When a test fails after scrubbing
265
+
266
+ If no recorded request matches any more (`VCR::Errors::UnhandledHTTPRequestError`, or vcrpy's
267
+ `CannotOverwriteExistingCassetteException`), the value was sent by the test itself and requests are matched
268
+ on something that changed: the body, or for vcrpy, which matches on the query by default, a query
269
+ parameter. Change the value in the test, scrub with `--only-responses`, or stop matching on it.
270
+
271
+ For secret headers, `scrub` prints the configuration that keeps them out of the next recording:
272
+ `filter_sensitive_data` for VCR, `filter_headers` and `before_record_response` for vcrpy.
273
+
274
+ ## Trimming large bodies
275
+
276
+ `trim` is opt-in, because any test that reads a trimmed body gets the stub instead.
277
+
278
+ ```bash
279
+ cassette_guard trim --dry-run
280
+ ```
281
+
282
+ - **Binary bodies** of at least `--min-size` (default 100 KB) are replaced with one line of text:
283
+ `cassette_guard: trimmed 1.9 MB application/pdf, sha256:63994a27c4c3ee1d`. The hash is of the original
284
+ body, so you can still tell two recordings apart. A body counts as binary when it is not valid UTF-8, or
285
+ when its content type is PDF, an image, audio, video, a font, `application/octet-stream`, an archive
286
+ (zip, gzip, tar), protobuf or a Microsoft Office document.
287
+ - **Base64 fields** inside JSON, XML or form bodies, such as a ticket PDF embedded in a JSON response, are
288
+ replaced with the base64 encoding of that same stub, so code that decodes them still gets text. Fields
289
+ under 1 KB are never trimmed, whatever `--min-size` says: below that, an alphanumeric value is as likely
290
+ to be a name or a token as a file.
291
+ - `Content-Length` is updated, and a gzipped binary body loses its `Content-Encoding` header, since the
292
+ stub is plain text.
293
+
294
+ ## The baseline
295
+
296
+ `.cassette_guard_baseline.yml` lists findings that `audit` and `check` leave out. Each entry holds a
297
+ fingerprint, where the finding is (rule, kind, cassette, location) and why it is there: `accepted` for
298
+ findings accepted with `cassette_guard baseline`, `scrubbed` for the fakes `scrub` wrote. It never holds a
299
+ value.
300
+
301
+ - The fingerprint covers the rule, kind, cassette path and location, and a SHA-256 of the value. A finding
302
+ comes back when its value changes (a re-recording brought new data) or when the cassette is moved or
303
+ renamed.
304
+ - Entries for findings that no longer exist stay until you run `cassette_guard baseline` again.
305
+ - Running `cassette_guard baseline` again accepts **everything** currently found, new findings included.
306
+ Run `check` first and deal with what it reports.
307
+
308
+ ## In CI
309
+
310
+ ```bash
311
+ cassette_guard baseline # once, commit .cassette_guard_baseline.yml
312
+ ```
313
+
314
+ ```bash
315
+ cassette_guard check # on every build
316
+ ```
317
+
318
+ GitHub code scanning shows findings as annotations on the pull request:
319
+
320
+ ```yaml
321
+ - run: bundle exec cassette_guard check --format sarif --output cassette_guard.sarif
322
+ - uses: github/codeql-action/upload-sarif@v3
323
+ if: always()
324
+ with:
325
+ sarif_file: cassette_guard.sarif
326
+ ```
327
+
328
+ ## Unused cassettes
329
+
330
+ Finding unused cassettes by grepping the code misses names built at runtime and names VCR derives from
331
+ example metadata, so `cassette_guard` records which cassettes your tests actually load:
332
+
333
+ ```ruby
334
+ # spec/spec_helper.rb, after VCR is configured
335
+ require "cassette_guard/usage"
336
+ CassetteGuard::Usage.track!
337
+ ```
338
+
339
+ For pytest, copy [`python/cassette_guard_usage.py`](https://gitlab.com/chrisgeeq/cassette_guard/-/blob/main/python/cassette_guard_usage.py)
340
+ from the repository next to your tests:
341
+
342
+ ```python
343
+ # conftest.py
344
+ import cassette_guard_usage
345
+
346
+ cassette_guard_usage.track()
347
+ ```
348
+
349
+ Paths are recorded as absolute, so it doesn't matter which directory the tests or the audit run from, as
350
+ long as both run in the same checkout.
351
+
352
+ Both trackers write to `tmp/cassette_guard`, or to `$CASSETTE_GUARD_USAGE_DIR` when it is set. Only the
353
+ trackers read that variable: if you change the directory, pass the same one to the audit with
354
+ `--usage-dir` or `usage_dir:`.
355
+
356
+ Run the **whole** suite (each process writes its own file, so parallel runners and pytest-xdist are fine), then
357
+ `cassette_guard audit`. Clear `tmp/cassette_guard` before the next full run; a partial run would make every
358
+ cassette it skipped look unused.
359
+
360
+ ## Configuration
361
+
362
+ `.cassette_guard.yml` in the directory you run `cassette_guard` from, or any file passed with `--config`.
363
+ All keys are optional:
364
+
365
+ ```yaml
366
+ cassettes: [spec/fixtures/vcr_cassettes] # directories, files or glob patterns
367
+ max_file_size_kb: 1024
368
+ max_body_size_kb: 256
369
+ max_age_days: 365
370
+ trim_min_size_kb: 100 # at least 1
371
+ usage_dir: tmp/cassette_guard
372
+ baseline: .cassette_guard_baseline.yml
373
+ pii_keys: [loyalty_number, nachname] # extra keys holding personal data
374
+ secret_keys: [partner_signature] # extra keys holding secrets
375
+ secret_headers: [x-partner-key]
376
+ secret_cookies: [partner_ref] # extra cookie names holding secrets
377
+ allow:
378
+ values: ["support@acme.io"]
379
+ emails: ["*@acme-sandbox.io"]
380
+ keys: [station_name]
381
+ ```
382
+
383
+ There is also a `salt:` key, but keep the salt out of this file; see [The salt](#the-salt).
384
+
385
+ Environment variables:
386
+
387
+ | Variable | Read by | What it does |
388
+ |---|---|---|
389
+ | `CASSETTE_GUARD_SALT` | `scrub` | The salt for fakes. Takes precedence over `salt:`. |
390
+ | `CASSETTE_GUARD_USAGE_DIR` | the usage trackers | Where they write. `audit` reads `--usage-dir` / `usage_dir:` instead. |
391
+
392
+ ## Limitations
393
+
394
+ Detection is a heuristic, not a guarantee. Review the diff of every scrub before you commit it.
395
+
396
+ - **Key names are English.** A generic `name` key is not treated as personal data, since it names stations
397
+ and products as often as people. Keys in other languages (`nom`, `vorname`, `cognome`) are not recognised:
398
+ add them with `pii_keys:`. Free text is only checked for emails, IBANs and card numbers, so a name in a
399
+ sentence is not found.
400
+ - **XML** is read through its leaf elements and attributes only; text mixed with child elements and CDATA
401
+ sections are not checked.
402
+ - **Binary bodies are not scanned.** A PDF ticket or a scanned passport can carry personal data that
403
+ `cassette_guard` never reads; `trim` them if your tests do not need them.
404
+ - **File names**: only `.yml`, `.yaml` and `.json` files are read; cassettes without an extension are skipped.
405
+ - **Ruby VCR**: the `:yaml`, `:psych` and `:json` serializers are supported. `:compressed` cassettes and
406
+ custom serializers are not.
407
+ - **vcrpy**: the `yaml` and `json` serializers are supported. Cassettes holding `!!python/...` tags, or written
408
+ by a custom serializer with its own tags, are reported but never rewritten.
409
+ - **Unused** cassettes are only as good as the run that was tracked: run the whole suite.
410
+
411
+ ## Tested with
412
+
413
+ - Ruby 3.3.5, VCR 6.3
414
+ - Python 3.11, vcrpy 9.4.2, PyYAML 6.0 with libyaml
415
+
416
+ CI is configured for Ruby 3.3, 3.4 and 4.0, and Python 3.12, but has not run yet.
417
+
418
+ ## Development
419
+
420
+ ```bash
421
+ bundle install
422
+ bundle exec rspec
423
+ ruby examples/generate.rb # rebuilds the demo cassettes (all data in them is made up)
424
+ exe/cassette_guard audit examples/cassettes
425
+ ```
426
+
427
+ ```bash
428
+ pip install vcrpy pytest requests
429
+ pytest python # the pytest usage tracker
430
+ python spec/fixtures/vcrpy/record.py # re-records the vcrpy fixtures against a local server
431
+ python spec/fixtures/vcrpy/edge_cases.py # rewrites the PyYAML edge-case fixtures
432
+ ```
433
+
434
+ ### Releasing
435
+
436
+ Bump `CassetteGuard::VERSION` in `lib/cassette_guard/version.rb`, add the release to `CHANGELOG.md`, commit, then:
437
+
438
+ ```bash
439
+ bundle exec rake release
440
+ ```
441
+
442
+ It builds the gem, tags the commit `vX.Y.Z`, pushes the commit and tag to GitLab, and pushes the gem to RubyGems
443
+ (which asks for your one-time code: the gem requires MFA). It refuses to run with uncommitted changes.
444
+
445
+ ## Roadmap
446
+
447
+ - A `pip install` package, so Python projects don't need Ruby
448
+ - Adapters for go-vcr and Polly.js recordings
449
+ - A GitHub App that comments on pull requests adding sensitive cassettes
450
+ - Drift detection: replay recorded requests against a sandbox and report changed response shapes
451
+
452
+ ## Licence
453
+
454
+ MIT; see [LICENSE.txt](LICENSE.txt).
@@ -0,0 +1,6 @@
1
+ #!/usr/bin/env ruby
2
+ # frozen_string_literal: true
3
+
4
+ require_relative "../lib/cassette_guard"
5
+
6
+ exit CassetteGuard::CLI.new(ARGV).run
@@ -0,0 +1,59 @@
1
+ # frozen_string_literal: true
2
+
3
+ module CassetteGuard
4
+ # Findings that `check` ignores. Entries hold a fingerprint and where the finding is, never the value.
5
+ class Baseline
6
+ HEADER = <<~YAML
7
+ # Findings accepted with `cassette_guard baseline`, or fakes written by `cassette_guard scrub`.
8
+ # `cassette_guard check` ignores them. Commit this file.
9
+ YAML
10
+
11
+ attr_reader :path
12
+
13
+ def self.load(path)
14
+ return new(path) unless File.exist?(path)
15
+
16
+ data = YAML.safe_load_file(path) || {}
17
+ raise Error, "#{path}: expected a mapping" unless data.is_a?(Hash)
18
+
19
+ new(path, Array(data["findings"]))
20
+ end
21
+
22
+ def initialize(path, entries = [])
23
+ @path = path
24
+ @entries = entries.to_h { |entry| [entry["fingerprint"], entry] }
25
+ end
26
+
27
+ def include?(finding)
28
+ @entries.key?(finding.fingerprint)
29
+ end
30
+
31
+ def reason_for(finding)
32
+ @entries.dig(finding.fingerprint, "reason")
33
+ end
34
+
35
+ def add(finding, reason:)
36
+ @entries[finding.fingerprint] = {
37
+ "fingerprint" => finding.fingerprint,
38
+ "rule" => finding.rule.to_s,
39
+ "kind" => finding.kind,
40
+ "cassette" => finding.cassette,
41
+ "location" => finding.location,
42
+ "reason" => reason,
43
+ }.compact
44
+ end
45
+
46
+ def size
47
+ @entries.size
48
+ end
49
+
50
+ def count(reason)
51
+ @entries.count { |_, entry| entry["reason"] == reason }
52
+ end
53
+
54
+ def save!
55
+ entries = @entries.values.sort_by { |entry| entry.values_at("cassette", "location", "kind", "fingerprint").map(&:to_s) }
56
+ File.write(path, HEADER + YAML.dump({ "version" => 1, "findings" => entries }).delete_prefix("---\n"))
57
+ end
58
+ end
59
+ end
@@ -0,0 +1,100 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "zlib"
4
+
5
+ module CassetteGuard
6
+ # A recorded body, decoded from its storage and from gzip if needed. Storage differs by recorder:
7
+ # Ruby VCR keeps {"string" => ...} or {"base64_string" => ...}; vcrpy keeps the request body as a bare
8
+ # string and the response body as {"string" => ...}, either one possibly `!!binary`.
9
+ class Body
10
+ GZIP_MAGIC = "\x1F\x8B".b.freeze
11
+ BINARY_TYPES = %r{
12
+ \A(?:image|audio|video|font)/ |
13
+ \Aapplication/(?:pdf|octet-stream|zip|gzip|x-gzip|x-tar|x-protobuf|msword|vnd\.ms-|vnd\.openxmlformats)
14
+ }x
15
+
16
+ attr_reader :headers
17
+
18
+ def self.for(part)
19
+ case part["body"]
20
+ when Hash then new(part["body"], part["body"].key?("base64_string") ? "base64_string" : "string", part["headers"])
21
+ when String then new(part, "body", part["headers"])
22
+ end
23
+ end
24
+
25
+ def initialize(container, key, headers)
26
+ @container = container
27
+ @key = key
28
+ @headers = headers.is_a?(Hash) ? headers : {}
29
+ end
30
+
31
+ def base64?
32
+ @key == "base64_string"
33
+ end
34
+
35
+ def raw_bytes
36
+ value = @container[@key].to_s
37
+ base64? ? value.unpack1("m") : value
38
+ end
39
+
40
+ def size
41
+ raw_bytes.bytesize
42
+ end
43
+
44
+ def gzipped?
45
+ raw_bytes.b.start_with?(GZIP_MAGIC)
46
+ end
47
+
48
+ def content_type
49
+ Headers.get(headers, "content-type")&.split(";")&.first&.strip&.downcase
50
+ end
51
+
52
+ def text
53
+ return @text if defined?(@text)
54
+
55
+ @text = decode
56
+ end
57
+
58
+ def binary?
59
+ text.nil? || content_type.to_s.match?(BINARY_TYPES)
60
+ end
61
+
62
+ # Re-encodes the way the body was stored: gzip, base64 and binary storage are kept.
63
+ def text=(new_text)
64
+ bytes = gzipped? ? Zlib.gzip(new_text) : new_text
65
+ @container[@key] =
66
+ if base64? then [bytes].pack("m").force_encoding(Encoding::UTF_8)
67
+ elsif binary_storage? then bytes.b
68
+ else bytes.dup.force_encoding(Encoding::UTF_8)
69
+ end
70
+ @text = new_text
71
+ end
72
+
73
+ # Stores plain text, dropping base64, binary storage and compression.
74
+ def replace(new_text)
75
+ if base64?
76
+ @container.delete("base64_string")
77
+ @key = "string"
78
+ end
79
+ @container["encoding"] = "UTF-8" if @container.key?("encoding")
80
+ @container[@key] = new_text.dup.force_encoding(Encoding::UTF_8)
81
+ @text = new_text
82
+ end
83
+
84
+ private
85
+
86
+ def binary_storage?
87
+ value = @container[@key]
88
+ !base64? && value.is_a?(String) && value.encoding == Encoding::BINARY
89
+ end
90
+
91
+ def decode
92
+ bytes = raw_bytes
93
+ bytes = Zlib.gunzip(bytes.b) if gzipped?
94
+ decoded = bytes.dup.force_encoding(Encoding::UTF_8)
95
+ decoded.valid_encoding? ? decoded : nil
96
+ rescue Zlib::Error
97
+ nil
98
+ end
99
+ end
100
+ end