residoo 0.4.4 → 0.4.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +36 -23
- package/package.json +1 -1
- package/src/cli.js +28 -19
- package/src/report.js +18 -3
- package/src/rotation.js +36 -21
- package/src/scan.js +84 -20
- package/src/verify.js +152 -18
package/README.md
CHANGED
|
@@ -123,18 +123,22 @@ won't be built into the tool that writes it.
|
|
|
123
123
|
in the Rotation section, instead of just "last seen." Only `exp` is ever
|
|
124
124
|
read; every other claim in the payload is decoded transiently and
|
|
125
125
|
discarded. See `src/jwtExpiry.js`.
|
|
126
|
-
- **`--verify`** (opt-in, makes a real network call):
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
126
|
+
- **`--verify`** (opt-in, makes a real network call): asks a credential's own
|
|
127
|
+
vendor whether it still authenticates, using the exact value found in your
|
|
128
|
+
transcript. Five vendors today: **AWS** (an access key id found paired with
|
|
129
|
+
its secret, checked via `sts:get-caller-identity`, the same free,
|
|
130
|
+
read-only, permission-less call the AWS CLI and tools like aws-vault use
|
|
131
|
+
for exactly this; shells out to your own `aws` CLI rather than
|
|
132
|
+
reimplementing AWS request signing, since residoo ships zero runtime
|
|
133
|
+
dependencies and a subtly wrong signing implementation would silently
|
|
134
|
+
report real keys as invalid, worse than not checking), and **Slack,
|
|
135
|
+
OpenAI, Anthropic, GitHub** (a direct API call to each vendor's own free
|
|
136
|
+
"list what I can see" endpoint, no CLI needed, no request signing to get
|
|
137
|
+
wrong). A verified-active credential is escalated to "rotate immediately";
|
|
138
|
+
a verified-invalid one is reported as already dead, no action needed, and
|
|
139
|
+
sorted out of the way. Off by default; every environment variable the
|
|
140
|
+
`aws` CLI reads is built from scratch, never inherited, so it can never
|
|
141
|
+
fall back to your own real AWS profile. See `src/verify.js`.
|
|
138
142
|
- With `--include-noisy`, filters the broad generic-secret rules by how
|
|
139
143
|
machine-random the matched value actually looks (a lightweight, offline
|
|
140
144
|
approximation of BPE-tokenization rarity checks): ordinary English, a
|
|
@@ -146,10 +150,10 @@ won't be built into the tool that writes it.
|
|
|
146
150
|
preview, never the real value, including in `--json` mode. A decoded or
|
|
147
151
|
rejoined secret is redacted exactly like a plain one.
|
|
148
152
|
- On an interactive terminal, prints who it is and where it lives before
|
|
149
|
-
scanning starts (`residoo v0.4.
|
|
153
|
+
scanning starts (`residoo v0.4.5 · find secrets your AI coding agent left
|
|
150
154
|
on disk` plus the repo URL), then a live spinner naming the current file
|
|
151
155
|
as it scans. Every report also opens with the exact version and timestamp
|
|
152
|
-
it was run with (`residoo v0.4.
|
|
156
|
+
it was run with (`residoo v0.4.5 · scanned 2026-01-01 12:00`; `--json`
|
|
153
157
|
carries the same as `residooVersion`/`scannedAt`), so a report pasted or
|
|
154
158
|
screenshotted later never leaves you guessing which build produced it.
|
|
155
159
|
When there are findings, the report closes with a "Next steps" pointer to
|
|
@@ -335,7 +339,14 @@ with the way out:
|
|
|
335
339
|
versus how many are already resolved. A machine with a lot of history can
|
|
336
340
|
report hundreds of raw findings that are really a handful of distinct
|
|
337
341
|
values echoed repeatedly; the summary is built around what's actually left
|
|
338
|
-
to triage, not the raw count.
|
|
342
|
+
to triage, not the raw count. A value `--verify` confirmed dead, or a JWT
|
|
343
|
+
whose own signed `exp` claim is already past, is subtracted from "needs
|
|
344
|
+
review" the same way an acked or dismissed one is, since residoo already
|
|
345
|
+
knows it needs no action, not just that nobody has said so yet. This is a
|
|
346
|
+
strictly per-VALUE fact: it is never rolled up into a whole rule's
|
|
347
|
+
confidence tag in the breakdown below, since `--verify` only ever checks
|
|
348
|
+
the specific values it can (a paired AWS credential, a bearer token), and
|
|
349
|
+
a rule's other, unchecked findings say nothing either way.
|
|
339
350
|
- **The rotation list is grouped by credential type**, so the rotation URL
|
|
340
351
|
prints once per type instead of once per finding. Each distinct value's own
|
|
341
352
|
line shows its redacted preview, which file it's in, and when it was last
|
|
@@ -373,7 +384,7 @@ As a GitHub Action (this repository doubles as a composite action):
|
|
|
373
384
|
```yaml
|
|
374
385
|
steps:
|
|
375
386
|
- uses: actions/checkout@v4
|
|
376
|
-
- uses: dandovdub/residoo@v0.4.
|
|
387
|
+
- uses: dandovdub/residoo@v0.4.5
|
|
377
388
|
```
|
|
378
389
|
|
|
379
390
|
As a pre-commit hook:
|
|
@@ -381,7 +392,7 @@ As a pre-commit hook:
|
|
|
381
392
|
```yaml
|
|
382
393
|
repos:
|
|
383
394
|
- repo: https://github.com/dandovdub/residoo
|
|
384
|
-
rev: v0.4.
|
|
395
|
+
rev: v0.4.5
|
|
385
396
|
hooks:
|
|
386
397
|
- id: residoo
|
|
387
398
|
```
|
|
@@ -398,12 +409,14 @@ documented in [docs/ci.md](docs/ci.md).
|
|
|
398
409
|
- **No network calls in the default path, and none at all unless you
|
|
399
410
|
explicitly pass `--upload-cloudroam` or `--verify`.** A secret scanner that
|
|
400
411
|
phones home is not a tool you should trust with your secrets. Verify this
|
|
401
|
-
yourself:
|
|
402
|
-
|
|
403
|
-
`--
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
|
|
412
|
+
yourself: every network-capable call in the codebase lives behind one of
|
|
413
|
+
those two flags. `src/sealvault.js` holds the one `fetch` call reachable
|
|
414
|
+
from `--upload-cloudroam`, and sends only encrypted bytes. `src/verify.js`
|
|
415
|
+
holds everything reachable from `--verify`: a `fetch` call per vendor
|
|
416
|
+
(Slack, OpenAI, Anthropic, GitHub), each sending nothing but the exact
|
|
417
|
+
credential a scan found to that credential's own vendor, plus a subprocess
|
|
418
|
+
call to your own `aws` CLI for AWS credentials, never a `fetch`. Neither
|
|
419
|
+
file's code runs unless you pass the matching flag.
|
|
407
420
|
- **Nothing destructive, ever.** Scanning is read-only. Sealing creates *new*
|
|
408
421
|
files and modifies or deletes nothing, not even the plaintext it just
|
|
409
422
|
encrypted a copy of. That last step is deliberately left to a human.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "residoo",
|
|
3
|
-
"version": "0.4.
|
|
3
|
+
"version": "0.4.5",
|
|
4
4
|
"description": "Find secrets leaking through your AI coding agent's session history. Zero network calls in the scan path, zero dependencies.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"author": "CloudRoam (https://cloudroam.io)",
|
package/src/cli.js
CHANGED
|
@@ -56,9 +56,10 @@ const HELP = `residoo: find secrets leaking through your AI agent's session hist
|
|
|
56
56
|
|
|
57
57
|
Scanning makes NO network calls by default and changes nothing on disk.
|
|
58
58
|
Findings are redacted in every output format. The one opt-in exception is
|
|
59
|
-
--verify, which asks
|
|
60
|
-
authenticates
|
|
61
|
-
only. It never modifies or
|
|
59
|
+
--verify, which asks a credential's own vendor whether it still
|
|
60
|
+
authenticates (AWS, Slack, OpenAI, Anthropic, GitHub today); see below.
|
|
61
|
+
Sealing (--seal) writes NEW encrypted files only. It never modifies or
|
|
62
|
+
deletes anything that already exists.
|
|
62
63
|
|
|
63
64
|
Usage:
|
|
64
65
|
residoo scan [options]
|
|
@@ -98,17 +99,24 @@ Scan options:
|
|
|
98
99
|
--no-integrity skip the integrity checks (planted hooks, dropper
|
|
99
100
|
files, auto-run tasks, hidden Unicode)
|
|
100
101
|
--no-color disable ANSI colour
|
|
101
|
-
--verify
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
102
|
+
--verify ask the credential's own vendor whether it still
|
|
103
|
+
authenticates, using the exact value found in
|
|
104
|
+
your transcript. THIS MAKES A REAL NETWORK CALL.
|
|
105
|
+
Off by default. Five vendors today:
|
|
106
|
+
AWS: every access key id found paired with its
|
|
107
|
+
secret (see Rotation below) is checked via
|
|
108
|
+
sts:get-caller-identity. Needs the aws CLI on
|
|
109
|
+
PATH; residoo shells out to it rather than
|
|
110
|
+
reimplementing AWS request signing.
|
|
111
|
+
Slack: every token via auth.test.
|
|
112
|
+
OpenAI, Anthropic, GitHub: every key/token via
|
|
113
|
+
that vendor's own models/user listing endpoint.
|
|
114
|
+
All four non-AWS vendors are a direct, dependency-
|
|
115
|
+
free API call, no CLI needed. A verified-invalid
|
|
110
116
|
credential is reported as already dead, not as
|
|
111
|
-
something to rotate
|
|
117
|
+
something to rotate; a JWT's own signed exp claim
|
|
118
|
+
is checked locally with no network call at all,
|
|
119
|
+
on by default, not part of --verify.
|
|
112
120
|
|
|
113
121
|
Rotation:
|
|
114
122
|
residoo explain <rule-id> full rotation runbook for one detection rule
|
|
@@ -449,11 +457,12 @@ async function main(argv) {
|
|
|
449
457
|
const failOnFind = args.includes("--fail-on-find");
|
|
450
458
|
const allowAcked = args.includes("--allow-acked");
|
|
451
459
|
// The one flag that makes residoo do something other than read local
|
|
452
|
-
// files: --verify
|
|
453
|
-
//
|
|
454
|
-
//
|
|
455
|
-
//
|
|
456
|
-
|
|
460
|
+
// files: --verify asks the credential's own vendor whether it still
|
|
461
|
+
// authenticates, for every vendor residoo knows how to check today (AWS
|
|
462
|
+
// access key + paired secret via the aws CLI, Slack tokens via a direct
|
|
463
|
+
// API call; see verify.js). Off by default; every other flag here only
|
|
464
|
+
// changes what is READ or how it is DISPLAYED.
|
|
465
|
+
const verify = args.includes("--verify");
|
|
457
466
|
|
|
458
467
|
// --project [dir]: the dir is optional (CI passes ".", a bare --project
|
|
459
468
|
// means the current directory). null means machine mode.
|
|
@@ -563,7 +572,7 @@ async function main(argv) {
|
|
|
563
572
|
}
|
|
564
573
|
|
|
565
574
|
const progress = makeProgressReporter(noColor);
|
|
566
|
-
const result = await scan({ sources, includeNoisy, includeSuppressed, onProgress: progress.onProgress,
|
|
575
|
+
const result = await scan({ sources, includeNoisy, includeSuppressed, onProgress: progress.onProgress, verify });
|
|
567
576
|
progress.stop();
|
|
568
577
|
const integrity = wantsIntegrity ? runIntegrity() : null;
|
|
569
578
|
const rotation = renderRotation(result.findings, acks, dismissed);
|
package/src/report.js
CHANGED
|
@@ -143,9 +143,15 @@ function renderRotationSection(rotation, { noColor = false, showAdvisory = false
|
|
|
143
143
|
// raw values, while these entries dedupe fingerprints (which include the
|
|
144
144
|
// basename, so one value in two differently-named files is two rotations to
|
|
145
145
|
// track). Two counts under one word would read as a contradiction.
|
|
146
|
+
// "pending" keeps its literal ledger meaning here (not yet acked or
|
|
147
|
+
// dismissed): every entry below tagged pending really is, status-wise.
|
|
148
|
+
// confirmedDead is folded into this note, not into the count itself, so
|
|
149
|
+
// this line explains rather than contradicts "Recommended actions"
|
|
150
|
+
// above, which DOES subtract it from what still needs a look.
|
|
146
151
|
const resolvedNote = [
|
|
147
152
|
counts.acked > 0 ? `${counts.acked} acknowledged` : null,
|
|
148
153
|
counts.dismissed > 0 ? `${counts.dismissed} dismissed` : null,
|
|
154
|
+
counts.confirmedDead > 0 ? `${counts.confirmedDead} confirmed inactive` : null,
|
|
149
155
|
].filter(Boolean).join(", ");
|
|
150
156
|
push(paint(c.bold, "Rotation:") +
|
|
151
157
|
` ${counts.pending} of ${counts.distinct} rotation${counts.distinct === 1 ? "" : "s"} pending` +
|
|
@@ -411,17 +417,26 @@ function render({ findings, filesScanned, sourcesScanned, bytesScanned, suppress
|
|
|
411
417
|
// everything else is either already handled or a re-exposure of a value
|
|
412
418
|
// already accounted for.
|
|
413
419
|
if (rotation && rotation.counts.distinct > 0) {
|
|
414
|
-
const { pending, distinct, acked, dismissed } = rotation.counts;
|
|
420
|
+
const { pending, distinct, acked, dismissed, confirmedDead } = rotation.counts;
|
|
421
|
+
// confirmedDead is a PER-VALUE fact (a real --verify rejection, or a
|
|
422
|
+
// JWT's own signed exp claim already past), never an aggregate guess
|
|
423
|
+
// about a whole rule (see the Rotation section note: a rule's other,
|
|
424
|
+
// unverified findings say nothing either way). Subtracted here, not
|
|
425
|
+
// folded into `pending` itself, so --fail-on-find/--allow-acked and
|
|
426
|
+
// every other consumer of pending's original meaning are unaffected;
|
|
427
|
+
// this only changes what this one summary line tells a human to do.
|
|
428
|
+
const needsReview = pending - confirmedDead;
|
|
415
429
|
push();
|
|
416
430
|
push(paint(c.bold, "Recommended actions:"));
|
|
417
|
-
if (
|
|
418
|
-
push(` ${paint(c.yellow, "→")} ${
|
|
431
|
+
if (needsReview > 0) {
|
|
432
|
+
push(` ${paint(c.yellow, "→")} ${needsReview} of ${distinct} distinct value${distinct === 1 ? "" : "s"} ${needsReview === 1 ? "needs" : "need"} review: rotate the real ones (residoo ack), dismiss the rest (residoo dismiss)`);
|
|
419
433
|
} else {
|
|
420
434
|
push(` ${paint(c.green, "✓")} Nothing new to review; every distinct value here has already been triaged`);
|
|
421
435
|
}
|
|
422
436
|
const resolvedParts = [
|
|
423
437
|
acked > 0 ? `${acked} acknowledged` : null,
|
|
424
438
|
dismissed > 0 ? `${dismissed} dismissed` : null,
|
|
439
|
+
confirmedDead > 0 ? `${confirmedDead} confirmed inactive (verified rejected, or expired)` : null,
|
|
425
440
|
].filter(Boolean);
|
|
426
441
|
if (resolvedParts.length > 0) {
|
|
427
442
|
push(paint(c.dim, ` ${resolvedParts.join(", ")} already, no action needed (see Rotation below for which)`));
|
package/src/rotation.js
CHANGED
|
@@ -903,12 +903,13 @@ function renderRotation(findings, acks, dismissed = {}) {
|
|
|
903
903
|
// payload. null for every non-JWT finding, and for a JWT that
|
|
904
904
|
// failed to decode or carries no exp claim.
|
|
905
905
|
jwtExpiresAtMs: null,
|
|
906
|
-
// --verify only (see verify.js): whether
|
|
907
|
-
//
|
|
908
|
-
// this value is one
|
|
909
|
-
// network calls otherwise.
|
|
910
|
-
|
|
911
|
-
|
|
906
|
+
// --verify only (see verify.js): whether the credential's own
|
|
907
|
+
// vendor (AWS, Slack) still accepts it. null unless the scan was
|
|
908
|
+
// run with --verify AND this value is one residoo knows how to
|
|
909
|
+
// check; residoo makes no network calls otherwise. Same two fields
|
|
910
|
+
// regardless of vendor: the ruleId already says which one answered.
|
|
911
|
+
verified: null,
|
|
912
|
+
verifiedDetail: null,
|
|
912
913
|
};
|
|
913
914
|
byFp.set(st.fingerprint, e);
|
|
914
915
|
}
|
|
@@ -922,7 +923,7 @@ function renderRotation(findings, acks, dismissed = {}) {
|
|
|
922
923
|
// credential was rotated or revoked, only that residoo hasn't seen it
|
|
923
924
|
// paste anywhere more recently than this. residoo makes no network
|
|
924
925
|
// calls in the default path, so this alone never checks a provider for
|
|
925
|
-
// whether a key is still live (see
|
|
926
|
+
// whether a key is still live (see verified above for the opt-in
|
|
926
927
|
// exception, and jwtExpiresAtMs for the zero-network JWT case).
|
|
927
928
|
if (typeof f.fileMTimeMs === "number" && (e.lastSeenMs === null || f.fileMTimeMs > e.lastSeenMs)) {
|
|
928
929
|
e.lastSeenMs = f.fileMTimeMs;
|
|
@@ -940,26 +941,31 @@ function renderRotation(findings, acks, dismissed = {}) {
|
|
|
940
941
|
if (e.jwtExpiresAtMs === null && typeof f.jwtExpiresAtMs === "number") {
|
|
941
942
|
e.jwtExpiresAtMs = f.jwtExpiresAtMs;
|
|
942
943
|
}
|
|
943
|
-
if (e.
|
|
944
|
-
e.
|
|
945
|
-
e.
|
|
944
|
+
if (e.verified === null && typeof f.verified === "string") {
|
|
945
|
+
e.verified = f.verified;
|
|
946
|
+
e.verifiedDetail = typeof f.verifiedDetail === "string" ? f.verifiedDetail : null;
|
|
946
947
|
}
|
|
947
948
|
}
|
|
948
949
|
|
|
949
950
|
// Within a status tier, order by how demonstrated-urgent an entry is, not
|
|
950
951
|
// just its rule id: a real pair (see pairing.js) is a DEMONSTRATED usable
|
|
951
|
-
// credential, and --verify confirming
|
|
952
|
-
// evidence still; either way this entry must never be the one
|
|
953
|
-
// cap (see renderRotationSection) pushes into "N more."
|
|
954
|
-
//
|
|
955
|
-
//
|
|
956
|
-
//
|
|
957
|
-
//
|
|
952
|
+
// credential, and --verify confirming the vendor still accepts it is
|
|
953
|
+
// stronger evidence still; either way this entry must never be the one
|
|
954
|
+
// the display cap (see renderRotationSection) pushes into "N more."
|
|
955
|
+
//
|
|
956
|
+
// isConfirmedDead is the other direction, PROOF rather than a guess:
|
|
957
|
+
// --verify got a real "no" from the vendor, or a JWT's own signed exp
|
|
958
|
+
// claim is already in the past (decoded locally, no network call, always
|
|
959
|
+
// attempted; see jwtExpiry.js). Deliberately NOT "unverified" or "no
|
|
960
|
+
// pairing found": those mean residoo doesn't know, a weaker claim than
|
|
961
|
+
// residoo knows this one specific value needs no action. Sorts LOWER
|
|
962
|
+
// than an ordinary finding within its tier, and (see confirmedDead below)
|
|
963
|
+
// is subtracted from what the report tells a human still needs a look.
|
|
964
|
+
const isConfirmedDead = (e) => e.verified === "invalid" || (e.jwtExpiresAtMs !== null && e.jwtExpiresAtMs < Date.now());
|
|
958
965
|
const priorityScore = (e) => {
|
|
959
|
-
if (e.
|
|
966
|
+
if (e.verified === "active") return -2;
|
|
960
967
|
if (e.pairedSecretPreview !== null || e.pairedAccessKeyPreview !== null) return -1;
|
|
961
|
-
if (e
|
|
962
|
-
if (e.jwtExpiresAtMs !== null && e.jwtExpiresAtMs < Date.now()) return 1;
|
|
968
|
+
if (isConfirmedDead(e)) return 1;
|
|
963
969
|
return 0;
|
|
964
970
|
};
|
|
965
971
|
const entries = [...byFp.values()].sort((a, b) => {
|
|
@@ -970,7 +976,16 @@ function renderRotation(findings, acks, dismissed = {}) {
|
|
|
970
976
|
return a.fingerprint < b.fingerprint ? -1 : 1;
|
|
971
977
|
});
|
|
972
978
|
|
|
973
|
-
|
|
979
|
+
// A per-VALUE fact, never rolled up into a per-RULE label: only the
|
|
980
|
+
// specific values residoo actually checked (or that carry their own
|
|
981
|
+
// signed exp claim) ever count here, so this can never overstate what was
|
|
982
|
+
// proven about the rest of a rule's unverified findings. Counted only
|
|
983
|
+
// among PENDING entries: one already acked or dismissed is excluded from
|
|
984
|
+
// "needs review" for its own reason already, and double-subtracting would
|
|
985
|
+
// make the arithmetic in the report not add up.
|
|
986
|
+
const confirmedDead = entries.filter((e) => e.status === "pending" && isConfirmedDead(e)).length;
|
|
987
|
+
|
|
988
|
+
return { counts: { ...counts, confirmedDead }, entries };
|
|
974
989
|
}
|
|
975
990
|
|
|
976
991
|
module.exports = {
|
package/src/scan.js
CHANGED
|
@@ -6,13 +6,32 @@ const { findDecodedMatches, findBoundaryMatches, contentProjection } = require("
|
|
|
6
6
|
const { findPairedSecret } = require("./pairing");
|
|
7
7
|
const { looksRandom } = require("./rarity");
|
|
8
8
|
const { decodeJwtExpiryMs } = require("./jwtExpiry");
|
|
9
|
-
const {
|
|
9
|
+
const {
|
|
10
|
+
isAwsCliAvailable, verifyAwsCredential,
|
|
11
|
+
verifySlackToken, verifyOpenAiKey, verifyAnthropicKey, verifyGithubToken,
|
|
12
|
+
} = require("./verify");
|
|
10
13
|
|
|
11
|
-
// Never verify more than this many distinct
|
|
12
|
-
// pathological transcript with dozens of distinct
|
|
13
|
-
// not turn --verify into a long burst of outbound
|
|
14
|
-
// 0-2; this is a backstop, not the expected path.
|
|
15
|
-
const
|
|
14
|
+
// Never verify more than this many distinct credentials of ONE vendor in a
|
|
15
|
+
// single scan: a pathological transcript with dozens of distinct
|
|
16
|
+
// credentials should not turn --verify into a long burst of outbound calls.
|
|
17
|
+
// Real scans see 0-2 per vendor; this is a backstop, not the expected path.
|
|
18
|
+
const MAX_VERIFICATIONS_PER_VENDOR = 10;
|
|
19
|
+
|
|
20
|
+
// Every vendor whose credential is a single, unpaired bearer token: no
|
|
21
|
+
// AWS-style "two halves make one credential" pairing step, so these all
|
|
22
|
+
// share one collection/verification path below (see pendingSimpleVerifications).
|
|
23
|
+
const SIMPLE_VERIFY_FNS = {
|
|
24
|
+
slack_token: verifySlackToken,
|
|
25
|
+
openai_key: verifyOpenAiKey,
|
|
26
|
+
anthropic_key: verifyAnthropicKey,
|
|
27
|
+
github_pat: verifyGithubToken,
|
|
28
|
+
};
|
|
29
|
+
const SIMPLE_VERIFY_VENDOR_LABEL = {
|
|
30
|
+
slack_token: "Slack's auth.test",
|
|
31
|
+
openai_key: "OpenAI's models endpoint",
|
|
32
|
+
anthropic_key: "Anthropic's models endpoint",
|
|
33
|
+
github_pat: "GitHub's user endpoint",
|
|
34
|
+
};
|
|
16
35
|
|
|
17
36
|
// Rule ids that findPairedSecret's window search applies to (see pairing.js):
|
|
18
37
|
// AWS access key ids and STS session tokens both pair with the same shape
|
|
@@ -135,7 +154,7 @@ function safeName(file) { return path.basename(file); }
|
|
|
135
154
|
* absolute path can itself carry a username or a project name the rest of
|
|
136
155
|
* this report is careful never to print.
|
|
137
156
|
*/
|
|
138
|
-
async function scan({ sources, includeNoisy = false, includeSuppressed = false, onProgress = null,
|
|
157
|
+
async function scan({ sources, includeNoisy = false, includeSuppressed = false, onProgress = null, verify = false } = {}) {
|
|
139
158
|
const rules = includeNoisy ? PATTERNS.concat(NOISY_PATTERNS) : PATTERNS;
|
|
140
159
|
// The decode pass (see decode.js) only applies high-confidence, vendor-
|
|
141
160
|
// prefixed rules to decoded bytes: random binary that decodes to printable
|
|
@@ -165,6 +184,12 @@ async function scan({ sources, includeNoisy = false, includeSuppressed = false,
|
|
|
165
184
|
// call; nothing in it is ever written to a finding until verification has
|
|
166
185
|
// REPLACED the raw values with a status string.
|
|
167
186
|
const pendingAwsVerifications = new Map();
|
|
187
|
+
// --verify only (see verify.js): ruleId -> (token value -> { refs }), for
|
|
188
|
+
// every SIMPLE_VERIFY_FNS vendor. Unlike AWS, none of these need pairing
|
|
189
|
+
// (the token itself is the complete credential), so this is simpler: one
|
|
190
|
+
// entry per distinct value per rule, `refs` accumulating every finding
|
|
191
|
+
// object that value produced.
|
|
192
|
+
const pendingSimpleVerifications = new Map();
|
|
168
193
|
|
|
169
194
|
// One place raw matched text turns into a recorded finding: counts the
|
|
170
195
|
// distinct value and pushes the redacted record. `extra` carries the
|
|
@@ -268,7 +293,7 @@ async function scan({ sources, includeNoisy = false, includeSuppressed = false,
|
|
|
268
293
|
const jwtExtra = (!suppressedReason && rule.id === "jwt")
|
|
269
294
|
? { jwtExpiresAtMs: decodeJwtExpiryMs(m[0]) }
|
|
270
295
|
: null;
|
|
271
|
-
const
|
|
296
|
+
const primaryFinding = record(rule, m[0], relFile, file, lineNo,
|
|
272
297
|
mtimeMs,
|
|
273
298
|
resolveConfidence(rule.id, m[0], rule.confidence, suppressedReason),
|
|
274
299
|
suppressedReason,
|
|
@@ -282,12 +307,30 @@ async function scan({ sources, includeNoisy = false, includeSuppressed = false,
|
|
|
282
307
|
// re-echoed across several lines gets several finding objects, and
|
|
283
308
|
// the eventual result is applied to every one of them, not only
|
|
284
309
|
// the first.
|
|
285
|
-
if (
|
|
286
|
-
if (!pendingAwsVerifications.has(m[0]) && pendingAwsVerifications.size <
|
|
310
|
+
if (verify && secretFinding && rawPairedSecret) {
|
|
311
|
+
if (!pendingAwsVerifications.has(m[0]) && pendingAwsVerifications.size < MAX_VERIFICATIONS_PER_VENDOR) {
|
|
287
312
|
pendingAwsVerifications.set(m[0], { secretValue: rawPairedSecret, refs: [] });
|
|
288
313
|
}
|
|
289
314
|
const entry = pendingAwsVerifications.get(m[0]);
|
|
290
|
-
if (entry) entry.refs.push({ akiaFinding, secretFinding });
|
|
315
|
+
if (entry) entry.refs.push({ akiaFinding: primaryFinding, secretFinding });
|
|
316
|
+
}
|
|
317
|
+
// --verify, single-token vendors (Slack, OpenAI, Anthropic,
|
|
318
|
+
// GitHub): none of these need pairing (the value IS the complete
|
|
319
|
+
// credential), so queue every unsuppressed match directly, same
|
|
320
|
+
// dedup-by-value / accumulate-all-refs shape as the AWS map above,
|
|
321
|
+
// just one level deeper (keyed by rule id too, since several
|
|
322
|
+
// vendors share this path).
|
|
323
|
+
if (verify && !suppressedReason && SIMPLE_VERIFY_FNS[rule.id]) {
|
|
324
|
+
let byValue = pendingSimpleVerifications.get(rule.id);
|
|
325
|
+
if (!byValue) {
|
|
326
|
+
byValue = new Map();
|
|
327
|
+
pendingSimpleVerifications.set(rule.id, byValue);
|
|
328
|
+
}
|
|
329
|
+
if (!byValue.has(m[0]) && byValue.size < MAX_VERIFICATIONS_PER_VENDOR) {
|
|
330
|
+
byValue.set(m[0], { refs: [] });
|
|
331
|
+
}
|
|
332
|
+
const entry = byValue.get(m[0]);
|
|
333
|
+
if (entry) entry.refs.push(primaryFinding);
|
|
291
334
|
}
|
|
292
335
|
}
|
|
293
336
|
if (m.index === rule.re.lastIndex) rule.re.lastIndex++; // guard zero-width matches
|
|
@@ -446,16 +489,22 @@ async function scan({ sources, includeNoisy = false, includeSuppressed = false,
|
|
|
446
489
|
|
|
447
490
|
// --verify: runs once, here, after every file has been scanned, never
|
|
448
491
|
// interleaved with the matching pass above. A real network call per
|
|
449
|
-
// distinct
|
|
492
|
+
// distinct credential, one at a time (not concurrent), so this is the one
|
|
450
493
|
// place a scan's wall-clock time depends on something other than disk
|
|
451
494
|
// I/O; that tradeoff only exists when a caller explicitly asked for it.
|
|
452
|
-
|
|
453
|
-
|
|
495
|
+
// Same field names (verified/verifiedDetail) regardless of which vendor
|
|
496
|
+
// produced the result: rotation.js and report.js render them identically,
|
|
497
|
+
// and the finding's own ruleId already says which vendor answered.
|
|
498
|
+
const applyVerifyResult = (refs, result) => {
|
|
499
|
+
for (const ref of refs) {
|
|
500
|
+
ref.verified = result.status;
|
|
501
|
+
ref.verifiedDetail = result.detail;
|
|
502
|
+
}
|
|
503
|
+
};
|
|
504
|
+
if (verify && pendingAwsVerifications.size > 0) {
|
|
505
|
+
const applyPair = (refs, result) => {
|
|
454
506
|
for (const ref of refs) {
|
|
455
|
-
ref.akiaFinding.
|
|
456
|
-
ref.akiaFinding.awsVerifiedDetail = result.detail;
|
|
457
|
-
ref.secretFinding.awsVerified = result.status;
|
|
458
|
-
ref.secretFinding.awsVerifiedDetail = result.detail;
|
|
507
|
+
applyVerifyResult([ref.akiaFinding, ref.secretFinding], result);
|
|
459
508
|
}
|
|
460
509
|
};
|
|
461
510
|
if (!isAwsCliAvailable()) {
|
|
@@ -465,7 +514,7 @@ async function scan({ sources, includeNoisy = false, includeSuppressed = false,
|
|
|
465
514
|
"Install it (https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) to use --verify.\n"
|
|
466
515
|
);
|
|
467
516
|
const result = { status: "error", detail: "aws CLI not found on PATH" };
|
|
468
|
-
for (const { refs } of pendingAwsVerifications.values())
|
|
517
|
+
for (const { refs } of pendingAwsVerifications.values()) applyPair(refs, result);
|
|
469
518
|
} else {
|
|
470
519
|
process.stderr.write(
|
|
471
520
|
`residoo --verify: calling AWS sts:get-caller-identity for ${pendingAwsVerifications.size} ` +
|
|
@@ -474,7 +523,22 @@ async function scan({ sources, includeNoisy = false, includeSuppressed = false,
|
|
|
474
523
|
);
|
|
475
524
|
for (const [accessKeyValue, { secretValue, refs }] of pendingAwsVerifications) {
|
|
476
525
|
const result = verifyAwsCredential(accessKeyValue, secretValue);
|
|
477
|
-
|
|
526
|
+
applyPair(refs, result);
|
|
527
|
+
}
|
|
528
|
+
}
|
|
529
|
+
}
|
|
530
|
+
if (verify) {
|
|
531
|
+
for (const [ruleId, byValue] of pendingSimpleVerifications) {
|
|
532
|
+
if (byValue.size === 0) continue;
|
|
533
|
+
const verifyFn = SIMPLE_VERIFY_FNS[ruleId];
|
|
534
|
+
process.stderr.write(
|
|
535
|
+
`residoo --verify: calling ${SIMPLE_VERIFY_VENDOR_LABEL[ruleId]} for ${byValue.size} ` +
|
|
536
|
+
"token(s) found in this scan. This is a real network request, using the exact " +
|
|
537
|
+
"token found in your transcript, one at a time.\n"
|
|
538
|
+
);
|
|
539
|
+
for (const [value, { refs }] of byValue) {
|
|
540
|
+
const result = await verifyFn(value);
|
|
541
|
+
applyVerifyResult(refs, result);
|
|
478
542
|
}
|
|
479
543
|
}
|
|
480
544
|
}
|
package/src/verify.js
CHANGED
|
@@ -6,31 +6,46 @@
|
|
|
6
6
|
* Everything else in residoo is detection only: a shape matched a pattern,
|
|
7
7
|
* nothing more, zero network calls, by design (see README's "What it does
|
|
8
8
|
* not do"). This module is the one deliberate exception, and only when a
|
|
9
|
-
* user explicitly passes --verify: it
|
|
10
|
-
*
|
|
11
|
-
*
|
|
12
|
-
* permission-less call AWS's own docs and tools like the AWS CLI and
|
|
13
|
-
* aws-vault use for exactly this "is this credential still alive" check.
|
|
9
|
+
* user explicitly passes --verify: it asks the credential's own vendor
|
|
10
|
+
* whether it still authenticates, via whatever free, read-only check that
|
|
11
|
+
* vendor documents for exactly this "is this still alive" question.
|
|
14
12
|
*
|
|
15
|
-
*
|
|
16
|
-
*
|
|
13
|
+
* Two different implementation strategies live in this one file, chosen
|
|
14
|
+
* per vendor by how risky it would be to get wrong:
|
|
15
|
+
*
|
|
16
|
+
* AWS (verifyAwsCredential) shells out to the user's own `aws` CLI rather
|
|
17
|
+
* than hand-rolling AWS SigV4 request signing. Two reasons, not one: first,
|
|
17
18
|
* residoo ships zero runtime dependencies, and a correct SigV4
|
|
18
19
|
* implementation is real, easy-to-get-subtly-wrong cryptographic code this
|
|
19
20
|
* project cannot verify against a live AWS account in CI; a signing bug
|
|
20
21
|
* here would silently report every real key as "invalid," which is actively
|
|
21
22
|
* worse than not verifying at all. Second, the AWS CLI is exactly the
|
|
22
|
-
* client AWS itself maintains and tests against its own service
|
|
23
|
-
*
|
|
24
|
-
*
|
|
25
|
-
*
|
|
26
|
-
* Every environment variable the aws CLI reads is built from scratch here,
|
|
27
|
-
* never inherited from process.env: AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY
|
|
28
|
-
* are set to the exact values found in the scan, and AWS_CONFIG_FILE/
|
|
23
|
+
* client AWS itself maintains and tests against its own service. Every
|
|
24
|
+
* environment variable the aws CLI reads is built from scratch, never
|
|
25
|
+
* inherited from process.env: AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY are
|
|
26
|
+
* set to the exact values found in the scan, and AWS_CONFIG_FILE/
|
|
29
27
|
* AWS_SHARED_CREDENTIALS_FILE point at /dev/null so the CLI cannot fall
|
|
30
28
|
* back to the user's own real default profile if the found credential is
|
|
31
|
-
* malformed in some way that would otherwise trigger a fallback.
|
|
32
|
-
*
|
|
33
|
-
*
|
|
29
|
+
* malformed in some way that would otherwise trigger a fallback.
|
|
30
|
+
*
|
|
31
|
+
* Slack (verifySlackToken) calls the API directly with the built-in fetch
|
|
32
|
+
* instead: unlike AWS, Slack's auth check (auth.test) is a single bearer-
|
|
33
|
+
* token HTTP call with no request signing at all, so there is no signing
|
|
34
|
+
* bug to be worried about, and no CLI most residoo users would already
|
|
35
|
+
* have installed the way they'd have the aws CLI. Direct fetch is both
|
|
36
|
+
* simpler and more portable here; shelling out to a hypothetical "slack
|
|
37
|
+
* CLI" would add a dependency for no safety benefit. This is the pattern
|
|
38
|
+
* for any future vendor: shell out to that vendor's own official CLI only
|
|
39
|
+
* when the auth scheme itself is complex enough to be worth not
|
|
40
|
+
* reimplementing (AWS's SigV4); call directly for a plain bearer token.
|
|
41
|
+
*
|
|
42
|
+
* OpenAI, Anthropic, and GitHub (verifyOpenAiKey, verifyAnthropicKey,
|
|
43
|
+
* verifyGithubToken) share one implementation (verifyByStatusCode): each is
|
|
44
|
+
* a plain GET to a free, side-effect-free, already-authenticated endpoint
|
|
45
|
+
* (that vendor's own "list what I can see" call), where the HTTP status
|
|
46
|
+
* code alone says whether the credential authenticated. Slack needed its
|
|
47
|
+
* own function because auth.test always returns HTTP 200 and signals
|
|
48
|
+
* failure inside the JSON body instead of the status code.
|
|
34
49
|
*/
|
|
35
50
|
|
|
36
51
|
const { spawnSync } = require("child_process");
|
|
@@ -129,4 +144,123 @@ function verifyAwsCredential(accessKeyId, secretAccessKey, { spawnFn = spawnSync
|
|
|
129
144
|
return { status: "error", detail: `could not verify: ${sanitizeDetail(stderr).slice(0, 120) || `aws exited ${r.status}`}` };
|
|
130
145
|
}
|
|
131
146
|
|
|
132
|
-
|
|
147
|
+
/**
|
|
148
|
+
* Test-only escape hatch, same purpose as RESIDOO_TEST_AWS_CLI above but for
|
|
149
|
+
* an HTTP call instead of a subprocess: when RESIDOO_TEST_SLACK_API_URL is
|
|
150
|
+
* set, verifySlackToken calls that URL instead of Slack's real API, so a
|
|
151
|
+
* test can point at a small local HTTP server and exercise the real fetch +
|
|
152
|
+
* header + JSON-parsing plumbing without ever reaching slack.com.
|
|
153
|
+
*/
|
|
154
|
+
function slackAuthTestUrl() {
|
|
155
|
+
return process.env.RESIDOO_TEST_SLACK_API_URL || "https://slack.com/api/auth.test";
|
|
156
|
+
}
|
|
157
|
+
|
|
158
|
+
// Slack's own documented error codes for auth.test that mean the token
|
|
159
|
+
// itself is dead (revoked, expired, or never valid), not merely rate
|
|
160
|
+
// limited or a transient server problem.
|
|
161
|
+
const SLACK_DEAD_TOKEN_ERRORS = new Set([
|
|
162
|
+
"invalid_auth", "not_authed", "token_revoked", "token_expired", "account_inactive",
|
|
163
|
+
]);
|
|
164
|
+
|
|
165
|
+
/**
|
|
166
|
+
* Ask Slack whether this exact token still authenticates, via auth.test
|
|
167
|
+
* (api.slack.com/methods/auth.test): a bearer-token-only call Slack's own
|
|
168
|
+
* docs recommend for checking token validity, needing no scope of its own.
|
|
169
|
+
* Same three-way { status, detail } contract as verifyAwsCredential.
|
|
170
|
+
*/
|
|
171
|
+
async function verifySlackToken(token, { fetchFn = fetch, timeoutMs = DEFAULT_TIMEOUT_MS } = {}) {
|
|
172
|
+
let res;
|
|
173
|
+
try {
|
|
174
|
+
res = await fetchFn(slackAuthTestUrl(), {
|
|
175
|
+
method: "POST",
|
|
176
|
+
headers: { Authorization: `Bearer ${token}` },
|
|
177
|
+
signal: AbortSignal.timeout(timeoutMs),
|
|
178
|
+
});
|
|
179
|
+
} catch (e) {
|
|
180
|
+
return { status: "error", detail: `could not reach Slack (${sanitizeDetail(e && e.message)})` };
|
|
181
|
+
}
|
|
182
|
+
let body;
|
|
183
|
+
try {
|
|
184
|
+
body = await res.json();
|
|
185
|
+
} catch (e) {
|
|
186
|
+
return { status: "error", detail: `Slack returned a non-JSON response (HTTP ${res.status})` };
|
|
187
|
+
}
|
|
188
|
+
if (body && body.ok === true) {
|
|
189
|
+
return { status: "active", detail: "Slack accepted this token (auth.test)" };
|
|
190
|
+
}
|
|
191
|
+
const err = body && typeof body.error === "string" ? body.error : null;
|
|
192
|
+
if (err && SLACK_DEAD_TOKEN_ERRORS.has(err)) {
|
|
193
|
+
return { status: "invalid", detail: `Slack rejected this token (${sanitizeDetail(err)})` };
|
|
194
|
+
}
|
|
195
|
+
return { status: "error", detail: `could not verify: ${sanitizeDetail(err) || `HTTP ${res.status}`}` };
|
|
196
|
+
}
|
|
197
|
+
|
|
198
|
+
/**
|
|
199
|
+
* Shared implementation for every vendor below Slack: a plain GET to a
|
|
200
|
+
* free, side-effect-free, already-authenticated endpoint (each vendor's own
|
|
201
|
+
* "list what I can see" call), where the HTTP status code alone says
|
|
202
|
+
* whether the credential authenticated. 200 is active; 401/403 is a real
|
|
203
|
+
* rejection; anything else (429 rate limited, 5xx, a network failure) is
|
|
204
|
+
* inconclusive, never guessed as either active or invalid. Slack needed its
|
|
205
|
+
* own function above because its auth.test always returns HTTP 200 and
|
|
206
|
+
* signals failure inside the JSON body instead.
|
|
207
|
+
*/
|
|
208
|
+
async function verifyByStatusCode(vendorName, url, buildHeaders, { fetchFn = fetch, timeoutMs = DEFAULT_TIMEOUT_MS } = {}) {
|
|
209
|
+
let res;
|
|
210
|
+
try {
|
|
211
|
+
res = await fetchFn(url, {
|
|
212
|
+
method: "GET",
|
|
213
|
+
headers: buildHeaders(),
|
|
214
|
+
signal: AbortSignal.timeout(timeoutMs),
|
|
215
|
+
});
|
|
216
|
+
} catch (e) {
|
|
217
|
+
return { status: "error", detail: `could not reach ${vendorName} (${sanitizeDetail(e && e.message)})` };
|
|
218
|
+
}
|
|
219
|
+
if (res.status === 200) {
|
|
220
|
+
return { status: "active", detail: `${vendorName} accepted this key` };
|
|
221
|
+
}
|
|
222
|
+
if (res.status === 401 || res.status === 403) {
|
|
223
|
+
return { status: "invalid", detail: `${vendorName} rejected this key (HTTP ${res.status})` };
|
|
224
|
+
}
|
|
225
|
+
return { status: "error", detail: `could not verify: HTTP ${res.status} from ${vendorName}` };
|
|
226
|
+
}
|
|
227
|
+
|
|
228
|
+
// Test-only escape hatches, same purpose and pattern as
|
|
229
|
+
// RESIDOO_TEST_SLACK_API_URL above: when set, the matching verify function
|
|
230
|
+
// calls that URL instead of the vendor's real one.
|
|
231
|
+
function openAiModelsUrl() {
|
|
232
|
+
return process.env.RESIDOO_TEST_OPENAI_API_URL || "https://api.openai.com/v1/models";
|
|
233
|
+
}
|
|
234
|
+
function anthropicModelsUrl() {
|
|
235
|
+
return process.env.RESIDOO_TEST_ANTHROPIC_API_URL || "https://api.anthropic.com/v1/models";
|
|
236
|
+
}
|
|
237
|
+
function githubUserUrl() {
|
|
238
|
+
return process.env.RESIDOO_TEST_GITHUB_API_URL || "https://api.github.com/user";
|
|
239
|
+
}
|
|
240
|
+
|
|
241
|
+
/** OpenAI: GET /v1/models, a free, read-only call that needs only a valid key, no usage cost. */
|
|
242
|
+
function verifyOpenAiKey(key, opts) {
|
|
243
|
+
return verifyByStatusCode("OpenAI", openAiModelsUrl(), () => ({ Authorization: `Bearer ${key}` }), opts);
|
|
244
|
+
}
|
|
245
|
+
|
|
246
|
+
/**
|
|
247
|
+
* Anthropic: GET /v1/models. Two headers, not one, and NOT an Authorization
|
|
248
|
+
* Bearer header: Anthropic's API takes the key as x-api-key, and every
|
|
249
|
+
* request needs an anthropic-version header regardless of endpoint.
|
|
250
|
+
*/
|
|
251
|
+
function verifyAnthropicKey(key, opts) {
|
|
252
|
+
return verifyByStatusCode("Anthropic", anthropicModelsUrl(), () => ({
|
|
253
|
+
"x-api-key": key,
|
|
254
|
+
"anthropic-version": "2023-06-01",
|
|
255
|
+
}), opts);
|
|
256
|
+
}
|
|
257
|
+
|
|
258
|
+
/** GitHub: GET /user with the token, a free, read-only call that needs no scopes. */
|
|
259
|
+
function verifyGithubToken(token, opts) {
|
|
260
|
+
return verifyByStatusCode("GitHub", githubUserUrl(), () => ({ Authorization: `Bearer ${token}` }), opts);
|
|
261
|
+
}
|
|
262
|
+
|
|
263
|
+
module.exports = {
|
|
264
|
+
isAwsCliAvailable, verifyAwsCredential, verifySlackToken,
|
|
265
|
+
verifyOpenAiKey, verifyAnthropicKey, verifyGithubToken,
|
|
266
|
+
};
|