@avi2dg/checks 0.31.0 → 0.33.0

package/docs/gates/checks-secrets.md

@@ -0,0 +1,93 @@
+---
+kind: reference
+audience: consumers
+---
+# checks-secrets
+
+`checks-secrets` is the gate that fails a range whose commits add a secret, such as an API token, a private key, a VPN key or a proxy link that carries its credential.
+
+## What it checks
+
+It scans the lines each commit in the range adds, in every file the commit adds or modifies, with gitleaks.
+In a merge commit it scans the lines the merge's own resolution adds, such as a key written while resolving a conflict, and not what the merge brings in from the other parent.
+It exits 2 when a commit it scans is an octopus merge, one with three or more parents, because git gives no resolution diff for one.
+It fails on each secret the scan finds, and names its file, its line, the rule that matched and the commit that added it.
+A secret one commit adds and a later commit in the range removes still fails, because the first commit still holds it.
+It guards new changes only, and never scans a commit before the range, so a secret already in the history takes no part.
+
+It runs gitleaks' default rules, which know the API keys and tokens of common services and private keys in PEM form, and eight rules of the kit's own that report a secret:
+
+| Rule | What it matches | What passes |
+| --- | --- | --- |
+| `wireguard-key` | a WireGuard or AmneziaWG private or pre-shared key value, 44 base64 characters, under a name that ends in `PrivateKey`, `PresharedKey` or `psk`, such as a real key after `PrivateKey =`, `"privateKey":`, `wg_private_key:` or `client_psk:` | a placeholder such as `<private-key>` or `${WG_PRIVATE_KEY}` |
+| `wireguard-key-file` | a line that holds only a WireGuard or AmneziaWG key, 44 base64 characters, in a file whose name holds `priv`, `preshared`, `pre_shared`, `pre-shared` or `psk`, such as the `privatekey` file `wg genkey \| tee privatekey` writes, whatever else the name holds | a bare key in a file with any other name, such as `publickey` or `peer.pub`, and a placeholder such as `<private-key>` |
+| `wireguard-dot-key-file` | a line that holds only a WireGuard or AmneziaWG key in a file whose name ends in `.key`, such as `wg0.key` | a file whose name ends in `pub.key` or `public.key`, such as `server-public.key`, unless `wireguard-key-file` matches its name |
+| `proxy-userinfo-link` | a `vless://`, `vmess://`, `ss://`, `trojan://`, `hysteria://`, `hysteria2://`, `hy2://`, `tuic://`, `socks://`, `socks5://`, `socks5h://`, `wireguard://` or `wg://` link whose credential before an `@` has a Shannon entropy above 2.5 bits per character, such as a real password before the `@` of a `trojan://` link or a real private key before the `@` of a `wireguard://` link, and a template reference with real text around it, such as a literal user before `:${PASS}` or a non-empty default in `${PASS:-…}` | a link with no credential such as `socks5://127.0.0.1:1080`, a low-entropy credential such as `pw@` or `user@`, a placeholder credential such as `vless://<uuid>@<host>`, a credential that is only template references, such as `{password}`, `$PASS`, `{{ password }}` or `%(password)s` naming a variable that starts with a letter or underscore, or any `${…}` interpolation except a shell default such as `${PASS:-value}`, such as `${pass}`, `${pass:-}`, `${pass:?message}` or `${encodeURIComponent(pass)}`, including a user part such as `$USER:$PASS@`, and a link to `example.com`, `example.net`, `example.org`, a host under `.example`, `.invalid` or `.test`, or `localhost` |
+| `proxy-base64-link` | a `vmess://`, `ss://` or `ssr://` link whose payload is 16 or more base64 characters with a Shannon entropy above 4.2 bits per character, whatever follows it | a placeholder such as `vmess://<base64-config>`, a link to a host and port such as `ss://vpn.home.net:8388`, and a payload an `@` follows, which `proxy-userinfo-link` judges |
+| `proxy-query-credential-first` | the first `auth`, `auth_str`, `obfsParam`, `obfs-password`, `password` or `pk` query value of one of the links `proxy-userinfo-link` names or an `ssr://` link, whose Shannon entropy is above 2.5 bits per character, such as a real password in the `auth` of a `hysteria://` link or a real private key in the `pk` of a `wg://` link, whatever parameters come before or after it | a low-entropy value such as `auth=pw`, a placeholder value such as `auth=<password>`, a value that is only template references as `proxy-userinfo-link` describes, such as `auth={auth}`, `auth=${HY_AUTH:?set HY_AUTH}` or `auth=${encodeURIComponent("a b")}`, and a link to one of the example hosts above |
+| `proxy-query-credential-last` | the last of these query values in such a link, judged the same way, so a real value after a low-entropy or template one still fails | what passes `proxy-query-credential-first` |
+| `proxy-subscription-url` | 16 or more token characters with a Shannon entropy above 3.0 bits per character after the `sub`, `subs`, `subscribe`, `subscription` or `link` path segment of an `http` or `https` URL, or in any `token`, `key`, `uuid` or `id` query value on a line that holds such a URL, each judged on its own | a `token`, `key`, `uuid` or `id` value on a line with no such URL, and a URL to one of the example hosts above |
+
+A ninth rule, `proxy-subscription-path`, reports nothing: it finds the URL `proxy-subscription-url` needs on the same line.
+
+These rules do not catch:
+
+- a real query value between two others of these parameters in one link whose values are low-entropy or template references, such as a real `auth` between an `obfsParam=pw` or `obfsParam=${OBFS}` and a `password=pw`
+- a real `auth`, `auth_str`, `obfsParam`, `obfs-password`, `password` or `pk` query value of an `http` or `https` URL, or of a scheme the table does not name, because the kit's query rules judge share links only, and gitleaks' default rules catch such a value only now and then
+- a credential that begins with `${` and is not a shell default such as `${PASS:-value}`, `${PASS-value}`, `${PASS:=value}` or `${PASS:+value}`, because it reads as an interpolation, even when it holds a fallback such as `${auth ?? "value"}`
+
+No setting accepts a finding.
+It ignores a repository's `.gitleaks.toml`, its `.gitleaksignore` and every `gitleaks:allow` comment.
+It skips the paths gitleaks' default config skips, such as lockfiles, images and `node_modules/`.
+
+## What it reads
+
+It reads the commits of the range from git, not the working tree.
+It needs git 2.36 or newer, which diffs a merge against the merge git would make on its own, and it exits 2 on an older git.
+It reads its rules from `src/delivery/gitleaks.toml` in the installed package.
+
+It runs gitleaks 8.30.1, pinned by the SHA-256 of each platform's archive and of the binary inside it.
+On first use it downloads the archive from the scanner's GitHub release and unpacks the binary into `~/.cache/avi2dg-checks/gitleaks/8.30.1/`.
+It checks the binary's SHA-256 again on every run and exits 2 on a copy that differs.
+Builds are pinned for macOS and Linux, each on x64 and arm64.
+On any other platform it exits 2, and no setting runs a scanner other than the pinned build.
+A cold cache downloads about 8 MB.
+
+## Arguments
+
+```sh
+checks-secrets <base-ref> <head-ref>
+checks-secrets <ref>
+```
+
+With two arguments it scans each commit from the merge base of the two refs to the head.
+With one it scans that commit alone.
+
+## Exit codes
+
+| Code | When |
+| --- | --- |
+| 0 | no commit in the range adds a secret |
+| 1 | a commit in the range adds a secret |
+| 2 | a ref does not resolve, or a commit it scans is an octopus merge, or no verified scanner is at hand, or git is older than 2.36, or gitleaks fails |
+
+## Sample output
+
+```
+secrets: the range adds 3 secret(s); put a placeholder such as <private-key> in its place in the commit that added it, and rotate any secret that left this machine:
+  vpn/links.txt:1 proxy-userinfo-link in e19e45c2: A proxy share link that carries its credential before the @, such as vless://, trojan://, socks5:// or wireguard://
+  vpn/wg0.conf:2 wireguard-key in e19e45c2: A WireGuard or AmneziaWG private or pre-shared key
+  vpn/wg0.conf:7 wireguard-key in e19e45c2: A WireGuard or AmneziaWG private or pre-shared key
+```
+
+The report never prints a secret.
+
+## When it runs
+
+`checks-lint` runs it over each pull request's range in every repository, as [checks-lint](checks-lint.md) says.
+A local `bun run lint` runs it over the commits the branch adds, so it fails before a push.
+
+## Related topics
+
+- [checks-lint](checks-lint.md)
+- [Why it is shaped this way](../design.md)

package/docs/gates/checks-vendor.md

@@ -60,19 +60,19 @@ With no arguments it pins nothing.
 ## Sample output
 
 ```
-checks-vendor: cloned effect@4.0.0-rc.115 from https://github.com/Effect-TS/effect.git and linked repos/effect
+checks-vendor: cloned effect@4.0.0 from https://github.com/Effect-TS/effect.git and linked repos/effect
 ```
 
 A fresh fetch reports the tag it cloned and the link it made.
 
 ```
-checks-vendor: repos/effect still holds effect@4.0.0-rc.115, verified against its recorded commit
+checks-vendor: repos/effect still holds effect@4.0.0, verified against its recorded commit
 ```
 
 A later run reports the link it kept.
 
 ```
-checks-vendor: found 1 path writable by its owner, starting with /home/runner/.cache/avi2dg-checks/repos/github.com/Effect-TS/effect/effect@4.0.0-rc.115, and froze the tree again, so repos/effect still holds effect@4.0.0-rc.115, verified against its recorded commit
+checks-vendor: found 1 path writable by its owner, starting with /home/runner/.cache/avi2dg-checks/repos/github.com/Effect-TS/effect/effect@4.0.0, and froze the tree again, so repos/effect still holds effect@4.0.0, verified against its recorded commit
 ```
 
 A run that found an owner write bit reports how many paths carried one and the first of them.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@avi2dg/checks",
-  "version": "0.31.0",
+  "version": "0.33.0",
   "description": "Deterministic checks shared across a set of TypeScript repositories",
   "license": "MIT",
   "repository": {
@@ -29,6 +29,10 @@
     "src/delivery/changelog-write.ts",
     "src/delivery/release-notes.ts",
     "src/delivery/release-report.ts",
+    "src/delivery/release.ts",
+    "src/delivery/release-pr.ts",
+    "src/delivery/release-tag.ts",
+    "src/delivery/github.ts",
     "src/testing/test-layout.ts",
     "src/testing/test.ts",
     "src/testing/test-skips.ts",
@@ -43,6 +47,9 @@
     "src/testing/subsumed-tests.ts",
     "src/delivery/ci-wiring.ts",
     "src/delivery/shell-command.ts",
+    "src/delivery/secrets.ts",
+    "src/delivery/gitleaks.ts",
+    "src/delivery/gitleaks.toml",
     "src/quality/comment-matchers.ts",
     "src/docs/prose-matchers.ts",
     "src/quality/comments.ts",
@@ -53,6 +60,7 @@
     "src/complexity/exports.ts",
     "src/complexity/knip.ts",
     "src/testing/quarantine-clock.ts",
+    "src/docs/sentence-length.ts",
     "src/docs/docs.ts",
     "src/dependencies/vendor.ts",
     "src/dependencies/vendor-args.ts",
@@ -60,6 +68,7 @@
     "src/dependencies/advisories.ts",
     "src/dependencies/advisory-rules.ts",
     "src/dependencies/osv-scanner.ts",
+    "src/dependencies/pinned-binary.ts",
     "src/docs/doc-agents.ts",
     "src/docs/doc-outline.ts",
     "src/docs/doc-rules.ts",
@@ -93,6 +102,8 @@
     "checks-changelog": "src/delivery/changelog-write.ts",
     "checks-release-notes": "src/delivery/release-notes.ts",
     "checks-release-report": "src/delivery/release-report.ts",
+    "checks-release-pr": "src/delivery/release-pr.ts",
+    "checks-release-tag": "src/delivery/release-tag.ts",
     "checks-lint-coverage": "src/quality/lint-coverage.sh",
     "checks-test-layout": "src/testing/test-layout.ts",
     "checks-test": "src/testing/test.ts",
@@ -110,7 +121,8 @@
     "checks-quarantine-clock": "src/testing/quarantine-clock.ts",
     "checks-docs": "src/docs/docs.ts",
     "checks-vendor": "src/dependencies/vendor.ts",
-    "checks-advisories": "src/dependencies/advisories.ts"
+    "checks-advisories": "src/dependencies/advisories.ts",
+    "checks-secrets": "src/delivery/secrets.ts"
   },
   "scripts": {
     "prepare": "bun src/dependencies/vendor.ts --library effect --package effect --repository https://github.com/Effect-TS/effect.git --tag 'effect@{version}' --path packages/effect/package.json",
@@ -125,7 +137,7 @@
     "@swc/core": "1.16.2",
     "dependency-cruiser": "18.4.0",
     "@effect/tsgo": "0.45.0",
-    "effect": "4.0.0-rc.115",
+    "effect": "4.0.0",
     "jscpd": "5.3.2",
     "oxlint": "1.83.0",
     "oxlint-tsgolint": "7.0.2002",
@@ -138,8 +150,9 @@
     "@stryker-mutator/core": "10.0.0",
     "@swc/core": "1.16.2",
     "@types/bun": "^1.3.0",
+    "@types/commonmark": "0.27.10",
     "dependency-cruiser": "18.4.0",
-    "effect": "4.0.0-rc.115",
+    "effect": "4.0.0",
     "jscpd": "5.3.2",
     "oxlint": "1.83.0",
     "oxlint-tsgolint": "7.0.2002",
@@ -148,13 +161,17 @@
   "dependencies": {
     "@commitlint/cli": "21.2.3",
     "@commitlint/config-conventional": "21.2.3",
-    "@effect/platform-bun": "4.0.0-rc.115",
-    "@effect/platform-node-shared": "4.0.0-rc.115",
+    "@effect/platform-bun": "4.0.0",
+    "@effect/platform-node-shared": "4.0.0",
     "@total-typescript/ts-reset": "0.6.1",
+    "commonmark": "0.31.2",
     "knip": "6.38.0"
   },
   "overrides": {
     "qs": "6.16.0",
     "smol-toml": "1.9.0"
+  },
+  "patchedDependencies": {
+    "@hughescr/stryker-bun-runner@1.4.0": "patches/@hughescr%2Fstryker-bun-runner@1.4.0.patch"
   }
 }

package/src/core/gates.ts

@@ -43,6 +43,7 @@ export const KIT_GATES = [
   { bin: "checks-comment-gate", vector: "quality", file: "comment-gate.ts", reads: "range", appliesTo: EVERY_REPOSITORY },
   { bin: "checks-suppressions-ratchet", vector: "complexity", file: "suppressions-ratchet.ts", reads: "range", appliesTo: EVERY_REPOSITORY },
   { bin: "checks-ci-wiring", vector: "delivery", file: "ci-wiring.ts", reads: "tree", appliesTo: EVERY_REPOSITORY },
+  { bin: "checks-secrets", vector: "delivery", file: "secrets.ts", reads: "range", appliesTo: EVERY_REPOSITORY },
   { bin: "checks-docs", vector: "docs", file: "docs.ts", reads: "range", alsoReads: "every agent file at the head commit", appliesTo: EVERY_REPOSITORY },
   { bin: "checks-repetition", vector: "complexity", file: "repetition.ts", reads: "range", appliesTo: TYPESCRIPT_SOURCE },
   { bin: "checks-unused", vector: "complexity", file: "unused.ts", reads: "tree", appliesTo: LINTED_SOURCE },

package/src/core/git.ts

@@ -1,5 +1,5 @@
 import { Config, Effect, FileSystem, Path, Schema, Stream } from "effect";
-import { ChildProcess, ChildProcessSpawner } from "effect/unstable/process";
+import { ChildProcess, ChildProcessSpawner } from "effect/process";
 import { DEFAULT_BRANCH } from "./gates.ts";
 import { Usage } from "./main.ts";
 

package/src/core/lint.ts

@@ -1,6 +1,6 @@
 #!/usr/bin/env bun
 import { Config, Console, Effect, FileSystem, Option, Path, Schema } from "effect";
-import { ChildProcess, ChildProcessSpawner } from "effect/unstable/process";
+import { ChildProcess, ChildProcessSpawner } from "effect/process";
 import { EVERY_REPOSITORY, KIT_GATES, type KitGate } from "./gates.ts";
 import { defaultBranch, git } from "./git.ts";
 import { runMain, Usage } from "./main.ts";

package/src/delivery/commit-identity.ts

@@ -1,5 +1,6 @@
 #!/usr/bin/env bun
 import { Console, Effect, FileSystem, Path, Schema } from "effect";
+import { releasedVersionOf } from "./release.ts";
 import { git, refArgs } from "../core/git.ts";
 import { runMain } from "../core/main.ts";
 
@@ -20,6 +21,17 @@ const DEFAULT_AUTHORS: readonly Identity[] = [{ name: "avi2d", email: "avi2dg@gm
 // GitHub writes the squash commit, so it commits what the owner authored and never authors.
 const SQUASH_COMMITTER: Identity = { name: "GitHub", email: "noreply@github.com" };
 
+// checks-release-pr commits through the workflow token, which GitHub attributes to its Actions bot.
+// GitHub may also name that bot as co-author when it squashes the pull request the bot opened.
+const RELEASE_AUTHOR: Identity = { name: "github-actions[bot]", email: "41898282+github-actions[bot]@users.noreply.github.com" };
+
+const CO_AUTHOR = /^co-authored-by:\s*(.*?)\s*<([^<>]*)>\s*$/i;
+
+function namesReleaseAuthor(trailer: string): boolean {
+  const [, name, email = ""] = CO_AUTHOR.exec(trailer) ?? [];
+  return name !== undefined && allows([RELEASE_AUTHOR], { name, email });
+}
+
 // git's own trailer parser, so only the trailer block counts and prose never does.
 const CO_AUTHORED_BY_FORMAT = "%(trailers:key=Co-authored-by)";
 
@@ -113,13 +125,16 @@ function allows(allowed: readonly Identity[], identity: Identity): boolean {
 
 function inspect(commit: Commit, allowed: readonly Identity[]): Offence | undefined {
   const reasons: string[] = [];
-  if (!allows(allowed, commit.author)) {
+  const release = releasedVersionOf(commit.subject) !== undefined;
+  const authors = release ? [...allowed, RELEASE_AUTHOR] : allowed;
+  if (!allows(authors, commit.author)) {
     reasons.push(`author ${render(commit.author)}`);
   }
   if (!allows([...allowed, SQUASH_COMMITTER], commit.committer)) {
     reasons.push(`committer ${render(commit.committer)}`);
   }
   for (const trailer of commit.coAuthoredBy) {
+    if (release && namesReleaseAuthor(trailer)) continue;
     reasons.push(`trailer ${trailer}`);
   }
   return reasons.length === 0 ? undefined : { commit, reasons };
@@ -144,6 +159,7 @@ const check = Effect.gen(function* () {
     }
     lines.push(`  allowed: ${allowed.map(render).join(", ")}`);
     lines.push(`  allowed as committer only: ${render(SQUASH_COMMITTER)}`);
+    lines.push(`  allowed as release author only: ${render(RELEASE_AUTHOR)}`);
     yield* Console.error(lines.join("\n"));
     return false;
   }

package/src/delivery/github.ts

@@ -0,0 +1,38 @@
+import { Effect, Schema } from "effect";
+import { ChildProcessSpawner } from "effect/process";
+import { collect } from "../core/git.ts";
+
+export class GitHubFailure extends Schema.TaggedError<GitHubFailure>()("GitHubFailure", {
+  message: Schema.String,
+}) {}
+
+type Method = "GET" | "POST" | "PATCH";
+
+type Decode<A> = (text: string) => Effect.Effect<A, Schema.SchemaError>;
+
+// gh fills {owner} and {repo} from the checkout's remote, so a workflow names no repository.
+export const REPOSITORY = "repos/{owner}/{repo}";
+
+const gitHub = Effect.fn("gitHub")(function* (method: Method, path: string, body?: unknown) {
+  const failed = (reason: string): GitHubFailure => new GitHubFailure({ message: `gh api ${method} ${path}: ${reason.trim()}` });
+  const input = body === undefined ? [] : ["--input", "-"];
+  const feed = body === undefined ? {} : { input: JSON.stringify(body) };
+  const { stdout, stderr, exitCode } = yield* collect("gh", ["api", "--method", method, path, ...input], undefined, feed).pipe(
+    Effect.mapError((cause) => failed(cause.message)),
+  );
+  if (exitCode !== ChildProcessSpawner.ExitCode(0)) return yield* failed(stderr);
+  return stdout;
+});
+
+export function gitHubJson<A>(decode: Decode<A>, method: Method, path: string, body?: unknown): Effect.Effect<A, GitHubFailure, ChildProcessSpawner.ChildProcessSpawner> {
+  return gitHub(method, path, body).pipe(
+    Effect.flatMap((text) =>
+      decode(text).pipe(Effect.mapError((cause) => new GitHubFailure({ message: `gh api ${method} ${path} answered ${cause.message}` }))),
+    ),
+  );
+}
+
+// A dispatch is the one run a workflow token starts at once: GitHub starts none for the token's push, and holds its pull request's runs for approval.
+export const dispatch = Effect.fn("dispatch")(function* (workflow: string, ref: string) {
+  yield* gitHub("POST", `${REPOSITORY}/actions/workflows/${workflow}/dispatches`, { ref });
+});

package/src/delivery/gitleaks.toml

@@ -0,0 +1,113 @@
+title = "checks-secrets"
+
+[extend]
+useDefault = true
+
+[[rules]]
+id = "wireguard-key"
+description = "A WireGuard or AmneziaWG private or pre-shared key"
+regex = '''(?i)(?:private[_-]?key|pre[_-]?shared[_-]?key|psk)["']?[ \t]*[:=][ \t]*["']?([A-Za-z0-9+/]{42}[AEIMQUYcgkosw048]=)'''
+secretGroup = 1
+entropy = 3.0
+keywords = ["private", "preshared", "pre_shared", "pre-shared", "psk"]
+
+[[rules]]
+id = "wireguard-key-file"
+description = "A WireGuard or AmneziaWG key alone on a line of a file whose name says private, priv, pre-shared or psk"
+path = '''(?i)(?:^|/)[^/]*(?:priv|pre[_-]?shared|psk)[^/]*$'''
+regex = '''(?m)^[ \t]*([A-Za-z0-9+/]{42}[AEIMQUYcgkosw048]=)[ \t]*\r?$'''
+secretGroup = 1
+entropy = 3.0
+
+[[rules]]
+id = "wireguard-dot-key-file"
+description = "A WireGuard or AmneziaWG key alone on a line of a .key file not named for a public key"
+path = '''(?i)\.key$'''
+regex = '''(?m)^[ \t]*([A-Za-z0-9+/]{42}[AEIMQUYcgkosw048]=)[ \t]*\r?$'''
+secretGroup = 1
+entropy = 3.0
+
+[[rules.allowlists]]
+description = "A .key file named for a public key"
+paths = ['''(?i)pub(?:lic)?\.key$''']
+
+[[rules]]
+id = "proxy-userinfo-link"
+description = "A proxy share link that carries its credential before the @, such as vless://, trojan://, socks5:// or wireguard://"
+regex = '''(?i)\b(?:vless|vmess|ss|trojan|hysteria2?|hy2|tuic|socks5?h?|wireguard|wg)://([^\s"'`<>@?#]+)@[^\s"'`<>]+'''
+secretGroup = 1
+entropy = 2.5
+keywords = ["vless://", "vmess://", "ss://", "trojan://", "hysteria://", "hysteria2://", "hy2://", "tuic://", "socks://", "socksh://", "socks5://", "socks5h://", "wireguard://", "wg://"]
+
+[[rules.allowlists]]
+description = "A link to a reserved example or test host"
+regexTarget = "match"
+regexes = ['''^(?i)[a-z0-9]+://(?:[^\s"'`@?#]*@)?(?:[\w-]+\.)*(?:example\.(?:com|net|org)|[\w-]+\.(?:example|invalid|test)|localhost)(?::\d+)?(?:[/?#]|$)''']
+
+[[rules]]
+id = "proxy-base64-link"
+description = "A proxy share link whose whole payload is base64, such as vmess://, ss:// or ssr://"
+regex = '''(?i)\b(?:vmess|ssr?)://([A-Za-z0-9+/_-]{16,}={0,2})@?'''
+secretGroup = 1
+entropy = 4.2
+keywords = ["vmess://", "ss://", "ssr://"]
+
+[[rules.allowlists]]
+description = "A payload an @ follows, which proxy-userinfo-link judges"
+regexTarget = "match"
+regexes = ['''@$''']
+
+[[rules]]
+id = "proxy-query-credential-first"
+description = "The first auth, auth_str, obfsParam, obfs-password, password or pk query parameter of a proxy share link, such as hysteria://host?auth="
+regex = '''(?i)\b(?:vless|vmess|ssr?|trojan|hysteria2?|hy2|tuic|socks5?h?|wireguard|wg)://[^\s"'`?#]*\?(?:[^\s"'`#]*?&)??(?:auth|auth_str|obfsParam|obfs-password|password|pk)=([^\s"'`<>&#]+)'''
+secretGroup = 1
+entropy = 2.5
+keywords = ["vless://", "vmess://", "ss://", "ssr://", "trojan://", "hysteria://", "hysteria2://", "hy2://", "tuic://", "socks://", "socksh://", "socks5://", "socks5h://", "wireguard://", "wg://"]
+
+[[rules.allowlists]]
+description = "A link to a reserved example or test host"
+regexTarget = "match"
+regexes = ['''^(?i)[a-z0-9]+://(?:[^\s"'`@?#]*@)?(?:[\w-]+\.)*(?:example\.(?:com|net|org)|[\w-]+\.(?:example|invalid|test)|localhost)(?::\d+)?(?:[/?#]|$)''']
+
+[[rules]]
+id = "proxy-query-credential-last"
+description = "The last auth, auth_str, obfsParam, obfs-password, password or pk query parameter of a proxy share link, such as hysteria://host?auth="
+regex = '''(?i)\b(?:vless|vmess|ssr?|trojan|hysteria2?|hy2|tuic|socks5?h?|wireguard|wg)://[^\s"'`?#]*\?(?:[^\s"'`#]*&)?(?:auth|auth_str|obfsParam|obfs-password|password|pk)=([^\s"'`<>&#]+)'''
+secretGroup = 1
+entropy = 2.5
+keywords = ["vless://", "vmess://", "ss://", "ssr://", "trojan://", "hysteria://", "hysteria2://", "hy2://", "tuic://", "socks://", "socksh://", "socks5://", "socks5h://", "wireguard://", "wg://"]
+
+[[rules.allowlists]]
+description = "A link to a reserved example or test host"
+regexTarget = "match"
+regexes = ['''^(?i)[a-z0-9]+://(?:[^\s"'`@?#]*@)?(?:[\w-]+\.)*(?:example\.(?:com|net|org)|[\w-]+\.(?:example|invalid|test)|localhost)(?::\d+)?(?:[/?#]|$)''']
+
+[[rules]]
+id = "proxy-subscription-url"
+description = "A proxy subscription URL whose path or query carries its token"
+regex = '''(?i)(?:\bhttps?://[^\s"'`<>/]+/(?:[^\s"'`<>?]*/)?(?:sub|subs|subscribe|subscription|link)/|[?&](?:token|key|uuid|id)=)([A-Za-z0-9_-]{16,})'''
+secretGroup = 1
+entropy = 3.0
+keywords = ["sub", "link"]
+
+[[rules.required]]
+id = "proxy-subscription-path"
+withinLines = 0
+
+[[rules]]
+id = "proxy-subscription-path"
+description = "An http or https URL whose last path segment before its token or query is sub, subs, subscribe, subscription or link"
+regex = '''(?i)\bhttps?://[^\s"'`<>/]+/(?:[^\s"'`<>?]*/)?(?:sub|subs|subscribe|subscription|link)[/?]'''
+keywords = ["sub", "link"]
+skipReport = true
+
+[[rules.allowlists]]
+description = "A link to a reserved example or test host"
+regexTarget = "match"
+regexes = ['''^(?i)https?://(?:[\w-]+\.)*(?:example\.(?:com|net|org)|[\w-]+\.(?:example|invalid|test)|localhost)(?:[:/?#]|$)''']
+
+[[allowlists]]
+description = "A link credential that is only template references, such as {password}, ${pass}, ${encodeURIComponent(pass)}, ${pass:-}, ${pass:?message}, $PASS, {{ password }} or %(password)s, with a user part that is one too; a last ${...} may be cut short, and one that gives a default or alternative value, such as ${pass:-value}, is not a reference"
+targetRules = ["proxy-userinfo-link", "proxy-query-credential-first", "proxy-query-credential-last"]
+regexes = ['''^(?:(?:\$\{(?:[^A-Za-z_}][^}]*|[A-Za-z_][\w.]*(?:[^\w.:=+}-][^}]*|:(?:[^-=+}][^}]*)?|:?[-=+])?)?\}|\$[A-Za-z_]\w*|\{\{\s*[A-Za-z_][\w.]*\s*\}\}|\{[A-Za-z_][\w.]*\}|%\([A-Za-z_][\w.]*\)s):)?(?:\$\{(?:[^A-Za-z_}][^}]*|[A-Za-z_][\w.]*(?:[^\w.:=+}-][^}]*|:(?:[^-=+}][^}]*)?|:?[-=+])?)?\}?|\$[A-Za-z_]\w*|\{\{\s*[A-Za-z_][\w.]*\s*\}\}|\{[A-Za-z_][\w.]*\}|%\([A-Za-z_][\w.]*\)s)$''']

package/src/delivery/gitleaks.ts

@@ -0,0 +1,111 @@
+import { Effect, Option, Path, Schema } from "effect";
+import { collect } from "../core/git.ts";
+import { installPinned } from "../dependencies/pinned-binary.ts";
+
+const GITLEAKS_VERSION = "8.30.1";
+const RELEASES = `https://github.com/gitleaks/gitleaks/releases/download/v${GITLEAKS_VERSION}`;
+const MEMBER = "gitleaks";
+
+type Build = { readonly archive: string; readonly sha256: string; readonly binarySha256: string };
+
+// The archive's SHA-256 as gitleaks_<version>_checksums.txt of the release lists it, and the SHA-256 of the binary inside.
+const BUILDS: Readonly<Record<string, Build>> = {
+  "darwin-arm64": {
+    archive: `gitleaks_${GITLEAKS_VERSION}_darwin_arm64.tar.gz`,
+    sha256: "b40ab0ae55c505963e365f271a8d3846efbc170aa17f2607f13df610a9aeb6a5",
+    binarySha256: "ba52fb1bfabbcde42f032afad3d6e0b19dff8ed105229a16e7caa338bbc0e84f",
+  },
+  "darwin-x64": {
+    archive: `gitleaks_${GITLEAKS_VERSION}_darwin_x64.tar.gz`,
+    sha256: "dfe101a4db2255fc85120ac7f3d25e4342c3c20cf749f2c20a18081af1952709",
+    binarySha256: "cee01fea7173f1b779dff188e1c26ecbcb4027d394acc573b23aaf0be260e291",
+  },
+  "linux-arm64": {
+    archive: `gitleaks_${GITLEAKS_VERSION}_linux_arm64.tar.gz`,
+    sha256: "e4a487ee7ccd7d3a7f7ec08657610aa3606637dab924210b3aee62570fb4b080",
+    binarySha256: "00e91bbe655bd7c47753e8cfe61cb76ea1a5d7e7702fe161ee40102b46b3823b",
+  },
+  "linux-x64": {
+    archive: `gitleaks_${GITLEAKS_VERSION}_linux_x64.tar.gz`,
+    sha256: "551f6fc83ea457d62a0d98237cbad105af8d557003051f41f3e7ca7b3f2470eb",
+    binarySha256: "88f91962aa2f93ac6ab281d553b9e125f5197bbbce38f9f2437f7299c32e5509",
+  },
+};
+
+class GitleaksError extends Schema.TaggedError<GitleaksError>()("GitleaksError", {
+  message: Schema.String,
+}) {}
+
+export function gitleaksBuildFor(platform: string, arch: string): Option.Option<Build> {
+  return Option.fromNullishOr(BUILDS[`${platform}-${arch}`]);
+}
+
+export const pinnedGitleaks = Effect.fn("pinnedGitleaks")(function* (cache: string) {
+  const build = gitleaksBuildFor(process.platform, process.arch);
+  if (Option.isNone(build)) {
+    return yield* new GitleaksError({ message: `no pinned gitleaks build runs on ${process.platform}-${process.arch}` });
+  }
+  const path = yield* Path.Path;
+  const { archive, sha256, binarySha256 } = build.value;
+  return yield* installPinned(
+    { kind: "tar.gz", url: `${RELEASES}/${archive}`, sha256, member: MEMBER, memberSha256: binarySha256 },
+    path.join(cache, "gitleaks", GITLEAKS_VERSION, MEMBER),
+  );
+});
+
+const Leak = Schema.Struct({
+  RuleID: Schema.String,
+  Description: Schema.String,
+  File: Schema.String,
+  StartLine: Schema.Int,
+  Commit: Schema.String,
+});
+
+export type Leak = typeof Leak.Type;
+
+const decodeGitleaksReport = Schema.decodeUnknownEffect(Schema.fromJsonString(Schema.Array(Leak)));
+
+export type Scan = {
+  readonly binary: string;
+  readonly gitDir: string;
+  readonly logOptions: string;
+  readonly config: string;
+  readonly ignoreDir: string;
+};
+
+// gitleaks exits 1 both on a leak and on a fatal error, so it exits 0 on a leak here and the report alone tells them apart.
+// It exits 0 with an empty report when git log fails, and logs only errors here, so any line it logs fails the scan.
+// It reads a .gitleaksignore at its source whatever --gitleaks-ignore-path says, so the source is the git directory, where no tracked file lands.
+export const scanCommits = Effect.fn("scanCommits")(function* ({ binary, gitDir, logOptions, config, ignoreDir }: Scan) {
+  const args = [
+    "git",
+    gitDir,
+    "--log-opts",
+    logOptions,
+    "--config",
+    config,
+    "--gitleaks-ignore-path",
+    ignoreDir,
+    "--ignore-gitleaks-allow",
+    "--redact",
+    "--no-banner",
+    "--no-color",
+    "--log-level",
+    "error",
+    "--exit-code",
+    "0",
+    "--report-format",
+    "json",
+    "--report-path",
+    "-",
+  ];
+  const { stdout, stderr, exitCode } = yield* collect(binary, args, ignoreDir).pipe(
+    Effect.mapError((cause) => new GitleaksError({ message: `cannot run ${binary}: ${cause.message}` })),
+  );
+  if (exitCode !== 0 || stderr.trim() !== "") {
+    return yield* new GitleaksError({ message: `${binary} failed: ${stderr.trim() || `exit code ${exitCode}`}` });
+  }
+  return yield* decodeGitleaksReport(stdout).pipe(
+    Effect.mapError((cause) => new GitleaksError({ message: `cannot read the report gitleaks wrote: ${cause.message}` })),
+  );
+});