aspora-threatlenscli 0.4.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,91 @@
1
+ # ── Python ────────────────────────────────────────────────────────────────────
2
+ __pycache__/
3
+ *.py[cod]
4
+ *.pyo
5
+ *.pyd
6
+ *.egg
7
+ *.egg-info/
8
+ dist/
9
+ build/
10
+ .eggs/
11
+ *.whl
12
+ .venv/
13
+ venv/
14
+ env/
15
+ .env
16
+ .env.*
17
+ !.env.example
18
+ # Claude Code: local agent config, skills and worktrees — never pushed.
19
+ .claude/
20
+ CLAUDE.md
21
+
22
+ # ── SQLite ────────────────────────────────────────────────────────────────────
23
+ *.db
24
+ *.db-shm
25
+ *.db-wal
26
+ *.sqlite
27
+ *.sqlite3
28
+
29
+ # ── Node / Next.js ────────────────────────────────────────────────────────────
30
+ node_modules/
31
+ .next/
32
+ out/
33
+ .vercel/
34
+ npm-debug.log*
35
+ yarn-debug.log*
36
+ yarn-error.log*
37
+ pnpm-debug.log*
38
+ *.tsbuildinfo
39
+ next-env.d.ts
40
+
41
+ # ── Docker ────────────────────────────────────────────────────────────────────
42
+ docker-compose.override.yml
43
+
44
+ # ── Temporal ─────────────────────────────────────────────────────────────────
45
+ temporal/dynamicconfig/*.yaml
46
+ !temporal/dynamicconfig/development.yaml
47
+
48
+ # ── OS / IDE ──────────────────────────────────────────────────────────────────
49
+ .DS_Store
50
+ .DS_Store?
51
+ ._*
52
+ .Spotlight-V100
53
+ .Trashes
54
+ Thumbs.db
55
+ .idea/
56
+ .vscode/
57
+ *.swp
58
+ *.swo
59
+
60
+ # ── Logs / coverage ───────────────────────────────────────────────────────────
61
+ *.log
62
+ logs/
63
+ coverage/
64
+ .coverage
65
+ htmlcov/
66
+ .pytest_cache/
67
+ .mypy_cache/
68
+ .ruff_cache/
69
+
70
+ # ── Secrets ───────────────────────────────────────────────────────────────────
71
+ *.pem
72
+ *.key
73
+ *.cert
74
+ secrets/
75
+ # SC-07 / SC-03 — credential material that has shown up in repos of this shape.
76
+ # `~/.threatlenscli/config.json` holds a live API key; the rest are the private-key and
77
+ # keystore formats the .pem/.key patterns above miss.
78
+ .threatlenscli/
79
+ *.p12
80
+ *.pfx
81
+ *.jks
82
+ *.keystore
83
+ *.crt
84
+ id_rsa
85
+ id_rsa.*
86
+ id_ed25519
87
+ id_ed25519.*
88
+ .envrc
89
+ # Never commit a real Fernet key for SECRETS_KEY (see backend/src/app/crypto.py).
90
+ secrets.key
91
+ *.secrets.json
@@ -0,0 +1,445 @@
1
+ Metadata-Version: 2.4
2
+ Name: aspora-threatlenscli
3
+ Version: 0.4.0
4
+ Summary: Image, code and open-source dependency scanning in your CI, reported to a ThreatLens server
5
+ Keywords: ci,container-scanning,gitleaks,sast,sca,security,semgrep,trivy
6
+ Classifier: Development Status :: 4 - Beta
7
+ Classifier: Environment :: Console
8
+ Classifier: Intended Audience :: Developers
9
+ Classifier: Operating System :: MacOS
10
+ Classifier: Operating System :: POSIX
11
+ Classifier: Programming Language :: Python :: 3
12
+ Classifier: Programming Language :: Python :: 3 :: Only
13
+ Classifier: Topic :: Security
14
+ Classifier: Topic :: Software Development :: Quality Assurance
15
+ Requires-Python: >=3.8
16
+ Description-Content-Type: text/markdown
17
+
18
+ # threatlenscli — ThreatLens CLI
19
+
20
+ Snyk-style scanning in **your** CI. The scanners run on the runner; only the
21
+ results go to your ThreatLens server. The server never builds your code, never pulls
22
+ your private images and never needs a Docker socket.
23
+
24
+ | Command | Scans | Engines (on this machine) | Snyk equivalent |
25
+ |---|---|---|---|
26
+ | `threatlenscli oss [PATH]` | open-source dependencies, incl. transitive | `trivy fs` (lockfiles) ∪ `trivy rootfs` (installed artefacts) ∪ optional build-tool SBOM | `snyk test` / `snyk monitor` |
27
+ | `threatlenscli code [PATH]` | SAST, code quality, secrets, IaC misconfig | Semgrep CE, Gitleaks (else Trivy secret), Trivy misconfig | `snyk code test` + `snyk iac test` |
28
+ | `threatlenscli image <REF>` (`scan`, `container`) | container image: OS + library vulns, secrets, misconfig | Trivy (same flags as the server) | `snyk container test` |
29
+
30
+ It is a single Python file with zero pip dependencies (Python 3.8+).
31
+
32
+ ## Install
33
+
34
+ From PyPI (the package is `aspora-threatlenscli`; the command is `threatlenscli`):
35
+
36
+ ```bash
37
+ pipx install aspora-threatlenscli # or: pip install aspora-threatlenscli
38
+ pipx install aspora-threatlenscli==0.4.0 # in CI: pin the version
39
+ ```
40
+
41
+ Or the single file straight from the repository:
42
+
43
+ ```bash
44
+ sudo install -m 755 cli/threatlenscli /usr/local/bin/threatlenscli # or: ./cli/threatlenscli install
45
+ threatlenscli --version # threatlenscli 0.2.0
46
+ threatlenscli doctor # checks everything below
47
+ ```
48
+
49
+ In CI, install threatlenscli from the commit you reviewed and verify it (SUP-1/SUP-5):
50
+
51
+ ```bash
52
+ curl -fsSLo threatlenscli https://raw.githubusercontent.com/<org>/<repo>/<40-char-sha>/cli/threatlenscli
53
+ echo "<sha256 from: git show <sha>:cli/threatlenscli | sha256sum> threatlenscli" | sha256sum -c -
54
+ ```
55
+
56
+ Requirements on the runner:
57
+
58
+ - **Trivy >= 0.53** on `PATH`. Install a pinned release and verify its checksum.
59
+ threatlenscli refuses the credential-stealing releases **0.69.4, 0.69.5 and 0.69.6**
60
+ (GHSA-69fq-xp46-6x23). If `doctor` ever reports one, remove it and rotate every
61
+ secret that runner can reach. Add more entries with
62
+ `THREATLENS_SCANNER_VERSION_DENYLIST=trivy:x.y.z,...`.
63
+ - For `code`: **semgrep**, installed hash-locked from the server's
64
+ `backend/semgrep.lock` of the reviewed commit (`python3.14 -m venv /opt/semgrep &&
65
+ /opt/semgrep/bin/pip install --require-hashes --only-binary=:all: -r backend/semgrep.lock`;
66
+ the lock is resolved for CPython 3.14 on Linux, so another interpreter fails
67
+ loudly instead of installing unhashed files) and, optionally, **gitleaks** (v8
68
+ with the `dir` command). Without semgrep the code scan has no SAST and no
69
+ quality rules. threatlenscli says so on stderr, and `--require-sast` turns that into
70
+ exit 2. semgrep and gitleaks are checked against the same denylist
71
+ (`THREATLENS_SCANNER_VERSION_DENYLIST=semgrep:x.y.z,gitleaks:x.y.z`); an unparseable
72
+ version exits 2 (fail closed).
73
+ - For `image`: the image must be reachable by Trivy (local Docker daemon or a
74
+ registry). Registry credentials go in `TRIVY_USERNAME` / `TRIVY_PASSWORD`, never on argv.
75
+
76
+ GitHub Actions: pin every third-party action (including any Trivy action) by
77
+ its full commit SHA. The trivy-action and setup-trivy tags were compromised in
78
+ the same incident.
79
+
80
+ ## Authenticate
81
+
82
+ An admin creates a key in **Settings → API Keys**. Give CI keys the narrowest scope:
83
+
84
+ | Command | Scope | Legacy scope (still accepted, audited as deprecated) |
85
+ |---|---|---|
86
+ | `threatlenscli oss` | `ingest:oss` | `code:write` |
87
+ | `threatlenscli code` | `ingest:code` | `code:write` |
88
+ | `threatlenscli image` | `ingest:image` | `images:write` |
89
+
90
+ An `ingest:*` key can upload results and read back its own runs. It cannot read
91
+ other findings, add images or trigger server scans. Keys expire after 90 days
92
+ by default.
93
+
94
+ **Allowed identities.** A key can be bound to the projects it may write, using
95
+ glob patterns over normalised identities: `github.com/acme/*` or
96
+ `github.com/acme/payments` (`*` matches one path segment, `**` any number).
97
+ With an empty list, the key may only write projects it created itself. An
98
+ upload for any other project gets 403, and the refusal is audited. Set these
99
+ bindings for every shared CI key, and again after rotating a key.
100
+
101
+ ```bash
102
+ threatlenscli login # asks for your ThreatLens URL, opens the browser: sign in, approve, done
103
+ ```
104
+
105
+ `login` opens the ThreatLens sign-in page in your browser (or prints the link, for
106
+ SSH sessions). Sign in, check that the code matches your terminal, and approve; the
107
+ terminal then says `Authenticated as you@company.com` and every command works. The
108
+ CLI receives its own upload-only key (90 days, revocable; `threatlenscli logout`
109
+ revokes it). Admins' CLIs may upload for any repository; a member's CLI only for
110
+ projects and branches it creates itself. To paste a key instead:
111
+ `threatlenscli login --with-api-key`.
112
+
113
+ No server is built in. In CI set both environment variables; they always win over
114
+ `~/.threatlenscli/config.json`. On a CI runner an env key is never paired with the
115
+ server a stale config on a shared runner names:
116
+
117
+ ```bash
118
+ export THREATLENS_SERVER=${{ vars.THREATLENS_SERVER }} # e.g. https://threatlens.example.com
119
+ export THREATLENS_API_KEY=${{ secrets.THREATLENS_API_KEY }}
120
+ ```
121
+
122
+
123
+ Transport rules:
124
+ - TLS certificates are always verified. Use `SSL_CERT_FILE` for a private CA.
125
+ - Plain `http://` works only for loopback. For another host, outside CI only,
126
+ pass `login --insecure` or set `THREATLENS_ALLOW_INSECURE=1`. It is never allowed in CI.
127
+ - Redirects to another host or port, or from https to http, are refused, so the
128
+ key is never forwarded. A key saved for server A is never sent to a different
129
+ `THREATLENS_SERVER`.
130
+ - Reads (GET) retry 429, 502, 503, 504 and network errors with backoff
131
+ (`Retry-After` honoured). The upload (POST) is retried only when the server
132
+ explicitly did not process it (429, or 503 `ingest_busy`/`scan_in_progress`)
133
+ or the request never left the runner (connection refused, DNS, send timeout).
134
+ A lost response (read timeout, reset, 502/504) is NOT retried: the server may
135
+ have stored the results, so the CLI exits 2 and says to check the UI.
136
+ `--upload-timeout` (default 300 s) sets how long to wait for the response.
137
+ Large uploads are gzip-compressed.
138
+
139
+ ## Exit codes (all scan commands)
140
+
141
+ | Code | Meaning |
142
+ |---|---|
143
+ | `0` | scan completed, nothing at or above the gate |
144
+ | `1` | scan completed, gating issues found |
145
+ | `2` | error: scanner failure or timeout, network/TLS, HTTP 4xx/5xx, upload refused, bad flag (`--fail-on patchable`, unknown `--fail-on` value), `--require-sast` without semgrep, denylisted or too-old Trivy |
146
+ | `3` | nothing scannable. `oss`: no package resolved by any pass (the unresolved manifests are listed). `code`: no file to scan. Never for images |
147
+
148
+ ## Gating (CI policy)
149
+
150
+ ```
151
+ --severity-threshold low|medium|high|critical issues at or ABOVE this severity gate (default low;
152
+ 'unknown' gates as medium)
153
+ --fail-on all fail only if a gating issue has a fix
154
+ --fail-on upgradable fail only if upgrading fixes it (unknown upgradability + a fix counts as upgradable)
155
+ --fail-on none report, never fail
156
+ (`threatlenscli code` ignores all/upgradable with a warning: secrets and SAST
157
+ findings have no upgrade fix, so those values would disable the gate)
158
+ --include-dev include dev dependencies (npm, yarn, gradle) — off by default, like `snyk test`.
159
+ Without it, dev dependencies are removed from every pass, including the copies
160
+ `npm ci` installed in node_modules (matched by purl, else name+version) and
161
+ binaries shipped inside a dev-only package (esbuild's Go binary), so they
162
+ neither gate nor upload; the count is printed and recorded in meta/coverage
163
+ --exclude-base-image-vulns image only: ignore OS vulns that come from the base image (needs --file)
164
+ --require-fresh-db exit 2 unless the Trivy DB was built within 48 h and is not past its NextUpdate
165
+ (useful with --offline runners that restore a cached DB)
166
+ ```
167
+
168
+ Code-quality and licence findings are reported (JSON, SARIF, the UI) but never fail
169
+ the build; `threatlenscli code --gate-quality` lets them gate too.
170
+
171
+ The gate uses the **server's triage**: findings marked false positive, accepted
172
+ risk or not affected (and not past their `suppress_until`) do not fail the build.
173
+ Nothing else carries over. A finding this scan reports is present, so a
174
+ default-branch row that the pipeline closed (`resolved`, `fixed`, `stale`,
175
+ `not_applicable`) still gates: a PR that brings back a vulnerability `main`
176
+ already fixed fails. Only an advisory withdrawn upstream (`rejected`) is exempt.
177
+ With `--no-upload` the gate runs on local results only, without triage; threatlenscli
178
+ notes this on stderr.
179
+
180
+ `--fail-on critical,high` (the old severity list) is still accepted. It now means
181
+ `--severity-threshold high`: at or above, so a critical fails a "high" gate.
182
+
183
+ ## Outputs
184
+
185
+ | Flag | Output |
186
+ |---|---|
187
+ | `--json` | exactly one JSON document on stdout (`schema: vss-cli-result/1`); all progress goes to stderr |
188
+ | `--json-file-output PATH` | the same document, written to a file |
189
+ | `--sarif-file-output PATH` | SARIF 2.1.0 for GitHub code scanning: one run per engine with distinct categories (`vss/oss/`, `vss/image/`, `vss/code/semgrep/`, `vss/code/gitleaks/`, `vss/code/trivy/`), text only, repo-relative paths, redacted |
190
+ | `--sbom-file-output PATH` | CycloneDX from `trivy convert` (inventory only, no vulnerabilities). Trivy 0.74 writes specVersion 1.7. For `oss` it covers every pass (lockfiles, installed artefacts, `--sbom`) after dev dependencies are excluded (unless `--include-dev`) |
191
+
192
+ JSON and SARIF are redacted: secrets never appear, even for public repositories
193
+ whose CI artefacts are public. No runner path leaves the machine: the report
194
+ `ArtifactName` and the SBOM `metadata.component.name` carry the project identity,
195
+ and absolute paths (target, `$VIRTUAL_ENV`, temp dir, `--artifact`, `--sbom`)
196
+ are replaced by placeholders in uploads and output files.
197
+
198
+ Trivy DB (C14): unless `--offline`, every scan first runs `--download-db-only`,
199
+ then `--download-java-db-only` (oss/image), then — for image and code — a
200
+ misconfig checks-bundle prefetch (a `trivy config` of an empty private dir).
201
+ Sources are pinned to `mirror.gcr.io` / `ghcr.io` unless the runner sets its own
202
+ `TRIVY_DB_REPOSITORY` / `TRIVY_JAVA_DB_REPOSITORY` / `TRIVY_CHECKS_BUNDLE_REPOSITORY`.
203
+ Every scan then runs with `--skip-db-update --skip-java-db-update
204
+ --skip-check-update`. A failed prefetch is a warning (the embedded checks are
205
+ used) and is recorded as `meta.check_bundle_prefetch` / `check_bundle_stale`.
206
+
207
+ ## `threatlenscli oss` — run it after the build
208
+
209
+ ```bash
210
+ npm ci # or: pip install -r requirements.txt into a venv / mvn package / gradle build
211
+ threatlenscli oss . --severity-threshold high --sarif-file-output oss.sarif
212
+ ```
213
+
214
+ - Pass 1, `trivy fs`: reads lockfiles with the dependency graph (relationship,
215
+ `introduced_through`, dev flag, licences). It runs in `precise` mode, so an
216
+ unpinned manifest yields no package rather than a made-up version.
217
+ - Pass 2, `trivy rootfs`: reads what is installed (`node_modules`,
218
+ `site-packages`, a venv (`$VIRTUAL_ENV` is added automatically), JAR/WAR
219
+ files, Go and Rust binaries). `--artifact PATH` adds built outputs;
220
+ `--lockfile-only` skips this pass.
221
+ - JVM builds keep dependencies in `~/.m2` / `~/.gradle`, outside the project:
222
+ - `--sbom target/bom.json` ingests a cyclonedx-maven-plugin or
223
+ cyclonedx-gradle-plugin SBOM (re-matched by `trivy sbom`, which takes no
224
+ `--skip-check-update`: it has no misconfig checks). The path is checked
225
+ (exists, <= 128 MiB, CycloneDX or SPDX JSON) before any Trivy pass runs.
226
+ - Manifests without a lockfile are resolved automatically **on a CI runner**,
227
+ like `snyk test` (elsewhere pass `--resolve`; `--no-resolve` turns it off):
228
+ 1. a workspace lockfile in a parent folder (npm/pnpm/yarn, uv/poetry/pdm, Cargo);
229
+ 2. the ecosystem's own tool, writing into a private dir, never the repo:
230
+ `npm install --package-lock-only --ignore-scripts`, `pip install --dry-run
231
+ --report`, `composer update --no-install --no-scripts --no-plugins`,
232
+ `bundle lock`, `cargo generate-lockfile`, `dotnet restore --use-lock-file`,
233
+ `mvn dependency:copy-dependencies`, a Gradle init-script task. The nearest
234
+ `gradlew`/`mvnw` up to the git root is used (multi-module builds);
235
+ 3. JVM only: the JARs built in a parent build root (Dockerfile in
236
+ `services/api`, `./gradlew assemble` at the repo root).
237
+ A missing tool or a failed run is a warning, never fatal. Build tools
238
+ execute your build configuration; the server never does.
239
+ - Detected manifests that did not resolve (`build.gradle` without
240
+ `gradle.lockfile`, unpinned `requirements.txt`, `.csproj` without
241
+ `packages.lock.json`, `requirements.lock`, ...) are listed with a fix. When no
242
+ package resolves at all, the exit code is 3, never a false "no
243
+ vulnerabilities".
244
+ - Identity is the normalised git remote (`github.com/acme/api`, credentials
245
+ stripped) plus the sub-path in a monorepo. Without a remote, pass
246
+ `--project-name`. CLI projects are separate from GitHub-imported ones; the UI
247
+ links them by remote.
248
+ - Only the **default branch** is monitored: its upload becomes the project's
249
+ inventory, findings are reconciled, the daily re-check watches it and Slack
250
+ alerts fire. Uploads from other branches (PRs, feature branches) are stored
251
+ as separate branch snapshots. They return triage-aware gate results and never
252
+ touch default-branch findings.
253
+ - The default branch comes from `refs/remotes/origin/HEAD`, `CI_DEFAULT_BRANCH`
254
+ (GitLab), `BUILDKITE_PIPELINE_DEFAULT_BRANCH`, or on GitHub Actions (whose
255
+ shallow checkout has no origin/HEAD) `repository.default_branch` in
256
+ `$GITHUB_EVENT_PATH`. Once the server has stored a project's default branch it
257
+ keeps it: a different claim from a client is logged, audited and ignored.
258
+ With no default known at all, only `main`/`master` are monitored, and that
259
+ guess is not stored.
260
+ - A pull/merge-request build is **never** monitored, whatever its head branch
261
+ is called (a fork PR from its own `main`). threatlenscli detects it from
262
+ `GITHUB_HEAD_REF`/`GITHUB_EVENT_NAME`, `CI_MERGE_REQUEST_IID`,
263
+ `BITBUCKET_PR_ID`, `BUILDKITE_PULL_REQUEST`, `CIRCLE_PULL_REQUEST`,
264
+ `SYSTEM_PULLREQUEST_PULLREQUESTID` or `CHANGE_ID`.
265
+ - The server re-derives everything from the raw Trivy reports (it never trusts
266
+ the client's merge) and unions them with a live OSV query of every versioned
267
+ package, so a finding carries `engines: trivy`, `osv` or both. An OSV outage
268
+ never resolves OSV-found findings.
269
+ - An upload resolves findings that disappeared only when it is provably
270
+ complete: fresh Trivy DB (not `--offline`), all requested passes present, no
271
+ package-count collapse, and no manifest that resolved last run and is
272
+ unresolved now (a monorepo sub-project whose install failed). Otherwise
273
+ nothing is resolved and the run's coverage says why.
274
+
275
+ ## `threatlenscli code`
276
+
277
+ ```bash
278
+ threatlenscli code . --require-sast --severity-threshold high --sarif-file-output code.sarif
279
+ ```
280
+
281
+ - **Semgrep** packs: the security base (`p/default`, `p/owasp-top-ten`,
282
+ `p/cwe-top-25`, `p/secrets`, `p/security-audit`), language and framework
283
+ packs from detected files, IaC/CI packs, and quality packs (`p/r2c-bug-scan`,
284
+ `p/r2c-best-practices`, `r/<lang>.lang.correctness`, ...). The server's list
285
+ (`GET /api/code/config?languages=…&frameworks=…&iac=…`, sent the local
286
+ detection) wins when it is available; `coverage.semgrep_pack_source` says which
287
+ list ran. The local fallback (`--no-upload`, or the server unreachable) uses
288
+ tables that mirror the server's default `select_configs` exactly (a parity
289
+ test compares them), including `p/ci` for CI configs (`.github/workflows`,
290
+ `.gitlab-ci.yml`, `Jenkinsfile`, ...); a server-side `CODE_SEMGREP_CONFIGS`
291
+ override is not applied locally (`coverage.semgrep_pack_note`). Semgrep always
292
+ runs with `--metrics=off`, never `--config auto`, `--disable-nosem` and
293
+ `--verbose` (only then does Semgrep report `paths.skipped`; its log goes to a
294
+ file in the private dir). Files over `--semgrep-max-target-bytes` (default
295
+ 1000000, the server's) or that time out / fail to parse are counted in
296
+ `coverage.semgrep` (`skipped_paths`, `skipped_by_reason`), warned about, and
297
+ uploaded as repo-relative `paths.skipped` so the server never resolves a
298
+ finding in a file Semgrep did not read. Extra configs:
299
+ `--semgrep-config p/jwt` or a local rules file.
300
+ - **Gitleaks** (when installed) runs without `--redact` into a 0600 report
301
+ inside a private temp dir. threatlenscli digests each secret (sha256), then drops
302
+ the raw value and masks Match/Line before anything is uploaded; the report is
303
+ deleted. gitleaks always honours a repo `.gitleaksignore`, and threatlenscli reports
304
+ when one exists. Without gitleaks, Trivy's secret scanner is used, and the
305
+ digest is recovered from the file, not from Trivy's masked match. Gitleaks
306
+ skips files over `--gitleaks-max-target-mb` (default 50, the server's
307
+ `GITLEAKS_MAX_TARGET_MB`) without a word, so threatlenscli lists them itself:
308
+ `coverage.bounds.GITLEAKS_MAX_TARGET_MB` (`value`, `hit`, `skipped_sample`),
309
+ `coverage.gitleaks_skipped.paths` and a warning.
310
+ - **Trivy** misconfig: Dockerfile, Kubernetes, Helm, CloudFormation, Azure ARM,
311
+ Terraform plan JSON and Ansible (`--misconfig-scanners
312
+ azure-arm,cloudformation,dockerfile,helm,kubernetes,terraformplan-json,ansible`),
313
+ with `**/<dir>` skip globs. **Trivy's Terraform scanner never runs in threatlenscli**:
314
+ it resolves `module` sources from the scanned tree, including loopback
315
+ addresses no proxy can block, so a fork PR's `.tf` could make the runner
316
+ connect where it chooses. When `.tf`, `.tf.json`, `.tofu` or `.tofu.json` files
317
+ are present, threatlenscli warns and records `coverage.terraform_not_scanned`; only
318
+ Semgrep `p/terraform` rules cover them. Full Terraform IaC coverage needs the
319
+ server code scan (which scans a guarded copy of the tree).
320
+ Trivy runs with **no network**: a dead loopback proxy (`HTTPS_PROXY`/`HTTP_PROXY`/
321
+ `ALL_PROXY` in both cases, empty `NO_PROXY`/`no_proxy`), `GIT_ALLOW_PROTOCOL=vss-none`
322
+ and a private `TMPDIR`.
323
+ - Semgrep's secret digest is sha256 of the credential literal (one quoted
324
+ literal, or the value of a single `NAME=VALUE` assignment), the same rule as
325
+ the server and Gitleaks/Trivy, so one secret found by several engines is one finding.
326
+ - `--tracked-only` reports only files git tracks (`git ls-files -z`, so
327
+ non-ASCII names match). Committed `.env` and `.github/**` files are kept. If
328
+ `git ls-files` fails or times out, `--tracked-only` exits 2 instead of
329
+ filtering against an unknown list.
330
+ - Secret findings from Semgrep are masked before upload, in `--json` and in
331
+ SARIF: the matched span, each of its lines, quoted literals inside it and
332
+ every metavariable value (what rule messages interpolate) are masked on the
333
+ full source line before it is clipped. If a literal would still be visible,
334
+ the snippet is dropped and the message becomes the rule title. The mask is the
335
+ server's: a known token prefix plus the last 2 characters for tokens of 16+
336
+ characters (`ghp_****…i2`), `****…xy` for 24+ characters, and nothing at all
337
+ for shorter secrets (`********`).
338
+ - Local findings (`--no-upload` gate, `--json`, SARIF) use the server's
339
+ identities: one secret found by several engines is ONE finding (merged on file
340
+ + secret digest, `engines` lists them); misconfig identity is rule + resource +
341
+ a hash of the cause lines + an occurrence index, never a line number, so SARIF
342
+ `partialFingerprints` survive edits above the finding. A secret's local
343
+ fingerprint is keyed with a per-run random key and SARIF omits
344
+ `partialFingerprints` for secrets, so a published artifact carries nothing
345
+ derived from a credential. SARIF tags quality results `correctness`/
346
+ `maintainability`/`performance` and licence results `license` (no `security`
347
+ tag or `security-severity`).
348
+ - `--offline`: Semgrep registry packs (`p/...`, `r/...`) are downloads from
349
+ semgrep.dev, so they are skipped. Only `--semgrep-config <local rules>` run;
350
+ with none, Semgrep is skipped with a warning (`--require-sast` then exits 2).
351
+ No checks-bundle prefetch runs (cached or embedded misconfig checks,
352
+ recorded as `check_bundle_stale`).
353
+
354
+ ## `threatlenscli image`
355
+
356
+ ```bash
357
+ threatlenscli image my-app:1.4.2 --file Dockerfile --platform linux/amd64 --severity-threshold critical
358
+ ```
359
+
360
+ This uses the server's exact Trivy argv, flag for flag and in order
361
+ (`GET /api/images/scan-profile` shows it): `--scanners vuln,secret,misconfig`,
362
+ `--image-config-scanners misconfig,secret`, `--pkg-types os,library`,
363
+ `--list-all-pkgs`, `--skip-check-update`, `--no-progress`,
364
+ `--misconfig-scanners dockerfile,kubernetes,helm,cloudformation,azure-arm,ansible`
365
+ (never Terraform: its module resolution would make the runner fetch
366
+ attacker-chosen URLs), `--detection-priority comprehensive`,
367
+ `--max-image-size 16384MiB` and `--image-src docker,remote -- <ref>`. The only
368
+ differences are no `--cache-dir` and `--offline-scan` with `--offline`. A Trivy
369
+ too old for `--max-image-size` drops it with a warning (`meta.flags_dropped`).
370
+ The scan runs with `GIT_ALLOW_PROTOCOL=vss-none` and a private `TMPDIR`
371
+ (registry egress stays: the image must be pulled). `--file` uploads the
372
+ Dockerfile's FROM lines for base-image attribution and advice.
373
+
374
+ ## CI recipes
375
+
376
+ ```yaml
377
+ # GitHub Actions — PRs gate, main monitors (same command; the branch decides)
378
+ - run: npm ci
379
+ - run: threatlenscli oss . --severity-threshold high --sarif-file-output oss.sarif
380
+ env: { THREATLENS_SERVER: ${{ vars.THREATLENS_SERVER }}, THREATLENS_API_KEY: ${{ secrets.THREATLENS_OSS_KEY }} }
381
+ - run: threatlenscli code . --require-sast --severity-threshold high --sarif-file-output code.sarif
382
+ env: { THREATLENS_SERVER: ${{ vars.THREATLENS_SERVER }}, THREATLENS_API_KEY: ${{ secrets.THREATLENS_CODE_KEY }} }
383
+ - uses: github/codeql-action/upload-sarif@<full-commit-sha>
384
+ if: always()
385
+ with: { sarif_file: oss.sarif, category: threatlens-oss }
386
+ ```
387
+
388
+ ```yaml
389
+ # GitLab CI
390
+ threatlens:
391
+ script:
392
+ - threatlenscli oss . --severity-threshold high --json-file-output threatlens-oss.json
393
+ artifacts: { paths: [threatlens-oss.json], when: always }
394
+ ```
395
+
396
+ Every upload carries provenance in `meta`: the Trivy, Semgrep and Gitleaks
397
+ versions, the Trivy binary sha256, the Trivy DB `UpdatedAt`, sanitised argv
398
+ (the target and temp dirs become `<target>`/`<tmp>`, any other absolute path keeps
399
+ only its basename, credential flag values are masked), git remote, branch and commit, and the scanner
400
+ environment variables that were ignored (for example a `TRIVY_SEVERITY` set in
401
+ CI).
402
+
403
+ ## Other commands
404
+
405
+ ```
406
+ threatlenscli login [--server URL] browser sign-in (--with-api-key: paste a key); 0600 config
407
+ threatlenscli logout remove saved config
408
+ threatlenscli config show config (key masked; env overrides flagged)
409
+ threatlenscli doctor scanners, denylist, DB age, server reachability, key scopes + identities
410
+ threatlenscli list [--type repos] images (default) or repositories on the server
411
+ threatlenscli install [--dest DIR] symlink this script onto PATH
412
+ ```
413
+
414
+ ## Measuring parity with Snyk
415
+
416
+ The JSON document written by `--json-file-output` (`schema: vss-cli-result/1`)
417
+ is the ThreatLens input of the parity harness in `backend/tests/parity/`. Run the scan
418
+ with `--no-upload` on the same machine and day as the Snyk run, for example:
419
+
420
+ ```bash
421
+ threatlenscli image nginx:1.25@sha256:<digest> --no-upload --json-file-output threatlens-image.json
422
+ snyk container test nginx:1.25@sha256:<digest> --json-file-output=snyk-container.json
423
+ python3 backend/tests/parity/parity_harness.py compare --snyk-container snyk-container.json --vss threatlens-image.json
424
+ ```
425
+
426
+ The corpus, the commands for every target, the match rules and the pass bar
427
+ (recall >= 80 % per product, precision reported) are in
428
+ [`docs/parity-benchmark.md`](../docs/parity-benchmark.md). No parity figure is
429
+ claimed until it has been measured that way.
430
+
431
+ ## Honest limitations
432
+
433
+ - Semgrep CE is intraprocedural: there is no cross-function or cross-file taint
434
+ (Snyk Code has it). Pro-only rules are unavailable.
435
+ - Registry rules are under the Semgrep Rules License and are fetched live;
436
+ threatlenscli bundles none.
437
+ - `trivy rootfs` finds installed packages but has no dependency graph, so those
438
+ findings have `relationship: unknown`. Rust binaries are found only when built
439
+ with cargo-auditable.
440
+ - `--fail-on upgradable` knows upgradability exactly only for direct
441
+ dependencies. For transitive ones, "has a fix" is treated as upgradable
442
+ (fails closed).
443
+ - The Trivy DB is the newest build available when the run starts, which can
444
+ lag upstream by up to about 24 h.
445
+ - A code scan does not scan git history for secrets.