@openai/codex-security 0.8.0 → 0.8.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +16 -10
- package/_bundled_plugin/.codex-plugin/plugin.json +1 -0
- package/_bundled_plugin/mcp/helpers.mjs.br.part-000 +0 -0
- package/_bundled_plugin/mcp/helpers.mjs.br.part-001 +0 -0
- package/_bundled_plugin/mcp/native/THIRD_PARTY_NOTICES.txt +4 -4
- package/_bundled_plugin/mcp/native/darwin-arm64/unix.node +0 -0
- package/_bundled_plugin/mcp/native/darwin-x64/unix.node +0 -0
- package/_bundled_plugin/mcp/native/linux-arm64-gnu/unix.node +0 -0
- package/_bundled_plugin/mcp/native/linux-arm64-musl/unix.node +0 -0
- package/_bundled_plugin/mcp/native/linux-x64-gnu/unix.node +0 -0
- package/_bundled_plugin/mcp/native/linux-x64-musl/unix.node +0 -0
- package/_bundled_plugin/mcp/native/win32-arm64/windows.node +0 -0
- package/_bundled_plugin/mcp/native/win32-x64/windows.node +0 -0
- package/_bundled_plugin/mcp/server.mjs.br.part-000 +0 -0
- package/_bundled_plugin/mcp/server.mjs.br.part-001 +0 -0
- package/_bundled_plugin/mcp/server.mjs.br.part-002 +0 -0
- package/_bundled_plugin/references/scan-artifacts.md +1 -1
- package/_bundled_plugin/scripts/finalize_scan_contract.py +37 -1
- package/_bundled_plugin/scripts/servicenow_avit.py +107 -0
- package/_bundled_plugin/scripts/workbench_coverage_receipts.py +0 -17
- package/_bundled_plugin/scripts/workbench_db.py +1 -3
- package/_bundled_plugin/scripts/workbench_saved_results.py +0 -66
- package/_bundled_plugin/scripts/workbench_scan_start.py +4 -1
- package/_bundled_plugin/scripts/workbench_target.py +0 -108
- package/_bundled_plugin/skills/publish-findings/SKILL.md +9 -3
- package/_bundled_plugin/skills/resolve-dependency-alert/SKILL.md +18 -0
- package/_bundled_plugin/skills/resolve-dependency-alert/agents/openai.yaml +4 -0
- package/_bundled_plugin/skills/security-scan/SKILL.md +1 -1
- package/_bundled_plugin/skills/threat-model-visualize/SKILL.md +83 -0
- package/_bundled_plugin/skills/threat-model-visualize/agents/openai.yaml +4 -0
- package/_bundled_plugin/skills/threat-model-visualize/assets/context-reference-overview.svg +366 -0
- package/_bundled_plugin/skills/threat-model-visualize/assets/context-reference-path.svg +558 -0
- package/_bundled_plugin/skills/threat-model-visualize/assets/control-effects.js +85 -0
- package/_bundled_plugin/skills/threat-model-visualize/assets/explorer.css +17 -0
- package/_bundled_plugin/skills/threat-model-visualize/assets/explorer.html +38 -0
- package/_bundled_plugin/skills/threat-model-visualize/assets/explorer.js +372 -0
- package/_bundled_plugin/skills/threat-model-visualize/assets/route-paths.js +71 -0
- package/_bundled_plugin/skills/threat-model-visualize/assets/source-roles.js +47 -0
- package/_bundled_plugin/skills/threat-model-visualize/assets/svg-layout.js +149 -0
- package/_bundled_plugin/skills/threat-model-visualize/references/context-explorer.md +21 -0
- package/_bundled_plugin/skills/threat-model-visualize/references/view-model.md +49 -0
- package/_bundled_plugin/skills/threat-model-visualize/scripts/build_explorer.py +563 -0
- package/dist/api.d.ts.map +1 -1
- package/dist/api.js +36 -15
- package/dist/api.js.map +1 -1
- package/dist/artifact-export.d.ts +12 -3
- package/dist/artifact-export.d.ts.map +1 -1
- package/dist/artifact-export.js +30 -10
- package/dist/artifact-export.js.map +1 -1
- package/dist/auth.js +1 -1
- package/dist/auth.js.map +1 -1
- package/dist/cli-help.d.ts.map +1 -1
- package/dist/cli-help.js +1 -0
- package/dist/cli-help.js.map +1 -1
- package/dist/cli.d.ts +1 -1
- package/dist/cli.d.ts.map +1 -1
- package/dist/cli.js +72 -50
- package/dist/cli.js.map +1 -1
- package/dist/contract.d.ts +1 -0
- package/dist/contract.d.ts.map +1 -1
- package/dist/contract.js +16 -12
- package/dist/contract.js.map +1 -1
- package/dist/custom-validation-prompt.d.ts.map +1 -1
- package/dist/custom-validation-prompt.js +12 -4
- package/dist/custom-validation-prompt.js.map +1 -1
- package/dist/custom-validation.js +2 -2
- package/dist/custom-validation.js.map +1 -1
- package/dist/deduplication/deduplication-reviewer.d.ts.map +1 -1
- package/dist/deduplication/deduplication-reviewer.js +1 -1
- package/dist/deduplication/deduplication-reviewer.js.map +1 -1
- package/dist/deduplication/records-protocol.d.ts.map +1 -1
- package/dist/deduplication/records-protocol.js +36 -22
- package/dist/deduplication/records-protocol.js.map +1 -1
- package/dist/deduplication/records.d.ts +3 -1
- package/dist/deduplication/records.d.ts.map +1 -1
- package/dist/deduplication/records.js +12 -8
- package/dist/deduplication/records.js.map +1 -1
- package/dist/endor-findings.d.ts.map +1 -1
- package/dist/endor-findings.js +46 -61
- package/dist/endor-findings.js.map +1 -1
- package/dist/external-findings-import.d.ts.map +1 -1
- package/dist/external-findings-import.js +8 -6
- package/dist/external-findings-import.js.map +1 -1
- package/dist/external-findings-local.d.ts +3 -1
- package/dist/external-findings-local.d.ts.map +1 -1
- package/dist/external-findings-local.js +136 -36
- package/dist/external-findings-local.js.map +1 -1
- package/dist/external-findings-publish.d.ts +3 -1
- package/dist/external-findings-publish.d.ts.map +1 -1
- package/dist/external-findings-publish.js +56 -31
- package/dist/external-findings-publish.js.map +1 -1
- package/dist/external-findings-read.d.ts +2 -0
- package/dist/external-findings-read.d.ts.map +1 -1
- package/dist/external-findings-read.js +9 -0
- package/dist/external-findings-read.js.map +1 -1
- package/dist/external-findings.d.ts +11 -0
- package/dist/external-findings.d.ts.map +1 -1
- package/dist/external-findings.js +13 -1
- package/dist/external-findings.js.map +1 -1
- package/dist/external-import-models.d.ts +1 -1
- package/dist/external-import-models.d.ts.map +1 -1
- package/dist/external-import-models.js.map +1 -1
- package/dist/github-advisories.d.ts.map +1 -1
- package/dist/github-advisories.js +5 -5
- package/dist/github-advisories.js.map +1 -1
- package/dist/github-dependabot-findings.d.ts +6 -0
- package/dist/github-dependabot-findings.d.ts.map +1 -0
- package/dist/github-dependabot-findings.js +247 -0
- package/dist/github-dependabot-findings.js.map +1 -0
- package/dist/github-findings.d.ts.map +1 -1
- package/dist/github-findings.js +11 -4
- package/dist/github-findings.js.map +1 -1
- package/dist/github-identifiers.d.ts +4 -0
- package/dist/github-identifiers.d.ts.map +1 -0
- package/dist/github-identifiers.js +9 -0
- package/dist/github-identifiers.js.map +1 -0
- package/dist/github.d.ts.map +1 -1
- package/dist/github.js +4 -6
- package/dist/github.js.map +1 -1
- package/dist/host-protocol.d.ts +5 -0
- package/dist/host-protocol.d.ts.map +1 -1
- package/dist/host-protocol.js +27 -3
- package/dist/host-protocol.js.map +1 -1
- package/dist/hosted-scan.d.ts.map +1 -1
- package/dist/hosted-scan.js +13 -5
- package/dist/hosted-scan.js.map +1 -1
- package/dist/index.d.ts +2 -0
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +2 -0
- package/dist/index.js.map +1 -1
- package/dist/inspector-findings.d.ts +9 -0
- package/dist/inspector-findings.d.ts.map +1 -0
- package/dist/inspector-findings.js +517 -0
- package/dist/inspector-findings.js.map +1 -0
- package/dist/patch-tui.d.ts +1 -1
- package/dist/patch-tui.d.ts.map +1 -1
- package/dist/patch-tui.js +4 -1
- package/dist/patch-tui.js.map +1 -1
- package/dist/runtime.d.ts +3 -2
- package/dist/runtime.d.ts.map +1 -1
- package/dist/runtime.js +15 -5
- package/dist/runtime.js.map +1 -1
- package/dist/scan-dashboard.d.ts.map +1 -1
- package/dist/scan-dashboard.js +25 -15
- package/dist/scan-dashboard.js.map +1 -1
- package/dist/scanner-import.d.ts +6 -2
- package/dist/scanner-import.d.ts.map +1 -1
- package/dist/scanner-import.js +23 -14
- package/dist/scanner-import.js.map +1 -1
- package/dist/semgrep-findings.d.ts.map +1 -1
- package/dist/semgrep-findings.js +39 -49
- package/dist/semgrep-findings.js.map +1 -1
- package/dist/snyk-findings.d.ts.map +1 -1
- package/dist/snyk-findings.js +78 -88
- package/dist/snyk-findings.js.map +1 -1
- package/docs/cli.md +52 -14
- package/docs/dedupe-records.md +17 -4
- package/docs/hosted-scans.md +14 -0
- package/package.json +5 -5
- package/schemas/external-findings.schema.json +1 -1
package/README.md
CHANGED
|
@@ -174,8 +174,8 @@ try {
|
|
|
174
174
|
|
|
175
175
|
Omitting `outputDir` saves results in the private Codex Security state directory.
|
|
176
176
|
To choose another location, use a new or empty directory outside the scanned
|
|
177
|
-
repository and its enclosing Git worktree.
|
|
178
|
-
|
|
177
|
+
repository and its enclosing Git worktree. Existing output directories keep their
|
|
178
|
+
permissions; newly created directories are private to you by default.
|
|
179
179
|
|
|
180
180
|
### Read results
|
|
181
181
|
|
|
@@ -667,6 +667,12 @@ options where supported, and changing formats can change record identities or
|
|
|
667
667
|
replace richer evidence with less detail. See the
|
|
668
668
|
[import guide](docs/cli.md#import-vendor-findings-locally) for format selection.
|
|
669
669
|
|
|
670
|
+
For Amazon Inspector Code Security, use `--provider inspector` with a complete
|
|
671
|
+
ListFindings JSON export and `--source-repository-url`. When the export identifies
|
|
672
|
+
the repository by an ARN and name, pass `--inspector-project-arn ARN` with the ARN
|
|
673
|
+
you verified for that Git URL in the SCM integration. See the
|
|
674
|
+
[Inspector guide](https://github.com/openai/codex-security/blob/main/docs/scanner-imports/inspector.md).
|
|
675
|
+
|
|
670
676
|
For OpenGrep, use `--provider opengrep` with native JSON or gzip and an explicit
|
|
671
677
|
`--source-repository-url`. Local and Cloud imports retain unassessed OpenGrep
|
|
672
678
|
evidence through the existing Semgrep source-provider contract. See the
|
|
@@ -679,13 +685,13 @@ repository package vulnerabilities (SCA) only; it does not add SAST, CSPM or
|
|
|
679
685
|
complete code-to-cloud coverage. Both formats support gzip. The acquisition path
|
|
680
686
|
still needs validation with an authorized Wiz tenant.
|
|
681
687
|
|
|
682
|
-
For GitHub, save complete JSON from
|
|
683
|
-
|
|
684
|
-
|
|
685
|
-
|
|
686
|
-
|
|
687
|
-
|
|
688
|
-
to GitHub.
|
|
688
|
+
For GitHub code scanning and GHSA advisories, save complete JSON from
|
|
689
|
+
`import github` or `import github-advisories`, respectively. **GHSA import
|
|
690
|
+
does not establish an installed dependency or applicability to the repository.**
|
|
691
|
+
For Dependabot, save the native REST alert records from one repository using the
|
|
692
|
+
[Dependabot export guide](https://github.com/openai/codex-security/blob/main/docs/scanner-imports/github-dependabot.md).
|
|
693
|
+
The GitHub file reader accepts all three families. Secret-scanning alerts and
|
|
694
|
+
writeback to GitHub are outside its scope.
|
|
689
695
|
|
|
690
696
|
Use `publish findings FILE --to local --repository PATH --provider wiz
|
|
691
697
|
--source-key KEY` to store the same vendor evidence in the local workbench SQLite
|
|
@@ -701,7 +707,7 @@ Use `--provider endor` with native Endor Finding JSON; existing endorctl SARIF i
|
|
|
701
707
|
accepted as a lossy compatibility format. See
|
|
702
708
|
[Endor imports](docs/cli.md#import-endor-labs-findings).
|
|
703
709
|
|
|
704
|
-
For saved GitHub code-scanning alerts
|
|
710
|
+
For saved GitHub code-scanning alerts, GHSA advisories, or Dependabot alerts, use
|
|
705
711
|
`--provider github` with a separate stable source key for each finding family.
|
|
706
712
|
See [local imports](docs/cli.md#import-vendor-findings-locally).
|
|
707
713
|
|
|
Binary file
|
|
Binary file
|
|
@@ -3137,7 +3137,7 @@ SOFTWARE.
|
|
|
3137
3137
|
|
|
3138
3138
|
---
|
|
3139
3139
|
|
|
3140
|
-
napi-derive-backend@6.1.
|
|
3140
|
+
napi-derive-backend@6.1.6
|
|
3141
3141
|
|
|
3142
3142
|
MIT License
|
|
3143
3143
|
|
|
@@ -3186,7 +3186,7 @@ SOFTWARE.
|
|
|
3186
3186
|
|
|
3187
3187
|
---
|
|
3188
3188
|
|
|
3189
|
-
napi-derive@3.6.
|
|
3189
|
+
napi-derive@3.6.13
|
|
3190
3190
|
|
|
3191
3191
|
MIT License
|
|
3192
3192
|
|
|
@@ -3235,7 +3235,7 @@ SOFTWARE.
|
|
|
3235
3235
|
|
|
3236
3236
|
---
|
|
3237
3237
|
|
|
3238
|
-
napi-sys@3.4.
|
|
3238
|
+
napi-sys@3.4.1
|
|
3239
3239
|
|
|
3240
3240
|
MIT License
|
|
3241
3241
|
|
|
@@ -3284,7 +3284,7 @@ SOFTWARE.
|
|
|
3284
3284
|
|
|
3285
3285
|
---
|
|
3286
3286
|
|
|
3287
|
-
napi@3.
|
|
3287
|
+
napi@3.15.0
|
|
3288
3288
|
|
|
3289
3289
|
MIT License
|
|
3290
3290
|
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
@@ -11,7 +11,7 @@ Follow `artifact-storage.md` for storage selection, file authoring and shared th
|
|
|
11
11
|
- `scan_id=<scanId returned by the MCP scan context, or CODEX_SECURITY_SCAN_ID when provided>`
|
|
12
12
|
- `scan_dir=<scanDir returned by the MCP scan context>`; when `CODEX_SECURITY_SCAN_DIR` is provided, use that existing SDK-owned directory. Never derive a scan directory from a repository name, revision, timestamp or temporary-directory formula.
|
|
13
13
|
- For a standalone phase, use the `directory` returned by `save_codex_security_artifact` for the selected `targetPath` and `storage`, as described in `artifact-storage.md`.
|
|
14
|
-
- `target_paths_file=<CODEX_SECURITY_TARGET_PATHS_FILE>` for SDK scoped-path scans; reference it as `"$CODEX_SECURITY_TARGET_PATHS_FILE"` in POSIX shells or `"$env:CODEX_SECURITY_TARGET_PATHS_FILE"` in PowerShell. This read-only scope
|
|
14
|
+
- `target_paths_file=<CODEX_SECURITY_TARGET_PATHS_FILE>` for SDK scoped-path scans; reference it as `"$CODEX_SECURITY_TARGET_PATHS_FILE"` in POSIX shells or `"$env:CODEX_SECURITY_TARGET_PATHS_FILE"` in PowerShell. This read-only scope file lives outside the scan directory. Pass it unchanged as `--scopes-file` to the SDK's primary inventory command and to `bind-repo-scopes` before finalization. For `make-hosted-repo-scope-input`, review the inventory at the path it reports. Use the supplied scan-directory recovery command only if a later shell cannot read that file. Do not print, evaluate, modify, or treat the scope file as shell syntax.
|
|
15
15
|
- `artifacts_dir=<scan_dir>/artifacts`
|
|
16
16
|
- `context_dir=<artifacts_dir>/01_context`
|
|
17
17
|
- `discovery_dir=<artifacts_dir>/02_discovery`
|
|
@@ -26,6 +26,7 @@ from urllib.parse import quote, urlsplit
|
|
|
26
26
|
# Keep sibling helpers importable when Python starts in isolated mode.
|
|
27
27
|
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
|
28
28
|
import report_projection
|
|
29
|
+
import servicenow_avit
|
|
29
30
|
import threat_model_projection
|
|
30
31
|
from review_state import checkpoint_review_state, review_state, review_summary
|
|
31
32
|
|
|
@@ -78,6 +79,7 @@ SOURCE_READ_CHUNK_SIZE = 64 * 1024
|
|
|
78
79
|
MAX_JSON_INTEGER = (1 << 53) - 1
|
|
79
80
|
EXPORT_PATHS = {
|
|
80
81
|
"csv": "exports/findings.csv",
|
|
82
|
+
"servicenow-avit": "exports/servicenow-avit.csv",
|
|
81
83
|
"json": "exports/findings.json",
|
|
82
84
|
"sarif": "exports/results.sarif",
|
|
83
85
|
}
|
|
@@ -2977,11 +2979,21 @@ def build_csv_projection(findings: dict[str, Any], coverage: dict[str, Any]) ->
|
|
|
2977
2979
|
return output.getvalue().encode("utf-8")
|
|
2978
2980
|
|
|
2979
2981
|
|
|
2982
|
+
def _avit_fields(finding: dict[str, Any]) -> dict[str, str]:
|
|
2983
|
+
attack_path = dict(finding.get("attackPath") or {})
|
|
2984
|
+
_normalize_legacy_string_list_fields(attack_path, ("steps",))
|
|
2985
|
+
steps = "\n".join(attack_path.get("steps", []))
|
|
2986
|
+
return servicenow_avit.finding_fields(
|
|
2987
|
+
finding, _sarif_primary_location(finding), csv_cell(steps)
|
|
2988
|
+
)
|
|
2989
|
+
|
|
2990
|
+
|
|
2980
2991
|
def build_findings_export(
|
|
2981
2992
|
scan_dir: Path,
|
|
2982
2993
|
export_format: str,
|
|
2983
2994
|
source_root: Path | None = None,
|
|
2984
2995
|
schema_dir: Path | None = None,
|
|
2996
|
+
avit_template: Path | None = None,
|
|
2985
2997
|
) -> bytes:
|
|
2986
2998
|
if export_format not in EXPORT_PATHS:
|
|
2987
2999
|
raise ContractError(f"unsupported export format: {export_format}")
|
|
@@ -2994,6 +3006,13 @@ def build_findings_export(
|
|
|
2994
3006
|
)
|
|
2995
3007
|
if export_format == "json":
|
|
2996
3008
|
return findings_bytes
|
|
3009
|
+
if export_format == "servicenow-avit":
|
|
3010
|
+
if avit_template is None:
|
|
3011
|
+
raise ContractError("--export-format servicenow-avit requires --avit-template")
|
|
3012
|
+
try:
|
|
3013
|
+
return servicenow_avit.render(avit_template, map(_avit_fields, findings["findings"]))
|
|
3014
|
+
except ValueError as exc:
|
|
3015
|
+
raise ContractError(str(exc)) from exc
|
|
2997
3016
|
return build_csv_projection(findings, coverage)
|
|
2998
3017
|
|
|
2999
3018
|
|
|
@@ -3407,6 +3426,7 @@ def main() -> int:
|
|
|
3407
3426
|
"--export-artifact", choices=("findings", "threat-model"), default="findings"
|
|
3408
3427
|
)
|
|
3409
3428
|
parser.add_argument("--export-format", choices=[*sorted(EXPORT_PATHS), "md"])
|
|
3429
|
+
parser.add_argument("--avit-template", type=Path)
|
|
3410
3430
|
parser.add_argument("--export-output", type=Path)
|
|
3411
3431
|
parser.add_argument("--export-metadata", action="store_true")
|
|
3412
3432
|
parser.add_argument("--describe-threat-model", action="store_true")
|
|
@@ -3414,6 +3434,18 @@ def main() -> int:
|
|
|
3414
3434
|
args = parser.parse_args()
|
|
3415
3435
|
try:
|
|
3416
3436
|
args.scan_dir = args.scan_dir.resolve()
|
|
3437
|
+
if args.avit_template is not None:
|
|
3438
|
+
if args.export_format != "servicenow-avit":
|
|
3439
|
+
parser.error(
|
|
3440
|
+
"--avit-template is only supported with --export-format servicenow-avit"
|
|
3441
|
+
)
|
|
3442
|
+
if (
|
|
3443
|
+
args.export_output
|
|
3444
|
+
and args.export_output.exists()
|
|
3445
|
+
and args.avit_template.exists()
|
|
3446
|
+
and args.export_output.samefile(args.avit_template)
|
|
3447
|
+
):
|
|
3448
|
+
parser.error("AVIT export output cannot overwrite the reusable template")
|
|
3417
3449
|
if args.describe_threat_model:
|
|
3418
3450
|
sys.stdout.buffer.write(
|
|
3419
3451
|
_json_bytes(describe_threat_model(args.scan_dir, args.schema_dir))
|
|
@@ -3448,7 +3480,11 @@ def main() -> int:
|
|
|
3448
3480
|
description, contents = saved
|
|
3449
3481
|
else:
|
|
3450
3482
|
contents = build_findings_export(
|
|
3451
|
-
args.scan_dir,
|
|
3483
|
+
args.scan_dir,
|
|
3484
|
+
args.export_format,
|
|
3485
|
+
args.source_root,
|
|
3486
|
+
args.schema_dir,
|
|
3487
|
+
avit_template=args.avit_template,
|
|
3452
3488
|
)
|
|
3453
3489
|
if args.export_output is None:
|
|
3454
3490
|
sys.stdout.buffer.write(contents)
|
|
@@ -0,0 +1,107 @@
|
|
|
1
|
+
"""ServiceNow Manual AVIT Ingestion CSV template and finding fields."""
|
|
2
|
+
|
|
3
|
+
from __future__ import annotations
|
|
4
|
+
|
|
5
|
+
import csv
|
|
6
|
+
import io
|
|
7
|
+
import re
|
|
8
|
+
import sys
|
|
9
|
+
from collections.abc import Iterable
|
|
10
|
+
from pathlib import Path
|
|
11
|
+
from typing import Any
|
|
12
|
+
|
|
13
|
+
AVIT_APP_FIELDS = {
|
|
14
|
+
"Application": 255,
|
|
15
|
+
"Requested by": 151,
|
|
16
|
+
"Purpose of application": 4000,
|
|
17
|
+
"Types of sensitive data": 40,
|
|
18
|
+
"List of compliance programs": 4000,
|
|
19
|
+
"Technology stack details": 4000,
|
|
20
|
+
"Application team": 100,
|
|
21
|
+
"URLs to test": 4000,
|
|
22
|
+
"Assigned to": 151,
|
|
23
|
+
"Assignment group": 151,
|
|
24
|
+
}
|
|
25
|
+
AVIT_FINDING_FIELDS = (
|
|
26
|
+
"Risk rating",
|
|
27
|
+
"CWE category",
|
|
28
|
+
"Vulnerability ID",
|
|
29
|
+
"Steps to reproduce",
|
|
30
|
+
"Technical details",
|
|
31
|
+
)
|
|
32
|
+
|
|
33
|
+
|
|
34
|
+
def read_template(path: Path) -> tuple[list[str], dict[str, str]]:
|
|
35
|
+
try:
|
|
36
|
+
with path.open(encoding="utf-8-sig", newline="") as source:
|
|
37
|
+
rows = (row for row in csv.reader(source, strict=True) if row)
|
|
38
|
+
headers = next(rows, [])
|
|
39
|
+
values = next(rows, [])
|
|
40
|
+
if len(headers) != len(values) or next(rows, None) is not None:
|
|
41
|
+
raise ValueError("AVIT template must have a header and one application row")
|
|
42
|
+
except (OSError, UnicodeError, csv.Error) as exc:
|
|
43
|
+
raise ValueError(f"Could not read AVIT template {path}: {exc}") from exc
|
|
44
|
+
|
|
45
|
+
row = {header.strip().casefold(): value for header, value in zip(headers, values, strict=True)}
|
|
46
|
+
if len(row) != len(headers):
|
|
47
|
+
raise ValueError("AVIT template has duplicate column names")
|
|
48
|
+
for field in (*AVIT_APP_FIELDS, *AVIT_FINDING_FIELDS):
|
|
49
|
+
if field.casefold() not in row:
|
|
50
|
+
raise ValueError(f"AVIT template is missing column {field!r}")
|
|
51
|
+
for field, limit in AVIT_APP_FIELDS.items():
|
|
52
|
+
value = row[field.casefold()]
|
|
53
|
+
if not value.strip() or len(value) > limit:
|
|
54
|
+
raise ValueError(
|
|
55
|
+
f"AVIT template {field!r} requires a value of at most {limit} characters"
|
|
56
|
+
)
|
|
57
|
+
return headers, row
|
|
58
|
+
|
|
59
|
+
|
|
60
|
+
def _narrative(text: str) -> str:
|
|
61
|
+
return text if len(text) <= 1000 else text[:997] + "..."
|
|
62
|
+
|
|
63
|
+
|
|
64
|
+
def finding_fields(finding: dict[str, Any], location: dict[str, Any], steps: str) -> dict[str, str]:
|
|
65
|
+
details = (
|
|
66
|
+
f"{finding['findingId']} | {location['path']}:{location['startLine']}\n"
|
|
67
|
+
f"{finding['title']}\n{finding['summary']}\n{finding['remediation']}"
|
|
68
|
+
)
|
|
69
|
+
cwe = next(
|
|
70
|
+
(
|
|
71
|
+
value.upper()
|
|
72
|
+
for value in finding["taxonomy"]["cwe"]
|
|
73
|
+
if len(value) <= 255 and re.fullmatch(r"CWE-[0-9]+", value, re.I)
|
|
74
|
+
),
|
|
75
|
+
"",
|
|
76
|
+
)
|
|
77
|
+
level = finding["severity"]["level"]
|
|
78
|
+
return {
|
|
79
|
+
"Risk rating": "None" if level == "informational" else level.title(),
|
|
80
|
+
"CWE category": cwe,
|
|
81
|
+
"Vulnerability ID": "" if cwe else finding["findingId"],
|
|
82
|
+
"Steps to reproduce": _narrative(steps),
|
|
83
|
+
"Technical details": _narrative(details),
|
|
84
|
+
}
|
|
85
|
+
|
|
86
|
+
|
|
87
|
+
def render(template: Path, rows: Iterable[dict[str, str]]) -> bytes:
|
|
88
|
+
headers, row = read_template(template)
|
|
89
|
+
output = io.StringIO(newline="")
|
|
90
|
+
writer = csv.writer(output)
|
|
91
|
+
writer.writerow(headers)
|
|
92
|
+
missing_steps = 0
|
|
93
|
+
has_findings = False
|
|
94
|
+
for fields in rows:
|
|
95
|
+
has_findings = True
|
|
96
|
+
if not fields["Steps to reproduce"]:
|
|
97
|
+
missing_steps += 1
|
|
98
|
+
for field, value in fields.items():
|
|
99
|
+
row[field.casefold()] = value
|
|
100
|
+
writer.writerow(row.values())
|
|
101
|
+
if has_findings:
|
|
102
|
+
print(
|
|
103
|
+
"Review Steps to reproduce before uploading. Populated steps describe the recorded "
|
|
104
|
+
f"attack path. Rows with no recorded steps: {missing_steps}.",
|
|
105
|
+
file=sys.stderr,
|
|
106
|
+
)
|
|
107
|
+
return output.getvalue().encode("utf-8")
|
|
@@ -4,7 +4,6 @@ from __future__ import annotations
|
|
|
4
4
|
|
|
5
5
|
import copy
|
|
6
6
|
import os
|
|
7
|
-
import re
|
|
8
7
|
from collections.abc import Callable, Sequence
|
|
9
8
|
from pathlib import Path, PurePosixPath
|
|
10
9
|
from typing import Any
|
|
@@ -20,22 +19,6 @@ from finalize_scan_contract import (
|
|
|
20
19
|
from workbench.saved_coverage import _project_worker_coverage_row, _saved_coverage_id
|
|
21
20
|
|
|
22
21
|
|
|
23
|
-
def _checkpoint_head_directory(relative: str) -> Path | None:
|
|
24
|
-
path = Path(relative)
|
|
25
|
-
if path.name == "checkpoint-head.json":
|
|
26
|
-
return path.parent
|
|
27
|
-
if path.parent.name == "checkpoint-heads":
|
|
28
|
-
return path.parent.parent
|
|
29
|
-
return None
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
def _is_source_order_snapshot(relative: str) -> bool:
|
|
33
|
-
path = Path(relative)
|
|
34
|
-
return path.parent == Path("source-order") and bool(
|
|
35
|
-
re.fullmatch(r"[0-9a-f]{64}\.json", path.name)
|
|
36
|
-
)
|
|
37
|
-
|
|
38
|
-
|
|
39
22
|
def capture_receipt_checkpoints(
|
|
40
23
|
relative: str,
|
|
41
24
|
kind: str | None,
|
|
@@ -3248,10 +3248,8 @@ def require_canonical_scan_directory(scan_dir: Path) -> Path:
|
|
|
3248
3248
|
scan_dir
|
|
3249
3249
|
):
|
|
3250
3250
|
raise SystemExit("Scan directory must be an existing canonical non-symlink directory.")
|
|
3251
|
-
# Re-check
|
|
3251
|
+
# Re-check ownership and parents so the scan directory cannot be substituted.
|
|
3252
3252
|
if os.name != "nt":
|
|
3253
|
-
if stat.S_IMODE(metadata.st_mode) & 0o077:
|
|
3254
|
-
raise SystemExit("Scan directory must not be accessible to other users (chmod 700).")
|
|
3255
3253
|
geteuid = getattr(os, "geteuid", None)
|
|
3256
3254
|
effective_uid = geteuid() if geteuid is not None else None
|
|
3257
3255
|
if effective_uid is not None and metadata.st_uid != effective_uid:
|
|
@@ -39,55 +39,7 @@ from finalize_scan_contract import (
|
|
|
39
39
|
)
|
|
40
40
|
from finalize_scan_contract import finding_candidate_id as finding_candidate_id
|
|
41
41
|
from review_state import checkpoint_review_state, review_state, review_summary, uses_legacy_review
|
|
42
|
-
from workbench.retained_coverage import (
|
|
43
|
-
_append_previous_finding as _append_previous_finding,
|
|
44
|
-
)
|
|
45
|
-
from workbench.retained_coverage import (
|
|
46
|
-
_coverage_rows as _coverage_rows,
|
|
47
|
-
)
|
|
48
|
-
from workbench.retained_coverage import (
|
|
49
|
-
_ensure_finding_identities as _ensure_finding_identities,
|
|
50
|
-
)
|
|
51
|
-
from workbench.retained_coverage import (
|
|
52
|
-
_ensure_finding_identity as _ensure_finding_identity,
|
|
53
|
-
)
|
|
54
|
-
from workbench.retained_coverage import (
|
|
55
|
-
_finding_content as _finding_content,
|
|
56
|
-
)
|
|
57
|
-
from workbench.retained_coverage import (
|
|
58
|
-
_finding_identity as _finding_identity,
|
|
59
|
-
)
|
|
60
|
-
from workbench.retained_coverage import (
|
|
61
|
-
_finding_key as _finding_key,
|
|
62
|
-
)
|
|
63
|
-
from workbench.retained_coverage import (
|
|
64
|
-
_has_candidate_metadata as _has_candidate_metadata,
|
|
65
|
-
)
|
|
66
|
-
from workbench.retained_coverage import (
|
|
67
|
-
_identity_candidate as _identity_candidate,
|
|
68
|
-
)
|
|
69
|
-
from workbench.retained_coverage import (
|
|
70
|
-
_inherit_coverage_sources as _inherit_coverage_sources,
|
|
71
|
-
)
|
|
72
|
-
from workbench.retained_coverage import (
|
|
73
|
-
_retained_findings as _retained_findings,
|
|
74
|
-
)
|
|
75
|
-
from workbench.retained_coverage import (
|
|
76
|
-
_semantic_identity_key as _semantic_identity_key,
|
|
77
|
-
)
|
|
78
|
-
from workbench.retained_coverage import (
|
|
79
|
-
_worker_candidate_key as _worker_candidate_key,
|
|
80
|
-
)
|
|
81
42
|
from workbench.retained_coverage import merge_saved_results as merge_saved_results
|
|
82
|
-
from workbench.saved_coverage import (
|
|
83
|
-
_deferred_rows as _deferred_rows,
|
|
84
|
-
)
|
|
85
|
-
from workbench.saved_coverage import (
|
|
86
|
-
_merge_tied_parent_observations as _merge_tied_parent_observations,
|
|
87
|
-
)
|
|
88
|
-
from workbench.saved_coverage import (
|
|
89
|
-
_resolved_deferred_rows as _resolved_deferred_rows,
|
|
90
|
-
)
|
|
91
43
|
from workbench.saved_coverage import (
|
|
92
44
|
_saved_coverage_id as _saved_coverage_id,
|
|
93
45
|
)
|
|
@@ -115,28 +67,10 @@ from workbench.saved_sources import (
|
|
|
115
67
|
_saved_result_paths,
|
|
116
68
|
_source_digests,
|
|
117
69
|
)
|
|
118
|
-
from workbench.saved_sources import (
|
|
119
|
-
_capture_parent_observation as _capture_parent_observation,
|
|
120
|
-
)
|
|
121
|
-
from workbench.saved_sources import (
|
|
122
|
-
_checkpoint_head_directory as _checkpoint_head_directory,
|
|
123
|
-
)
|
|
124
70
|
from workbench.saved_sources import (
|
|
125
71
|
_digest as _digest,
|
|
126
72
|
)
|
|
127
|
-
from workbench.saved_sources import (
|
|
128
|
-
_latest_successful_reducer as _latest_successful_reducer,
|
|
129
|
-
)
|
|
130
|
-
from workbench.saved_sources import (
|
|
131
|
-
_parent_publication_ancestors as _parent_publication_ancestors,
|
|
132
|
-
)
|
|
133
|
-
from workbench.saved_sources import (
|
|
134
|
-
_worker_outputs as _worker_outputs,
|
|
135
|
-
)
|
|
136
73
|
from workbench_constants import PHASES
|
|
137
|
-
from workbench_coverage_receipts import (
|
|
138
|
-
_deferred_candidate_id as _deferred_candidate_id,
|
|
139
|
-
)
|
|
140
74
|
from workbench_coverage_receipts import (
|
|
141
75
|
_generic_surface_updates as _generic_surface_updates,
|
|
142
76
|
)
|
|
@@ -30,7 +30,10 @@ from workbench_validation import optional_text
|
|
|
30
30
|
|
|
31
31
|
|
|
32
32
|
def safe_segment(value: str) -> str:
|
|
33
|
-
|
|
33
|
+
segment = re.sub(r"[^\w.-]", "-", value)
|
|
34
|
+
if segment != value:
|
|
35
|
+
segment = segment.strip("-")
|
|
36
|
+
return segment or "scan"
|
|
34
37
|
|
|
35
38
|
|
|
36
39
|
def compact_timestamp() -> str:
|
|
@@ -3,7 +3,6 @@
|
|
|
3
3
|
from __future__ import annotations
|
|
4
4
|
|
|
5
5
|
import argparse
|
|
6
|
-
import codecs
|
|
7
6
|
import hashlib
|
|
8
7
|
import os
|
|
9
8
|
import re
|
|
@@ -13,7 +12,6 @@ import stat
|
|
|
13
12
|
import subprocess
|
|
14
13
|
import sys
|
|
15
14
|
import tempfile
|
|
16
|
-
from concurrent.futures import ThreadPoolExecutor
|
|
17
15
|
from pathlib import Path, PurePosixPath
|
|
18
16
|
from typing import Any, BinaryIO, Callable
|
|
19
17
|
|
|
@@ -84,112 +82,6 @@ def git_bytes(
|
|
|
84
82
|
return completed.stdout if completed.returncode == 0 else None
|
|
85
83
|
|
|
86
84
|
|
|
87
|
-
def git_blob_samples(
|
|
88
|
-
target: Path,
|
|
89
|
-
object_names: list[str],
|
|
90
|
-
*,
|
|
91
|
-
git_dir: Path | None = None,
|
|
92
|
-
work_tree: Path | None = None,
|
|
93
|
-
) -> list[tuple[bytes, bool] | None]:
|
|
94
|
-
"""Sample ordered blobs and classify all their bytes without buffering the batch."""
|
|
95
|
-
if not object_names:
|
|
96
|
-
return []
|
|
97
|
-
|
|
98
|
-
# Git before 2.38 rejects -z, so only names with line breaks need it.
|
|
99
|
-
nul_input = any("\n" in name or "\r" in name for name in object_names)
|
|
100
|
-
separator = b"\0" if nul_input else b"\n"
|
|
101
|
-
request = separator.join(os.fsencode(name) for name in object_names) + separator
|
|
102
|
-
read_fd, write_fd = os.pipe()
|
|
103
|
-
with (
|
|
104
|
-
ThreadPoolExecutor(max_workers=1) as reader,
|
|
105
|
-
os.fdopen(read_fd, "rb") as output,
|
|
106
|
-
os.fdopen(write_fd, "wb") as sink,
|
|
107
|
-
):
|
|
108
|
-
|
|
109
|
-
def read_samples() -> list[tuple[bytes, bool] | None]:
|
|
110
|
-
# Closing the reader also unblocks Git if framing is invalid.
|
|
111
|
-
with output:
|
|
112
|
-
return _read_git_batch_samples(output, object_names)
|
|
113
|
-
|
|
114
|
-
samples = reader.submit(read_samples)
|
|
115
|
-
try:
|
|
116
|
-
completed = git_command(
|
|
117
|
-
target,
|
|
118
|
-
"cat-file",
|
|
119
|
-
"--batch",
|
|
120
|
-
*(["-z"] if nul_input else []),
|
|
121
|
-
text=False,
|
|
122
|
-
input_data=request,
|
|
123
|
-
git_dir=git_dir,
|
|
124
|
-
work_tree=work_tree,
|
|
125
|
-
stdout_file=sink,
|
|
126
|
-
)
|
|
127
|
-
finally:
|
|
128
|
-
# Git may fail before producing all requested records.
|
|
129
|
-
sink.close()
|
|
130
|
-
if completed.returncode != 0:
|
|
131
|
-
return [None] * len(object_names)
|
|
132
|
-
try:
|
|
133
|
-
return samples.result()
|
|
134
|
-
except ValueError:
|
|
135
|
-
return [None] * len(object_names)
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
def _read_git_batch_samples(
|
|
139
|
-
output: BinaryIO, object_names: list[str]
|
|
140
|
-
) -> list[tuple[bytes, bool] | None]:
|
|
141
|
-
"""Read the older-Git-compatible -z protocol without buffering blob bodies."""
|
|
142
|
-
samples: list[tuple[bytes, bool] | None] = []
|
|
143
|
-
for name in object_names:
|
|
144
|
-
header = output.readline()
|
|
145
|
-
raw_name = os.fsencode(name)
|
|
146
|
-
# Missing lookups echo the whole request, including any embedded newlines.
|
|
147
|
-
if header and raw_name.startswith(header):
|
|
148
|
-
header += output.read(len(raw_name) - len(header)) + output.readline()
|
|
149
|
-
if header in (raw_name + b" missing\n", raw_name + b" ambiguous\n"):
|
|
150
|
-
samples.append(None)
|
|
151
|
-
continue
|
|
152
|
-
fields = header.split(b" ")
|
|
153
|
-
if len(fields) != 3 or not header.endswith(b"\n"):
|
|
154
|
-
raise ValueError("invalid batch header")
|
|
155
|
-
size = int(fields[2])
|
|
156
|
-
if size < 0:
|
|
157
|
-
raise ValueError("invalid blob size")
|
|
158
|
-
# Consume non-blob objects too, so subsequent records stay aligned.
|
|
159
|
-
sample = _read_git_blob_sample(output, size)
|
|
160
|
-
samples.append(sample if fields[1] == b"blob" else None)
|
|
161
|
-
return samples
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
def _is_binary_sample(data: bytes) -> bool:
|
|
165
|
-
if data.startswith((codecs.BOM_UTF16_LE, codecs.BOM_UTF16_BE)):
|
|
166
|
-
return "\0" in data.decode("utf-16", errors="ignore")
|
|
167
|
-
return b"\0" in data
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
def _read_git_blob_sample(output: BinaryIO, size: int) -> tuple[bytes, bool]:
|
|
171
|
-
read_size = 64 * 1024
|
|
172
|
-
sample_size = min(size, read_size)
|
|
173
|
-
sample = output.read(sample_size)
|
|
174
|
-
if len(sample) != sample_size:
|
|
175
|
-
raise ValueError("truncated blob")
|
|
176
|
-
binary = _is_binary_sample(sample)
|
|
177
|
-
bom = sample[:2] if sample.startswith((codecs.BOM_UTF16_LE, codecs.BOM_UTF16_BE)) else b""
|
|
178
|
-
remaining = size - sample_size
|
|
179
|
-
while remaining:
|
|
180
|
-
chunk_size = min(remaining, read_size)
|
|
181
|
-
chunk = output.read(chunk_size)
|
|
182
|
-
if len(chunk) != chunk_size:
|
|
183
|
-
raise ValueError("truncated blob")
|
|
184
|
-
# Even-sized reads keep UTF-16 code units aligned; preserve the original byte order.
|
|
185
|
-
if not binary:
|
|
186
|
-
binary = _is_binary_sample(bom + chunk) if bom else b"\0" in chunk
|
|
187
|
-
remaining -= chunk_size
|
|
188
|
-
if output.read(1) != b"\n":
|
|
189
|
-
raise ValueError("missing blob terminator")
|
|
190
|
-
return (b"", True) if binary else (sample, False)
|
|
191
|
-
|
|
192
|
-
|
|
193
85
|
def _protected_repository_root(target: Path) -> Path:
|
|
194
86
|
root = target.resolve()
|
|
195
87
|
if root.is_file():
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: publish-findings
|
|
3
|
-
description: Requires explicit opt-in with CODEX_SECURITY_ENABLE_EXTERNAL_IMPORTS=1. Preview and import
|
|
3
|
+
description: Requires explicit opt-in with CODEX_SECURITY_ENABLE_EXTERNAL_IMPORTS=1. Preview and import scanner findings into the local workbench or Codex Security Cloud. Use the selected provider's documented export format.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Publish external findings
|
|
@@ -31,6 +31,10 @@ Read the stored evidence with `codex-security findings imported /path/to/project
|
|
|
31
31
|
|
|
32
32
|
Local records remain unassessed source evidence, separate from `findings list` and completed scans. Repeated imports use the same repository/provider/source-key/source-ID identity; omitted records remain stored. When a source repository URL and local Git remotes are available, the URL must match a remote. Local import does not publish to Cloud or automatically investigate or patch findings.
|
|
33
33
|
|
|
34
|
+
## Amazon Inspector repository findings
|
|
35
|
+
|
|
36
|
+
Use `--provider inspector` and `--source-repository-url` with a complete Inspector2 ListFindings export for one Code Security repository. If it contains only a project ARN and name, pass the full `--inspector-project-arn ARN` verified for that Git URL in the SCM integration. Supply the same ARN when resuming a pending Cloud submission. Self-managed GitLab is supported locally.
|
|
37
|
+
|
|
34
38
|
## Endor Labs findings
|
|
35
39
|
|
|
36
40
|
For Endor Labs imports, prefer native Finding JSON from authenticated `endorctl`, scoped to one project and context. Omit `--field-mask` and fetch all pages with `--list-all`:
|
|
@@ -86,10 +90,12 @@ Import native OpenGrep JSON with `--provider opengrep`, `--source-repository-url
|
|
|
86
90
|
|
|
87
91
|
## GitHub alerts and advisories
|
|
88
92
|
|
|
89
|
-
Acquire code-scanning findings with `codex-security import github OWNER/REPO --format json` and save its complete JSON output for local import or Cloud publication. This existing collector completes pagination, fetches alert details, and retains the selected ref's instance. In Codex, use the official [GitHub Codex plugin](https://github.com/openai/plugins/tree/main/plugins/github) for discovery and selecting alerts, then pass that selection to the collector. Its GitHub MCP server paginates each list call separately. Do not ask the user to download JSON from a website or reconstruct
|
|
93
|
+
Acquire code-scanning findings with `codex-security import github OWNER/REPO --format json` and save its complete JSON output for local import or Cloud publication. This existing collector completes pagination, fetches alert details, and retains the selected ref's instance. In Codex, use the official [GitHub Codex plugin](https://github.com/openai/plugins/tree/main/plugins/github) for discovery and selecting alerts, then pass that selection to the collector. Its GitHub MCP server paginates each list call separately. Do not ask the user to download JSON from a website or reconstruct a code-scanning import envelope from MCP summaries. Use the saved collector output for code-scanning alerts.
|
|
90
94
|
|
|
91
95
|
The importer uses the user's GitHub CLI authentication and `GH_HOST`; plugin authentication is separate. It defaults to open alerts on the default branch. Use repeated `--github-alert NUMBER` for exact alerts or `--github-ref REF` for another ref. Save complete output without field/token filters or `--full-output`. Reading code-scanning alerts requires a token with Code scanning alerts read permission (classic: `security_events`, or `public_repo` for public repositories), along with repository access.
|
|
92
96
|
|
|
97
|
+
For Dependabot, follow the [repository alert export guide](https://github.com/openai/codex-security/blob/main/docs/scanner-imports/github-dependabot.md) to save the native REST alert records from one repository. The read-only export requires Dependabot alerts read permission. Preview the file with `publish findings FILE --provider github`, the intended destination and a separate source key such as `github.com/example/project/dependabot`.
|
|
98
|
+
|
|
93
99
|
**GHSA import stages advisory evidence for investigation; it does not establish an installed dependency or applicability to the destination repository.** Keep advisories separate from code scanning. `codex-security import github-advisories OWNER/REPO --format json` reads published repository disclosures by default. Repeated `--github-advisory GHSA-xxxx-xxxx-xxxx` selects exact IDs; `--github-state draft|triage|closed` explicitly selects an unpublished list and cannot be combined with exact IDs. Add `--github-advisory-source global` for exact public database IDs. This does not enumerate the global database. Private advisory access needs Repository security advisories read permission and access to those records. Stop on a read failure instead of publishing an incomplete selection.
|
|
94
100
|
|
|
95
101
|
```sh
|
|
@@ -111,7 +117,7 @@ codex-security publish findings /path/to/advisories.json \
|
|
|
111
117
|
|
|
112
118
|
Choose the local or Cloud preview for the intended destination. GHSA input supports the same choice: use `--to local --repository /path/to/project` with the advisory file and source key. Follow [local imports](#import-locally) for confirmation and readback or [Cloud publication](#publish-to-cloud) for upload review.
|
|
113
119
|
|
|
114
|
-
Use separate stable source
|
|
120
|
+
Use a separate stable source key for each GitHub finding family. Code-scanning identities include the ref; repository alerts and advisories must match the selected Cloud repository or, when available, a configured local Git remote. Global advisories omit source-repository provenance and are bound to the explicit local or Cloud destination for investigation. Preserve original evidence, package ranges and patches; do not infer installed packages or source locations from an advisory. Missing security severity stays `unknown`. Dismissal, resolution and withdrawal are source state, not Codex assessment. Secret-scanning alerts and writeback to GitHub are outside this workflow.
|
|
115
121
|
|
|
116
122
|
## Publish to Cloud
|
|
117
123
|
|