@openai/codex-security 0.6.0 → 0.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (243) hide show
  1. package/README.md +68 -37
  2. package/_bundled_plugin/.mcp.json +1 -0
  3. package/_bundled_plugin/mcp/helpers.mjs.br.part-000 +0 -0
  4. package/_bundled_plugin/mcp/helpers.mjs.br.part-001 +0 -0
  5. package/_bundled_plugin/mcp/native/THIRD_PARTY_NOTICES.txt +1 -1
  6. package/_bundled_plugin/mcp/native/darwin-arm64/unix.node +0 -0
  7. package/_bundled_plugin/mcp/native/darwin-x64/unix.node +0 -0
  8. package/_bundled_plugin/mcp/native/linux-arm64-gnu/unix.node +0 -0
  9. package/_bundled_plugin/mcp/native/linux-arm64-musl/unix.node +0 -0
  10. package/_bundled_plugin/mcp/native/linux-x64-gnu/unix.node +0 -0
  11. package/_bundled_plugin/mcp/native/win32-arm64/windows.node +0 -0
  12. package/_bundled_plugin/mcp/native/win32-x64/windows.node +0 -0
  13. package/_bundled_plugin/mcp/server.mjs.br.part-000 +0 -0
  14. package/_bundled_plugin/mcp/server.mjs.br.part-001 +0 -0
  15. package/_bundled_plugin/mcp/server.mjs.br.part-002 +0 -0
  16. package/_bundled_plugin/references/access-advisory.md +11 -0
  17. package/_bundled_plugin/references/core-scan.md +30 -77
  18. package/_bundled_plugin/references/final-report.md +45 -189
  19. package/_bundled_plugin/references/scan-contract.md +2 -0
  20. package/_bundled_plugin/references/scan-prologue.md +1 -1
  21. package/_bundled_plugin/references/scan-runtime.md +1 -5
  22. package/_bundled_plugin/references/threat-model.md +33 -41
  23. package/_bundled_plugin/schemas/coverage.schema.json +12 -1
  24. package/_bundled_plugin/schemas/scan-manifest.schema.json +1 -0
  25. package/_bundled_plugin/schemas/tools/deep-reducer.schema.json +38 -1
  26. package/_bundled_plugin/scripts/deep_scan_config.py +39 -39
  27. package/_bundled_plugin/scripts/deep_scan_workbench.py +145 -85
  28. package/_bundled_plugin/scripts/finalize_scan_contract.py +189 -57
  29. package/_bundled_plugin/scripts/report_projection.py +44 -15
  30. package/_bundled_plugin/scripts/threat_model_projection.py +4 -2
  31. package/_bundled_plugin/scripts/windows_scan_local_files.py +43 -14
  32. package/_bundled_plugin/scripts/workbench/retained_coverage.py +3218 -0
  33. package/_bundled_plugin/scripts/workbench/retained_records.py +627 -0
  34. package/_bundled_plugin/scripts/workbench/saved_coverage.py +644 -1
  35. package/_bundled_plugin/scripts/workbench/saved_outputs.py +72 -0
  36. package/_bundled_plugin/scripts/workbench/saved_sources.py +712 -0
  37. package/_bundled_plugin/scripts/workbench_cli.py +3 -0
  38. package/_bundled_plugin/scripts/workbench_coverage_receipts.py +479 -0
  39. package/_bundled_plugin/scripts/workbench_db.py +51 -22
  40. package/_bundled_plugin/scripts/workbench_feedback.py +0 -1
  41. package/_bundled_plugin/scripts/workbench_finding_workflows.py +51 -9
  42. package/_bundled_plugin/scripts/workbench_native_indexes.py +56 -44
  43. package/_bundled_plugin/scripts/workbench_saved_results.py +641 -2387
  44. package/_bundled_plugin/scripts/workbench_scan_history.py +72 -7
  45. package/_bundled_plugin/scripts/workbench_schema.py +1 -1
  46. package/_bundled_plugin/scripts/workbench_source_scopes.py +4 -1
  47. package/_bundled_plugin/scripts/workbench_target.py +142 -17
  48. package/_bundled_plugin/scripts/workbench_validation.py +9 -2
  49. package/_bundled_plugin/shared/workbench-history-repairs.json +21 -0
  50. package/_bundled_plugin/shared/workbench-migrations.json +28 -0
  51. package/_bundled_plugin/skills/assess-patch-risk/SKILL.md +3 -14
  52. package/_bundled_plugin/skills/assess-patch-risk/references/windows-validation.md +14 -0
  53. package/_bundled_plugin/skills/attack-path-analysis/SKILL.md +38 -103
  54. package/_bundled_plugin/skills/attack-path-analysis/references/attack-path-facts.md +11 -73
  55. package/_bundled_plugin/skills/attack-path-analysis/references/severity-policy.md +57 -116
  56. package/_bundled_plugin/skills/deep-security-scan/SKILL.md +28 -51
  57. package/_bundled_plugin/skills/define-security-policy/SKILL.md +9 -9
  58. package/_bundled_plugin/skills/finding-discovery/SKILL.md +26 -140
  59. package/_bundled_plugin/skills/fix-finding/SKILL.md +9 -28
  60. package/_bundled_plugin/skills/propose-security-hardening/SKILL.md +28 -218
  61. package/_bundled_plugin/skills/propose-security-hardening/references/proposal-format.md +29 -424
  62. package/_bundled_plugin/skills/publish-findings/SKILL.md +28 -24
  63. package/_bundled_plugin/skills/publish-findings/references/wiz-import.md +64 -0
  64. package/_bundled_plugin/skills/security-diff-scan/SKILL.md +12 -12
  65. package/_bundled_plugin/skills/security-scan/SKILL.md +28 -16
  66. package/_bundled_plugin/skills/security-scan/references/desktop-scan.md +11 -17
  67. package/_bundled_plugin/skills/security-scan/references/scan-artifacts-and-ledger.md +52 -69
  68. package/_bundled_plugin/skills/threat-model/SKILL.md +16 -9
  69. package/_bundled_plugin/skills/triage-finding/SKILL.md +5 -15
  70. package/_bundled_plugin/skills/triage-finding/references/ticket-intake.md +6 -6
  71. package/_bundled_plugin/skills/validation/SKILL.md +42 -104
  72. package/_bundled_plugin/skills/validation/references/validation-guidance.md +21 -106
  73. package/_bundled_plugin/skills/verify-fix/SKILL.md +4 -2
  74. package/_bundled_plugin/skills/vulnerability-writeup/SKILL.md +21 -159
  75. package/_bundled_plugin/skills/vulnerability-writeup/references/report-format.md +18 -60
  76. package/dist/api.d.ts +5 -3
  77. package/dist/api.d.ts.map +1 -1
  78. package/dist/api.js +341 -37
  79. package/dist/api.js.map +1 -1
  80. package/dist/auth.d.ts.map +1 -1
  81. package/dist/auth.js +4 -1
  82. package/dist/auth.js.map +1 -1
  83. package/dist/cli.d.ts +2 -2
  84. package/dist/cli.d.ts.map +1 -1
  85. package/dist/cli.js +674 -419
  86. package/dist/cli.js.map +1 -1
  87. package/dist/component-scan.d.ts +8 -2
  88. package/dist/component-scan.d.ts.map +1 -1
  89. package/dist/component-scan.js +2 -0
  90. package/dist/component-scan.js.map +1 -1
  91. package/dist/contract.d.ts +2 -0
  92. package/dist/contract.d.ts.map +1 -1
  93. package/dist/contract.js +67 -33
  94. package/dist/contract.js.map +1 -1
  95. package/dist/cost.d.ts +18 -0
  96. package/dist/cost.d.ts.map +1 -1
  97. package/dist/cost.js +107 -12
  98. package/dist/cost.js.map +1 -1
  99. package/dist/coverage-presentation.d.ts +6 -0
  100. package/dist/coverage-presentation.d.ts.map +1 -0
  101. package/dist/coverage-presentation.js +34 -0
  102. package/dist/coverage-presentation.js.map +1 -0
  103. package/dist/custom-validation-prompt.js +3 -3
  104. package/dist/custom-validation-prompt.js.map +1 -1
  105. package/dist/custom-validation.d.ts.map +1 -1
  106. package/dist/custom-validation.js +30 -4
  107. package/dist/custom-validation.js.map +1 -1
  108. package/dist/deduplication/checkpointed-review.d.ts.map +1 -1
  109. package/dist/deduplication/checkpointed-review.js +3 -1
  110. package/dist/deduplication/checkpointed-review.js.map +1 -1
  111. package/dist/deduplication/scan.d.ts.map +1 -1
  112. package/dist/deduplication/scan.js +7 -5
  113. package/dist/deduplication/scan.js.map +1 -1
  114. package/dist/endor-findings.d.ts +4 -3
  115. package/dist/endor-findings.d.ts.map +1 -1
  116. package/dist/endor-findings.js +16 -11
  117. package/dist/endor-findings.js.map +1 -1
  118. package/dist/external-findings-import.d.ts.map +1 -1
  119. package/dist/external-findings-import.js +13 -5
  120. package/dist/external-findings-import.js.map +1 -1
  121. package/dist/external-findings-local.d.ts.map +1 -1
  122. package/dist/external-findings-local.js +8 -1
  123. package/dist/external-findings-local.js.map +1 -1
  124. package/dist/external-findings-publish.d.ts +1 -1
  125. package/dist/external-findings-publish.d.ts.map +1 -1
  126. package/dist/external-findings-publish.js +11 -3
  127. package/dist/external-findings-publish.js.map +1 -1
  128. package/dist/external-findings-read.d.ts +11 -1
  129. package/dist/external-findings-read.d.ts.map +1 -1
  130. package/dist/external-findings-read.js +15 -6
  131. package/dist/external-findings-read.js.map +1 -1
  132. package/dist/external-import-contract.d.ts +3 -1
  133. package/dist/external-import-contract.d.ts.map +1 -1
  134. package/dist/external-import-contract.js +33 -25
  135. package/dist/external-import-contract.js.map +1 -1
  136. package/dist/external-imports-feature.d.ts +6 -0
  137. package/dist/external-imports-feature.d.ts.map +1 -0
  138. package/dist/external-imports-feature.js +51 -0
  139. package/dist/external-imports-feature.js.map +1 -0
  140. package/dist/finding-workflow.d.ts +22 -5
  141. package/dist/finding-workflow.d.ts.map +1 -1
  142. package/dist/finding-workflow.js +21 -9
  143. package/dist/finding-workflow.js.map +1 -1
  144. package/dist/github-advisories.d.ts.map +1 -1
  145. package/dist/github-advisories.js +3 -1
  146. package/dist/github-advisories.js.map +1 -1
  147. package/dist/github-findings.d.ts +4 -3
  148. package/dist/github-findings.d.ts.map +1 -1
  149. package/dist/github-findings.js +16 -14
  150. package/dist/github-findings.js.map +1 -1
  151. package/dist/hosted-remediation.d.ts.map +1 -1
  152. package/dist/hosted-remediation.js +19 -2
  153. package/dist/hosted-remediation.js.map +1 -1
  154. package/dist/hosted-scan.d.ts +1 -1
  155. package/dist/hosted-scan.d.ts.map +1 -1
  156. package/dist/hosted-scan.js +0 -5
  157. package/dist/hosted-scan.js.map +1 -1
  158. package/dist/import-scan.js +1 -1
  159. package/dist/import-scan.js.map +1 -1
  160. package/dist/index.d.ts +3 -3
  161. package/dist/index.d.ts.map +1 -1
  162. package/dist/index.js +1 -1
  163. package/dist/index.js.map +1 -1
  164. package/dist/knowledge-base.d.ts +1 -0
  165. package/dist/knowledge-base.d.ts.map +1 -1
  166. package/dist/knowledge-base.js +17 -8
  167. package/dist/knowledge-base.js.map +1 -1
  168. package/dist/models.d.ts +1 -0
  169. package/dist/models.d.ts.map +1 -1
  170. package/dist/models.js.map +1 -1
  171. package/dist/owner-evidence.d.ts +2 -0
  172. package/dist/owner-evidence.d.ts.map +1 -1
  173. package/dist/owner-evidence.js +2 -1
  174. package/dist/owner-evidence.js.map +1 -1
  175. package/dist/publish.d.ts +2 -0
  176. package/dist/publish.d.ts.map +1 -1
  177. package/dist/publish.js +1 -1
  178. package/dist/publish.js.map +1 -1
  179. package/dist/remediation.d.ts +1 -25
  180. package/dist/remediation.d.ts.map +1 -1
  181. package/dist/remediation.js +3 -24
  182. package/dist/remediation.js.map +1 -1
  183. package/dist/runtime.d.ts.map +1 -1
  184. package/dist/runtime.js +7 -7
  185. package/dist/runtime.js.map +1 -1
  186. package/dist/scan-dashboard.d.ts +5 -3
  187. package/dist/scan-dashboard.d.ts.map +1 -1
  188. package/dist/scan-dashboard.js +79 -26
  189. package/dist/scan-dashboard.js.map +1 -1
  190. package/dist/scan-history-renderer.d.ts.map +1 -1
  191. package/dist/scan-history-renderer.js +8 -2
  192. package/dist/scan-history-renderer.js.map +1 -1
  193. package/dist/scan-owner-suggestions.d.ts +1 -1
  194. package/dist/scan-owner-suggestions.d.ts.map +1 -1
  195. package/dist/scan-owner-suggestions.js +4 -4
  196. package/dist/scan-owner-suggestions.js.map +1 -1
  197. package/dist/scan-usage-display.d.ts +8 -0
  198. package/dist/scan-usage-display.d.ts.map +1 -0
  199. package/dist/scan-usage-display.js +46 -0
  200. package/dist/scan-usage-display.js.map +1 -0
  201. package/dist/scanner-import.d.ts +1 -1
  202. package/dist/scanner-import.d.ts.map +1 -1
  203. package/dist/scanner-import.js +7 -3
  204. package/dist/scanner-import.js.map +1 -1
  205. package/dist/security-policy.d.ts.map +1 -1
  206. package/dist/security-policy.js +1 -20
  207. package/dist/security-policy.js.map +1 -1
  208. package/dist/semgrep-findings.d.ts +4 -3
  209. package/dist/semgrep-findings.d.ts.map +1 -1
  210. package/dist/semgrep-findings.js +9 -9
  211. package/dist/semgrep-findings.js.map +1 -1
  212. package/dist/skill-authentication.d.ts +18 -0
  213. package/dist/skill-authentication.d.ts.map +1 -0
  214. package/dist/skill-authentication.js +82 -0
  215. package/dist/skill-authentication.js.map +1 -0
  216. package/dist/snyk-findings.d.ts +5 -4
  217. package/dist/snyk-findings.d.ts.map +1 -1
  218. package/dist/snyk-findings.js +18 -17
  219. package/dist/snyk-findings.js.map +1 -1
  220. package/dist/suggest-owners.d.ts +4 -1
  221. package/dist/suggest-owners.d.ts.map +1 -1
  222. package/dist/suggest-owners.js +53 -21
  223. package/dist/suggest-owners.js.map +1 -1
  224. package/dist/targets.d.ts.map +1 -1
  225. package/dist/targets.js +4 -1
  226. package/dist/targets.js.map +1 -1
  227. package/dist/wiz-csv.d.ts +2 -1
  228. package/dist/wiz-csv.d.ts.map +1 -1
  229. package/dist/wiz-csv.js +3 -3
  230. package/dist/wiz-csv.js.map +1 -1
  231. package/dist/wiz-findings.d.ts +2 -1
  232. package/dist/wiz-findings.d.ts.map +1 -1
  233. package/dist/wiz-findings.js +27 -23
  234. package/dist/wiz-findings.js.map +1 -1
  235. package/docs/classify-severity-records.md +3 -4
  236. package/docs/cli.md +155 -38
  237. package/docs/findings-service.md +119 -118
  238. package/docs/hosted-remediation.md +19 -17
  239. package/docs/hosted-scans.md +37 -11
  240. package/package.json +8 -8
  241. package/_bundled_plugin/scripts/generate_in_scope_files.py +0 -277
  242. package/_bundled_plugin/scripts/generate_rank_input.py +0 -350
  243. package/_bundled_plugin/scripts/rank_preview.py +0 -129
@@ -2,6 +2,7 @@
2
2
 
3
3
  import argparse
4
4
  import fnmatch
5
+ import hashlib
5
6
  import json
6
7
  import os
7
8
  import sqlite3
@@ -14,7 +15,13 @@ from urllib.parse import urlsplit
14
15
 
15
16
  # Some plugin hosts launch Python with safe-path isolation enabled.
16
17
  sys.path.insert(0, str(Path(__file__).resolve().parent))
17
- from finalize_scan_contract import ContractError, _prepare_scan_finalization
18
+ from finalize_scan_contract import (
19
+ ContractError,
20
+ _prepare_scan_finalization,
21
+ _read_scan_local_json_bytes,
22
+ _read_sealed_scan,
23
+ manifest_is_sealed,
24
+ )
18
25
  from report_projection import SEVERITY_ORDER
19
26
  from workbench_constants import ARTIFACTS, FINDINGS_PAGE_MAX
20
27
  from workbench_scan_start import scan_target_identity
@@ -122,10 +129,7 @@ def cli_scan_resume(
122
129
  if manifest_path is not None:
123
130
  manifest = wb.read_json_object(manifest_path)
124
131
  manifest_scan = manifest.get("scan")
125
- if isinstance(manifest_scan, dict) and (
126
- manifest_scan.get("sealedAt") is not None
127
- or manifest_scan.get("artifacts") is not None
128
- ):
132
+ if isinstance(manifest_scan, dict) and manifest_is_sealed(manifest_scan):
129
133
  try:
130
134
  binding = wb.workbench_completion_binding(scan, scan["started_at"], manifest)
131
135
  _prepare_scan_finalization(
@@ -139,6 +143,65 @@ def cli_scan_resume(
139
143
  return result
140
144
 
141
145
 
146
+ def coverage_summary_for_history(
147
+ wb: Any, scan: sqlite3.Row, canonical_scan_directory: Callable[[Path], Path]
148
+ ) -> dict[str, Any]:
149
+ scan_dir = canonical_scan_directory(Path(scan["scan_dir"]))
150
+ coverage_ref = ARTIFACTS["coverage"]
151
+ try:
152
+ if scan["seal_manifest_digest"] is None:
153
+ # Older workbench rows can have valid seals without a pinned digest.
154
+ manifest, _, coverage, _ = _read_sealed_scan(scan_dir, None, "history coverage")
155
+ wb.verify_manifest_binding(scan, manifest)
156
+ if coverage["mode"] != wb.expected_coverage_mode(scan):
157
+ raise ContractError("coverage.mode: must match the workbench scan")
158
+ else:
159
+ # Completion already validated the full scan. History only needs the
160
+ # pinned manifest and its coverage artifact, not findings or receipts.
161
+ manifest, manifest_bytes = _read_scan_local_json_bytes(
162
+ scan_dir, ARTIFACTS["manifest"], ARTIFACTS["manifest"]
163
+ )
164
+ if (
165
+ f"sha256:{hashlib.sha256(manifest_bytes).hexdigest()}"
166
+ != scan["seal_manifest_digest"]
167
+ ):
168
+ raise ContractError("The sealed scan manifest changed after completion.")
169
+ sealed_scan = manifest["scan"]
170
+ if (
171
+ sealed_scan["id"] != scan["id"]
172
+ or not sealed_scan.get("sealedAt")
173
+ or sealed_scan["coverageRef"] != coverage_ref
174
+ ):
175
+ raise ContractError("The sealed coverage does not belong to this scan.")
176
+ coverage, coverage_bytes = _read_scan_local_json_bytes(
177
+ scan_dir, coverage_ref, coverage_ref
178
+ )
179
+ expected = next(
180
+ (
181
+ artifact["sha256"]
182
+ for artifact in sealed_scan["artifacts"]
183
+ if artifact["path"] == coverage_ref
184
+ ),
185
+ None,
186
+ )
187
+ if (
188
+ hashlib.sha256(coverage_bytes).hexdigest() != expected
189
+ or coverage.get("scanId") != scan["id"]
190
+ ):
191
+ raise ContractError("The sealed coverage changed after completion.")
192
+ except ContractError as exc:
193
+ raise SystemExit(str(exc)) from exc
194
+ return {
195
+ key: coverage[key]
196
+ for key in (
197
+ "mode",
198
+ "includePaths",
199
+ "excludePaths",
200
+ "explicitExclusions",
201
+ )
202
+ }
203
+
204
+
142
205
  def _windows_path_key(value: str) -> str:
143
206
  return os.path.normcase(os.path.realpath(value))
144
207
 
@@ -202,8 +265,7 @@ def _repository_origin(target: Path) -> tuple[str, str] | None:
202
265
  return (host.lower(), path) if host and path else None
203
266
 
204
267
 
205
- SCAN_SUMMARY_QUERY = """
206
- SELECT
268
+ SCAN_SUMMARY_COLUMNS = """
207
269
  scans.*,
208
270
  progress.reportable_findings_count,
209
271
  progress.scope_file_count,
@@ -215,6 +277,9 @@ SELECT
215
277
  FROM finding_occurrences AS occurrences
216
278
  WHERE occurrences.scan_id = scans.id
217
279
  ) AS finding_count
280
+ """
281
+ SCAN_SUMMARY_QUERY = f"""
282
+ SELECT {SCAN_SUMMARY_COLUMNS}
218
283
  FROM scans
219
284
  JOIN scan_progress AS progress ON progress.scan_id = scans.id
220
285
  """
@@ -75,7 +75,7 @@ def apply_migrations(
75
75
  elif version == 16:
76
76
  should_backfill_targets = repair_stable_targets_migration(connection)
77
77
  elif version in applied:
78
- if version in (2, 12, 13, 26, 28, 31, 32, 42, 47, 50):
78
+ if version in (2, 12, 13, 26, 28, 31, 32, 42, 47, 50, 54, 55):
79
79
  repair_additive_migration(connection, version)
80
80
  elif version == 11:
81
81
  repair_deep_scan_migration(connection)
@@ -275,7 +275,10 @@ def capture_source_scopes(
275
275
  def load_source_scopes(
276
276
  scan: sqlite3.Row, target: Path, requested: list[str]
277
277
  ) -> tuple[Path, str, list[dict[str, str]]] | None:
278
- if "diff_target_kind" in scan.keys() and scan["diff_target_kind"] == "working_tree":
278
+ kind = scan["diff_target_kind"] if "diff_target_kind" in scan.keys() else None
279
+ if kind == "working_tree" or (
280
+ "mode" in scan.keys() and scan["mode"] == "diff" and kind not in ("commit", "range")
281
+ ):
279
282
  return None
280
283
  saved = scan["source_scopes_json"] if "source_scopes_json" in scan.keys() else None
281
284
  paths = {PurePosixPath(path).as_posix() for path in requested}
@@ -20,7 +20,6 @@ from typing import Any, BinaryIO, Callable
20
20
  # Some plugin hosts launch Python with safe-path isolation enabled.
21
21
  sys.path.insert(0, str(Path(__file__).resolve().parent))
22
22
  from filesystem_identity import stored_filesystem_identity_matches
23
- from rank_preview import DEFAULT_PREVIEW_READ_BYTES, is_binary_sample
24
23
  from windows_scan_local_files import copy_directory_junction
25
24
  from workbench_constants import GIT_REPOSITORY_ENVIRONMENT
26
25
 
@@ -28,6 +27,14 @@ SOURCE_METADATA_DIRECTORIES = {".git", ".svn"}
28
27
  _WINDOWS = os.name == "nt"
29
28
 
30
29
 
30
+ class UnreadableLocalFile(SystemExit):
31
+ """Source contents are unavailable for an optional review cache."""
32
+
33
+
34
+ class UnsupportedLocalFileType(SystemExit):
35
+ """An entry cannot be read as source content for a review cache."""
36
+
37
+
31
38
  def committed_diff_snapshot_digest(kind: str, base_revision: str, head_revision: str) -> str:
32
39
  digest = hashlib.sha256(
33
40
  f"codex-security-diff/v1\0{kind}\0{base_revision}\0{head_revision}".encode()
@@ -154,22 +161,29 @@ def _read_git_batch_samples(
154
161
  return samples
155
162
 
156
163
 
164
+ def _is_binary_sample(data: bytes) -> bool:
165
+ if data.startswith((codecs.BOM_UTF16_LE, codecs.BOM_UTF16_BE)):
166
+ return "\0" in data.decode("utf-16", errors="ignore")
167
+ return b"\0" in data
168
+
169
+
157
170
  def _read_git_blob_sample(output: BinaryIO, size: int) -> tuple[bytes, bool]:
158
- sample_size = min(size, DEFAULT_PREVIEW_READ_BYTES)
171
+ read_size = 64 * 1024
172
+ sample_size = min(size, read_size)
159
173
  sample = output.read(sample_size)
160
174
  if len(sample) != sample_size:
161
175
  raise ValueError("truncated blob")
162
- binary = is_binary_sample(sample)
176
+ binary = _is_binary_sample(sample)
163
177
  bom = sample[:2] if sample.startswith((codecs.BOM_UTF16_LE, codecs.BOM_UTF16_BE)) else b""
164
178
  remaining = size - sample_size
165
179
  while remaining:
166
- chunk_size = min(remaining, DEFAULT_PREVIEW_READ_BYTES)
180
+ chunk_size = min(remaining, read_size)
167
181
  chunk = output.read(chunk_size)
168
182
  if len(chunk) != chunk_size:
169
183
  raise ValueError("truncated blob")
170
184
  # Even-sized reads keep UTF-16 code units aligned; preserve the original byte order.
171
185
  if not binary:
172
- binary = is_binary_sample(bom + chunk) if bom else b"\0" in chunk
186
+ binary = _is_binary_sample(bom + chunk) if bom else b"\0" in chunk
173
187
  remaining -= chunk_size
174
188
  if output.read(1) != b"\n":
175
189
  raise ValueError("missing blob terminator")
@@ -302,6 +316,71 @@ def git_command(
302
316
  return subprocess.CompletedProcess(full_command, 127, empty_output, empty_output)
303
317
 
304
318
 
319
+ def run_git_changed_paths(repo: Path, diff_args: list[str]) -> list[tuple[Path, str]]:
320
+ """Return changed regular files from the selected side of each change."""
321
+ result = git_command(
322
+ repo,
323
+ "diff",
324
+ "--ignore-submodules=all",
325
+ "--raw",
326
+ "-z",
327
+ "--diff-filter=ACMRDT",
328
+ *diff_args,
329
+ "--",
330
+ text=False,
331
+ )
332
+ result.check_returncode()
333
+ fields = result.stdout.split(b"\0")
334
+ if fields and not fields[-1]:
335
+ fields.pop()
336
+
337
+ changed: list[tuple[Path, str]] = []
338
+ index = 0
339
+ while index < len(fields):
340
+ metadata = fields[index].split()
341
+ status = chr(metadata[-1][0])
342
+ index += 1
343
+ if status in {"C", "R"}:
344
+ index += 1
345
+ path = repo / os.fsdecode(fields[index])
346
+ index += 1
347
+ selected_mode = metadata[0].removeprefix(b":") if status == "D" else metadata[1]
348
+ if selected_mode.startswith(b"100"):
349
+ changed.append((path, status))
350
+ return changed
351
+
352
+
353
+ def git_changed_paths(repo: Path, base: str, head: str, mode: str) -> list[tuple[Path, str]]:
354
+ if mode == "revisions":
355
+ return run_git_changed_paths(repo, [f"{base}..{head}"])
356
+ if mode == "local-patch":
357
+ unstaged = run_git_changed_paths(repo, [base])
358
+ staged = run_git_changed_paths(repo, ["--cached", base])
359
+ untracked = git_command(
360
+ repo,
361
+ "ls-files",
362
+ "--others",
363
+ "--exclude-standard",
364
+ "-z",
365
+ text=False,
366
+ )
367
+ untracked.check_returncode()
368
+ combined = dict(staged)
369
+ combined.update(unstaged)
370
+ combined.update(
371
+ (repo / os.fsdecode(relative), "A")
372
+ for relative in untracked.stdout.split(b"\0")
373
+ if relative and not relative.endswith(b"/")
374
+ )
375
+ return sorted(
376
+ (path, status)
377
+ for path, status in combined.items()
378
+ if SOURCE_METADATA_DIRECTORIES.isdisjoint(path.relative_to(repo).parts)
379
+ and (status == "D" or (not path.is_symlink() and path.is_file()))
380
+ )
381
+ raise SystemExit(f"Unknown diff mode: {mode}")
382
+
383
+
305
384
  def candidate_source_lines(
306
385
  target: Path,
307
386
  diff_target: dict[str, str],
@@ -310,8 +389,6 @@ def candidate_source_lines(
310
389
  case_insensitive: bool = False,
311
390
  ) -> dict[str, dict[str, Any]]:
312
391
  """Resolve candidate sources in the same selected view as the diff inventory."""
313
- from generate_rank_input import git_changed_paths
314
-
315
392
  base, head = diff_target["baseRevision"], diff_target["headRevision"]
316
393
  local = diff_target["kind"] == "working_tree"
317
394
  changed = {
@@ -422,15 +499,18 @@ def worktree_content_digest(target: Path) -> str:
422
499
 
423
500
 
424
501
  def remediation_checkout_snapshot(
425
- scan: sqlite3.Row, *, expected_revision: str | None = None
502
+ scan: sqlite3.Row,
503
+ *,
504
+ expected_revision: str | None = None,
505
+ head_changed_message: str = (
506
+ "Repository HEAD changed. Regenerate the remediation patch against the current checkout."
507
+ ),
426
508
  ) -> tuple[str, str | None]:
427
509
  target = require_scan_target_identity(scan)
428
510
  revision = git_revision(target)
429
511
  required_revision = expected_revision or scan["target_revision"]
430
512
  if revision != required_revision:
431
- raise SystemExit(
432
- "Repository HEAD changed. Regenerate the remediation patch against the current checkout."
433
- )
513
+ raise SystemExit(head_changed_message)
434
514
  content_digest = (
435
515
  worktree_content_digest(target)
436
516
  if revision != "unversioned"
@@ -439,6 +519,44 @@ def remediation_checkout_snapshot(
439
519
  return revision, content_digest
440
520
 
441
521
 
522
+ def require_scan_target_unchanged(scan: sqlite3.Row) -> None:
523
+ head_changed_message = "Repository HEAD changed. Run a new scan before validating its findings."
524
+ if (
525
+ scan["diff_target_kind"] not in {"working_tree", "commit", "range"}
526
+ and scan["target_snapshot_digest"] is None
527
+ and scan["mode"] != "diff"
528
+ and scan["target_revision"] != "unversioned"
529
+ ):
530
+ # Legacy Git records bind only the revision, not working-tree contents.
531
+ target = require_scan_target_identity(scan)
532
+ if git_revision(target) != scan["target_revision"]:
533
+ raise SystemExit(head_changed_message)
534
+ require_clean_submodule_worktrees(target)
535
+ return
536
+ _, current_digest = remediation_checkout_snapshot(
537
+ scan, head_changed_message=head_changed_message
538
+ )
539
+ if scan["diff_target_kind"] == "working_tree":
540
+ expected_digest = scan["diff_content_digest"]
541
+ elif scan["diff_target_kind"] in {"commit", "range"}:
542
+ tracked = git_bytes(Path(scan["target_path"]), "ls-files", "-t", "-z")
543
+ if tracked is None:
544
+ raise SystemExit("Could not inspect tracked files in the scan target.")
545
+ if any(entry.startswith(b"S ") for entry in tracked.split(b"\0")):
546
+ raise SystemExit(
547
+ "Committed-diff scans require a full repository checkout. "
548
+ "Sparse checkouts are not supported; materialize skipped tracked files and retry."
549
+ )
550
+ expected_digest = clean_worktree_content_digest()
551
+ else:
552
+ expected_digest = scan["target_snapshot_digest"]
553
+ if current_digest != expected_digest:
554
+ raise SystemExit(
555
+ "Scan target contents changed since the scan. "
556
+ "Run a new scan before validating its findings."
557
+ )
558
+
559
+
442
560
  def worktree_content_digest_for_context(
443
561
  repository: Path,
444
562
  pathspec: str,
@@ -815,6 +933,7 @@ def source_directory_snapshot_paths(
815
933
  target: Path,
816
934
  excluded: tuple[Path, ...] = (),
817
935
  *,
936
+ include_git_metadata: bool = False,
818
937
  onerror: Callable[[OSError], None] | None = None,
819
938
  ) -> list[Path]:
820
939
  paths: list[Path] = []
@@ -824,7 +943,9 @@ def source_directory_snapshot_paths(
824
943
  parent = Path(directory)
825
944
  for name in directories[:]:
826
945
  path = parent / name
827
- if name in SOURCE_METADATA_DIRECTORIES or path in excluded:
946
+ if (
947
+ name in SOURCE_METADATA_DIRECTORIES and not include_git_metadata
948
+ ) or path in excluded:
828
949
  directories.remove(name)
829
950
  continue
830
951
  paths.append(path)
@@ -834,7 +955,8 @@ def source_directory_snapshot_paths(
834
955
  paths.extend(
835
956
  parent / name
836
957
  for name in files
837
- if name not in SOURCE_METADATA_DIRECTORIES and parent / name not in excluded
958
+ if (name not in SOURCE_METADATA_DIRECTORIES or include_git_metadata)
959
+ and parent / name not in excluded
838
960
  )
839
961
  return sorted(paths)
840
962
 
@@ -844,6 +966,7 @@ def directory_content_digest(
844
966
  *,
845
967
  excluded: tuple[Path, ...] = (),
846
968
  include_ignored: bool = False,
969
+ include_git_metadata: bool = False,
847
970
  ) -> str:
848
971
  def raise_walk_error(error: OSError) -> None:
849
972
  raise error
@@ -852,7 +975,9 @@ def directory_content_digest(
852
975
  path.relative_to(target) for path in excluded if path.is_relative_to(target)
853
976
  ]
854
977
  paths = (
855
- source_directory_snapshot_paths(target, excluded, onerror=raise_walk_error)
978
+ source_directory_snapshot_paths(
979
+ target, excluded, include_git_metadata=include_git_metadata, onerror=raise_walk_error
980
+ )
856
981
  if include_ignored
857
982
  else git_directory_snapshot_paths(target)
858
983
  )
@@ -867,7 +992,7 @@ def directory_content_digest(
867
992
  try:
868
993
  metadata = path.lstat()
869
994
  except OSError as exc:
870
- raise SystemExit(f"Could not read local file: {relative_path}") from exc
995
+ raise UnreadableLocalFile(f"Could not read local file: {relative_path}") from exc
871
996
  mode = metadata.st_mode
872
997
  link_target = (
873
998
  os.readlink(path)
@@ -891,12 +1016,12 @@ def directory_content_digest(
891
1016
  content_digest.update(chunk)
892
1017
  content_size += len(chunk)
893
1018
  except OSError as exc:
894
- raise SystemExit(f"Could not read local file: {relative_path}") from exc
1019
+ raise UnreadableLocalFile(f"Could not read local file: {relative_path}") from exc
895
1020
  update_digest_field(digest, b"kind", b"file")
896
1021
  update_digest_field(digest, b"size", str(content_size).encode())
897
1022
  update_digest_field(digest, b"content-sha256", content_digest.digest())
898
1023
  else:
899
- raise SystemExit(f"Unsupported local file type: {relative_path}")
1024
+ raise UnsupportedLocalFileType(f"Unsupported local file type: {relative_path}")
900
1025
  return f"codex-security-snapshot/v1:sha256:{digest.hexdigest()}"
901
1026
 
902
1027
 
@@ -25,6 +25,15 @@ from workbench_scan_usage import _timestamp as usage_timestamp
25
25
  reject_non_finite_json = finalizer._reject_non_finite_json
26
26
 
27
27
 
28
+ def javascript_utf8(value: str) -> bytes:
29
+ """Match Node's UTF-8 encoding of UTF-16 strings without changing stored text."""
30
+ return (
31
+ value.encode("utf-16-le", errors="surrogatepass")
32
+ .decode("utf-16-le", errors="replace")
33
+ .encode("utf-8")
34
+ )
35
+
36
+
28
37
  def timestamp_key(value: str) -> datetime:
29
38
  return finalizer.parse_timestamp(value)
30
39
 
@@ -189,8 +198,6 @@ def _valid_measured_scan_usage(usage: object) -> bool:
189
198
  def parse_scan_cost(value: str | None) -> str | None:
190
199
  if value is None:
191
200
  return None
192
- if len(value.encode("utf-8")) > 8192:
193
- raise SystemExit("Scan cost must be no larger than 8 KiB.")
194
201
  try:
195
202
  cost = json.loads(value, parse_constant=reject_nonstandard_json_number)
196
203
  except (TypeError, UnicodeError, ValueError) as exc:
@@ -1,4 +1,13 @@
1
1
  [
2
+ [
3
+ "-- Move checkpoint previews before source-scope history checks.",
4
+ "WITH input AS (SELECT :history AS history)",
5
+ "SELECT 'sql' AS operation,",
6
+ " 'UPDATE schema_migrations SET version = 57 WHERE version = ' || version AS value",
7
+ "FROM schema_migrations",
8
+ "WHERE version IN (47, 48, 50, 52, 53, 54)",
9
+ " AND name = 'freeze stopped scan checkpoint selections';"
10
+ ],
2
11
  [
3
12
  "-- Move source-scope previews before the shared history repairs.",
4
13
  "WITH input AS (SELECT :history AS history)",
@@ -57,6 +66,18 @@
57
66
  " FROM moves JOIN schema_migrations USING (version, name)",
58
67
  "ORDER BY position;"
59
68
  ],
69
+ [
70
+ "-- A main-branch reader may have added discovery context under the old checkpoint number.",
71
+ "SELECT 'timestamp' AS operation,",
72
+ " 'INSERT INTO schema_migrations (version, name, applied_at) VALUES (47, ' ||",
73
+ " quote(json_extract(value, '$.name')) || ', ?)' AS value",
74
+ "FROM json_each(:history) WHERE json_extract(value, '$.version') = 47",
75
+ " AND EXISTS (SELECT 1 FROM schema_migrations WHERE version = 57",
76
+ " AND name = 'freeze stopped scan checkpoint selections')",
77
+ " AND NOT EXISTS (SELECT 1 FROM schema_migrations WHERE version = 47)",
78
+ " AND EXISTS (SELECT 1 FROM pragma_table_info('deep_scan_runs')",
79
+ " WHERE name = 'discovery_user_context_json');"
80
+ ],
60
81
  [
61
82
  "-- profiles: preserve legacy columns before the later repairs.",
62
83
  "WITH",
@@ -441,5 +441,33 @@
441
441
  "statements": [
442
442
  "CREATE TABLE external_source_reports (\n id TEXT PRIMARY KEY,\n repository TEXT NOT NULL,\n provider TEXT NOT NULL,\n source_key TEXT NOT NULL,\n source_finding_id TEXT NOT NULL,\n evidence_json TEXT NOT NULL,\n version INTEGER NOT NULL,\n created_at TEXT NOT NULL,\n updated_at TEXT NOT NULL,\n UNIQUE (repository, provider, source_key, source_finding_id)\n);"
443
443
  ]
444
+ },
445
+ {
446
+ "version": 54,
447
+ "name": "snapshot deep scan setup configuration",
448
+ "statements": [
449
+ "ALTER TABLE workspaces ADD COLUMN deep_scan_config_json TEXT;"
450
+ ]
451
+ },
452
+ {
453
+ "version": 55,
454
+ "name": "retain undelivered headless deep scan results",
455
+ "statements": [
456
+ "ALTER TABLE deep_scan_runs ADD COLUMN pending_target_project_path TEXT;"
457
+ ]
458
+ },
459
+ {
460
+ "version": 56,
461
+ "name": "recover unfinished legacy headless deep scans",
462
+ "statements": [
463
+ "UPDATE deep_scan_runs\nSET pending_target_project_path = ''\nWHERE pending_target_project_path IS NULL\n AND status IN ('running', 'succeeded') AND cancel_requested = 0\n AND discovery_user_context_json IS NOT NULL\n AND EXISTS (\n SELECT 1 FROM scans JOIN workspaces ON workspaces.id = scans.workspace_id\n WHERE scans.id = deep_scan_runs.scan_id AND scans.mode = 'deep'\n AND scans.status = 'running' AND scans.canceled_at IS NULL\n AND scans.recipe_json IS NULL AND scans.handoff_status = 'delivered'\n AND scans.handoff_claim_token IS NULL AND scans.continuation_thread_id IS NULL\n AND workspaces.active_scan_id = scans.id AND workspaces.deep_scan_config_json IS NULL\n AND workspaces.created_at = scans.created_at AND scans.created_at = deep_scan_runs.created_at\n );"
464
+ ]
465
+ },
466
+ {
467
+ "version": 57,
468
+ "name": "freeze stopped scan checkpoint selections",
469
+ "statements": [
470
+ "ALTER TABLE scans ADD COLUMN retained_checkpoint_heads_json TEXT;"
471
+ ]
444
472
  }
445
473
  ]
@@ -25,10 +25,10 @@ Read [references/risk-rubric.md](references/risk-rubric.md) before assigning rat
25
25
  4. **Describe the semantic change.** Separate production, test, generated, configuration, dependency, migration, documentation, and build changes. Identify changed behavior, defaults, errors, side effects, state, and contracts.
26
26
  5. **Map program impact from source.** Trace changed symbols through direct callers and affected callees to production entrypoints, jobs, routes, registries, package exports, deployment paths, or supported external consumers. Check dynamic dispatch and configuration-selected paths. Do not call code dead from text search alone.
27
27
  6. **Inspect material boundaries.** Check authentication and authorization, tenant isolation, parsing, filesystem and network access, sandboxing, public APIs, serialized data, configuration defaults, migrations, persistence, concurrency, retries, performance, and rollout behavior when affected.
28
- 7. **Try to falsify safety.** For each material changed boundary, state one concrete counterexample and one legitimate control grounded in base source, callers, or an authoritative contract. Trace both through the patched source. Reclassify redirects, callbacks, embedded URLs, cached authority, and other derived trust decisions at the point of use instead of inheriting trust from their origin. When policy aggregates multiple subjects, bind each decision to the same identity, route, resource, or record rather than transferring one subject's properties to the set. Trace validated values, authority, and state through later mutation or re-resolution to the first sensitive sink. Treat UI, discovery, prompt, instruction, and visibility controls as exposure controls unless they remove the underlying capability or an independent downstream control enforces the same boundary. A changed test or implementation list cannot by itself define the supported contract.
28
+ 7. **Try to falsify safety.** For each material changed boundary, trace a concrete counterexample and a legitimate control grounded in base source, callers, or an authoritative contract. Use the boundary challenges in `references/risk-rubric.md`, including identity binding, derived trust, and changes between validation and use. A changed test or implementation list cannot by itself define the supported contract.
29
29
  8. **Evaluate regression protection.** Distinguish changed-path, caller, integration, and rollout coverage. Inspect what assertions actually observe, whether the relevant check ran at the exact head, and whether platform or deployment-specific validation is missing. Tests lower likelihood or raise confidence; they never lower the impact if failure occurs.
30
30
  9. **Assess applicability and recovery.** Establish that the patch affects an owned runtime or supported consumer. Use `no_op` when evidence proves no live effect, wrong ownership, duplication, or supersession. Describe rollback, persistent-state effects, migrations, and operational recovery. Report the risk of not merging separately; use `unknown` when motivating context is unavailable.
31
- 10. **Resolve available unknowns now.** Inspect accessible source, exact-head checks, and focused deterministic local tests when safe. If a decision-critical unknown remains, return `hold_for_evidence` with at most three concrete actions, the evidence each action seeks, and how each possible result changes the recommendation. Do not wait or poll indefinitely.
31
+ 10. **Resolve available unknowns now.** Inspect accessible source, exact-head checks, and focused deterministic local tests when safe. If a decision-critical unknown remains, return `hold_for_evidence` with at most three next actions as required by the schema, the evidence each seeks, and how the possible results change the recommendation. Do not wait or poll indefinitely.
32
32
 
33
33
  ## Recommendation
34
34
 
@@ -70,18 +70,7 @@ Before returning the result, validate the JSON from any working directory with:
70
70
  <plugin-root>/scripts/launch_codex_security_mcp --helper validate-patch-risk-assessment <assessment.json>
71
71
  ```
72
72
 
73
- On Windows, use this PowerShell invocation. Replace the placeholders inside the single quotes with literal paths, doubling any single quote in a path:
74
-
75
- ```powershell
76
- $env:patchRiskPluginRoot = Convert-Path -LiteralPath '<plugin-root>' -ErrorAction Stop
77
- $env:patchRiskAssessment = '<assessment.json>'
78
- if ($env:patchRiskAssessment -ne '-') {
79
- $env:patchRiskAssessment = Convert-Path -LiteralPath $env:patchRiskAssessment -ErrorAction Stop
80
- }
81
- cmd.exe /d /v:off /s /c '""%patchRiskPluginRoot%\scripts\launch_codex_security_mcp.cmd" --helper validate-patch-risk-assessment "%patchRiskAssessment%""'
82
- ```
83
-
84
- These two environment variables are temporary values in the calling shell, not application settings. Resolve paths against PowerShell's current location before CMD starts, including when that location is a UNC share. Missing paths stop the invocation with PowerShell's path error. CMD expands the references once, preserving literal `%` and `!` in the paths. Pass `-` as `<assessment.json>` to read the assessment from standard input without creating a file. The validator uses the bundled Node runtime helper and does not require Python.
73
+ On Windows, use the literal-path PowerShell invocation in [references/windows-validation.md](references/windows-validation.md). The validator also accepts `-` for standard input and uses the bundled Node helper without requiring Python.
85
74
 
86
75
  Correct structural or invariant errors by revisiting the evidence; never change a recommendation merely to make validation pass. Return the validated JSON in the response. Write it to disk only when the caller requests an artifact, and keep every assessment-created file outside the subject checkout and its Git directories.
87
76
 
@@ -0,0 +1,14 @@
1
+ # Windows assessment validation
2
+
3
+ On Windows, use this PowerShell invocation. Replace the placeholders inside the single quotes with literal paths, doubling any single quote in a path:
4
+
5
+ ```powershell
6
+ $env:patchRiskPluginRoot = Convert-Path -LiteralPath '<plugin-root>' -ErrorAction Stop
7
+ $env:patchRiskAssessment = '<assessment.json>'
8
+ if ($env:patchRiskAssessment -ne '-') {
9
+ $env:patchRiskAssessment = Convert-Path -LiteralPath $env:patchRiskAssessment -ErrorAction Stop
10
+ }
11
+ cmd.exe /d /v:off /s /c '""%patchRiskPluginRoot%\scripts\launch_codex_security_mcp.cmd" --helper validate-patch-risk-assessment "%patchRiskAssessment%""'
12
+ ```
13
+
14
+ These two environment variables are temporary values in the calling shell, not application settings. Resolve paths against PowerShell's current location before CMD starts, including when that location is a UNC share. Missing paths stop the invocation with PowerShell's path error. CMD expands the references once, preserving literal `%` and `!` in the paths. Pass `-` as `<assessment.json>` to read the assessment from standard input without creating a file. The validator uses the bundled Node runtime helper and does not require Python.