@1aboveio/skills 0.20.2 → 0.20.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +14 -6
- package/package.json +1 -1
- package/runtime/skills/distribution/generated/recipes.json +220 -25
- package/runtime/skills/distribution/scripts/bundles.mjs +167 -48
- package/runtime/skills/engineering/engineering-runtime/scripts/workflow-coherence.mjs +4 -0
- package/runtime/skills/engineering/engineering-runtime/scripts/workflow-policy.mjs +1 -1
- package/skills/compliance/compliance-audit/LICENSE +3 -0
- package/skills/compliance/compliance-audit/SKILL.md +466 -0
- package/skills/compliance/compliance-audit/references/findings-schema.md +290 -0
- package/skills/compliance/compliance-audit/references/vars-checklist.md +1068 -0
- package/skills/compliance/compliance-audit/references/vars-findings-schema.md +395 -0
- package/skills/compliance/compliance-audit/scripts/reconcile_tracker.py +454 -0
- package/skills/compliance/compliance-audit/scripts/test_validate_tracker.py +90 -0
- package/skills/compliance/compliance-audit/scripts/validate_tracker.py +752 -0
- package/skills/compliance/doc-authenticity/LICENSE +3 -0
- package/skills/compliance/doc-authenticity/SKILL.md +158 -0
- package/skills/compliance/doc-authenticity/references/agreement.md +66 -0
- package/skills/compliance/doc-authenticity/references/bank-statement.md +150 -0
- package/skills/compliance/doc-authenticity/references/forensic-signals.md +157 -0
- package/skills/compliance/doc-authenticity/references/image-signals.md +114 -0
- package/skills/compliance/doc-authenticity/references/invoice.md +66 -0
- package/skills/compliance/doc-authenticity/references/issuer-profiles.md +86 -0
- package/skills/compliance/doc-authenticity/references/verdict-and-report.md +95 -0
- package/skills/compliance/doc-authenticity/references/verdict.schema.json +81 -0
- package/skills/compliance/doc-authenticity/scripts/image_forensics.py +294 -0
- package/skills/compliance/doc-authenticity/scripts/pdf_forensics.py +401 -0
- package/skills/compliance/doc-authenticity/scripts/validate_codes.py +224 -0
- package/skills/compliance/doc-authenticity/scripts/validate_verdict.py +308 -0
- package/skills/compliance/report-writing/LICENSE +3 -0
- package/skills/compliance/report-writing/SKILL.md +206 -0
- package/skills/compliance/report-writing/references/audit-report-template.md +121 -0
- package/skills/compliance/report-writing/references/external-assessment-template.md +167 -0
- package/skills/compliance/report-writing/references/management-report-template.md +145 -0
- package/skills/compliance/report-writing/references/vars-report-template.md +224 -0
- package/skills/document-utils/doc-to-md/LICENSE +3 -0
- package/skills/document-utils/doc-to-md/SKILL.md +77 -0
- package/skills/document-utils/doc-to-md/scripts/convert.py +157 -0
- package/skills/document-utils/reg-parser/LICENSE +3 -0
- package/skills/document-utils/reg-parser/SKILL.md +201 -0
- package/skills/document-utils/reg-parser/references/checklist-schema.md +223 -0
- package/skills/document-utils/reg-parser/references/checklists/hk-amlo-checklist.json +1306 -0
- package/skills/document-utils/reg-parser/references/checklists/hk-amlo-checklist.md +469 -0
- package/skills/document-utils/reg-parser/references/checklists/hk-fi-corporate-onboarding-cdd-checklist.json +634 -0
- package/skills/document-utils/reg-parser/references/checklists/hk-fi-corporate-onboarding-cdd-checklist.md +233 -0
- package/skills/document-utils/reg-parser/references/checklists/hk-mso-guideline-checklist.json +2194 -0
- package/skills/document-utils/reg-parser/references/checklists/hk-mso-guideline-checklist.md +771 -0
- package/skills/document-utils/reg-parser/references/checklists/kyc-aml-checklist.md +258 -0
- package/skills/document-utils/reg-parser/references/checklists/visa-vars-checklist.md +1058 -0
- package/skills/document-utils/reg-parser/references/mandates/hk-amlo-cap615.md +389 -0
- package/skills/document-utils/reg-parser/references/mandates/visa-vars.md +205 -0
- package/skills/document-utils/reg-parser/scripts/compare_checklist.py +228 -0
- package/skills/document-utils/reg-parser/scripts/dump_checklist.py +348 -0
- package/skills/document-utils/reg-parser/scripts/preprocess-mandate.sh +171 -0
- package/skills/document-utils/reg-parser/scripts/test_quarantine.sh +102 -0
- package/skills/document-utils/reg-parser/scripts/validate_checklist.py +532 -0
- package/skills/document-utils/reg-parser/scripts/validate_mandate.py +242 -0
- package/skills/engineering/engineering-runtime/coherence/workflow.json +16 -16
- package/skills/engineering/engineering-runtime/scripts/workflow-coherence.mjs +4 -0
- package/skills/engineering/engineering-runtime/scripts/workflow-policy.mjs +1 -1
- package/skills/payment/similar-domain/LICENSE +3 -0
- package/skills/payment/similar-domain/SKILL.md +178 -0
- package/skills/payment/similar-domain/references/entities.md +58 -0
- package/skills/payment/similar-domain/references/fingerprints.md +82 -0
- package/skills/payment/similar-domain/references/pivots.md +66 -0
- package/skills/payment/similar-domain/references/report-template.md +76 -0
- package/skills/payment/similar-domain/references/scoring.md +61 -0
- package/skills/payment/similar-domain/references/terminology.md +37 -0
- package/skills/payment/similar-domain/references/workflow.md +79 -0
- package/skills/payment/tld-detection/SKILL.md +76 -21
- package/skills/payment/tld-detection/references/clean.md +1 -1
- package/skills/payment/tld-detection/references/clustering.md +21 -6
- package/skills/payment/tld-detection/references/decision.md +7 -2
- package/skills/payment/tld-detection/references/fx-conversion.md +229 -0
- package/skills/payment/tld-detection/references/report-template.md +63 -27
- package/skills/payment/tld-detection/references/streams.md +90 -0
- package/skills/payment/tld-detection/references/terminology.md +15 -7
- package/skills/payment/tld-detection/references/workflow.md +1 -1
- package/skills/payment/tld-detection/scripts/detect_streams.py +542 -0
- package/skills/engineering/ensure-coverage/evals/evals.json +0 -125
- package/skills/engineering/rush-issues/evals/evals.json +0 -65
- package/skills/engineering/rush-release/evals/evals.json +0 -58
- package/skills/fullstack/shadcn/evals/evals.json +0 -90
- package/skills/payment/fraud-analysis/evals/evals.json +0 -40
- package/skills/payment/tld-detection/evals/evals.json +0 -107
- package/skills/payment/tld-detection/tests/test_detect_ladders.py +0 -132
|
@@ -0,0 +1,308 @@
|
|
|
1
|
+
# /// script
|
|
2
|
+
# requires-python = ">=3.10"
|
|
3
|
+
# dependencies = []
|
|
4
|
+
# ///
|
|
5
|
+
"""Validate a schema-version 2.2 routed dual-axis authenticity verdict."""
|
|
6
|
+
|
|
7
|
+
from __future__ import annotations
|
|
8
|
+
|
|
9
|
+
import json
|
|
10
|
+
import sys
|
|
11
|
+
from datetime import datetime
|
|
12
|
+
from pathlib import Path
|
|
13
|
+
|
|
14
|
+
AUTH_VERDICTS = {
|
|
15
|
+
"NO_EVIDENCE_OF_MODIFICATION",
|
|
16
|
+
"SUSPICIOUS",
|
|
17
|
+
"LIKELY_MODIFIED",
|
|
18
|
+
"INCONCLUSIVE",
|
|
19
|
+
}
|
|
20
|
+
VALIDITY_VERDICTS = {"VALID", "INVALID", "INCONCLUSIVE"}
|
|
21
|
+
CONFIDENCES = {"HIGH", "MEDIUM", "LOW"}
|
|
22
|
+
RESULTS = {"PASS", "FLAG", "FAIL", "NA"}
|
|
23
|
+
DOC_TYPES = {"bank_statement", "conversation_log", "invoice", "agreement", "other"}
|
|
24
|
+
SCHEMA_VERSION = "2.2"
|
|
25
|
+
PDF_AUTH_IDS = [f"F{i}" for i in range(1, 10)]
|
|
26
|
+
IMAGE_AUTH_IDS = [f"I{i}" for i in range(1, 10)]
|
|
27
|
+
VALIDITY_IDS = {
|
|
28
|
+
"bank_statement": [f"C{i}" for i in range(1, 6)] + [f"D{i}" for i in range(1, 6)],
|
|
29
|
+
"invoice": [f"N{i}" for i in range(1, 9)],
|
|
30
|
+
"agreement": [f"A{i}" for i in range(1, 9)],
|
|
31
|
+
"conversation_log": [],
|
|
32
|
+
"other": [],
|
|
33
|
+
}
|
|
34
|
+
EXPECTED = {
|
|
35
|
+
**{f"F{i}": ("authenticity", "file_forensics") for i in range(1, 10)},
|
|
36
|
+
**{f"I{i}": ("authenticity", "file_forensics") for i in range(1, 10)},
|
|
37
|
+
**{f"C{i}": ("validity", "content") for i in range(1, 6)},
|
|
38
|
+
**{f"D{i}": ("validity", "domain") for i in range(1, 6)},
|
|
39
|
+
**{f"N{i}": ("validity", "content") for i in (2, 3, 4, 7)},
|
|
40
|
+
**{f"N{i}": ("validity", "domain") for i in (1, 5, 6, 8)},
|
|
41
|
+
**{f"A{i}": ("validity", "content") for i in (2, 4, 5, 7)},
|
|
42
|
+
**{f"A{i}": ("validity", "domain") for i in (1, 3, 6, 8)},
|
|
43
|
+
}
|
|
44
|
+
|
|
45
|
+
|
|
46
|
+
def nonempty_string(value: object) -> bool:
|
|
47
|
+
return isinstance(value, str) and bool(value.strip())
|
|
48
|
+
|
|
49
|
+
|
|
50
|
+
def expected_authenticity(
|
|
51
|
+
seen: dict[str, dict], auth_ids: list[str], file_type: str
|
|
52
|
+
) -> str:
|
|
53
|
+
results = [seen[cid]["result"] for cid in auth_ids]
|
|
54
|
+
fails = results.count("FAIL")
|
|
55
|
+
flags = results.count("FLAG")
|
|
56
|
+
decisive_fail = any(
|
|
57
|
+
seen[cid]["result"] == "FAIL" and seen[cid]["decisive"] for cid in auth_ids
|
|
58
|
+
)
|
|
59
|
+
if decisive_fail or fails >= 2 or (fails == 1 and flags >= 2):
|
|
60
|
+
return "LIKELY_MODIFIED"
|
|
61
|
+
if (
|
|
62
|
+
fails == 1
|
|
63
|
+
or flags >= 3
|
|
64
|
+
or (file_type == "image" and seen["I2"]["result"] == "FLAG")
|
|
65
|
+
):
|
|
66
|
+
return "SUSPICIOUS"
|
|
67
|
+
if results.count("NA") * 2 >= len(results):
|
|
68
|
+
return "INCONCLUSIVE"
|
|
69
|
+
return "NO_EVIDENCE_OF_MODIFICATION"
|
|
70
|
+
|
|
71
|
+
|
|
72
|
+
def expected_validity(seen: dict[str, dict], validity_ids: list[str]) -> str:
|
|
73
|
+
if not validity_ids:
|
|
74
|
+
return "INCONCLUSIVE"
|
|
75
|
+
results = [seen[cid]["result"] for cid in validity_ids]
|
|
76
|
+
fails = results.count("FAIL")
|
|
77
|
+
decisive_fail = any(
|
|
78
|
+
seen[cid]["result"] == "FAIL" and seen[cid]["decisive"] for cid in validity_ids
|
|
79
|
+
)
|
|
80
|
+
if decisive_fail or fails >= 2:
|
|
81
|
+
return "INVALID"
|
|
82
|
+
if fails or results.count("FLAG") > 2 or results.count("NA") * 2 >= len(results):
|
|
83
|
+
return "INCONCLUSIVE"
|
|
84
|
+
return "VALID"
|
|
85
|
+
|
|
86
|
+
|
|
87
|
+
def main() -> int:
|
|
88
|
+
if len(sys.argv) != 2:
|
|
89
|
+
print("Usage: validate_verdict.py <verdict.json>", file=sys.stderr)
|
|
90
|
+
return 2
|
|
91
|
+
try:
|
|
92
|
+
verdict = json.loads(Path(sys.argv[1]).read_text(encoding="utf-8"))
|
|
93
|
+
except Exception as exc: # noqa: BLE001
|
|
94
|
+
print(f"ERROR: not valid JSON: {exc}")
|
|
95
|
+
return 1
|
|
96
|
+
|
|
97
|
+
if not isinstance(verdict, dict):
|
|
98
|
+
print("ERROR: verdict root must be an object")
|
|
99
|
+
return 1
|
|
100
|
+
|
|
101
|
+
errors: list[str] = []
|
|
102
|
+
required = {
|
|
103
|
+
"schema_version",
|
|
104
|
+
"artifact",
|
|
105
|
+
"authenticity",
|
|
106
|
+
"validity",
|
|
107
|
+
"checks",
|
|
108
|
+
"summary",
|
|
109
|
+
"checked_at",
|
|
110
|
+
}
|
|
111
|
+
missing = required - set(verdict)
|
|
112
|
+
extra = set(verdict) - required
|
|
113
|
+
if missing:
|
|
114
|
+
errors.append(f"missing keys: {sorted(missing)}")
|
|
115
|
+
if extra:
|
|
116
|
+
errors.append(f"unknown top-level keys: {sorted(extra)}")
|
|
117
|
+
if verdict.get("schema_version") != SCHEMA_VERSION:
|
|
118
|
+
errors.append(f"schema_version must be {SCHEMA_VERSION!r}")
|
|
119
|
+
|
|
120
|
+
artifact = verdict.get("artifact", {})
|
|
121
|
+
artifact_keys = {
|
|
122
|
+
"path",
|
|
123
|
+
"file_type",
|
|
124
|
+
"media_type",
|
|
125
|
+
"size_bytes",
|
|
126
|
+
"sha256",
|
|
127
|
+
"extension_magic_match",
|
|
128
|
+
"doc_type",
|
|
129
|
+
"detail",
|
|
130
|
+
"claimed_issuer",
|
|
131
|
+
"supplied_claim",
|
|
132
|
+
}
|
|
133
|
+
if not isinstance(artifact, dict):
|
|
134
|
+
errors.append("artifact must be an object")
|
|
135
|
+
artifact = {}
|
|
136
|
+
else:
|
|
137
|
+
missing_artifact = artifact_keys - set(artifact)
|
|
138
|
+
extra_artifact = set(artifact) - artifact_keys
|
|
139
|
+
if missing_artifact:
|
|
140
|
+
errors.append(f"artifact missing keys: {sorted(missing_artifact)}")
|
|
141
|
+
if extra_artifact:
|
|
142
|
+
errors.append(f"artifact has unknown keys: {sorted(extra_artifact)}")
|
|
143
|
+
file_type = artifact.get("file_type")
|
|
144
|
+
doc_type = artifact.get("doc_type")
|
|
145
|
+
media_type = artifact.get("media_type")
|
|
146
|
+
if file_type not in {"pdf", "image"}:
|
|
147
|
+
errors.append("artifact.file_type must be pdf or image")
|
|
148
|
+
if doc_type not in DOC_TYPES:
|
|
149
|
+
errors.append(f"artifact.doc_type must be one of {sorted(DOC_TYPES)}")
|
|
150
|
+
for key in ("path", "media_type"):
|
|
151
|
+
if not nonempty_string(artifact.get(key)):
|
|
152
|
+
errors.append(f"artifact.{key} must be a non-empty string")
|
|
153
|
+
size_bytes = artifact.get("size_bytes")
|
|
154
|
+
if (
|
|
155
|
+
isinstance(size_bytes, bool)
|
|
156
|
+
or not isinstance(size_bytes, int)
|
|
157
|
+
or size_bytes < 1
|
|
158
|
+
):
|
|
159
|
+
errors.append("artifact.size_bytes must be a positive integer")
|
|
160
|
+
sha256 = artifact.get("sha256")
|
|
161
|
+
if (
|
|
162
|
+
not isinstance(sha256, str)
|
|
163
|
+
or len(sha256) != 64
|
|
164
|
+
or any(char not in "0123456789abcdef" for char in sha256)
|
|
165
|
+
):
|
|
166
|
+
errors.append("artifact.sha256 must be 64 lowercase hexadecimal characters")
|
|
167
|
+
if not isinstance(artifact.get("extension_magic_match"), bool):
|
|
168
|
+
errors.append("artifact.extension_magic_match must be boolean")
|
|
169
|
+
if file_type == "pdf" and media_type != "application/pdf":
|
|
170
|
+
errors.append("pdf file_type requires media_type application/pdf")
|
|
171
|
+
if file_type == "image" and (
|
|
172
|
+
not isinstance(media_type, str) or not media_type.startswith("image/")
|
|
173
|
+
):
|
|
174
|
+
errors.append("image file_type requires an image/* media_type")
|
|
175
|
+
for key in ("detail", "claimed_issuer", "supplied_claim"):
|
|
176
|
+
if (
|
|
177
|
+
key in artifact
|
|
178
|
+
and artifact[key] is not None
|
|
179
|
+
and not isinstance(artifact[key], str)
|
|
180
|
+
):
|
|
181
|
+
errors.append(f"artifact.{key} must be a string or null")
|
|
182
|
+
|
|
183
|
+
auth = verdict.get("authenticity", {})
|
|
184
|
+
validity = verdict.get("validity", {})
|
|
185
|
+
if not isinstance(auth, dict):
|
|
186
|
+
errors.append("authenticity must be an object")
|
|
187
|
+
auth = {}
|
|
188
|
+
if not isinstance(validity, dict):
|
|
189
|
+
errors.append("validity must be an object")
|
|
190
|
+
validity = {}
|
|
191
|
+
if auth.get("verdict") not in AUTH_VERDICTS:
|
|
192
|
+
errors.append(f"invalid authenticity verdict: {auth.get('verdict')!r}")
|
|
193
|
+
if validity.get("verdict") not in VALIDITY_VERDICTS:
|
|
194
|
+
errors.append(f"invalid validity verdict: {validity.get('verdict')!r}")
|
|
195
|
+
axis_keys = {"verdict", "confidence", "summary"}
|
|
196
|
+
for label, axis in (("authenticity", auth), ("validity", validity)):
|
|
197
|
+
missing_axis = axis_keys - set(axis)
|
|
198
|
+
extra_axis = set(axis) - axis_keys
|
|
199
|
+
if missing_axis:
|
|
200
|
+
errors.append(f"{label} missing keys: {sorted(missing_axis)}")
|
|
201
|
+
if extra_axis:
|
|
202
|
+
errors.append(f"{label} has unknown keys: {sorted(extra_axis)}")
|
|
203
|
+
if axis.get("confidence") not in CONFIDENCES:
|
|
204
|
+
errors.append(f"{label}.confidence must be HIGH, MEDIUM, or LOW")
|
|
205
|
+
if not nonempty_string(axis.get("summary")):
|
|
206
|
+
errors.append(f"{label}.summary must be non-empty")
|
|
207
|
+
|
|
208
|
+
auth_ids = (
|
|
209
|
+
PDF_AUTH_IDS
|
|
210
|
+
if file_type == "pdf"
|
|
211
|
+
else IMAGE_AUTH_IDS
|
|
212
|
+
if file_type == "image"
|
|
213
|
+
else []
|
|
214
|
+
)
|
|
215
|
+
validity_ids = VALIDITY_IDS.get(doc_type, [])
|
|
216
|
+
wanted = auth_ids + validity_ids
|
|
217
|
+
checks = verdict.get("checks", [])
|
|
218
|
+
seen: dict[str, dict] = {}
|
|
219
|
+
if not isinstance(checks, list):
|
|
220
|
+
errors.append("checks must be an array")
|
|
221
|
+
checks = []
|
|
222
|
+
check_keys = {"id", "name", "axis", "dimension", "result", "decisive", "evidence"}
|
|
223
|
+
for index, check in enumerate(checks):
|
|
224
|
+
if not isinstance(check, dict):
|
|
225
|
+
errors.append(f"checks[{index}] must be an object")
|
|
226
|
+
continue
|
|
227
|
+
missing_check = check_keys - set(check)
|
|
228
|
+
extra_check = set(check) - check_keys
|
|
229
|
+
if missing_check:
|
|
230
|
+
errors.append(f"checks[{index}] missing keys: {sorted(missing_check)}")
|
|
231
|
+
if extra_check:
|
|
232
|
+
errors.append(f"checks[{index}] has unknown keys: {sorted(extra_check)}")
|
|
233
|
+
cid = check.get("id")
|
|
234
|
+
if cid not in EXPECTED:
|
|
235
|
+
errors.append(f"checks[{index}]: unknown id {cid!r}")
|
|
236
|
+
continue
|
|
237
|
+
if cid in seen:
|
|
238
|
+
errors.append(f"checks[{index}]: duplicate id {cid}")
|
|
239
|
+
seen[cid] = check
|
|
240
|
+
want_axis, want_dimension = EXPECTED[cid]
|
|
241
|
+
if check.get("axis") != want_axis:
|
|
242
|
+
errors.append(f"{cid}: axis must be {want_axis}")
|
|
243
|
+
if check.get("dimension") != want_dimension:
|
|
244
|
+
errors.append(f"{cid}: dimension must be {want_dimension}")
|
|
245
|
+
if check.get("result") not in RESULTS:
|
|
246
|
+
errors.append(f"{cid}: invalid result {check.get('result')!r}")
|
|
247
|
+
if not isinstance(check.get("decisive"), bool):
|
|
248
|
+
errors.append(f"{cid}: decisive must be boolean")
|
|
249
|
+
elif check["decisive"] and check.get("result") != "FAIL":
|
|
250
|
+
errors.append(f"{cid}: only a FAIL can be decisive")
|
|
251
|
+
if not nonempty_string(check.get("name")):
|
|
252
|
+
errors.append(f"{cid}: name must be non-empty")
|
|
253
|
+
if not nonempty_string(check.get("evidence")):
|
|
254
|
+
errors.append(f"{cid}: evidence must be non-empty")
|
|
255
|
+
absent = [cid for cid in wanted if cid not in seen]
|
|
256
|
+
unexpected = [cid for cid in seen if cid not in wanted]
|
|
257
|
+
if absent:
|
|
258
|
+
errors.append(f"missing checks: {absent}")
|
|
259
|
+
if unexpected:
|
|
260
|
+
errors.append(f"checks do not belong to the selected route: {unexpected}")
|
|
261
|
+
|
|
262
|
+
if not nonempty_string(verdict.get("summary")):
|
|
263
|
+
errors.append("summary must be non-empty")
|
|
264
|
+
try:
|
|
265
|
+
checked_at = datetime.fromisoformat(
|
|
266
|
+
str(verdict.get("checked_at", "")).replace("Z", "+00:00")
|
|
267
|
+
)
|
|
268
|
+
if checked_at.tzinfo is None:
|
|
269
|
+
raise ValueError("timezone missing")
|
|
270
|
+
except ValueError:
|
|
271
|
+
errors.append("checked_at must be a valid ISO 8601 timestamp with timezone")
|
|
272
|
+
|
|
273
|
+
if not errors:
|
|
274
|
+
wanted_authenticity = expected_authenticity(seen, auth_ids, file_type)
|
|
275
|
+
wanted_validity = expected_validity(seen, validity_ids)
|
|
276
|
+
if auth["verdict"] != wanted_authenticity:
|
|
277
|
+
errors.append(
|
|
278
|
+
"authenticity verdict must be "
|
|
279
|
+
f"{wanted_authenticity} for these check results"
|
|
280
|
+
)
|
|
281
|
+
if validity["verdict"] != wanted_validity:
|
|
282
|
+
errors.append(
|
|
283
|
+
f"validity verdict must be {wanted_validity} for these check results"
|
|
284
|
+
)
|
|
285
|
+
if (
|
|
286
|
+
auth["verdict"] == "NO_EVIDENCE_OF_MODIFICATION"
|
|
287
|
+
and auth["confidence"] == "HIGH"
|
|
288
|
+
):
|
|
289
|
+
provenance_id = "F8" if file_type == "pdf" else "I9"
|
|
290
|
+
if seen[provenance_id]["result"] != "PASS":
|
|
291
|
+
errors.append(
|
|
292
|
+
f"HIGH confidence requires trusted provenance ({provenance_id} PASS)"
|
|
293
|
+
)
|
|
294
|
+
|
|
295
|
+
for error in errors:
|
|
296
|
+
print(f"ERROR: {error}")
|
|
297
|
+
if errors:
|
|
298
|
+
print(f"invalid: {len(errors)} error(s)")
|
|
299
|
+
return 1
|
|
300
|
+
print(
|
|
301
|
+
f"valid: {file_type}/{doc_type} route, {len(wanted)} checks, "
|
|
302
|
+
f"authenticity {auth['verdict']}, validity {validity['verdict']}"
|
|
303
|
+
)
|
|
304
|
+
return 0
|
|
305
|
+
|
|
306
|
+
|
|
307
|
+
if __name__ == "__main__":
|
|
308
|
+
raise SystemExit(main())
|
|
@@ -0,0 +1,206 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: report-writing
|
|
3
|
+
description: "Convert compliance audit findings into audience-formatted reports. Four styles: VARS Review Report (Visa's official 2025 template, required verbatim for VARS engagements), Audit Report (compliance team), External Assessment (regulators/external auditors), Management Report (board/C-suite). Use whenever audit output needs to become a deliverable, even just 'write up the findings'."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Report Writing
|
|
7
|
+
|
|
8
|
+
Generate compliance reports from structured audit findings. Four report styles target
|
|
9
|
+
different audiences — same data, different depth and tone.
|
|
10
|
+
|
|
11
|
+
## Input Format
|
|
12
|
+
|
|
13
|
+
Structured findings in Markdown with per-item:
|
|
14
|
+
- **Status:** Pass / Fail / Partial / N/A
|
|
15
|
+
- **Severity:** Critical / Major / Minor / Observation
|
|
16
|
+
- **Category:** e.g., Access Control, Data Protection, Incident Response
|
|
17
|
+
- **Observations:** What was found
|
|
18
|
+
- **Evidence:** Logs, configs, screenshots, code references
|
|
19
|
+
- **Recommendations:** Suggested remediation
|
|
20
|
+
|
|
21
|
+
If the input deviates, normalize it to the above structure before generating the report.
|
|
22
|
+
|
|
23
|
+
### VARS-specific input requirement
|
|
24
|
+
|
|
25
|
+
The VARS Review Report must be generated from the **full** per-control assessment —
|
|
26
|
+
typically the `tracker.md` produced by the `compliance-audit` skill running against the
|
|
27
|
+
VARS checklist (`skills/compliance/compliance-audit/references/vars-checklist.md`, or a
|
|
28
|
+
parsed equivalent from `reg-parser`). Do **not** start a VARS report from a summary,
|
|
29
|
+
verbal findings, or a partial sub-domain review — Visa's report aggregates a complete
|
|
30
|
+
control-by-control assessment, and Section 5 of the deliverable explicitly references
|
|
31
|
+
the *VARS Control Testing and Remediation Tracking* spreadsheet (which is the same
|
|
32
|
+
tracker, in Visa's Excel format).
|
|
33
|
+
|
|
34
|
+
If the user asks for a VARS report and you don't see a complete tracker covering all
|
|
35
|
+
nine sub-domains, **stop and ask** for it — or offer to run `compliance-audit` first.
|
|
36
|
+
A VARS report without the underlying control-level evidence is not a deliverable Visa
|
|
37
|
+
will accept.
|
|
38
|
+
|
|
39
|
+
## Report Style Selection
|
|
40
|
+
|
|
41
|
+
| Style | Audience | When to use |
|
|
42
|
+
|---|---|---|
|
|
43
|
+
| **VARS Review Report** | Visa, acquirer leadership | Visa Acceptance Risk Standards engagements — uses Visa's official 2025 template |
|
|
44
|
+
| **Audit Report** | Compliance team, internal auditors | Day-to-day remediation tracking, technical deep-dives |
|
|
45
|
+
| **External Assessment** | Regulators, external auditors, certification bodies | Formal submissions, certification audits, regulatory filings |
|
|
46
|
+
| **Management Report** | C-suite, board, leadership | Quarterly reviews, risk committee briefings, budget justification |
|
|
47
|
+
|
|
48
|
+
Select the style based on the stated audience or purpose. If unclear, ask. Default to
|
|
49
|
+
Audit Report when the requestor is technical. **If the framework is VARS / Visa
|
|
50
|
+
Acceptance Risk Standards, always use the VARS Review Report style** — Visa requires
|
|
51
|
+
its template verbatim.
|
|
52
|
+
|
|
53
|
+
## Workflow
|
|
54
|
+
|
|
55
|
+
### 1. Ingest Findings
|
|
56
|
+
|
|
57
|
+
1. Read the audit findings file (Markdown). For VARS, this is the `compliance-audit`
|
|
58
|
+
tracker produced against the VARS checklist — it must cover all nine sub-domains.
|
|
59
|
+
2. Parse each finding into: ID, title, category, status, severity, observations, evidence, recommendations.
|
|
60
|
+
3. Validate completeness — flag any finding missing severity or status. For VARS,
|
|
61
|
+
additionally verify every sub-domain is represented; if any are missing, stop and
|
|
62
|
+
surface the gap to the user before drafting.
|
|
63
|
+
|
|
64
|
+
### 2. Select Style
|
|
65
|
+
|
|
66
|
+
Confirm the target style with the user or infer from context:
|
|
67
|
+
- "VARS" / "Visa Acceptance Risk Standards" / "for Visa" → **VARS Review Report**
|
|
68
|
+
- "for the regulator" → External Assessment
|
|
69
|
+
- "for the board" / "executive summary" → Management Report
|
|
70
|
+
- "full technical report" / default → Audit Report
|
|
71
|
+
|
|
72
|
+
### 3. Generate Report
|
|
73
|
+
|
|
74
|
+
Follow the style-specific template. Reference files provide the full structure:
|
|
75
|
+
|
|
76
|
+
- **VARS Review Report:** Read `references/vars-report-template.md` (Visa's official 2025 template)
|
|
77
|
+
- **Audit Report:** Read `references/audit-report-template.md`
|
|
78
|
+
- **External Assessment:** Read `references/external-assessment-template.md`
|
|
79
|
+
- **Management Report:** Read `references/management-report-template.md`
|
|
80
|
+
|
|
81
|
+
### 4. Review and Finalize
|
|
82
|
+
|
|
83
|
+
1. Verify coverage:
|
|
84
|
+
- Audit Report: every finding from the input has its own section.
|
|
85
|
+
- External Assessment: every finding is mapped to a regulatory requirement.
|
|
86
|
+
- Management Report / VARS: findings are aggregated by category/sub-domain with no category dropped.
|
|
87
|
+
2. Confirm severity counts match the input data. For VARS: Section 3.d is **executive**
|
|
88
|
+
(a headline plus a 2–3 sentence business posture per sub-domain, no operational
|
|
89
|
+
detail); Section 4 is **department-manager** detail, with "Compliance Status" carrying
|
|
90
|
+
the tracker's Control Effectiveness Rating (Satisfactory / Improvement Recommended /
|
|
91
|
+
Unsatisfactory / N/A; worst rating across the row) and "Findings and Comments" written
|
|
92
|
+
as an inline observed → finding → comments narrative in the report's own words (no
|
|
93
|
+
source-document/clause/sample citations — those stay in the tracker) with the Actions
|
|
94
|
+
Required / Recommendations reproduced verbatim. When the ATPA archetype is in scope,
|
|
95
|
+
confirm all TPA controls are consolidated into a single "Third Party Agent" row rather
|
|
96
|
+
than scattered across the AACQ sub-domains.
|
|
97
|
+
3. Check that no placeholder text remains (e.g. `{{...}}`, `____`, `MM/YEAR`).
|
|
98
|
+
4. Output as Markdown. If DOCX is requested (the VARS deliverable is typically distributed as
|
|
99
|
+
DOCX to match Visa's template), **do not rely on a plain `pandoc` Markdown→DOCX conversion
|
|
100
|
+
for the Section 4 table** — pandoc collapses the `<br>` line breaks inside the wide
|
|
101
|
+
Findings-and-Comments cells, so the enumerated Actions Required / Recommendations items run
|
|
102
|
+
together and appear to be missing. Instead render the DOCX so each cell's `<br>`-separated
|
|
103
|
+
lines become **separate paragraphs** (e.g. build it with `python-docx`, or post-process the
|
|
104
|
+
pandoc output). In Section 4 specifically:
|
|
105
|
+
- split every `<br>` into its own paragraph in the cell, so each Actions Required /
|
|
106
|
+
Recommendations item is on its own visible line;
|
|
107
|
+
- render `**…**` labels (`Actions Required:`, `Recommendations:`) as real **bold**, not
|
|
108
|
+
literal asterisks;
|
|
109
|
+
- shade the **Compliance Status** cell by rating (Satisfactory = green, Improvement
|
|
110
|
+
Recommended = amber, Unsatisfactory = red, N/A = grey);
|
|
111
|
+
- set the page to **landscape** to give the table room.
|
|
112
|
+
For PDF, the same line-break requirement applies.
|
|
113
|
+
|
|
114
|
+
---
|
|
115
|
+
|
|
116
|
+
## Style Summaries
|
|
117
|
+
|
|
118
|
+
### VARS Review Report
|
|
119
|
+
|
|
120
|
+
Visa's official 2025 template, used verbatim. Six fixed sections with mandated
|
|
121
|
+
boilerplate (Purpose paragraph, reliance statement, recommendation summary stem) and
|
|
122
|
+
nine fixed sub-domains in canonical order — plus a consolidated **Third Party Agent**
|
|
123
|
+
row when the ATPA archetype is in scope. Detailed per-control findings live in the
|
|
124
|
+
companion *VARS Control Testing and Remediation Tracking* Excel — the report itself
|
|
125
|
+
aggregates one row per sub-domain. The two summary layers serve different audiences:
|
|
126
|
+
**Section 3.d is executive** (business posture, 2–3 sentences per sub-domain) and
|
|
127
|
+
**Section 4 is for department managers / control owners** (an inline observed → finding
|
|
128
|
+
→ comments narrative in the report's own words, with verbatim Actions Required /
|
|
129
|
+
Recommendations and no duplication of the tracker's evidence citations).
|
|
130
|
+
|
|
131
|
+
**Key sections:** Cover, Document History, Distribution List, Executive Summary
|
|
132
|
+
(Purpose / Background / VARS Control Assessment / Summary of Findings / Recommendation
|
|
133
|
+
Summary), Observations Overview (sub-domain table), Detailed Information (pointer to
|
|
134
|
+
Excel tracker), Next Steps.
|
|
135
|
+
|
|
136
|
+
**Tone:** Formal, third-person, Visa-facing. Preserve the boilerplate paragraphs
|
|
137
|
+
verbatim — paraphrasing breaks reviewer expectations.
|
|
138
|
+
|
|
139
|
+
### Audit Report
|
|
140
|
+
|
|
141
|
+
Full technical detail. Every finding gets its own section with evidence, root cause analysis,
|
|
142
|
+
remediation steps, priority, and suggested timeline. Appendices contain raw evidence.
|
|
143
|
+
|
|
144
|
+
**Key sections:**
|
|
145
|
+
1. Report metadata (scope, period, assessor, framework)
|
|
146
|
+
2. Executive summary (pass/fail counts)
|
|
147
|
+
3. Findings (one per item — full detail)
|
|
148
|
+
4. Remediation plan (prioritized)
|
|
149
|
+
5. Appendices (evidence, methodology, glossary)
|
|
150
|
+
|
|
151
|
+
**Tone:** Technical, precise, neutral. Use passive voice sparingly. Reference specific
|
|
152
|
+
controls, configurations, and code paths.
|
|
153
|
+
|
|
154
|
+
### External Assessment
|
|
155
|
+
|
|
156
|
+
Formal language suitable for regulatory submission. Organized by regulatory requirement,
|
|
157
|
+
not by internal finding ID. Includes gap analysis matrix and attestation-ready statements.
|
|
158
|
+
|
|
159
|
+
**Key sections:**
|
|
160
|
+
1. Cover page and scope statement
|
|
161
|
+
2. Assessment methodology
|
|
162
|
+
3. Compliance status by requirement (article/section mapping)
|
|
163
|
+
4. Gap analysis matrix (requirement × status × severity)
|
|
164
|
+
5. Remediation roadmap
|
|
165
|
+
6. Attestation statement
|
|
166
|
+
7. Appendices (evidence index, assessor qualifications)
|
|
167
|
+
|
|
168
|
+
**Tone:** Formal, objective, structured. Reference regulation articles explicitly
|
|
169
|
+
(e.g., "Article 32(1)(a) of Regulation (EU) 2016/679"). Avoid colloquial language.
|
|
170
|
+
|
|
171
|
+
### Management Report
|
|
172
|
+
|
|
173
|
+
Concise, business-oriented. Leads with a one-page executive summary. Uses visual
|
|
174
|
+
indicators (counts, heatmap descriptions) rather than technical evidence.
|
|
175
|
+
|
|
176
|
+
**Key sections:**
|
|
177
|
+
1. Executive summary (1 page max — overall posture, key risks, headline numbers)
|
|
178
|
+
2. Risk heatmap (critical/major/minor counts by category)
|
|
179
|
+
3. Business impact analysis (what gaps mean operationally and financially)
|
|
180
|
+
4. Recommended actions with effort estimates (cost, time, owner)
|
|
181
|
+
5. Progress since last assessment (if prior data available)
|
|
182
|
+
6. Appendix (methodology note, detailed data reference)
|
|
183
|
+
|
|
184
|
+
**Tone:** Business language, action-oriented. Quantify impact where possible.
|
|
185
|
+
Avoid jargon — explain technical concepts in business terms.
|
|
186
|
+
|
|
187
|
+
---
|
|
188
|
+
|
|
189
|
+
## Output Conventions
|
|
190
|
+
|
|
191
|
+
- **File naming:** `{style}-{framework}-{YYYY-MM-DD}.md`
|
|
192
|
+
- Example: `audit-report-SOC2-2026-03-30.md`
|
|
193
|
+
- VARS example: `vars-review-report-{acquirer}-2026-03-30.md`
|
|
194
|
+
- **Severity color coding** (for Markdown rendering):
|
|
195
|
+
- 🔴 Critical | 🟠 Major | 🟡 Minor | 🔵 Observation
|
|
196
|
+
- **Finding IDs:** Preserve original IDs from the input. If absent, assign sequential IDs: `F-001`, `F-002`, etc.
|
|
197
|
+
- **Cross-references:** Link findings to remediation items and evidence appendices by ID.
|
|
198
|
+
|
|
199
|
+
## Edge Cases
|
|
200
|
+
|
|
201
|
+
- **No critical findings:** Lead the executive summary with overall compliance posture, not absence of criticals.
|
|
202
|
+
- **Mixed frameworks:** If findings span multiple frameworks (e.g., SOC 2 + PCI DSS), organize by framework first, then by requirement.
|
|
203
|
+
- **Incomplete evidence:** Flag as "Evidence Pending" — do not omit the finding.
|
|
204
|
+
- **Prior assessment data unavailable:** Omit the "Progress" section in Management Report; note "Baseline assessment — no prior data" in methodology.
|
|
205
|
+
- **VARS sub-domain with no findings:** Still include the row in Section 4 and the bullet in Section 3.d — Compliance Status "Satisfactory", Findings and Comments an inline observed-then-finding narrative closing with "No remediation required." (matching the tracker's Satisfactory Issue Description) — Visa expects all nine sub-domains to appear (plus the Third Party Agent row when ATPA is in scope).
|
|
206
|
+
- **VARS legacy numbering:** The official 2025 template skips index 9 in the Section 4 table (Regulatory Risk is labelled #10). The `references/vars-report-template.md` uses sequential 1–9; switch to legacy numbering only if the Visa engagement lead asks.
|
|
@@ -0,0 +1,121 @@
|
|
|
1
|
+
# Audit Report Template
|
|
2
|
+
|
|
3
|
+
Use this structure for the full technical audit report (audience: compliance team).
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## Report Metadata
|
|
8
|
+
|
|
9
|
+
| Field | Value |
|
|
10
|
+
|---|---|
|
|
11
|
+
| Report Title | {Framework} Compliance Audit Report |
|
|
12
|
+
| Assessment Period | {Start Date} – {End Date} |
|
|
13
|
+
| Report Date | {YYYY-MM-DD} |
|
|
14
|
+
| Assessor(s) | {Name(s), Role(s)} |
|
|
15
|
+
| Framework / Standard | {e.g., SOC 2 Type II, ISO 27001, PCI DSS v4.0} |
|
|
16
|
+
| Scope | {Systems, processes, and data in scope} |
|
|
17
|
+
| Classification | {Confidential / Internal / Restricted} |
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## 1. Executive Summary
|
|
22
|
+
|
|
23
|
+
Brief overview (1–2 paragraphs) of the audit scope, approach, and headline results.
|
|
24
|
+
|
|
25
|
+
### Finding Summary
|
|
26
|
+
|
|
27
|
+
| Severity | Count | % of Total |
|
|
28
|
+
|---|---|---|
|
|
29
|
+
| 🔴 Critical | {n} | {%} |
|
|
30
|
+
| 🟠 Major | {n} | {%} |
|
|
31
|
+
| 🟡 Minor | {n} | {%} |
|
|
32
|
+
| 🔵 Observation | {n} | {%} |
|
|
33
|
+
| **Total** | **{n}** | **100%** |
|
|
34
|
+
|
|
35
|
+
### Overall Compliance Posture
|
|
36
|
+
|
|
37
|
+
{One sentence: e.g., "The organization demonstrates a mature control environment with
|
|
38
|
+
two critical gaps requiring immediate remediation."}
|
|
39
|
+
|
|
40
|
+
---
|
|
41
|
+
|
|
42
|
+
## 2. Findings
|
|
43
|
+
|
|
44
|
+
Repeat this block for each finding, ordered by severity (Critical → Observation).
|
|
45
|
+
|
|
46
|
+
### F-{NNN}: {Finding Title}
|
|
47
|
+
|
|
48
|
+
| Field | Detail |
|
|
49
|
+
|---|---|
|
|
50
|
+
| **ID** | F-{NNN} |
|
|
51
|
+
| **Category** | {e.g., Access Control} |
|
|
52
|
+
| **Control Reference** | {e.g., CC6.1, A.9.2.3} |
|
|
53
|
+
| **Status** | {Pass / Fail / Partial} |
|
|
54
|
+
| **Severity** | {Critical / Major / Minor / Observation} |
|
|
55
|
+
|
|
56
|
+
**Observation:**
|
|
57
|
+
{Detailed description of what was found. Reference specific systems, configurations,
|
|
58
|
+
or processes. Be precise — include paths, parameter names, versions where relevant.}
|
|
59
|
+
|
|
60
|
+
**Evidence:**
|
|
61
|
+
{Reference evidence items: E-001, E-002. Describe what each evidence item shows.
|
|
62
|
+
Full evidence in Appendix A.}
|
|
63
|
+
|
|
64
|
+
**Root Cause:**
|
|
65
|
+
{Why the gap exists. Distinguish between process failure, technical misconfiguration,
|
|
66
|
+
resource constraint, or design gap.}
|
|
67
|
+
|
|
68
|
+
**Risk:**
|
|
69
|
+
{Impact if unaddressed. Quantify where possible: number of records exposed,
|
|
70
|
+
blast radius, regulatory penalty exposure.}
|
|
71
|
+
|
|
72
|
+
**Recommendation:**
|
|
73
|
+
{Specific remediation steps.}
|
|
74
|
+
|
|
75
|
+
| Priority | Timeline | Estimated Effort | Owner |
|
|
76
|
+
|---|---|---|---|
|
|
77
|
+
| {P1/P2/P3} | {e.g., 30 days} | {e.g., 2 engineer-weeks} | {Role/Team} |
|
|
78
|
+
|
|
79
|
+
---
|
|
80
|
+
|
|
81
|
+
## 3. Remediation Plan
|
|
82
|
+
|
|
83
|
+
Consolidated view of all findings requiring action, sorted by priority.
|
|
84
|
+
|
|
85
|
+
| ID | Title | Severity | Priority | Timeline | Owner | Status |
|
|
86
|
+
|---|---|---|---|---|---|---|
|
|
87
|
+
| F-001 | {Title} | 🔴 Critical | P1 | 14 days | {Owner} | Open |
|
|
88
|
+
| F-002 | {Title} | 🟠 Major | P2 | 30 days | {Owner} | Open |
|
|
89
|
+
| ... | | | | | | |
|
|
90
|
+
|
|
91
|
+
---
|
|
92
|
+
|
|
93
|
+
## Appendix A: Evidence Index
|
|
94
|
+
|
|
95
|
+
| Evidence ID | Description | Type | Source | Date Collected |
|
|
96
|
+
|---|---|---|---|---|
|
|
97
|
+
| E-001 | {Description} | {Screenshot / Log / Config / Interview} | {System/Person} | {Date} |
|
|
98
|
+
| E-002 | {Description} | {Type} | {Source} | {Date} |
|
|
99
|
+
|
|
100
|
+
{Include raw evidence below or attach as separate files.}
|
|
101
|
+
|
|
102
|
+
### E-001: {Description}
|
|
103
|
+
|
|
104
|
+
```
|
|
105
|
+
{Raw evidence: log excerpt, configuration snippet, or screenshot reference}
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
---
|
|
109
|
+
|
|
110
|
+
## Appendix B: Assessment Methodology
|
|
111
|
+
|
|
112
|
+
{Describe the audit approach: document review, technical testing, interviews,
|
|
113
|
+
sampling methodology, tools used.}
|
|
114
|
+
|
|
115
|
+
---
|
|
116
|
+
|
|
117
|
+
## Appendix C: Glossary
|
|
118
|
+
|
|
119
|
+
| Term | Definition |
|
|
120
|
+
|---|---|
|
|
121
|
+
| {Term} | {Definition} |
|