@1aboveio/skills 0.20.2 → 0.20.3

package/skills/compliance/compliance-audit/references/findings-schema.md

@@ -0,0 +1,290 @@
+# Assessment Tracker Schema (Generic)
+
+Output schema for non-VARS compliance audits (AML, KYC, financial reporting,
+local-regulator frameworks, etc.). The deliverable is a **tracker** — a
+worksheet with one row per Control ID — not a narrative findings report.
+
+For VARS audits use `vars-findings-schema.md` instead.
+
+## Output Files
+
+Two files per audit:
+
+1. `tracker.md` — the deliverable. One row per `Control ID`.
+2. `phase-trail.md` — audit work product. Per-Control-ID record showing what each phase (`document_review`, `onsite_review`, `sample_test`) found, preserved as-is for audit defensibility.
+
+The tracker is the artifact handed to the client; the phase trail is internal.
+
+## Tracker Columns
+
+| Column | Owner | Stage | Notes |
+|---|---|---|---|
+| Control ID | — | Checklist | e.g. `KYC.C3.2` |
+| Title | — | Checklist | Short control title |
+| Status | Reviewer | Final assessment | `PASS` / `FAIL` / `PARTIAL` / `NOT_APPLICABLE` |
+| Effectiveness | Reviewer | Final assessment | `Green` / `Amber` / `Red` / `N/A` |
+| Severity | Reviewer | Final assessment | From checklist if assigned, else derived (rubric below) |
+| Observation | Reviewer | Final assessment | One sentence — what was found |
+| Gap | Reviewer | Final assessment | One sentence — what's missing or weak. `—` for `PASS`. |
+| Recommendation | Reviewer | Final assessment | One-line remediation pointer or `Recommended Action Ref` |
+| Recommended Action Ref | Reviewer | Final assessment | Optional `best-practice://[Control ID]` link if a co-located repository exists |
+
+## Audit Metadata Header
+
+`tracker.md` opens with a metadata block:
+
+```markdown
+# Compliance Assessment Tracker
+
+- **Date:** YYYY-MM-DD
+- **Auditor:** [name]
+- **Scope:** full | targeted: [control groups]
+- **Checklist:** [checklist name / version / file path]
+- **System/Process Under Audit:** [description]
+- **Audit Phases Completed:** [document_review, onsite_review, sample_test]
+```
+
+No executive summary, no Critical & Major Gaps roll-up, no Recommendations Summary section. The tracker is filterable; aggregation happens off the tracker.
+
+## Tracker Format
+
+Single markdown table. One row per Control ID.
+
+```markdown
+| Control ID | Title | Status | Effectiveness | Severity | Observation | Gap | Recommendation | Recommended Action Ref |
+|------------|-------|--------|---------------|----------|-------------|-----|----------------|------------------------|
+| KYC.C3.2 | Sanctions Screening at Onboarding | PARTIAL | Amber | MAJOR | OFAC screening mandated in SOP section 4.3; vendor scope confirms OFAC + UN only. | EU sanctions list and local PEP list coverage absent from policy and vendor scope. | Extend vendor scope and revise SOP section 4.3 to include EU and local PEP lists. | — |
+| KYC.C5.1 | Beneficial Ownership Verification | FAIL | Red | CRITICAL | No procedure for verifying UBO above 25% threshold. | Policy silent on UBO verification entirely. | Adopt UBO verification SOP aligned with FATF guidance. | best-practice://KYC.C5.1 |
+```
+
+## Phase Trail Companion (`phase-trail.md`)
+
+Audit work product. One block per Control ID, preserves what each phase saw before the final tracker row was assembled.
+
+```markdown
+# Phase Trail
+
+### [Control ID]: [Control Title]
+
+- **Archetype:** [if applicable, else N/A]
+- **Requirement:** [short requirement statement]
+
+#### document_review
+- **Status:** PASS | FAIL | PARTIAL | NOT_APPLICABLE
+- **Control Effectiveness:** Green | Amber | Red | N/A
+- **Observation:** [What was found]
+- **Evidence:** [Documents reviewed]
+- **Gap:** [Missing or weak elements]
+- **Adjustment Note:** [How this phase changed the assessment, if applicable]
+
+#### onsite_review
+- **Status:** …
+- **Control Effectiveness:** …
+- **Observation:** …
+- **Evidence:** [Interview records / walkthrough evidence]
+- **Gap:** …
+- **Adjustment Note:** …
+
+#### sample_test
+- **Status:** …
+- **Control Effectiveness:** …
+- **Observation:** …
+- **Evidence:** [Sample IDs, transactions, screenshots]
+- **Gap:** …
+- **Adjustment Note:** …
+```
+
+Only include the phases that ran. Do not erase phase records when the tracker row is finalised.
+
+## Field Definitions
+
+### Status
+- `PASS` — evidence satisfies the control.
+- `FAIL` — evidence is missing, insufficient, or contradicts the requirement.
+- `PARTIAL` — some aspects met, gaps remain.
+- `NOT_APPLICABLE` — control does not apply to scope.
+
+### Severity
+
+If the source checklist assigns a severity per control, carry that value through unchanged. Otherwise derive from final Status × Effectiveness:
+
+| Status | Effectiveness | Severity |
+|--------|---------------|----------|
+| FAIL | Red | CRITICAL if regulatory / customer / systemic risk; otherwise MAJOR |
+| PARTIAL | Red | MAJOR |
+| PARTIAL | Amber | MINOR |
+| PASS | Amber | OBSERVATION |
+| PASS | Green | — (omit cell) |
+| NOT_APPLICABLE | N/A | — |
+
+CRITICAL is reserved for failures needing immediate action (regulator-facing, license-threatening, or active customer harm). When in doubt between CRITICAL and MAJOR, ask the user.
+
+This rubric does **not** apply to VARS audits — VARS uses `vars-findings-schema.md`.
+
+### Control Effectiveness
+- `Green` — designed appropriately and operating effectively.
+- `Amber` — exists but needs improvement in design or operation.
+- `Red` — absent, materially ineffective, or insufficient.
+- `N/A` — not applicable.
+
+### Observation
+- State what was found, not what should exist.
+- One sentence. Detail belongs in `phase-trail.md`.
+
+### Gap
+- One sentence describing the deficiency.
+- `—` (or empty) for `PASS`.
+
+### Recommendation / Recommended Action Ref
+- Inline a one-line remediation pointer in `Recommendation`.
+- If a co-located best-practice repository keyed by `Control ID` exists, fill `Recommended Action Ref` with `best-practice://[Control ID]`. Do not invent links.
+
+## Subagent Prompt Template
+
+Used by `SKILL.md` Step 6B (per-phase chunking) **and** Step 7 (synthesis chunking).
+
+### Variant A — Phase Review (Step 6B)
+
+Each subagent reviews one phase for one control group and outputs phase-trail blocks for those Control IDs.
+
+```text
+You are a compliance auditor. Review the following controls for the specified audit phase.
+
+Audit phase: <document_review | onsite_review | sample_test>
+Controls:
+<paste only the checklist rows for these Control IDs>
+
+Evidence:
+<paste mapped evidence only>
+
+Output format — one block per Control ID:
+
+### <Control ID>: <Control Title>
+- **Status:** PASS | FAIL | PARTIAL | NOT_APPLICABLE
+- **Control Effectiveness:** Green | Amber | Red | N/A
+- **Observation:** What you found in this phase.
+- **Evidence:** Exact artifacts reviewed (document name + section / file path / sample ID).
+- **Gap:** What remains missing or weak. Use "—" for PASS.
+- **Adjustment Note:** How this phase changes or confirms the prior view, if applicable.
+
+Rules:
+- Never mark PASS / Green without direct evidence.
+- If evidence is absent or ambiguous, mark the gap and explain why.
+- Output observations only.
+```
+
+Each subagent writes to `phase-<phase>-<group>.md`.
+
+### Variant B — Synthesis (Step 7)
+
+Each subagent receives the phase-trail blocks for one control group (across all completed phases) and emits **tracker rows** + the merged phase-trail block per Control ID.
+
+```text
+You are a compliance auditor consolidating an assessment.
+
+Controls in scope (this group):
+<paste only the checklist rows for these Control IDs>
+
+Phase trail blocks (all completed phases for these Control IDs):
+<paste the phase-trail blocks from phase-<phase>-<group>.md files>
+
+Produce two outputs.
+
+OUTPUT 1 — tracker rows (markdown table rows only, no header):
+One row per Control ID. Columns:
+| Control ID | Title | Status | Effectiveness | Severity | Observation | Gap | Recommendation | Recommended Action Ref |
+
+Rules:
+- Final Status reflects the most operationally-grounded phase available (sample_test > onsite_review > document_review).
+- Severity: carry from checklist if assigned; otherwise derive from Status × Effectiveness using the rubric in findings-schema.md.
+- Observation and Gap: one sentence each. Detail belongs in the merged phase-trail block, not the tracker cell.
+- Recommendation: one-line pointer. Use Recommended Action Ref column for `best-practice://` links only if a repository exists; do not invent links.
+
+OUTPUT 2 — merged phase-trail blocks (markdown):
+For each Control ID in this group, output the consolidated phase-trail block in the format defined in findings-schema.md. Preserve all phase records.
+
+Output OUTPUT 1 first, then a `---` separator, then OUTPUT 2. No commentary.
+```
+
+Each synthesis subagent writes to `synth-<group>.md` containing both outputs. The main agent splits on `---` and concatenates into `tracker.md` and `phase-trail.md` respectively.
+
+### Variant C — Citation Judge (Step 9C)
+
+Independent grounding pass — verifies that every coverage claim's citation actually addresses the requirement. Clean context.
+
+```text
+You are an evidence-grounding reviewer. You verify each coverage claim independently against the cited evidence.
+
+Controls in scope (this group):
+<paste checklist rows + requirement text>
+
+Tracker rows (Observation + Gap columns):
+<paste rows>
+
+Evidence excerpts:
+<paste cited clauses with context>
+
+For each Observation bullet citing a clause:
+1. Locate the cited clause.
+2. Compare to the Requirement.
+3. Emit a verdict per cited item:
+   - "Confirmed" — directly addresses the requirement.
+   - "Weak" — references the topic but does not operationalize.
+   - "Off-target" — does not address the requirement.
+   - "Missing" — cited clause cannot be located.
+
+Produce **two files**, both required so Step 9E can reconcile findings deterministically.
+
+`judge-<group>.md` — human-readable bullets, one per cited item under each Control ID.
+
+`judge-<group>.json` — machine-readable verdicts, consumed by `scripts/reconcile_tracker.py`:
+
+[
+  {"control_id": "KYC.C3.2", "item": "1", "verdict": "Confirmed", "reason": ""},
+  {"control_id": "KYC.C3.2", "item": "2", "verdict": "Off-target", "reason": "cited section X covers screening cadence, not threshold"}
+]
+
+Rules: use exactly one of Confirmed | Weak | Off-target | Missing. The `item` field is the bare item identifier from the checklist (no "Control " prefix). Skip "Not addressed" bullets — those are not coverage claims. No commentary outside the structured outputs.
+```
+
+Each judge subagent writes both files: `judge-<group>.md` (human) and `judge-<group>.json` (machine).
+
+### Variant D — Adversarial Challenger (Step 9D)
+
+Adversarial second pass — clean context, "assume over-claiming" framing.
+
+```text
+You are a senior challenge reviewer. Assume the prior auditor was too generous — they marked items PASS that should be PARTIAL or FAIL.
+
+Lens: bare references without operational procedure are weak; vendor contracts without scope are weak; cross-references count only if the target is present and substantive; design-only coverage without operating evidence is design-only.
+
+Controls in scope:
+<paste checklist rows + requirement>
+
+Tracker rows:
+<paste rows>
+
+Evidence excerpts:
+<paste cited clauses + context>
+
+For each PASS or PARTIAL row, re-examine the citation against the requirement. If you find a credible challenge, emit it; otherwise write "No challenge" for that control.
+
+Output format per Control ID:
+- **Challenge:** <specific weakness with requirement reference and recommended downgrade>
+
+Surface concrete, actionable downgrades only.
+```
+
+Each challenger subagent writes to `challenge-<group>.md`.
+
+## Notes for Auditors
+
+- **Evidence not yet produced is still PARTIAL.** When a clause is referenced but the readable exhibit hasn't yet been delivered (online policy URL with no attached PDF, signed copy expected but not provided, sample log promised next round), mark the row **PARTIAL** / Amber with a phase note in `phase-trail.md` saying "exhibit not produced — to be reviewed in next round". When the evidence arrives, run a fresh phase (another `document_review`, `onsite_review`, or `sample_test`) — synthesis produces an updated tracker. Do **not** invent a "Pending" status; the phase model is the temporal axis. The conservative rating treatment (PARTIAL lowers severity at minimum to MINOR) keeps fieldwork honest — outstanding exhibits don't quietly inflate a PASS.
+
+## Worked Example — Tracker Slice
+
+```markdown
+| Control ID | Title | Status | Effectiveness | Severity | Observation | Gap | Recommendation | Recommended Action Ref |
+|------------|-------|--------|---------------|----------|-------------|-----|----------------|------------------------|
+| KYC.C3.2 | Sanctions Screening at Onboarding | PARTIAL | Amber | MAJOR | OFAC screening mandated in SOP section 4.3; vendor scope confirms OFAC + UN only. | EU sanctions list and local PEP list coverage absent. | Extend vendor scope and revise SOP section 4.3 to add EU and local PEP lists. | — |
+```