@query-ai/digital-workers 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +27 -0
- package/.claude-plugin/plugin.json +11 -0
- package/README.md +430 -0
- package/hooks/hooks.json +16 -0
- package/hooks/run-hook.cmd +4 -0
- package/hooks/session-start +32 -0
- package/package.json +16 -0
- package/skills/alert-classifier/SKILL.md +111 -0
- package/skills/alert-investigation/SKILL.md +838 -0
- package/skills/detection-engineer/SKILL.md +170 -0
- package/skills/evidence-quality-checker/SKILL.md +109 -0
- package/skills/fsql-expert/SKILL.md +308 -0
- package/skills/fsql-expert/fsql-reference.md +525 -0
- package/skills/hunt-pattern-analyzer/SKILL.md +150 -0
- package/skills/hunt-quality-checker/SKILL.md +105 -0
- package/skills/hypothesis-builder/SKILL.md +303 -0
- package/skills/identity-investigator/SKILL.md +172 -0
- package/skills/itdr/SKILL.md +1178 -0
- package/skills/network-investigator/SKILL.md +196 -0
- package/skills/report-writer/SKILL.md +158 -0
- package/skills/senior-analyst-review/SKILL.md +199 -0
- package/skills/severity-scorer/SKILL.md +131 -0
- package/skills/templates/org-policy-template.md +516 -0
- package/skills/templates/runbook-template.md +300 -0
- package/skills/threat-hunt/SKILL.md +628 -0
- package/skills/threat-intel-enricher/SKILL.md +127 -0
- package/skills/using-digital-workers/SKILL.md +76 -0
|
@@ -0,0 +1,196 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: network-investigator
|
|
3
|
+
description: Use when investigation involves network activity, lateral movement, C2 communication, unusual traffic patterns, or network-based IOCs
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Network Investigator
|
|
7
|
+
|
|
8
|
+
## Iron Law
|
|
9
|
+
|
|
10
|
+
**MAP THE FULL COMMUNICATION PATTERN, NOT JUST THE SINGLE CONNECTION.**
|
|
11
|
+
|
|
12
|
+
A single suspicious connection means nothing without context. How often does it occur? What's the pattern? What other hosts are involved? Build the full picture.
|
|
13
|
+
|
|
14
|
+
## When to Invoke
|
|
15
|
+
|
|
16
|
+
Called by `alert-investigation` when the `alert-classifier` identifies:
|
|
17
|
+
- Alert type: Network/Lateral
|
|
18
|
+
- IOCs include: IP addresses, domains, ports, hostnames
|
|
19
|
+
- MITRE techniques: T1071 (Application Layer Protocol), T1572 (Protocol Tunneling), T1021 (Remote Services), T1570 (Lateral Tool Transfer)
|
|
20
|
+
|
|
21
|
+
## Investigation Process
|
|
22
|
+
|
|
23
|
+
Use `digital-workers:fsql-expert` for ALL queries below.
|
|
24
|
+
|
|
25
|
+
### Step 1: Map the Connection
|
|
26
|
+
|
|
27
|
+
From the alert, identify the primary connection:
|
|
28
|
+
- Source endpoint (IP, hostname, port)
|
|
29
|
+
- Destination endpoint (IP, hostname, port)
|
|
30
|
+
- Protocol and service
|
|
31
|
+
- Direction (inbound, outbound, internal)
|
|
32
|
+
- Timestamp and duration
|
|
33
|
+
|
|
34
|
+
### Step 2: Source Endpoint Activity
|
|
35
|
+
|
|
36
|
+
What else has the source endpoint been doing?
|
|
37
|
+
|
|
38
|
+
```
|
|
39
|
+
QUERY #network.src_endpoint.ip, #network.dst_endpoint.ip, #network.dst_endpoint.port,
|
|
40
|
+
#network.message, #network.time
|
|
41
|
+
WITH #network.src_endpoint.ip = '<source_ip>' AFTER 24h
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
Look for:
|
|
45
|
+
- **Volume**: How many connections from this host? Normal baseline?
|
|
46
|
+
- **Destinations**: How many unique destinations? Any known-bad?
|
|
47
|
+
- **Ports**: What destination ports? Any unusual (4444, 8080, 6666)?
|
|
48
|
+
- **Protocols**: Expected protocols for this host type?
|
|
49
|
+
- **Data volume**: Unusual bytes transferred?
|
|
50
|
+
|
|
51
|
+
**SUMMARIZE for network pattern analysis:**
|
|
52
|
+
|
|
53
|
+
Use SUMMARIZE to quantify network behavior before examining individual connections:
|
|
54
|
+
|
|
55
|
+
```
|
|
56
|
+
-- Outbound connection volume by source (scanning detection)
|
|
57
|
+
SUMMARIZE COUNT network_activity.message GROUP BY network_activity.src_endpoint.ip
|
|
58
|
+
WITH network_activity.src_endpoint.ip IN '<ip1>', '<ip2>' AFTER 7d
|
|
59
|
+
|
|
60
|
+
-- Port scan breadth (how many unique ports targeted?)
|
|
61
|
+
SUMMARIZE COUNT DISTINCT network_activity.dst_endpoint.port
|
|
62
|
+
WITH network_activity.src_endpoint.ip = '<ip>' AFTER 7d
|
|
63
|
+
|
|
64
|
+
-- Destination diversity (how many unique IPs contacted?)
|
|
65
|
+
SUMMARIZE COUNT DISTINCT network_activity.dst_endpoint.ip
|
|
66
|
+
WITH network_activity.src_endpoint.ip = '<ip>' AFTER 7d
|
|
67
|
+
|
|
68
|
+
-- Lateral movement scope (internal destinations per source)
|
|
69
|
+
SUMMARIZE COUNT DISTINCT network_activity.dst_endpoint.ip
|
|
70
|
+
GROUP BY network_activity.src_endpoint.ip
|
|
71
|
+
WITH network_activity.dst_endpoint.port IN 3389, 445, 22, 5985 AFTER 7d
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
SUMMARIZE answers "how much?" and "how broad?". QUERY answers "to where exactly?" and "when?".
|
|
75
|
+
|
|
76
|
+
> **Constraints:** SUMMARIZE has known execution limits — `status_id` filtering fails on detection_finding (use GROUP BY instead), `FROM` not supported, high-cardinality GROUP BY can overflow. If SUMMARIZE returns empty, fall back to QUERY. See fsql-expert Layer 1c for workarounds and check `summarize_support` in the environment profile.
|
|
77
|
+
|
|
78
|
+
### Step 3: Destination Analysis
|
|
79
|
+
|
|
80
|
+
Is the destination known-good, known-bad, or unknown?
|
|
81
|
+
|
|
82
|
+
```
|
|
83
|
+
QUERY #network.src_endpoint.ip, #network.dst_endpoint.ip, #network.dst_endpoint.port,
|
|
84
|
+
#network.message, #network.time
|
|
85
|
+
WITH #network.dst_endpoint.ip = '<dest_ip>' AFTER 7d
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
Look for:
|
|
89
|
+
- How many internal hosts connect to this destination?
|
|
90
|
+
- Is this a known service/infrastructure IP?
|
|
91
|
+
- First-seen: is this a new destination?
|
|
92
|
+
- Pass to `threat-intel-enricher` for reputation
|
|
93
|
+
|
|
94
|
+
### Step 4: Lateral Movement Detection
|
|
95
|
+
|
|
96
|
+
Search for the source endpoint connecting to internal systems:
|
|
97
|
+
|
|
98
|
+
```
|
|
99
|
+
QUERY #network.src_endpoint.ip, #network.dst_endpoint.ip, #network.dst_endpoint.port,
|
|
100
|
+
#network.message, #network.time
|
|
101
|
+
WITH #network.src_endpoint.ip = '<source_ip>'
|
|
102
|
+
AND #network.dst_endpoint.port IN 22, 135, 139, 445, 3389, 5985, 5986 AFTER 48h
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
These ports indicate remote access attempts (SSH, SMB, RDP, WinRM). Look for:
|
|
106
|
+
- Number of unique internal destinations
|
|
107
|
+
- Success vs. failure of connections
|
|
108
|
+
- Time pattern (scanning = rapid sequential connections)
|
|
109
|
+
- Whether the source is authorized for this access
|
|
110
|
+
|
|
111
|
+
### Step 5: DNS Activity
|
|
112
|
+
|
|
113
|
+
If domain names are involved:
|
|
114
|
+
|
|
115
|
+
```
|
|
116
|
+
QUERY dns_activity.message, dns_activity.time, dns_activity.query.hostname,
|
|
117
|
+
dns_activity.src_endpoint.ip
|
|
118
|
+
WITH dns_activity.query.hostname = '<domain>' AFTER 7d
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
Look for:
|
|
122
|
+
- Which internal hosts queried this domain?
|
|
123
|
+
- Query frequency (beaconing = regular intervals)
|
|
124
|
+
- DNS record types (TXT records may indicate tunneling)
|
|
125
|
+
- Resolution to known-bad IPs
|
|
126
|
+
|
|
127
|
+
### Step 6: Timeline Reconstruction
|
|
128
|
+
|
|
129
|
+
Build a chronological view of all network activity for the involved endpoints:
|
|
130
|
+
|
|
131
|
+
```
|
|
132
|
+
-- Layer 1a: Discover all event types for this IP
|
|
133
|
+
QUERY *.message, *.time WITH %ip = '<source_ip>' AFTER 48h
|
|
134
|
+
|
|
135
|
+
-- Layer 1b: Query specific event types found in Layer 1a with targeted fields
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
Order by timestamp. Identify:
|
|
139
|
+
- Initial compromise point
|
|
140
|
+
- Reconnaissance/scanning phase
|
|
141
|
+
- Lateral movement attempts
|
|
142
|
+
- Data staging/exfiltration
|
|
143
|
+
- C2 communication pattern (interval, data volume, protocol)
|
|
144
|
+
|
|
145
|
+
## C2 Beaconing Indicators
|
|
146
|
+
|
|
147
|
+
Look for these patterns in connection data:
|
|
148
|
+
- **Regular intervals**: Connections every 30s, 60s, 5m (with jitter)
|
|
149
|
+
- **Small data transfers**: < 1KB per connection (heartbeat)
|
|
150
|
+
- **Unusual protocols**: HTTP/HTTPS to non-standard ports
|
|
151
|
+
- **Long connections**: Persistent connections to external IPs
|
|
152
|
+
- **DNS anomalies**: High volume of DNS queries, TXT record lookups, long subdomain strings
|
|
153
|
+
|
|
154
|
+
## Output
|
|
155
|
+
|
|
156
|
+
```
|
|
157
|
+
NETWORK INVESTIGATION:
|
|
158
|
+
Primary Connection: [src] -> [dst] ([protocol]:[port])
|
|
159
|
+
Direction: [Inbound/Outbound/Internal]
|
|
160
|
+
|
|
161
|
+
Source Endpoint Profile:
|
|
162
|
+
Hostname: [name]
|
|
163
|
+
Total connections (24h): [count]
|
|
164
|
+
Unique destinations: [count]
|
|
165
|
+
Unusual ports: [list or "None"]
|
|
166
|
+
|
|
167
|
+
Destination Assessment:
|
|
168
|
+
IP/Domain: [value]
|
|
169
|
+
Reputation: [from threat-intel-enricher]
|
|
170
|
+
Internal hosts connecting: [count]
|
|
171
|
+
First seen: [date]
|
|
172
|
+
|
|
173
|
+
Lateral Movement: [Detected/Not detected]
|
|
174
|
+
Internal targets: [list of IPs/hostnames]
|
|
175
|
+
Ports targeted: [list]
|
|
176
|
+
Success rate: [X/Y attempts]
|
|
177
|
+
|
|
178
|
+
C2 Indicators: [Present/Not present]
|
|
179
|
+
Pattern: [description if present]
|
|
180
|
+
Interval: [if beaconing detected]
|
|
181
|
+
|
|
182
|
+
Timeline: [chronological summary]
|
|
183
|
+
|
|
184
|
+
IOCs Discovered: [new indicators]
|
|
185
|
+
Recommended Follow-Up: [additional queries or skills]
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
**Return this investigation output to the calling orchestrator and continue. Do not present to the user or wait for input — the orchestrator will incorporate findings into the evidence package.**
|
|
189
|
+
|
|
190
|
+
## Red Flags
|
|
191
|
+
|
|
192
|
+
| Red Flag | Correct Action |
|
|
193
|
+
|----------|---------------|
|
|
194
|
+
| "Just one connection to a suspicious IP" | Complete the pattern analysis. One connection may be the tip. |
|
|
195
|
+
| "Internal traffic, probably safe" | STOP. Lateral movement IS internal traffic. Investigate. |
|
|
196
|
+
| "No data on this destination" | That's a finding. Unknown destinations connecting to internal hosts is suspicious. Note as unknown, flag for enrichment. |
|
|
@@ -0,0 +1,158 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: report-writer
|
|
3
|
+
description: Use when investigation findings need to be formatted as a response-ready report — produces technical investigation with business summary, switchable to full executive format on request
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Report Writer
|
|
7
|
+
|
|
8
|
+
## Iron Law
|
|
9
|
+
|
|
10
|
+
**EVERY REPORT IS RESPONSE-READY. IF THE READER CAN'T ACT ON IT, REWRITE IT.**
|
|
11
|
+
|
|
12
|
+
The person reading this report should know exactly what happened, how confident we are, and what to do next — without needing to re-investigate.
|
|
13
|
+
|
|
14
|
+
## Recommendation Model — MANDATORY
|
|
15
|
+
|
|
16
|
+
**NEVER write "INCIDENT DECLARED" in any report.** The Digital Worker recommends — it does not declare incidents. Incident declaration has compliance and business implications that require human judgment.
|
|
17
|
+
|
|
18
|
+
Use these phrases:
|
|
19
|
+
- "Proposed Disposition: Critical Threat — Recommend Incident Escalation"
|
|
20
|
+
- "RECOMMEND ESCALATION" (not "INCIDENT DECLARED")
|
|
21
|
+
- "Evidence supports incident declaration — analyst action required"
|
|
22
|
+
- "All findings require human analyst validation before action"
|
|
23
|
+
|
|
24
|
+
The report header should use "RECOMMEND ESCALATION" or "Proposed Disposition: Critical Threat", never "INCIDENT DECLARED" or "Status: INCIDENT".
|
|
25
|
+
|
|
26
|
+
## When to Invoke
|
|
27
|
+
|
|
28
|
+
Called by `alert-investigation` at Gate 6 (Incident Notification) after the investigation is complete.
|
|
29
|
+
|
|
30
|
+
## Default Output: Technical Report with Business Summary
|
|
31
|
+
|
|
32
|
+
### Section 1: Business Summary (ALWAYS PRESENT)
|
|
33
|
+
|
|
34
|
+
Write in plain English. No jargon. No acronyms without explanation. A non-technical executive should understand this section completely.
|
|
35
|
+
|
|
36
|
+
Include:
|
|
37
|
+
- **What happened** — one sentence
|
|
38
|
+
- **Impact** — what systems, data, or operations are affected
|
|
39
|
+
- **Current status** — contained? under investigation? resolved?
|
|
40
|
+
- **Risk level** — plain language (e.g., "moderate risk to customer data")
|
|
41
|
+
- **What's being done** — immediate actions taken or recommended
|
|
42
|
+
- **What's needed** — any decisions or actions required from leadership
|
|
43
|
+
|
|
44
|
+
**Length:** 3-6 sentences. Concise. Direct.
|
|
45
|
+
|
|
46
|
+
### Section 2: Technical Investigation
|
|
47
|
+
|
|
48
|
+
#### Proposed Disposition
|
|
49
|
+
|
|
50
|
+
The disposition below is the worker's **recommended** disposition based on evidence gathered. The analyst confirms or overrides before the disposition is finalized.
|
|
51
|
+
|
|
52
|
+
- **Critical Threat** — confirmed malicious activity; recommend immediate response and incident escalation
|
|
53
|
+
- **Policy Violation** — real activity violating policy, not necessarily malicious
|
|
54
|
+
- **False Positive** — detection triggered incorrectly
|
|
55
|
+
- **Benign Activity** — real activity that is expected/authorized
|
|
56
|
+
|
|
57
|
+
#### Five W's
|
|
58
|
+
|
|
59
|
+
| Question | Answer | Confidence |
|
|
60
|
+
|----------|--------|------------|
|
|
61
|
+
| **Who** | [user, account, actor] | [Confirmed/High/Medium/Low] |
|
|
62
|
+
| **What** | [action or event] | [Confirmed/High/Medium/Low] |
|
|
63
|
+
| **When** | [timestamp range] | [Confirmed/High/Medium/Low] |
|
|
64
|
+
| **Where** | [systems, IPs, locations] | [Confirmed/High/Medium/Low] |
|
|
65
|
+
| **Why** | [assessed motive or cause] | [Confirmed/High/Medium/Low] |
|
|
66
|
+
|
|
67
|
+
#### MITRE ATT&CK Mapping
|
|
68
|
+
|
|
69
|
+
- **Tactic:** [tactic name]
|
|
70
|
+
- **Technique:** [technique name] ([technique ID])
|
|
71
|
+
- **Sub-technique:** [if applicable]
|
|
72
|
+
|
|
73
|
+
#### Indicators of Compromise
|
|
74
|
+
|
|
75
|
+
| IOC | Type | Reputation | Context |
|
|
76
|
+
|-----|------|-----------|---------|
|
|
77
|
+
| [value] | [IP/Hash/Domain/etc.] | [Known Malicious/Suspicious/Unknown/Clean] | [where found, significance] |
|
|
78
|
+
|
|
79
|
+
#### Evidence Chain
|
|
80
|
+
|
|
81
|
+
For each key finding, document:
|
|
82
|
+
1. **FSQL query executed** (exact query text)
|
|
83
|
+
2. **What the results showed** (summary)
|
|
84
|
+
3. **Conclusion drawn** (with confidence level)
|
|
85
|
+
4. **How this connects** to the overall investigation narrative
|
|
86
|
+
|
|
87
|
+
#### Correlated Alerts
|
|
88
|
+
|
|
89
|
+
List any related alerts discovered during investigation:
|
|
90
|
+
- Alert title, source, timestamp
|
|
91
|
+
- Relationship to primary alert
|
|
92
|
+
|
|
93
|
+
#### Recommended Next Steps
|
|
94
|
+
|
|
95
|
+
Specific, actionable items:
|
|
96
|
+
- For **Critical Threat**: Containment actions, escalation targets, notification requirements. Present incident criteria assessment — map evidence against criteria and recommend whether to formally declare an incident.
|
|
97
|
+
- For **Policy Violation**: Remediation steps, policy owner to notify
|
|
98
|
+
- For **False Positive**: Detection tuning recommendation, what to adjust
|
|
99
|
+
- For **Benign Activity**: Baseline update, suppression rule suggestion
|
|
100
|
+
|
|
101
|
+
Include customer operating procedures if a custom skill is loaded (e.g., `acme-escalation-policy`).
|
|
102
|
+
|
|
103
|
+
### Section 3: Show Your Work (On Request)
|
|
104
|
+
|
|
105
|
+
Only include when asked or when confidence is low on key findings:
|
|
106
|
+
- Complete list of FSQL queries executed with results
|
|
107
|
+
- Reasoning chain: how each finding led to the next query
|
|
108
|
+
- What was ruled out and why
|
|
109
|
+
- **Knowns**: facts supported by data
|
|
110
|
+
- **Unknowns**: what couldn't be determined (and why — data unavailable, insufficient time range, connector down)
|
|
111
|
+
- **Assumptions**: any inferences made without direct evidence (with rationale)
|
|
112
|
+
|
|
113
|
+
## Executive Report (On Request)
|
|
114
|
+
|
|
115
|
+
When asked to produce a business/executive version:
|
|
116
|
+
|
|
117
|
+
Restructure the entire report for a non-technical audience:
|
|
118
|
+
- Lead with business impact and risk
|
|
119
|
+
- Translate all technical findings to business language
|
|
120
|
+
- Remove FSQL queries, IOC tables, ATT&CK mappings
|
|
121
|
+
- Add: financial impact estimate (if applicable), regulatory implications, reputational risk
|
|
122
|
+
- Add: what decisions leadership needs to make
|
|
123
|
+
- Keep it to one page / one screen
|
|
124
|
+
|
|
125
|
+
## Org-Policy Integration
|
|
126
|
+
|
|
127
|
+
If an org-policy skill is loaded, check it for:
|
|
128
|
+
- **Report Format preferences** — which sections to include/exclude, whether to expand Show Your Work by default
|
|
129
|
+
- **Escalation Targets** — include org-specific notification routing in Recommended Next Steps
|
|
130
|
+
- **Custom escalation procedures** — reference org-specific runbooks or escalation paths
|
|
131
|
+
|
|
132
|
+
If no org-policy is loaded, use the default format documented above.
|
|
133
|
+
|
|
134
|
+
## Quality Checklist
|
|
135
|
+
|
|
136
|
+
Before finalizing any report, verify:
|
|
137
|
+
|
|
138
|
+
- [ ] Business summary is present and jargon-free
|
|
139
|
+
- [ ] Disposition is one of the four valid values
|
|
140
|
+
- [ ] All Five W's are answered (or explicitly marked as Unknown)
|
|
141
|
+
- [ ] Every conclusion cites specific evidence (query + result)
|
|
142
|
+
- [ ] Confidence level is stated for each finding
|
|
143
|
+
- [ ] IOCs are listed with type and reputation
|
|
144
|
+
- [ ] Next steps are specific and actionable
|
|
145
|
+
- [ ] Unknowns are explicitly stated
|
|
146
|
+
- [ ] Report is response-ready (reader can act without re-investigating)
|
|
147
|
+
|
|
148
|
+
**Return the completed report to the calling orchestrator. Do NOT present it directly to the analyst — the orchestrator will run senior-analyst-review first (if triggered) before presenting. Do not wait for user input.**
|
|
149
|
+
|
|
150
|
+
## Red Flags
|
|
151
|
+
|
|
152
|
+
| Red Flag | Correct Action |
|
|
153
|
+
|----------|---------------|
|
|
154
|
+
| "Investigation complete" (without business summary) | STOP. Business summary is mandatory on every report. |
|
|
155
|
+
| "Probably a false positive" (without evidence) | STOP. State the evidence that supports the disposition. |
|
|
156
|
+
| "Recommend further investigation" (as only next step) | STOP. Be specific. What should be investigated? Which queries? Which systems? |
|
|
157
|
+
| Writing technical jargon in the business summary | STOP. Rewrite. If your manager's manager can't understand it, simplify. |
|
|
158
|
+
| Skipping the confidence column | STOP. Every finding gets a confidence level. No exceptions. |
|
|
@@ -0,0 +1,199 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: senior-analyst-review
|
|
3
|
+
description: Use when a completed investigation needs quality review — checks evidence completeness, logic, missed indicators, severity calibration, and blind spots. Auto-triggered on HIGH/CRITICAL, manually invocable on any investigation.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Senior Analyst Review
|
|
7
|
+
|
|
8
|
+
## Iron Law
|
|
9
|
+
|
|
10
|
+
**REVIEW THE EVIDENCE, NOT THE CONCLUSION. Re-examine what was found and what was missed — then evaluate whether the disposition follows.**
|
|
11
|
+
|
|
12
|
+
## When to Invoke
|
|
13
|
+
|
|
14
|
+
- **Automatically** by `alert-investigation` after Gate 6 when the investigation's composite severity is >= 3.0 (DEEP) or the proposed disposition is Critical Threat
|
|
15
|
+
- **Manually** by analyst: `skill: "digital-workers:senior-analyst-review"`
|
|
16
|
+
- **Configurable** via org-policy skill (`review_trigger: all | high_critical | critical_only | manual`)
|
|
17
|
+
|
|
18
|
+
**Never** during an investigation. Always after the investigation is complete (post-Gate 6). The review must be independent of the investigation process.
|
|
19
|
+
|
|
20
|
+
## Evidence Sources
|
|
21
|
+
|
|
22
|
+
The investigation writes artifacts to the investigation directory (`docs/investigations/YYYY-MM-DD-<description>/`). Use the **Read tool** to review these files during the quality check:
|
|
23
|
+
|
|
24
|
+
| File | Use During |
|
|
25
|
+
|------|-----------|
|
|
26
|
+
| `queries.md` | Check 1 (evidence completeness) — were all relevant queries run? |
|
|
27
|
+
| `iocs.md` | Check 3 (missed indicators) — were all IOCs followed up? |
|
|
28
|
+
| `gate-1-intake.md` | Check 1 — what was the starting scope? |
|
|
29
|
+
| `gate-2-enrichment.md` | Check 1, 3 — what was enriched, what was found? |
|
|
30
|
+
| `gate-3-severity.md` | Check 4 (severity calibration) — are scores justified? |
|
|
31
|
+
| `gate-4-investigation.md` | Check 2 (logic) — does disposition follow from evidence? |
|
|
32
|
+
| `report.md` | Check 5, 6 — are recommendations actionable? Are blind spots documented? |
|
|
33
|
+
|
|
34
|
+
## The Nine Checks
|
|
35
|
+
|
|
36
|
+
### Check 1: Evidence Completeness
|
|
37
|
+
|
|
38
|
+
Were all relevant data sources queried?
|
|
39
|
+
|
|
40
|
+
- Cross-reference the connectors available in the mesh (via `FSQL_Connectors`) against the connectors actually queried during the investigation
|
|
41
|
+
- Were all extracted IOCs followed up? Every IOC from Gate 1 should have at least one enrichment query
|
|
42
|
+
- Are the Five W's fully populated? Each "W" answered or explicitly marked Unknown with a reason
|
|
43
|
+
- Were time ranges appropriate? Not too narrow (missed context), not too broad (noise)
|
|
44
|
+
- Were observable searches run? (`%ip`, `%hash`, `%email`, `%domain`) for key indicators
|
|
45
|
+
|
|
46
|
+
**Flag:** "Data source X is connected but was not queried — relevant because [reason]"
|
|
47
|
+
**Flag:** "IOC [value] was extracted in Gate 1 but has no follow-up query"
|
|
48
|
+
|
|
49
|
+
### Check 2: Logic Check
|
|
50
|
+
|
|
51
|
+
Does the proposed disposition follow from the evidence?
|
|
52
|
+
|
|
53
|
+
- For each conclusion in the report, can you trace it back to a specific query result?
|
|
54
|
+
- Are there unsupported leaps? (Conclusions stated without corresponding evidence)
|
|
55
|
+
- Is the confidence calibration justified? Each finding's confidence level should match evidence quality
|
|
56
|
+
- Were alternative explanations considered? (Could a Critical Threat be a Policy Violation? Could a False Positive be a True Positive with insufficient data?)
|
|
57
|
+
- Audit test: if this disposition were challenged in an incident review, would the evidence chain hold?
|
|
58
|
+
|
|
59
|
+
**Flag:** "Conclusion X is stated with HIGH confidence but the supporting query returned only [N] records"
|
|
60
|
+
**Flag:** "Alternative explanation not considered: [scenario]"
|
|
61
|
+
|
|
62
|
+
### Check 3: Missed Indicators
|
|
63
|
+
|
|
64
|
+
Are there IOCs in the evidence that weren't followed up?
|
|
65
|
+
|
|
66
|
+
Review all query results from the investigation:
|
|
67
|
+
- IPs that appeared in results but weren't searched across the mesh
|
|
68
|
+
- Usernames mentioned but no authentication pattern analysis run
|
|
69
|
+
- File hashes found but no threat intel lookup performed
|
|
70
|
+
- Domains referenced but no DNS activity check
|
|
71
|
+
- Pivot points: query results that contained new IOCs not extracted and pursued
|
|
72
|
+
|
|
73
|
+
**Flag:** "IP [value] appears in query 3 results but was not searched as an IOC"
|
|
74
|
+
**Flag:** "User [name] appears in correlated alerts but identity-investigator was not invoked"
|
|
75
|
+
|
|
76
|
+
### Check 4: Severity Calibration
|
|
77
|
+
|
|
78
|
+
Is the scoring justified by the evidence?
|
|
79
|
+
|
|
80
|
+
- Review each of the five dimension scores against the evidence
|
|
81
|
+
- Would different reasonable weights change the depth routing?
|
|
82
|
+
- Are override rules correctly applied (or correctly not applied)?
|
|
83
|
+
- Check for anchoring bias — was the score influenced by the source tool's severity label rather than contextual evidence?
|
|
84
|
+
- If an org-policy defines custom weights or crown jewels, were they applied?
|
|
85
|
+
|
|
86
|
+
**Flag:** "Asset Criticality scored 3/5 but the compromised host [name] appears in the org crown jewels list"
|
|
87
|
+
**Flag:** "Override condition 'lateral movement detected' is met by [evidence] but DEEP was not triggered"
|
|
88
|
+
|
|
89
|
+
### Check 5: Recommendations Check
|
|
90
|
+
|
|
91
|
+
Are the response actions specific and actionable?
|
|
92
|
+
|
|
93
|
+
- "Investigate further" is not actionable. "Query VPN logs for user X in the 24h before the alert" is.
|
|
94
|
+
- Are containment recommendations proportional to the threat?
|
|
95
|
+
- Are notification targets identified (or flagged as needing org-policy)?
|
|
96
|
+
- For Critical Threat: are incident criteria clearly mapped against evidence?
|
|
97
|
+
- Can the analyst act on every recommendation without additional research?
|
|
98
|
+
|
|
99
|
+
**Flag:** "Recommendation 'monitor the situation' lacks specificity — what to monitor, for how long, what triggers escalation?"
|
|
100
|
+
|
|
101
|
+
### Check 6: Blind Spots
|
|
102
|
+
|
|
103
|
+
What couldn't be determined — and was that called out?
|
|
104
|
+
|
|
105
|
+
- Were all unknowns explicitly documented in the report?
|
|
106
|
+
- Are there data sources that were unavailable? (Connector down, no data for time range, query returned zero results)
|
|
107
|
+
- Are there investigation paths not taken? Should they have been?
|
|
108
|
+
- Were critical gaps called out with impact assessment? ("No auth logs available — cannot verify MFA bypass" is good. Silently not checking auth is bad.)
|
|
109
|
+
|
|
110
|
+
**Flag:** "Query for DNS activity returned zero results but this gap is not mentioned in the report"
|
|
111
|
+
**Flag:** "Network flow data was not queried — if available, would show exfiltration volume"
|
|
112
|
+
|
|
113
|
+
### Check 7: Status Verification
|
|
114
|
+
|
|
115
|
+
Were all findings cited as evidence verified for `status_id`?
|
|
116
|
+
|
|
117
|
+
- Review every detection finding cited in the report. Check its `status_id` and `status_detail`.
|
|
118
|
+
- Are any RESOLVED/Benign findings cited as active threats?
|
|
119
|
+
- Does the alert count in the report include a status breakdown (NEW vs. RESOLVED vs. null)?
|
|
120
|
+
- Were `status_detail` values like "UnsupportedAlertType" interpreted correctly?
|
|
121
|
+
|
|
122
|
+
**Flag:** "Report cites [N] APT detections but [M] are RESOLVED/Benign — these do not support the narrative"
|
|
123
|
+
**Flag:** "Alert count of [N] has no status breakdown — [M] are resolved, only [K] are NEW"
|
|
124
|
+
|
|
125
|
+
### Check 8: Attribution Integrity
|
|
126
|
+
|
|
127
|
+
Is threat actor attribution supported by independently verified evidence?
|
|
128
|
+
|
|
129
|
+
- Is the attribution based on NEW/unresolved detections, or on RESOLVED/Benign vendor labels?
|
|
130
|
+
- Was the attribution independently verified via threat intel enrichment (hash lookups, IP reputation)?
|
|
131
|
+
- If threat intel enrichment was not possible (no hashes, no external IPs), is the confidence ceiling stated?
|
|
132
|
+
- Are vendor labels from the same platform counted as independent corroboration? (They shouldn't be.)
|
|
133
|
+
|
|
134
|
+
**Flag:** "APT28 attribution based entirely on vendor labels — [N] of [M] supporting detections are RESOLVED/Benign"
|
|
135
|
+
**Flag:** "Attribution at HIGH confidence but no independent threat intel verification was possible"
|
|
136
|
+
|
|
137
|
+
### Check 9: Specialist Invocation
|
|
138
|
+
|
|
139
|
+
If the investigation was routed to DEEP depth, were the appropriate specialist skills invoked?
|
|
140
|
+
|
|
141
|
+
- Cross-reference alert types from classification against specialist skills available
|
|
142
|
+
- Identity/Access alerts → was `identity-investigator` invoked?
|
|
143
|
+
- Network/Lateral alerts → was `network-investigator` invoked?
|
|
144
|
+
- If specialists were not invoked: is there a documented reason?
|
|
145
|
+
|
|
146
|
+
**Flag:** "Investigation routed DEEP but identity-investigator was never invoked despite [N] identity-type alerts"
|
|
147
|
+
**Flag:** "No specialist skills invoked on a DEEP investigation — Gate 4 specialist requirement was skipped"
|
|
148
|
+
|
|
149
|
+
## Output
|
|
150
|
+
|
|
151
|
+
```
|
|
152
|
+
SENIOR ANALYST REVIEW
|
|
153
|
+
━━━━━━━━━━━━━━━━━━━━
|
|
154
|
+
Investigation: [alert title / investigation ID]
|
|
155
|
+
Proposed Disposition: [disposition]
|
|
156
|
+
Review Verdict: ✅ APPROVED | ⚠️ GAPS IDENTIFIED
|
|
157
|
+
|
|
158
|
+
Evidence Completeness: [PASS | GAPS]
|
|
159
|
+
Logic Check: [PASS | CONCERNS]
|
|
160
|
+
Missed Indicators: [NONE FOUND | FOUND]
|
|
161
|
+
Severity Calibration: [APPROPRIATE | ADJUSTMENT SUGGESTED]
|
|
162
|
+
Recommendations: [ACTIONABLE | NEEDS IMPROVEMENT]
|
|
163
|
+
Blind Spots: [DOCUMENTED | UNDOCUMENTED GAPS]
|
|
164
|
+
Status Verification: [PASS | FINDINGS ON RESOLVED ALERTS]
|
|
165
|
+
Attribution Integrity: [PASS | UNSUPPORTED CLAIMS]
|
|
166
|
+
Specialist Invocation: [PASS | SKILLS NOT INVOKED]
|
|
167
|
+
|
|
168
|
+
[If GAPS IDENTIFIED:]
|
|
169
|
+
|
|
170
|
+
Issues Found:
|
|
171
|
+
1. [Specific issue with category and suggested action]
|
|
172
|
+
2. [Specific issue with category and suggested action]
|
|
173
|
+
|
|
174
|
+
Suggested Follow-Up Queries:
|
|
175
|
+
1. [Specific FSQL query or investigation action]
|
|
176
|
+
2. [Specific FSQL query or investigation action]
|
|
177
|
+
|
|
178
|
+
[If APPROVED:]
|
|
179
|
+
|
|
180
|
+
Investigation is thorough and the proposed disposition is well-supported
|
|
181
|
+
by the evidence. Ready for analyst review.
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
## What Happens After Review
|
|
185
|
+
|
|
186
|
+
**Return the review output to the calling orchestrator immediately. Do not present review results directly to the user or wait for input.**
|
|
187
|
+
|
|
188
|
+
- **APPROVED**: The orchestrator proceeds immediately to analyst presentation (Gate 6 Step 4).
|
|
189
|
+
- **GAPS IDENTIFIED**: The orchestrator runs the suggested follow-up queries, updates the evidence package and Five W's, regenerates the report, and re-submits for review. Maximum 2 review cycles — if gaps persist after 2 cycles, present the investigation with the review notes attached so the analyst sees both the findings and the reviewer's concerns.
|
|
190
|
+
|
|
191
|
+
## Red Flags
|
|
192
|
+
|
|
193
|
+
| Red Flag | Correct Action |
|
|
194
|
+
|----------|---------------|
|
|
195
|
+
| "The investigation looks fine, approved" without running all nine checks | STOP. Run every check. A cursory review is worse than no review. |
|
|
196
|
+
| Changing the disposition directly | STOP. The reviewer flags concerns and suggests — it doesn't override. The analyst decides. |
|
|
197
|
+
| Reviewing while the investigation is still in progress | STOP. Review runs only on completed investigations. Independence matters. |
|
|
198
|
+
| "No gaps found" on a 2-query investigation | STOP. A DEEP investigation with only 2 queries almost certainly has evidence gaps. |
|
|
199
|
+
| Skipping Check 3 (Missed Indicators) because "the disposition seems right" | STOP. Missed indicators are how real threats slip through correct-seeming investigations. |
|
|
@@ -0,0 +1,131 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: severity-scorer
|
|
3
|
+
description: Use when an alert needs multi-factor risk scoring to determine investigation depth — combines raw severity with asset criticality, business impact, and confidence
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Severity Scorer
|
|
7
|
+
|
|
8
|
+
## Iron Law
|
|
9
|
+
|
|
10
|
+
**RAW SEVERITY IS NOT RISK. CONTEXT DETERMINES DEPTH.**
|
|
11
|
+
|
|
12
|
+
A Critical alert on a dev test server may be lower risk than a Medium alert on a production database holding customer PII. Always score contextually.
|
|
13
|
+
|
|
14
|
+
## When to Invoke
|
|
15
|
+
|
|
16
|
+
Called by `alert-investigation` at Gate 3 (Analyze Situation) after enrichment is complete.
|
|
17
|
+
|
|
18
|
+
## Org-Policy Integration
|
|
19
|
+
|
|
20
|
+
If an org-policy skill is loaded, check it for:
|
|
21
|
+
- **Custom dimension weights** (Severity Weights section) — use instead of defaults
|
|
22
|
+
- **Custom depth routing thresholds** — use instead of 1.0-1.9 / 2.0-2.9 / 3.0-5.0
|
|
23
|
+
- **Crown jewels list** (Crown Jewels section) — use for Asset Criticality scoring instead of inferring
|
|
24
|
+
- **Additional override rules** — add to the built-in override list
|
|
25
|
+
|
|
26
|
+
If no org-policy is loaded, use all defaults documented below.
|
|
27
|
+
|
|
28
|
+
## Scoring Dimensions
|
|
29
|
+
|
|
30
|
+
Score each dimension 1-5, then compute composite:
|
|
31
|
+
|
|
32
|
+
### 1. Alert Severity (from source tool)
|
|
33
|
+
|
|
34
|
+
| Score | Level | Description |
|
|
35
|
+
|-------|-------|-------------|
|
|
36
|
+
| 5 | FATAL/CRITICAL | Highest severity from detection tool |
|
|
37
|
+
| 4 | HIGH | Significant threat indicated |
|
|
38
|
+
| 3 | MEDIUM | Moderate concern |
|
|
39
|
+
| 2 | LOW | Minor or informational |
|
|
40
|
+
| 1 | INFO | Informational only |
|
|
41
|
+
|
|
42
|
+
### 2. Asset Criticality
|
|
43
|
+
|
|
44
|
+
| Score | Level | Indicators |
|
|
45
|
+
|-------|-------|-----------|
|
|
46
|
+
| 5 | Crown Jewel | Production customer data, financial systems, domain controllers, CA servers |
|
|
47
|
+
| 4 | High Value | Production application servers, email systems, VPN gateways |
|
|
48
|
+
| 3 | Standard | Standard workstations, internal applications |
|
|
49
|
+
| 2 | Low Value | Dev/test systems, sandbox environments |
|
|
50
|
+
| 1 | Minimal | Decommissioned, isolated lab systems |
|
|
51
|
+
|
|
52
|
+
If an org-policy skill defines a Crown Jewels list, use it for scoring. If a separate `crown-jewels` skill is loaded, use that. Otherwise, infer from hostname, IP range, and system role.
|
|
53
|
+
|
|
54
|
+
### 3. Business Impact (Potential)
|
|
55
|
+
|
|
56
|
+
| Score | Level | Description |
|
|
57
|
+
|-------|-------|-------------|
|
|
58
|
+
| 5 | Severe | PII/PCI exposure, regulatory notification required, major service outage |
|
|
59
|
+
| 4 | High | Internal sensitive data at risk, partial service impact |
|
|
60
|
+
| 3 | Moderate | Business process disruption, limited data exposure |
|
|
61
|
+
| 2 | Low | Minimal operational impact, no data exposure |
|
|
62
|
+
| 1 | Negligible | No measurable business impact |
|
|
63
|
+
|
|
64
|
+
### 4. Confidence
|
|
65
|
+
|
|
66
|
+
| Score | Level | Description |
|
|
67
|
+
|-------|-------|-------------|
|
|
68
|
+
| 5 | Confirmed | Multiple corroborating data sources, clear evidence |
|
|
69
|
+
| 4 | High | Strong single-source evidence, consistent with known patterns |
|
|
70
|
+
| 3 | Medium | Plausible but incomplete evidence, some ambiguity |
|
|
71
|
+
| 2 | Low | Weak evidence, high chance of false positive |
|
|
72
|
+
| 1 | Minimal | Single low-fidelity signal, no corroboration |
|
|
73
|
+
|
|
74
|
+
**Anchoring guard**: If the primary evidence for scoring Confidence >= 4 (High) is vendor detection labels alone (e.g., "YTTRIUM malicious file detected", "Dukozy malware"), cap Confidence at 3 (Medium). Vendor labels from the same platform are not independent corroboration — five detections from SecLake are one source, not five. To score Confidence >= 4, require at least one of:
|
|
75
|
+
- File hash confirmed malicious via independent threat intel
|
|
76
|
+
- Behavioral correlation across two or more independent tools
|
|
77
|
+
- Network IOC (IP/domain) confirmed malicious via independent reputation service
|
|
78
|
+
- Manual analyst confirmation
|
|
79
|
+
|
|
80
|
+
### 5. Threat Context
|
|
81
|
+
|
|
82
|
+
| Score | Level | Description |
|
|
83
|
+
|-------|-------|-------------|
|
|
84
|
+
| 5 | Active Campaign | Matches known active threat campaign targeting this industry |
|
|
85
|
+
| 4 | Known APT TTP | Technique matches known advanced persistent threat |
|
|
86
|
+
| 3 | Known Technique | Common attack technique, well-understood |
|
|
87
|
+
| 2 | Generic | Commodity scanning, automated probing |
|
|
88
|
+
| 1 | Unknown | No threat intelligence correlation |
|
|
89
|
+
|
|
90
|
+
## Composite Score and Depth Routing
|
|
91
|
+
|
|
92
|
+
**Composite = (Alert Severity x 0.20) + (Asset Criticality x 0.25) + (Business Impact x 0.25) + (Confidence x 0.15) + (Threat Context x 0.15)**
|
|
93
|
+
|
|
94
|
+
| Composite Score | Depth | Action |
|
|
95
|
+
|----------------|-------|--------|
|
|
96
|
+
| **1.0 - 1.9** | AUTO-CLOSE | Document disposition, update FP patterns |
|
|
97
|
+
| **2.0 - 2.9** | STANDARD | Five W's, basic enrichment, verdict, report |
|
|
98
|
+
| **3.0 - 5.0** | DEEP | Full timeline, multi-source correlation, IOC analysis, ATT&CK mapping |
|
|
99
|
+
|
|
100
|
+
**Override rules (always DEEP regardless of score):**
|
|
101
|
+
- Lateral movement detected
|
|
102
|
+
- Persistence mechanism discovered
|
|
103
|
+
- Credential compromise indicators
|
|
104
|
+
- Data exfiltration signals
|
|
105
|
+
- Multiple correlated alerts from same actor/campaign
|
|
106
|
+
- Novel or previously unseen technique
|
|
107
|
+
|
|
108
|
+
## Output
|
|
109
|
+
|
|
110
|
+
```
|
|
111
|
+
SEVERITY ASSESSMENT:
|
|
112
|
+
Alert Severity: [score] — [rationale]
|
|
113
|
+
Asset Criticality: [score] — [rationale]
|
|
114
|
+
Business Impact: [score] — [rationale]
|
|
115
|
+
Confidence: [score] — [rationale]
|
|
116
|
+
Threat Context: [score] — [rationale]
|
|
117
|
+
|
|
118
|
+
Composite Score: [X.X]
|
|
119
|
+
Investigation Depth: [AUTO-CLOSE | STANDARD | DEEP]
|
|
120
|
+
Override Applied: [Yes/No — reason if yes]
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
**Return this severity assessment to the calling orchestrator and continue. Do not present to the user or wait for input.**
|
|
124
|
+
|
|
125
|
+
## Red Flags
|
|
126
|
+
|
|
127
|
+
| Red Flag | Correct Action |
|
|
128
|
+
|----------|---------------|
|
|
129
|
+
| "I don't know the asset criticality, so I'll assume low" | STOP. Query the mesh for the asset. Check hostname patterns, IP ranges. Note as Unknown, not Low. |
|
|
130
|
+
| "This is clearly a false positive" | STOP. Score it. If the score says auto-close, then auto-close. Don't prejudge. |
|
|
131
|
+
| "Critical severity = deep investigation always" | STOP. Critical on a test box with low confidence may be standard depth. Score all dimensions. |
|