@vantagesec/socc 0.1.13 → 0.1.15
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +8 -8
- package/dist/cli.mjs +10734 -10807
- package/package.json +7 -8
- package/scripts/bootstrap-socc-soul.mjs +23 -1
- package/socc-canonical/.agents/generated/socc-agent-manifest.json +0 -231
- package/socc-canonical/.agents/generated/socc-agent.md +0 -256
- package/socc-canonical/.agents/rules/AGENT.md +0 -109
- package/socc-canonical/.agents/rules/AQL_REFERENCE.md +0 -40
- package/socc-canonical/.agents/rules/MEMORY.md +0 -19
- package/socc-canonical/.agents/rules/TOOLS.md +0 -48
- package/socc-canonical/.agents/soc-copilot/AGENTS.md +0 -28
- package/socc-canonical/.agents/soc-copilot/MEMORY.md +0 -20
- package/socc-canonical/.agents/soc-copilot/SKILL.md +0 -51
- package/socc-canonical/.agents/soc-copilot/SOUL.md +0 -31
- package/socc-canonical/.agents/soc-copilot/TOOLS.md +0 -33
- package/socc-canonical/.agents/soc-copilot/USER.md +0 -31
- package/socc-canonical/.agents/soc-copilot/identity.md +0 -7
- package/socc-canonical/.agents/soc-copilot/references/evidence-rules.md +0 -30
- package/socc-canonical/.agents/soc-copilot/references/intelligence-source-registry.md +0 -32
- package/socc-canonical/.agents/soc-copilot/references/ioc-extraction.md +0 -25
- package/socc-canonical/.agents/soc-copilot/references/knowledge-ingestion-policy.md +0 -34
- package/socc-canonical/.agents/soc-copilot/references/mitre-guidance.md +0 -21
- package/socc-canonical/.agents/soc-copilot/references/output-contract.md +0 -31
- package/socc-canonical/.agents/soc-copilot/references/security-json-patterns.md +0 -129
- package/socc-canonical/.agents/soc-copilot/references/telemetry-investigation-patterns.md +0 -39
- package/socc-canonical/.agents/soc-copilot/schemas/analysis_response.json +0 -119
- package/socc-canonical/.agents/soc-copilot/skills/code-review-excellence/SKILL.md +0 -538
- package/socc-canonical/.agents/soc-copilot/skills/cybersecurity-analyst/QUICK_REFERENCE.md +0 -263
- package/socc-canonical/.agents/soc-copilot/skills/cybersecurity-analyst/README.md +0 -243
- package/socc-canonical/.agents/soc-copilot/skills/cybersecurity-analyst/SKILL.md +0 -1707
- package/socc-canonical/.agents/soc-copilot/skills/cybersecurity-analyst/tests/quiz.md +0 -472
- package/socc-canonical/.agents/soc-copilot/skills/data-visualization/SKILL.md +0 -304
- package/socc-canonical/.agents/soc-copilot/skills/deep-research/SKILL.md +0 -192
- package/socc-canonical/.agents/soc-copilot/skills/excel-analysis/SKILL.md +0 -247
- package/socc-canonical/.agents/soc-copilot/skills/find-skills/SKILL.md +0 -133
- package/socc-canonical/.agents/soc-copilot/skills/humanizer/README.md +0 -120
- package/socc-canonical/.agents/soc-copilot/skills/humanizer/SKILL.md +0 -439
- package/socc-canonical/.agents/soc-copilot/skills/malware-behavior/SKILL.md +0 -54
- package/socc-canonical/.agents/soc-copilot/skills/mitre/SKILL.md +0 -200
- package/socc-canonical/.agents/soc-copilot/skills/observability-logs-search/SKILL.md +0 -237
- package/socc-canonical/.agents/soc-copilot/skills/observability-logs-search/references/log-search-reference.md +0 -76
- package/socc-canonical/.agents/soc-copilot/skills/payload-triage/SKILL.md +0 -53
- package/socc-canonical/.agents/soc-copilot/skills/phishing-analysis/SKILL.md +0 -51
- package/socc-canonical/.agents/soc-copilot/skills/prd/SKILL.md +0 -143
- package/socc-canonical/.agents/soc-copilot/skills/remembering-conversations/MCP-TOOLS.md +0 -137
- package/socc-canonical/.agents/soc-copilot/skills/remembering-conversations/SKILL.md +0 -65
- package/socc-canonical/.agents/soc-copilot/skills/sequential-thinking/README.md +0 -118
- package/socc-canonical/.agents/soc-copilot/skills/sequential-thinking/SKILL.md +0 -93
- package/socc-canonical/.agents/soc-copilot/skills/sequential-thinking/references/advanced.md +0 -122
- package/socc-canonical/.agents/soc-copilot/skills/sequential-thinking/references/examples.md +0 -274
- package/socc-canonical/.agents/soc-copilot/skills/soc-generalist/SKILL.md +0 -53
- package/socc-canonical/.agents/soc-copilot/skills/suspicious-url/SKILL.md +0 -51
- package/socc-canonical/.agents/soc-copilot/skills/systematic-debugging/CREATION-LOG.md +0 -119
- package/socc-canonical/.agents/soc-copilot/skills/systematic-debugging/SKILL.md +0 -296
- package/socc-canonical/.agents/soc-copilot/skills/systematic-debugging/condition-based-waiting-example.ts +0 -158
- package/socc-canonical/.agents/soc-copilot/skills/systematic-debugging/condition-based-waiting.md +0 -115
- package/socc-canonical/.agents/soc-copilot/skills/systematic-debugging/defense-in-depth.md +0 -122
- package/socc-canonical/.agents/soc-copilot/skills/systematic-debugging/find-polluter.sh +0 -63
- package/socc-canonical/.agents/soc-copilot/skills/systematic-debugging/root-cause-tracing.md +0 -169
- package/socc-canonical/.agents/soc-copilot/skills/systematic-debugging/test-academic.md +0 -14
- package/socc-canonical/.agents/soc-copilot/skills/systematic-debugging/test-pressure-1.md +0 -58
- package/socc-canonical/.agents/soc-copilot/skills/systematic-debugging/test-pressure-2.md +0 -68
- package/socc-canonical/.agents/soc-copilot/skills/systematic-debugging/test-pressure-3.md +0 -69
- package/socc-canonical/.agents/soc-copilot/skills/translation-expertise/SKILL.md +0 -284
- package/socc-canonical/.agents/soc-copilot/skills/translation-expertise/chinese-traditional.md +0 -535
- package/socc-canonical/.agents/soc-copilot/skills/translation-expertise/english.md +0 -372
- package/socc-canonical/.agents/soc-copilot/skills/translation-expertise/japanese.md +0 -515
- package/socc-canonical/.agents/soc-copilot/skills/translation-expertise/tools-resources.md +0 -527
- package/socc-canonical/.agents/soc-copilot/skills/translation-expertise/translation-challenges.md +0 -603
- package/socc-canonical/.agents/soc-copilot/skills/web-search/SKILL.md +0 -322
- package/socc-canonical/.agents/soc-copilot/skills.md +0 -21
- package/socc-canonical/.agents/workflows/SOP.md +0 -137
- package/socc-canonical/README.md +0 -10
- /package/{.claude → .socc}/agents/socc.md +0 -0
- /package/{.claude → .socc}/references/evidence-rules.md +0 -0
- /package/{.claude → .socc}/references/intelligence-source-registry.md +0 -0
- /package/{.claude → .socc}/references/ioc-extraction.md +0 -0
- /package/{.claude → .socc}/references/knowledge-ingestion-policy.md +0 -0
- /package/{.claude → .socc}/references/mitre-guidance.md +0 -0
- /package/{.claude → .socc}/references/output-contract.md +0 -0
- /package/{.claude → .socc}/references/security-json-patterns.md +0 -0
- /package/{.claude → .socc}/references/telemetry-investigation-patterns.md +0 -0
- /package/{.claude → .socc}/rules/socc-business-rules.md +0 -0
- /package/{.claude → .socc}/skills/code-review-excellence/SKILL.md +0 -0
- /package/{.claude → .socc}/skills/cybersecurity-analyst/QUICK_REFERENCE.md +0 -0
- /package/{.claude → .socc}/skills/cybersecurity-analyst/README.md +0 -0
- /package/{.claude → .socc}/skills/cybersecurity-analyst/SKILL.md +0 -0
- /package/{.claude → .socc}/skills/cybersecurity-analyst/tests/quiz.md +0 -0
- /package/{.claude → .socc}/skills/data-visualization/SKILL.md +0 -0
- /package/{.claude → .socc}/skills/deep-research/SKILL.md +0 -0
- /package/{.claude → .socc}/skills/excel-analysis/SKILL.md +0 -0
- /package/{.claude → .socc}/skills/find-skills/SKILL.md +0 -0
- /package/{.claude → .socc}/skills/humanizer/README.md +0 -0
- /package/{.claude → .socc}/skills/humanizer/SKILL.md +0 -0
- /package/{.claude → .socc}/skills/malware-behavior/SKILL.md +0 -0
- /package/{.claude → .socc}/skills/mitre/SKILL.md +0 -0
- /package/{.claude → .socc}/skills/observability-logs-search/SKILL.md +0 -0
- /package/{.claude → .socc}/skills/observability-logs-search/references/log-search-reference.md +0 -0
- /package/{.claude → .socc}/skills/payload-triage/SKILL.md +0 -0
- /package/{.claude → .socc}/skills/phishing-analysis/SKILL.md +0 -0
- /package/{.claude → .socc}/skills/prd/SKILL.md +0 -0
- /package/{.claude → .socc}/skills/remembering-conversations/MCP-TOOLS.md +0 -0
- /package/{.claude → .socc}/skills/remembering-conversations/SKILL.md +0 -0
- /package/{.claude → .socc}/skills/sequential-thinking/README.md +0 -0
- /package/{.claude → .socc}/skills/sequential-thinking/SKILL.md +0 -0
- /package/{.claude → .socc}/skills/sequential-thinking/references/advanced.md +0 -0
- /package/{.claude → .socc}/skills/sequential-thinking/references/examples.md +0 -0
- /package/{.claude → .socc}/skills/soc-generalist/SKILL.md +0 -0
- /package/{.claude → .socc}/skills/suspicious-url/SKILL.md +0 -0
- /package/{.claude → .socc}/skills/systematic-debugging/CREATION-LOG.md +0 -0
- /package/{.claude → .socc}/skills/systematic-debugging/SKILL.md +0 -0
- /package/{.claude → .socc}/skills/systematic-debugging/condition-based-waiting-example.ts +0 -0
- /package/{.claude → .socc}/skills/systematic-debugging/condition-based-waiting.md +0 -0
- /package/{.claude → .socc}/skills/systematic-debugging/defense-in-depth.md +0 -0
- /package/{.claude → .socc}/skills/systematic-debugging/find-polluter.sh +0 -0
- /package/{.claude → .socc}/skills/systematic-debugging/root-cause-tracing.md +0 -0
- /package/{.claude → .socc}/skills/systematic-debugging/test-academic.md +0 -0
- /package/{.claude → .socc}/skills/systematic-debugging/test-pressure-1.md +0 -0
- /package/{.claude → .socc}/skills/systematic-debugging/test-pressure-2.md +0 -0
- /package/{.claude → .socc}/skills/systematic-debugging/test-pressure-3.md +0 -0
- /package/{.claude → .socc}/skills/translation-expertise/SKILL.md +0 -0
- /package/{.claude → .socc}/skills/translation-expertise/chinese-traditional.md +0 -0
- /package/{.claude → .socc}/skills/translation-expertise/english.md +0 -0
- /package/{.claude → .socc}/skills/translation-expertise/japanese.md +0 -0
- /package/{.claude → .socc}/skills/translation-expertise/tools-resources.md +0 -0
- /package/{.claude → .socc}/skills/translation-expertise/translation-challenges.md +0 -0
- /package/{.claude → .socc}/skills/web-search/SKILL.md +0 -0
|
@@ -1,237 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: observability-logs-search
|
|
3
|
-
description: >
|
|
4
|
-
Search and filter Observability logs using ES|QL. Use when investigating log spikes,
|
|
5
|
-
errors, or anomalies; getting volume and trends; or drilling into services or containers
|
|
6
|
-
during incidents.
|
|
7
|
-
metadata:
|
|
8
|
-
author: elastic
|
|
9
|
-
version: 0.2.0
|
|
10
|
-
---
|
|
11
|
-
|
|
12
|
-
# Logs Search
|
|
13
|
-
|
|
14
|
-
Search and filter logs to support incident investigation. The workflow mirrors Kibana Discover: apply a time range and
|
|
15
|
-
scope filter, then **iteratively add exclusion filters (NOT)** until a small, interesting subset of logs remains—either
|
|
16
|
-
the root cause or the key document. Optionally view logs in context (preceding and following that document) or pivot to
|
|
17
|
-
another entity and start a fresh search. Use ES|QL only (`POST /_query`); do not use Query DSL.
|
|
18
|
-
|
|
19
|
-
## When NOT to use
|
|
20
|
-
|
|
21
|
-
- **Metrics or traces** — use the dedicated metric or trace tools.
|
|
22
|
-
|
|
23
|
-
## Parameter conventions
|
|
24
|
-
|
|
25
|
-
Use consistent names for Observability log search:
|
|
26
|
-
|
|
27
|
-
| Parameter | Type | Description |
|
|
28
|
-
| ----------- | ------ | --------------------------------------------------------------------------- |
|
|
29
|
-
| `start` | string | Start of time range (Elasticsearch date math, e.g. `now-1h`) |
|
|
30
|
-
| `end` | string | End of time range (e.g. `now`) |
|
|
31
|
-
| `kqlFilter` | string | KQL query string to narrow results. Not `query`, `filter`, or `kql`. |
|
|
32
|
-
| `limit` | number | Maximum log samples to return (e.g. 10–100) |
|
|
33
|
-
| `groupBy` | string | Optional field to group the histogram by (e.g. `log.level`, `service.name`) |
|
|
34
|
-
|
|
35
|
-
For entity filters, use ECS field names: `service.name`, `host.name`, `service.environment`, `kubernetes.pod.name`,
|
|
36
|
-
`kubernetes.namespace`. Query ECS names only; OpenTelemetry aliases map automatically in Observability indices.
|
|
37
|
-
|
|
38
|
-
### Context minimization
|
|
39
|
-
|
|
40
|
-
Keep the context window small. In the sample branch of the query, **KEEP only a subset of fields**; do not return full
|
|
41
|
-
documents by default. A small summary (e.g. 10 docs with KEEP) stays under ~1000 tokens; a single full JSON doc can
|
|
42
|
-
exceed 4000 tokens.
|
|
43
|
-
|
|
44
|
-
**Recommended KEEP list for sample logs:**
|
|
45
|
-
`message`, `error.message`, `service.name`, `container.name`, `host.name`, `container.id`, `agent.name`,
|
|
46
|
-
`kubernetes.container.name`, `kubernetes.node.name`, `kubernetes.namespace`, `kubernetes.pod.name`
|
|
47
|
-
|
|
48
|
-
**Message fallback:** If present, use the first non-empty of: `body.text` (OTel), `message`, `error.message`,
|
|
49
|
-
`event.original`, `exception.message`, `error.exception.message`, `attributes.exception.message` (OTel). Observability
|
|
50
|
-
index templates often alias these; when building a single “message” for display, prefer that order.
|
|
51
|
-
|
|
52
|
-
**Limit samples:** Default to a small sample (10–20 logs) per query. Cap at 500; do not fetch thousands in one call.
|
|
53
|
-
Each funnel step is only to decide the next call—only the final narrowed result is the one to keep in context and
|
|
54
|
-
summarize.
|
|
55
|
-
|
|
56
|
-
## The funnel workflow
|
|
57
|
-
|
|
58
|
-
**You must iterate.** Do not stop after one query. Keep excluding noise with `NOT` until **fewer than 20 log patterns**
|
|
59
|
-
(distinct message categories) remain. **Always keep the full filter when iterating:** concatenate new NOTs to the
|
|
60
|
-
previous KQL; do not “zoom out” or drop earlier exclusions.
|
|
61
|
-
|
|
62
|
-
1. **Round 1 — broad:** Run a query with only the scope filter (e.g. `service.name: advertService`) and time range. Get
|
|
63
|
-
total count, histogram, sample logs, and message categorization (common + rare patterns).
|
|
64
|
-
2. **Inspect:** Look at the **histogram** (when spikes or drops occur), the **sample messages**, and the **categorized
|
|
65
|
-
patterns** (fork4 = top patterns by count, fork5 = rare patterns). If the histogram shows a sharp spike at a specific
|
|
66
|
-
time, narrow the time range (t_start, t_end) around that spike for the next round. Count how many distinct log
|
|
67
|
-
patterns remain (from the categorization); identify high-volume noise to exclude.
|
|
68
|
-
3. **Round 2 — exclude noise:** Add `NOT` clauses to the KQL filter for the dominant noise patterns. Run the query again
|
|
69
|
-
with the **full** filter (all previous NOTs plus new ones).
|
|
70
|
-
4. **Repeat:** Keep adding `NOT` clauses and re-running with the full filter. Do **not** stop after one or two rounds.
|
|
71
|
-
Continue until **fewer than 20 log patterns remain** (use the categorization result to count distinct message
|
|
72
|
-
categories). Then the remaining set is small enough to interpret as the interesting bits (errors, anomalies, root
|
|
73
|
-
cause).
|
|
74
|
-
5. **Pivot (optional):** Once the funnel isolates a specific entity (e.g. `container.id`, `host.name`), run one more
|
|
75
|
-
query focused on that entity to see its “dying words” or surrounding context.
|
|
76
|
-
6. **Step back (if needed):** If the funnel does not reveal the root cause, consider viewing logs in context (preceding
|
|
77
|
-
and following the key document) or a different entity and start a fresh search.
|
|
78
|
-
|
|
79
|
-
If you stop before reaching fewer than 20 log patterns, you will report noise instead of the actual failures. Each
|
|
80
|
-
intermediate result is only for deciding the next call; only the final narrowed result should be kept in context and
|
|
81
|
-
summarized.
|
|
82
|
-
|
|
83
|
-
## ES|QL patterns for log search
|
|
84
|
-
|
|
85
|
-
Use ES|QL (`POST /_query`) only; do not use Query DSL. **Always** return in one request: a time-series histogram, total
|
|
86
|
-
count, a small sample of logs, and **message categorization** (common and rare patterns). The histogram is the primary
|
|
87
|
-
signal—it shows when spikes or drops occur and guides the next filter. Use `FORK` to compute trend, total, samples, and
|
|
88
|
-
categorization in a single query.
|
|
89
|
-
|
|
90
|
-
**FORK output interpretation:** The response contains multiple result sets identified by a `_fork` column (or
|
|
91
|
-
equivalent). Map them as: **fork1** = trend (count per time bucket), **fork2** = total count (single row), **fork3** =
|
|
92
|
-
sample logs, **fork4** = common message patterns (top 20 by count, from up to 10k logs), **fork5** = rare message
|
|
93
|
-
patterns (bottom 20 by count, from up to 10k logs). Use fork1 to spot when to narrow the time range; use fork2 to see
|
|
94
|
-
how much noise remains; use fork3 to decide which NOTs to add next; use fork4 and fork5 to see how many distinct log
|
|
95
|
-
patterns remain and to choose the next exclusions—**continue iterating until fewer than 20 log patterns remain**.
|
|
96
|
-
|
|
97
|
-
### KQL guidance
|
|
98
|
-
|
|
99
|
-
- Prefer **phrase queries** for specificity when the target text is tokenized as you expect (e.g.
|
|
100
|
-
`message: "GET /health"`, `service.name: "advertService"`).
|
|
101
|
-
- If the target would not be tokenized as a single term, use a **wildcard** (e.g. `message: *Returning*`,
|
|
102
|
-
`message: *WARNING*`). Do **not** put wildcard characters inside quoted phrases.
|
|
103
|
-
- Use **explicit fielded KQL**: `service.name: "payment-api"`, `message: "GET /health"`,
|
|
104
|
-
`NOT kubernetes.namespace: "kube-system"`, `error.message: * AND NOT message: "Known benign warning"`.
|
|
105
|
-
- **Filtering on `log.level`** (e.g. `log.level: error`) can be useful, but it is **often flawed**: many logs have
|
|
106
|
-
missing or incorrect level metadata (e.g. everything as "info", or level only in the message text). Prefer funneling
|
|
107
|
-
by message content or `error.message` when hunting failures; treat `log.level` as a hint, not a reliable filter.
|
|
108
|
-
- **Random full-text searches for words like "error"** are also **often flawed**: they match harmless mentions (e.g. "no
|
|
109
|
-
error", "error code 0", stack traces that reference the word). Prefer scoping by service/entity and iterating with NOT
|
|
110
|
-
exclusions on actual message patterns rather than relying on a single keyword.
|
|
111
|
-
|
|
112
|
-
### Basic log search with histogram, samples, and categorization
|
|
113
|
-
|
|
114
|
-
Include message categorization so you can count distinct log patterns and iterate until fewer than 20 remain. Use a
|
|
115
|
-
five-way FORK: trend, total, samples, common patterns, rare patterns.
|
|
116
|
-
|
|
117
|
-
```json
|
|
118
|
-
POST /_query
|
|
119
|
-
{
|
|
120
|
-
"query": "FROM logs-* METADATA _id, _index | WHERE @timestamp >= TO_DATETIME(\"2025-03-06T10:00:00.000Z\") AND @timestamp <= TO_DATETIME(\"2025-03-06T11:00:00.000Z\") | FORK (STATS count = COUNT(*) BY bucket = BUCKET(@timestamp, 1m) | SORT bucket) (STATS total = COUNT(*)) (SORT @timestamp DESC | LIMIT 10 | KEEP _id, _index, message, error.message, service.name, container.name, host.name, kubernetes.container.name, kubernetes.node.name, kubernetes.namespace, kubernetes.pod.name) (LIMIT 10000 | STATS COUNT(*) BY CATEGORIZE(message) | SORT `COUNT(*)` DESC | LIMIT 20) (LIMIT 10000 | STATS COUNT(*) BY CATEGORIZE(message) | SORT `COUNT(*)` ASC | LIMIT 20)"
|
|
121
|
-
}
|
|
122
|
-
```
|
|
123
|
-
|
|
124
|
-
- **fork4** (common): top 20 message patterns by count, from up to 10,000 logs—use to add NOTs for dominant noise.
|
|
125
|
-
- **fork5** (rare): bottom 20 message patterns by count—helps spot needles in the haystack.
|
|
126
|
-
Count distinct patterns across fork4/fork5 (and the overall categorization) and **continue iterating until fewer than
|
|
127
|
-
20 log patterns remain**.
|
|
128
|
-
|
|
129
|
-
Adjust the index pattern (e.g. `logs-*`, `logs-*-*`), time range, and bucket size (e.g. `30s`, `5m`, `1h`). Keep sample
|
|
130
|
-
LIMIT small (10–20 by default; cap at 500). Use KEEP so the sample branch returns only summary fields, not full
|
|
131
|
-
documents.
|
|
132
|
-
|
|
133
|
-
### Adding a KQL filter
|
|
134
|
-
|
|
135
|
-
Narrow results with `KQL("...")`. The KQL expression is a single double-quoted string in ES|QL.
|
|
136
|
-
|
|
137
|
-
**Escaping in the request body:** The query is sent inside JSON, so every double quote that is part of the ES|QL string
|
|
138
|
-
must be escaped. Use `\"` for the quotes that wrap the KQL expression. If the KQL expression itself contains double
|
|
139
|
-
quotes (e.g. a phrase like `message: "GET /health"`), escape those in the JSON as `\\\"` so the KQL parser receives
|
|
140
|
-
literal quote characters.
|
|
141
|
-
|
|
142
|
-
```json
|
|
143
|
-
POST /_query
|
|
144
|
-
{
|
|
145
|
-
"query": "FROM logs-* METADATA _id, _index | WHERE @timestamp >= TO_DATETIME(\"2025-03-06T10:00:00.000Z\") AND @timestamp <= TO_DATETIME(\"2025-03-06T11:00:00.000Z\") | WHERE KQL(\"service.name: checkout AND log.level: error\") | FORK (STATS count = COUNT(*) BY bucket = BUCKET(@timestamp, 1m) | SORT bucket) (STATS total = COUNT(*)) (SORT @timestamp DESC | LIMIT 10 | KEEP _id, _index, message, error.message, service.name, host.name, kubernetes.pod.name) (LIMIT 10000 | STATS COUNT(*) BY CATEGORIZE(message) | SORT `COUNT(*)` DESC | LIMIT 20) (LIMIT 10000 | STATS COUNT(*) BY CATEGORIZE(message) | SORT `COUNT(*)` ASC | LIMIT 20)"
|
|
146
|
-
}
|
|
147
|
-
```
|
|
148
|
-
|
|
149
|
-
### Excluding noise with NOT
|
|
150
|
-
|
|
151
|
-
Build the funnel by excluding known noise. In the request body, wrap the KQL string in `\"...\"` and escape any quotes
|
|
152
|
-
inside the KQL expression as `\\\"`:
|
|
153
|
-
|
|
154
|
-
```json
|
|
155
|
-
"query": "... | WHERE KQL(\"NOT message: \\\"GET /health\\\" AND NOT kubernetes.namespace: \\\"kube-system\\\"\") | ..."
|
|
156
|
-
```
|
|
157
|
-
|
|
158
|
-
```json
|
|
159
|
-
"query": "... | WHERE KQL(\"error.message: * AND NOT message: \\\"Known benign warning\\\"\") | ..."
|
|
160
|
-
```
|
|
161
|
-
|
|
162
|
-
### Histogram grouped by a dimension
|
|
163
|
-
|
|
164
|
-
Break down the trend by a second dimension (e.g. `log.level`, `service.name`) to see which level or entity drives the
|
|
165
|
-
spike:
|
|
166
|
-
|
|
167
|
-
```text
|
|
168
|
-
STATS count = COUNT(*) BY bucket = BUCKET(@timestamp, 1m), log.level
|
|
169
|
-
```
|
|
170
|
-
|
|
171
|
-
Use a limited set of group values in the response to avoid explosion (e.g. top N by count, rest as `_other`).
|
|
172
|
-
|
|
173
|
-
## Examples
|
|
174
|
-
|
|
175
|
-
### Last hour of logs for a service
|
|
176
|
-
|
|
177
|
-
```json
|
|
178
|
-
POST /_query
|
|
179
|
-
{
|
|
180
|
-
"query": "FROM logs-* METADATA _id, _index | WHERE @timestamp >= NOW() - 1 hour AND @timestamp <= NOW() | WHERE KQL(\"service.name: api-gateway\") | SORT @timestamp DESC | LIMIT 20"
|
|
181
|
-
}
|
|
182
|
-
```
|
|
183
|
-
|
|
184
|
-
### Error logs with trend and samples
|
|
185
|
-
|
|
186
|
-
```json
|
|
187
|
-
POST /_query
|
|
188
|
-
{
|
|
189
|
-
"query": "FROM logs-* METADATA _id, _index | WHERE @timestamp >= NOW() - 2 hours AND @timestamp <= NOW() | WHERE KQL(\"log.level: error\") | FORK (STATS count = COUNT(*) BY bucket = BUCKET(@timestamp, 5m) | SORT bucket) (STATS total = COUNT(*)) (SORT @timestamp DESC | LIMIT 15)"
|
|
190
|
-
}
|
|
191
|
-
```
|
|
192
|
-
|
|
193
|
-
### Iterative funnel: NOT and NOT and NOT until the interesting bits
|
|
194
|
-
|
|
195
|
-
Do not stop after one exclusion. Each round, add more NOTs for the current top noise, then run again.
|
|
196
|
-
|
|
197
|
-
**Round 1:** `KQL("service.name: advertService")` → e.g. 55k logs; samples show "Returning N ads", "WARNING:
|
|
198
|
-
request...", "received ad request".
|
|
199
|
-
|
|
200
|
-
**Round 2:** Exclude the biggest noise:
|
|
201
|
-
`KQL("service.name: advertService AND NOT message: *Returning* AND NOT message: *WARNING*")` → re-run, check new total
|
|
202
|
-
and samples.
|
|
203
|
-
|
|
204
|
-
**Round 3:** Exclude next noise (e.g. request/cache chatter):
|
|
205
|
-
`KQL("service.name: advertService AND NOT message: *Returning* AND NOT message: *WARNING* AND NOT message: *received ad request* AND NOT message: *Adding* AND NOT message: *Cache miss*")` →
|
|
206
|
-
re-run.
|
|
207
|
-
|
|
208
|
-
**Round 4+:** Keep adding NOTs for whatever still dominates the samples (use fork4/fork5 categorization to see
|
|
209
|
-
patterns). Continue until **fewer than 20 log patterns remain**; then what remains is the signal to report (e.g. "error
|
|
210
|
-
fetching ads", encoding issues).
|
|
211
|
-
|
|
212
|
-
Escaping: wrap the KQL string in `\"...\"` in the JSON; for quoted phrases inside KQL use `\\\"`.
|
|
213
|
-
|
|
214
|
-
## Guidelines
|
|
215
|
-
|
|
216
|
-
- **Funnel: iterate with NOT.** Do not report findings after a single broad query. Add NOT clauses for dominant noise,
|
|
217
|
-
re-run with the **full** filter (keep all previous NOTs), and repeat until **fewer than 20 log patterns remain** (use
|
|
218
|
-
categorization fork4/fork5 to count). Stopping early yields noise, not signal.
|
|
219
|
-
- **Histogram first:** Use the trend (fork1) to see when spikes or drops occur; narrow the time range around the spike
|
|
220
|
-
if needed before adding more NOTs.
|
|
221
|
-
- **Context minimization:** KEEP only summary fields in the sample branch; default LIMIT 10–20, cap at 500. Each funnel
|
|
222
|
-
step is for deciding the next call; only the final narrowed result is for context and summary.
|
|
223
|
-
- **Request body escaping:** The `query` value is JSON. Escape double quotes in the ES|QL string: `\"` for the KQL
|
|
224
|
-
wrapper, `\\\"` for quotes inside the KQL expression (e.g. phrase values).
|
|
225
|
-
- Use Elasticsearch date math for `start` and `end` (e.g. `now-1h`, `now-15m`) when building queries programmatically.
|
|
226
|
-
- Choose bucket size from the time range: aim for roughly 20–50 buckets (e.g. 1h window → `1m` or `2m`).
|
|
227
|
-
- Prefer ECS field names. In Observability index templates, OTel fields are aliased to ECS; see
|
|
228
|
-
[references/log-search-reference.md](references/log-search-reference.md) for resource metadata field fallbacks
|
|
229
|
-
(container, host, cluster, namespace, pod, workload).
|
|
230
|
-
- **`log.level`:** Filtering or grouping by it can be OK but is often unreliable when levels are missing or mis-set;
|
|
231
|
-
prefer message content or `error.message` for finding failures.
|
|
232
|
-
- **Keyword searches:** Searching only for words like "error" or "fail" is often flawed (e.g. "no error", "error code
|
|
233
|
-
0"); prefer scoping by entity and funneling with NOT on real message patterns.
|
|
234
|
-
|
|
235
|
-
## References
|
|
236
|
-
|
|
237
|
-
- [references/log-search-reference.md](references/log-search-reference.md) — ECS/OTel field mapping and index patterns
|
|
@@ -1,76 +0,0 @@
|
|
|
1
|
-
# Log Search Reference
|
|
2
|
-
|
|
3
|
-
Reference for Observability log search: index patterns, ECS/OpenTelemetry field mapping, and resource metadata
|
|
4
|
-
fallbacks.
|
|
5
|
-
|
|
6
|
-
## Log index patterns
|
|
7
|
-
|
|
8
|
-
- `logs-*-*,logs-*,filebeat-*` — common pattern for log data streams
|
|
9
|
-
- `logs.*`- to also include wired streams
|
|
10
|
-
|
|
11
|
-
## ECS and OpenTelemetry
|
|
12
|
-
|
|
13
|
-
Observability index templates provide **field aliases** that map OpenTelemetry fields to ECS. Query using **ECS field
|
|
14
|
-
names** only; aliases handle the mapping.
|
|
15
|
-
|
|
16
|
-
| ECS field | OTel / other notes |
|
|
17
|
-
| ---------------------- | ------------------------ |
|
|
18
|
-
| `message` | `body.text` (OTel) |
|
|
19
|
-
| `log.level` | `severity_text` |
|
|
20
|
-
| `trace.id` | `trace_id` |
|
|
21
|
-
| `span.id` | `span_id` |
|
|
22
|
-
| `service.name` | Service name |
|
|
23
|
-
| `service.environment` | `deployment.environment` |
|
|
24
|
-
| `host.name` | Host name |
|
|
25
|
-
| `kubernetes.pod.name` | `k8s.pod.name` |
|
|
26
|
-
| `kubernetes.namespace` | `k8s.namespace.name` |
|
|
27
|
-
| `@timestamp` | Event time |
|
|
28
|
-
|
|
29
|
-
## Resource metadata field fallbacks
|
|
30
|
-
|
|
31
|
-
For display or grouping, use the first available in each line (ECS then OTel aliases):
|
|
32
|
-
|
|
33
|
-
| Resource | Preferred field(s) — try in order |
|
|
34
|
-
| --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
35
|
-
| Service | `service.name` |
|
|
36
|
-
| Container | `kubernetes.container.name` → `k8s.container.name` → `container.name` |
|
|
37
|
-
| Host/Node | `kubernetes.node.name` → `k8s.node.name` → `host.name` |
|
|
38
|
-
| Cluster | `orchestrator.cluster.name` → `k8s.cluster.name` |
|
|
39
|
-
| Namespace | `kubernetes.namespace` → `k8s.namespace.name` |
|
|
40
|
-
| Pod | `kubernetes.pod.name` → `k8s.pod.name` |
|
|
41
|
-
| Workload | One of: `kubernetes.deployment.name`, `k8s.deployment.name`, `kubernetes.replicaset.name`, `k8s.replicaset.name`, `kubernetes.statefulset.name`, `k8s.statefulset.name`, `kubernetes.daemonset.name`, `k8s.daemonset.name`, `kubernetes.job.name`, `k8s.job.name`, `kubernetes.cronjob.name`, `k8s.cronjob.name` |
|
|
42
|
-
|
|
43
|
-
## Message field fallback order
|
|
44
|
-
|
|
45
|
-
When building a single “message” for display from a log document, use the first non-empty of:
|
|
46
|
-
|
|
47
|
-
1. `body.text` (OTel message)
|
|
48
|
-
2. `message`
|
|
49
|
-
3. `error.message`
|
|
50
|
-
4. `event.original`
|
|
51
|
-
5. `exception.message`
|
|
52
|
-
6. `error.exception.message`
|
|
53
|
-
7. `attributes.exception.message` (OTel)
|
|
54
|
-
|
|
55
|
-
## ES|QL query shape for logs
|
|
56
|
-
|
|
57
|
-
Typical shape when using `POST /_query`:
|
|
58
|
-
|
|
59
|
-
1. `FROM <index_pattern> METADATA _id, _index` — include index and id for samples
|
|
60
|
-
2. `WHERE @timestamp >= ... AND @timestamp <= ...` — time range (ISO or date math)
|
|
61
|
-
3. Optional: `| WHERE KQL("...")` — KQL filter (escape `\"` in JSON)
|
|
62
|
-
4. For histogram + total + samples in one call:
|
|
63
|
-
`| FORK (STATS ... BY bucket = BUCKET(@timestamp, <size>) ...) (STATS total = COUNT(*)) (SORT @timestamp DESC | LIMIT n | KEEP ...)`
|
|
64
|
-
|
|
65
|
-
Bucket size examples: `30s`, `1m`, `5m`, `1h`. Keep sample LIMIT small (10–20 default; cap 500).
|
|
66
|
-
|
|
67
|
-
## Related documentation
|
|
68
|
-
|
|
69
|
-
- [ES|QL FORK command](https://www.elastic.co/docs/reference/query-languages/esql/commands/fork) — branch limits,
|
|
70
|
-
default LIMIT behavior, preview status
|
|
71
|
-
- [ES|QL CATEGORIZE function](https://www.elastic.co/docs/reference/query-languages/esql/functions-operators/grouping-functions/categorize)
|
|
72
|
-
— license requirement, grouping constraints
|
|
73
|
-
- [ES|QL KQL function](https://www.elastic.co/docs/reference/query-languages/esql/functions-operators/search-functions/kql)
|
|
74
|
-
— syntax and options
|
|
75
|
-
- [Use the ES|QL REST API](https://www.elastic.co/docs/reference/query-languages/esql/esql-rest) — `POST /_query`
|
|
76
|
-
endpoint, async queries, response formats
|
|
@@ -1,53 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: payload-triage
|
|
3
|
-
description: |
|
|
4
|
-
Default SOC Copilot skill for analyzing raw payloads, mixed artifacts, suspicious snippets, and generic alerts.
|
|
5
|
-
Use when the artifact type is unknown or when no specialized skill is a better fit.
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
# Payload Triage
|
|
9
|
-
|
|
10
|
-
Default analysis workflow for raw or mixed security artifacts.
|
|
11
|
-
|
|
12
|
-
## When to Use
|
|
13
|
-
|
|
14
|
-
- artifact type is unknown
|
|
15
|
-
- input is a raw payload, alert body, suspicious snippet, or mixed text
|
|
16
|
-
- no specialized playbook is clearly a better match
|
|
17
|
-
|
|
18
|
-
## Workflow
|
|
19
|
-
|
|
20
|
-
### 1. Establish the artifact context
|
|
21
|
-
|
|
22
|
-
- identify whether the input looks like log data, command content, email text, URL-heavy content, or generic payload
|
|
23
|
-
- note missing context that limits confidence
|
|
24
|
-
|
|
25
|
-
### 2. Extract observable evidence
|
|
26
|
-
|
|
27
|
-
Read [`../../references/ioc-extraction.md`](../../references/ioc-extraction.md) when extraction details matter.
|
|
28
|
-
|
|
29
|
-
- identify visible IOCs
|
|
30
|
-
- capture suspicious strings, commands, or infrastructure hints
|
|
31
|
-
- keep direct observations separate from assumptions
|
|
32
|
-
|
|
33
|
-
### 3. Assess likely risk
|
|
34
|
-
|
|
35
|
-
Read [`../../references/evidence-rules.md`](../../references/evidence-rules.md) for verdict and confidence guidance.
|
|
36
|
-
|
|
37
|
-
- determine whether the artifact is benign, suspicious, malicious, or inconclusive
|
|
38
|
-
- justify the verdict with explicit risk reasons
|
|
39
|
-
|
|
40
|
-
### 4. Suggest next analyst actions
|
|
41
|
-
|
|
42
|
-
- prioritize low-risk validation steps first
|
|
43
|
-
- recommend containment only when supported by evidence
|
|
44
|
-
|
|
45
|
-
## Output Contract
|
|
46
|
-
|
|
47
|
-
Read [`../../references/output-contract.md`](../../references/output-contract.md).
|
|
48
|
-
|
|
49
|
-
## Guardrails
|
|
50
|
-
|
|
51
|
-
- Do not force ATT&CK mapping for weak evidence.
|
|
52
|
-
- Prefer `inconclusivo` over false precision.
|
|
53
|
-
- Keep the result operational and concise.
|
|
@@ -1,51 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: phishing-analysis
|
|
3
|
-
description: |
|
|
4
|
-
Specialized SOC Copilot skill for analyzing suspicious emails, headers, senders, embedded links, and attachment clues.
|
|
5
|
-
Use when the user wants phishing triage or the artifact clearly contains email context.
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
# Phishing Analysis
|
|
9
|
-
|
|
10
|
-
Focused workflow for phishing and email-borne social engineering analysis.
|
|
11
|
-
|
|
12
|
-
## When to Use
|
|
13
|
-
|
|
14
|
-
- input contains sender, recipient, headers, subject, message body, or attachment details
|
|
15
|
-
- user asks whether a message is phishing
|
|
16
|
-
- links inside an email are central to the analysis
|
|
17
|
-
|
|
18
|
-
## Workflow
|
|
19
|
-
|
|
20
|
-
### 1. Identify email evidence
|
|
21
|
-
|
|
22
|
-
- extract sender, reply-to, recipient, domains, URLs, attachment names, and mail-routing clues when present
|
|
23
|
-
- distinguish message content from transport metadata
|
|
24
|
-
|
|
25
|
-
### 2. Evaluate phishing indicators
|
|
26
|
-
|
|
27
|
-
- look for sender and domain mismatch
|
|
28
|
-
- inspect suspicious links, urgent requests, spoofing language, and attachment risk
|
|
29
|
-
- weigh header or link mismatch more heavily than tone alone
|
|
30
|
-
|
|
31
|
-
### 3. Assess likely abuse pattern
|
|
32
|
-
|
|
33
|
-
Read [`../../references/evidence-rules.md`](../../references/evidence-rules.md) for verdict discipline.
|
|
34
|
-
|
|
35
|
-
- determine whether the artifact supports phishing, impersonation, malware delivery, or remains inconclusive
|
|
36
|
-
- explain why the evidence supports that conclusion
|
|
37
|
-
|
|
38
|
-
### 4. Recommend safe handling
|
|
39
|
-
|
|
40
|
-
- propose safe validation steps
|
|
41
|
-
- suggest reporting, isolation, or user outreach when justified
|
|
42
|
-
|
|
43
|
-
## Output Contract
|
|
44
|
-
|
|
45
|
-
Read [`../../references/output-contract.md`](../../references/output-contract.md).
|
|
46
|
-
|
|
47
|
-
## Guardrails
|
|
48
|
-
|
|
49
|
-
- Do not mark an email as phishing based only on generic urgency language.
|
|
50
|
-
- Separate visible evidence from reputation-dependent conclusions.
|
|
51
|
-
- If no header evidence exists, state that limitation clearly.
|
|
@@ -1,143 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: prd
|
|
3
|
-
description: 'Generate high-quality Product Requirements Documents (PRDs) for software systems and AI-powered features. Includes executive summaries, user stories, technical specifications, and risk analysis.'
|
|
4
|
-
license: MIT
|
|
5
|
-
---
|
|
6
|
-
|
|
7
|
-
# Product Requirements Document (PRD)
|
|
8
|
-
|
|
9
|
-
## Overview
|
|
10
|
-
|
|
11
|
-
Design comprehensive, production-grade Product Requirements Documents (PRDs) that bridge the gap between business vision and technical execution. This skill works for modern software systems, ensuring that requirements are clearly defined.
|
|
12
|
-
|
|
13
|
-
## When to Use
|
|
14
|
-
|
|
15
|
-
Use this skill when:
|
|
16
|
-
|
|
17
|
-
- Starting a new product or feature development cycle
|
|
18
|
-
- Translating a vague idea into a concrete technical specification
|
|
19
|
-
- Defining requirements for AI-powered features
|
|
20
|
-
- Stakeholders need a unified "source of truth" for project scope
|
|
21
|
-
- User asks to "write a PRD", "document requirements", or "plan a feature"
|
|
22
|
-
|
|
23
|
-
---
|
|
24
|
-
|
|
25
|
-
## Operational Workflow
|
|
26
|
-
|
|
27
|
-
### Phase 1: Discovery (The Interview)
|
|
28
|
-
|
|
29
|
-
Before writing a single line of the PRD, you **MUST** interrogate the user to fill knowledge gaps. Do not assume context.
|
|
30
|
-
|
|
31
|
-
**Ask about:**
|
|
32
|
-
|
|
33
|
-
- **The Core Problem**: Why are we building this now?
|
|
34
|
-
- **Success Metrics**: How do we know it worked?
|
|
35
|
-
- **Constraints**: Budget, tech stack, or deadline?
|
|
36
|
-
|
|
37
|
-
### Phase 2: Analysis & Scoping
|
|
38
|
-
|
|
39
|
-
Synthesize the user's input. Identify dependencies and hidden complexities.
|
|
40
|
-
|
|
41
|
-
- Map out the **User Flow**.
|
|
42
|
-
- Define **Non-Goals** to protect the timeline.
|
|
43
|
-
|
|
44
|
-
### Phase 3: Technical Drafting
|
|
45
|
-
|
|
46
|
-
Generate the document using the **Strict PRD Schema** below.
|
|
47
|
-
|
|
48
|
-
---
|
|
49
|
-
|
|
50
|
-
## PRD Quality Standards
|
|
51
|
-
|
|
52
|
-
### Requirements Quality
|
|
53
|
-
|
|
54
|
-
Use concrete, measurable criteria. Avoid "fast", "easy", or "intuitive".
|
|
55
|
-
|
|
56
|
-
```diff
|
|
57
|
-
# Vague (BAD)
|
|
58
|
-
- The search should be fast and return relevant results.
|
|
59
|
-
- The UI must look modern and be easy to use.
|
|
60
|
-
|
|
61
|
-
# Concrete (GOOD)
|
|
62
|
-
+ The search must return results within 200ms for a 10k record dataset.
|
|
63
|
-
+ The search algorithm must achieve >= 85% Precision@10 in benchmark evals.
|
|
64
|
-
+ The UI must follow the 'Vercel/Next.js' design system and achieve 100% Lighthouse Accessibility score.
|
|
65
|
-
```
|
|
66
|
-
|
|
67
|
-
---
|
|
68
|
-
|
|
69
|
-
## Strict PRD Schema
|
|
70
|
-
|
|
71
|
-
You **MUST** follow this exact structure for the output:
|
|
72
|
-
|
|
73
|
-
### 1. Executive Summary
|
|
74
|
-
|
|
75
|
-
- **Problem Statement**: 1-2 sentences on the pain point.
|
|
76
|
-
- **Proposed Solution**: 1-2 sentences on the fix.
|
|
77
|
-
- **Success Criteria**: 3-5 measurable KPIs.
|
|
78
|
-
|
|
79
|
-
### 2. User Experience & Functionality
|
|
80
|
-
|
|
81
|
-
- **User Personas**: Who is this for?
|
|
82
|
-
- **User Stories**: `As a [user], I want to [action] so that [benefit].`
|
|
83
|
-
- **Acceptance Criteria**: Bulleted list of "Done" definitions for each story.
|
|
84
|
-
- **Non-Goals**: What are we NOT building?
|
|
85
|
-
|
|
86
|
-
### 3. AI System Requirements (If Applicable)
|
|
87
|
-
|
|
88
|
-
- **Tool Requirements**: What tools and APIs are needed?
|
|
89
|
-
- **Evaluation Strategy**: How to measure output quality and accuracy.
|
|
90
|
-
|
|
91
|
-
### 4. Technical Specifications
|
|
92
|
-
|
|
93
|
-
- **Architecture Overview**: Data flow and component interaction.
|
|
94
|
-
- **Integration Points**: APIs, DBs, and Auth.
|
|
95
|
-
- **Security & Privacy**: Data handling and compliance.
|
|
96
|
-
|
|
97
|
-
### 5. Risks & Roadmap
|
|
98
|
-
|
|
99
|
-
- **Phased Rollout**: MVP -> v1.1 -> v2.0.
|
|
100
|
-
- **Technical Risks**: Latency, cost, or dependency failures.
|
|
101
|
-
|
|
102
|
-
---
|
|
103
|
-
|
|
104
|
-
## Implementation Guidelines
|
|
105
|
-
|
|
106
|
-
### DO (Always)
|
|
107
|
-
|
|
108
|
-
- **Define Testing**: For AI systems, specify how to test and validate output quality.
|
|
109
|
-
- **Iterate**: Present a draft and ask for feedback on specific sections.
|
|
110
|
-
|
|
111
|
-
### DON'T (Avoid)
|
|
112
|
-
|
|
113
|
-
- **Skip Discovery**: Never write a PRD without asking at least 2 clarifying questions first.
|
|
114
|
-
- **Hallucinate Constraints**: If the user didn't specify a tech stack, ask or label it as `TBD`.
|
|
115
|
-
|
|
116
|
-
---
|
|
117
|
-
|
|
118
|
-
## Example: Intelligent Search System
|
|
119
|
-
|
|
120
|
-
### 1. Executive Summary
|
|
121
|
-
|
|
122
|
-
**Problem**: Users struggle to find specific documentation snippets in massive repositories.
|
|
123
|
-
**Solution**: An intelligent search system that provides direct answers with source citations.
|
|
124
|
-
**Success**:
|
|
125
|
-
|
|
126
|
-
- Reduce search time by 50%.
|
|
127
|
-
- Citation accuracy >= 95%.
|
|
128
|
-
|
|
129
|
-
### 2. User Stories
|
|
130
|
-
|
|
131
|
-
- **Story**: As a developer, I want to ask natural language questions so I don't have to guess keywords.
|
|
132
|
-
- **AC**:
|
|
133
|
-
- Supports multi-turn clarification.
|
|
134
|
-
- Returns code blocks with "Copy" button.
|
|
135
|
-
|
|
136
|
-
### 3. AI System Architecture
|
|
137
|
-
|
|
138
|
-
- **Tools Required**: `codesearch`, `grep`, `webfetch`.
|
|
139
|
-
|
|
140
|
-
### 4. Evaluation
|
|
141
|
-
|
|
142
|
-
- **Benchmark**: Test with 50 common developer questions.
|
|
143
|
-
- **Pass Rate**: 90% must match expected citations.
|