@vantagesec/socc 0.1.13 → 0.1.15

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (127) hide show
  1. package/README.md +8 -8
  2. package/dist/cli.mjs +10734 -10807
  3. package/package.json +7 -8
  4. package/scripts/bootstrap-socc-soul.mjs +23 -1
  5. package/socc-canonical/.agents/generated/socc-agent-manifest.json +0 -231
  6. package/socc-canonical/.agents/generated/socc-agent.md +0 -256
  7. package/socc-canonical/.agents/rules/AGENT.md +0 -109
  8. package/socc-canonical/.agents/rules/AQL_REFERENCE.md +0 -40
  9. package/socc-canonical/.agents/rules/MEMORY.md +0 -19
  10. package/socc-canonical/.agents/rules/TOOLS.md +0 -48
  11. package/socc-canonical/.agents/soc-copilot/AGENTS.md +0 -28
  12. package/socc-canonical/.agents/soc-copilot/MEMORY.md +0 -20
  13. package/socc-canonical/.agents/soc-copilot/SKILL.md +0 -51
  14. package/socc-canonical/.agents/soc-copilot/SOUL.md +0 -31
  15. package/socc-canonical/.agents/soc-copilot/TOOLS.md +0 -33
  16. package/socc-canonical/.agents/soc-copilot/USER.md +0 -31
  17. package/socc-canonical/.agents/soc-copilot/identity.md +0 -7
  18. package/socc-canonical/.agents/soc-copilot/references/evidence-rules.md +0 -30
  19. package/socc-canonical/.agents/soc-copilot/references/intelligence-source-registry.md +0 -32
  20. package/socc-canonical/.agents/soc-copilot/references/ioc-extraction.md +0 -25
  21. package/socc-canonical/.agents/soc-copilot/references/knowledge-ingestion-policy.md +0 -34
  22. package/socc-canonical/.agents/soc-copilot/references/mitre-guidance.md +0 -21
  23. package/socc-canonical/.agents/soc-copilot/references/output-contract.md +0 -31
  24. package/socc-canonical/.agents/soc-copilot/references/security-json-patterns.md +0 -129
  25. package/socc-canonical/.agents/soc-copilot/references/telemetry-investigation-patterns.md +0 -39
  26. package/socc-canonical/.agents/soc-copilot/schemas/analysis_response.json +0 -119
  27. package/socc-canonical/.agents/soc-copilot/skills/code-review-excellence/SKILL.md +0 -538
  28. package/socc-canonical/.agents/soc-copilot/skills/cybersecurity-analyst/QUICK_REFERENCE.md +0 -263
  29. package/socc-canonical/.agents/soc-copilot/skills/cybersecurity-analyst/README.md +0 -243
  30. package/socc-canonical/.agents/soc-copilot/skills/cybersecurity-analyst/SKILL.md +0 -1707
  31. package/socc-canonical/.agents/soc-copilot/skills/cybersecurity-analyst/tests/quiz.md +0 -472
  32. package/socc-canonical/.agents/soc-copilot/skills/data-visualization/SKILL.md +0 -304
  33. package/socc-canonical/.agents/soc-copilot/skills/deep-research/SKILL.md +0 -192
  34. package/socc-canonical/.agents/soc-copilot/skills/excel-analysis/SKILL.md +0 -247
  35. package/socc-canonical/.agents/soc-copilot/skills/find-skills/SKILL.md +0 -133
  36. package/socc-canonical/.agents/soc-copilot/skills/humanizer/README.md +0 -120
  37. package/socc-canonical/.agents/soc-copilot/skills/humanizer/SKILL.md +0 -439
  38. package/socc-canonical/.agents/soc-copilot/skills/malware-behavior/SKILL.md +0 -54
  39. package/socc-canonical/.agents/soc-copilot/skills/mitre/SKILL.md +0 -200
  40. package/socc-canonical/.agents/soc-copilot/skills/observability-logs-search/SKILL.md +0 -237
  41. package/socc-canonical/.agents/soc-copilot/skills/observability-logs-search/references/log-search-reference.md +0 -76
  42. package/socc-canonical/.agents/soc-copilot/skills/payload-triage/SKILL.md +0 -53
  43. package/socc-canonical/.agents/soc-copilot/skills/phishing-analysis/SKILL.md +0 -51
  44. package/socc-canonical/.agents/soc-copilot/skills/prd/SKILL.md +0 -143
  45. package/socc-canonical/.agents/soc-copilot/skills/remembering-conversations/MCP-TOOLS.md +0 -137
  46. package/socc-canonical/.agents/soc-copilot/skills/remembering-conversations/SKILL.md +0 -65
  47. package/socc-canonical/.agents/soc-copilot/skills/sequential-thinking/README.md +0 -118
  48. package/socc-canonical/.agents/soc-copilot/skills/sequential-thinking/SKILL.md +0 -93
  49. package/socc-canonical/.agents/soc-copilot/skills/sequential-thinking/references/advanced.md +0 -122
  50. package/socc-canonical/.agents/soc-copilot/skills/sequential-thinking/references/examples.md +0 -274
  51. package/socc-canonical/.agents/soc-copilot/skills/soc-generalist/SKILL.md +0 -53
  52. package/socc-canonical/.agents/soc-copilot/skills/suspicious-url/SKILL.md +0 -51
  53. package/socc-canonical/.agents/soc-copilot/skills/systematic-debugging/CREATION-LOG.md +0 -119
  54. package/socc-canonical/.agents/soc-copilot/skills/systematic-debugging/SKILL.md +0 -296
  55. package/socc-canonical/.agents/soc-copilot/skills/systematic-debugging/condition-based-waiting-example.ts +0 -158
  56. package/socc-canonical/.agents/soc-copilot/skills/systematic-debugging/condition-based-waiting.md +0 -115
  57. package/socc-canonical/.agents/soc-copilot/skills/systematic-debugging/defense-in-depth.md +0 -122
  58. package/socc-canonical/.agents/soc-copilot/skills/systematic-debugging/find-polluter.sh +0 -63
  59. package/socc-canonical/.agents/soc-copilot/skills/systematic-debugging/root-cause-tracing.md +0 -169
  60. package/socc-canonical/.agents/soc-copilot/skills/systematic-debugging/test-academic.md +0 -14
  61. package/socc-canonical/.agents/soc-copilot/skills/systematic-debugging/test-pressure-1.md +0 -58
  62. package/socc-canonical/.agents/soc-copilot/skills/systematic-debugging/test-pressure-2.md +0 -68
  63. package/socc-canonical/.agents/soc-copilot/skills/systematic-debugging/test-pressure-3.md +0 -69
  64. package/socc-canonical/.agents/soc-copilot/skills/translation-expertise/SKILL.md +0 -284
  65. package/socc-canonical/.agents/soc-copilot/skills/translation-expertise/chinese-traditional.md +0 -535
  66. package/socc-canonical/.agents/soc-copilot/skills/translation-expertise/english.md +0 -372
  67. package/socc-canonical/.agents/soc-copilot/skills/translation-expertise/japanese.md +0 -515
  68. package/socc-canonical/.agents/soc-copilot/skills/translation-expertise/tools-resources.md +0 -527
  69. package/socc-canonical/.agents/soc-copilot/skills/translation-expertise/translation-challenges.md +0 -603
  70. package/socc-canonical/.agents/soc-copilot/skills/web-search/SKILL.md +0 -322
  71. package/socc-canonical/.agents/soc-copilot/skills.md +0 -21
  72. package/socc-canonical/.agents/workflows/SOP.md +0 -137
  73. package/socc-canonical/README.md +0 -10
  74. /package/{.claude → .socc}/agents/socc.md +0 -0
  75. /package/{.claude → .socc}/references/evidence-rules.md +0 -0
  76. /package/{.claude → .socc}/references/intelligence-source-registry.md +0 -0
  77. /package/{.claude → .socc}/references/ioc-extraction.md +0 -0
  78. /package/{.claude → .socc}/references/knowledge-ingestion-policy.md +0 -0
  79. /package/{.claude → .socc}/references/mitre-guidance.md +0 -0
  80. /package/{.claude → .socc}/references/output-contract.md +0 -0
  81. /package/{.claude → .socc}/references/security-json-patterns.md +0 -0
  82. /package/{.claude → .socc}/references/telemetry-investigation-patterns.md +0 -0
  83. /package/{.claude → .socc}/rules/socc-business-rules.md +0 -0
  84. /package/{.claude → .socc}/skills/code-review-excellence/SKILL.md +0 -0
  85. /package/{.claude → .socc}/skills/cybersecurity-analyst/QUICK_REFERENCE.md +0 -0
  86. /package/{.claude → .socc}/skills/cybersecurity-analyst/README.md +0 -0
  87. /package/{.claude → .socc}/skills/cybersecurity-analyst/SKILL.md +0 -0
  88. /package/{.claude → .socc}/skills/cybersecurity-analyst/tests/quiz.md +0 -0
  89. /package/{.claude → .socc}/skills/data-visualization/SKILL.md +0 -0
  90. /package/{.claude → .socc}/skills/deep-research/SKILL.md +0 -0
  91. /package/{.claude → .socc}/skills/excel-analysis/SKILL.md +0 -0
  92. /package/{.claude → .socc}/skills/find-skills/SKILL.md +0 -0
  93. /package/{.claude → .socc}/skills/humanizer/README.md +0 -0
  94. /package/{.claude → .socc}/skills/humanizer/SKILL.md +0 -0
  95. /package/{.claude → .socc}/skills/malware-behavior/SKILL.md +0 -0
  96. /package/{.claude → .socc}/skills/mitre/SKILL.md +0 -0
  97. /package/{.claude → .socc}/skills/observability-logs-search/SKILL.md +0 -0
  98. /package/{.claude → .socc}/skills/observability-logs-search/references/log-search-reference.md +0 -0
  99. /package/{.claude → .socc}/skills/payload-triage/SKILL.md +0 -0
  100. /package/{.claude → .socc}/skills/phishing-analysis/SKILL.md +0 -0
  101. /package/{.claude → .socc}/skills/prd/SKILL.md +0 -0
  102. /package/{.claude → .socc}/skills/remembering-conversations/MCP-TOOLS.md +0 -0
  103. /package/{.claude → .socc}/skills/remembering-conversations/SKILL.md +0 -0
  104. /package/{.claude → .socc}/skills/sequential-thinking/README.md +0 -0
  105. /package/{.claude → .socc}/skills/sequential-thinking/SKILL.md +0 -0
  106. /package/{.claude → .socc}/skills/sequential-thinking/references/advanced.md +0 -0
  107. /package/{.claude → .socc}/skills/sequential-thinking/references/examples.md +0 -0
  108. /package/{.claude → .socc}/skills/soc-generalist/SKILL.md +0 -0
  109. /package/{.claude → .socc}/skills/suspicious-url/SKILL.md +0 -0
  110. /package/{.claude → .socc}/skills/systematic-debugging/CREATION-LOG.md +0 -0
  111. /package/{.claude → .socc}/skills/systematic-debugging/SKILL.md +0 -0
  112. /package/{.claude → .socc}/skills/systematic-debugging/condition-based-waiting-example.ts +0 -0
  113. /package/{.claude → .socc}/skills/systematic-debugging/condition-based-waiting.md +0 -0
  114. /package/{.claude → .socc}/skills/systematic-debugging/defense-in-depth.md +0 -0
  115. /package/{.claude → .socc}/skills/systematic-debugging/find-polluter.sh +0 -0
  116. /package/{.claude → .socc}/skills/systematic-debugging/root-cause-tracing.md +0 -0
  117. /package/{.claude → .socc}/skills/systematic-debugging/test-academic.md +0 -0
  118. /package/{.claude → .socc}/skills/systematic-debugging/test-pressure-1.md +0 -0
  119. /package/{.claude → .socc}/skills/systematic-debugging/test-pressure-2.md +0 -0
  120. /package/{.claude → .socc}/skills/systematic-debugging/test-pressure-3.md +0 -0
  121. /package/{.claude → .socc}/skills/translation-expertise/SKILL.md +0 -0
  122. /package/{.claude → .socc}/skills/translation-expertise/chinese-traditional.md +0 -0
  123. /package/{.claude → .socc}/skills/translation-expertise/english.md +0 -0
  124. /package/{.claude → .socc}/skills/translation-expertise/japanese.md +0 -0
  125. /package/{.claude → .socc}/skills/translation-expertise/tools-resources.md +0 -0
  126. /package/{.claude → .socc}/skills/translation-expertise/translation-challenges.md +0 -0
  127. /package/{.claude → .socc}/skills/web-search/SKILL.md +0 -0
@@ -1,237 +0,0 @@
1
- ---
2
- name: observability-logs-search
3
- description: >
4
- Search and filter Observability logs using ES|QL. Use when investigating log spikes,
5
- errors, or anomalies; getting volume and trends; or drilling into services or containers
6
- during incidents.
7
- metadata:
8
- author: elastic
9
- version: 0.2.0
10
- ---
11
-
12
- # Logs Search
13
-
14
- Search and filter logs to support incident investigation. The workflow mirrors Kibana Discover: apply a time range and
15
- scope filter, then **iteratively add exclusion filters (NOT)** until a small, interesting subset of logs remains—either
16
- the root cause or the key document. Optionally view logs in context (preceding and following that document) or pivot to
17
- another entity and start a fresh search. Use ES|QL only (`POST /_query`); do not use Query DSL.
18
-
19
- ## When NOT to use
20
-
21
- - **Metrics or traces** — use the dedicated metric or trace tools.
22
-
23
- ## Parameter conventions
24
-
25
- Use consistent names for Observability log search:
26
-
27
- | Parameter | Type | Description |
28
- | ----------- | ------ | --------------------------------------------------------------------------- |
29
- | `start` | string | Start of time range (Elasticsearch date math, e.g. `now-1h`) |
30
- | `end` | string | End of time range (e.g. `now`) |
31
- | `kqlFilter` | string | KQL query string to narrow results. Not `query`, `filter`, or `kql`. |
32
- | `limit` | number | Maximum log samples to return (e.g. 10–100) |
33
- | `groupBy` | string | Optional field to group the histogram by (e.g. `log.level`, `service.name`) |
34
-
35
- For entity filters, use ECS field names: `service.name`, `host.name`, `service.environment`, `kubernetes.pod.name`,
36
- `kubernetes.namespace`. Query ECS names only; OpenTelemetry aliases map automatically in Observability indices.
37
-
38
- ### Context minimization
39
-
40
- Keep the context window small. In the sample branch of the query, **KEEP only a subset of fields**; do not return full
41
- documents by default. A small summary (e.g. 10 docs with KEEP) stays under ~1000 tokens; a single full JSON doc can
42
- exceed 4000 tokens.
43
-
44
- **Recommended KEEP list for sample logs:**
45
- `message`, `error.message`, `service.name`, `container.name`, `host.name`, `container.id`, `agent.name`,
46
- `kubernetes.container.name`, `kubernetes.node.name`, `kubernetes.namespace`, `kubernetes.pod.name`
47
-
48
- **Message fallback:** If present, use the first non-empty of: `body.text` (OTel), `message`, `error.message`,
49
- `event.original`, `exception.message`, `error.exception.message`, `attributes.exception.message` (OTel). Observability
50
- index templates often alias these; when building a single “message” for display, prefer that order.
51
-
52
- **Limit samples:** Default to a small sample (10–20 logs) per query. Cap at 500; do not fetch thousands in one call.
53
- Each funnel step is only to decide the next call—only the final narrowed result is the one to keep in context and
54
- summarize.
55
-
56
- ## The funnel workflow
57
-
58
- **You must iterate.** Do not stop after one query. Keep excluding noise with `NOT` until **fewer than 20 log patterns**
59
- (distinct message categories) remain. **Always keep the full filter when iterating:** concatenate new NOTs to the
60
- previous KQL; do not “zoom out” or drop earlier exclusions.
61
-
62
- 1. **Round 1 — broad:** Run a query with only the scope filter (e.g. `service.name: advertService`) and time range. Get
63
- total count, histogram, sample logs, and message categorization (common + rare patterns).
64
- 2. **Inspect:** Look at the **histogram** (when spikes or drops occur), the **sample messages**, and the **categorized
65
- patterns** (fork4 = top patterns by count, fork5 = rare patterns). If the histogram shows a sharp spike at a specific
66
- time, narrow the time range (t_start, t_end) around that spike for the next round. Count how many distinct log
67
- patterns remain (from the categorization); identify high-volume noise to exclude.
68
- 3. **Round 2 — exclude noise:** Add `NOT` clauses to the KQL filter for the dominant noise patterns. Run the query again
69
- with the **full** filter (all previous NOTs plus new ones).
70
- 4. **Repeat:** Keep adding `NOT` clauses and re-running with the full filter. Do **not** stop after one or two rounds.
71
- Continue until **fewer than 20 log patterns remain** (use the categorization result to count distinct message
72
- categories). Then the remaining set is small enough to interpret as the interesting bits (errors, anomalies, root
73
- cause).
74
- 5. **Pivot (optional):** Once the funnel isolates a specific entity (e.g. `container.id`, `host.name`), run one more
75
- query focused on that entity to see its “dying words” or surrounding context.
76
- 6. **Step back (if needed):** If the funnel does not reveal the root cause, consider viewing logs in context (preceding
77
- and following the key document) or a different entity and start a fresh search.
78
-
79
- If you stop before reaching fewer than 20 log patterns, you will report noise instead of the actual failures. Each
80
- intermediate result is only for deciding the next call; only the final narrowed result should be kept in context and
81
- summarized.
82
-
83
- ## ES|QL patterns for log search
84
-
85
- Use ES|QL (`POST /_query`) only; do not use Query DSL. **Always** return in one request: a time-series histogram, total
86
- count, a small sample of logs, and **message categorization** (common and rare patterns). The histogram is the primary
87
- signal—it shows when spikes or drops occur and guides the next filter. Use `FORK` to compute trend, total, samples, and
88
- categorization in a single query.
89
-
90
- **FORK output interpretation:** The response contains multiple result sets identified by a `_fork` column (or
91
- equivalent). Map them as: **fork1** = trend (count per time bucket), **fork2** = total count (single row), **fork3** =
92
- sample logs, **fork4** = common message patterns (top 20 by count, from up to 10k logs), **fork5** = rare message
93
- patterns (bottom 20 by count, from up to 10k logs). Use fork1 to spot when to narrow the time range; use fork2 to see
94
- how much noise remains; use fork3 to decide which NOTs to add next; use fork4 and fork5 to see how many distinct log
95
- patterns remain and to choose the next exclusions—**continue iterating until fewer than 20 log patterns remain**.
96
-
97
- ### KQL guidance
98
-
99
- - Prefer **phrase queries** for specificity when the target text is tokenized as you expect (e.g.
100
- `message: "GET /health"`, `service.name: "advertService"`).
101
- - If the target would not be tokenized as a single term, use a **wildcard** (e.g. `message: *Returning*`,
102
- `message: *WARNING*`). Do **not** put wildcard characters inside quoted phrases.
103
- - Use **explicit fielded KQL**: `service.name: "payment-api"`, `message: "GET /health"`,
104
- `NOT kubernetes.namespace: "kube-system"`, `error.message: * AND NOT message: "Known benign warning"`.
105
- - **Filtering on `log.level`** (e.g. `log.level: error`) can be useful, but it is **often flawed**: many logs have
106
- missing or incorrect level metadata (e.g. everything as "info", or level only in the message text). Prefer funneling
107
- by message content or `error.message` when hunting failures; treat `log.level` as a hint, not a reliable filter.
108
- - **Random full-text searches for words like "error"** are also **often flawed**: they match harmless mentions (e.g. "no
109
- error", "error code 0", stack traces that reference the word). Prefer scoping by service/entity and iterating with NOT
110
- exclusions on actual message patterns rather than relying on a single keyword.
111
-
112
- ### Basic log search with histogram, samples, and categorization
113
-
114
- Include message categorization so you can count distinct log patterns and iterate until fewer than 20 remain. Use a
115
- five-way FORK: trend, total, samples, common patterns, rare patterns.
116
-
117
- ```json
118
- POST /_query
119
- {
120
- "query": "FROM logs-* METADATA _id, _index | WHERE @timestamp >= TO_DATETIME(\"2025-03-06T10:00:00.000Z\") AND @timestamp <= TO_DATETIME(\"2025-03-06T11:00:00.000Z\") | FORK (STATS count = COUNT(*) BY bucket = BUCKET(@timestamp, 1m) | SORT bucket) (STATS total = COUNT(*)) (SORT @timestamp DESC | LIMIT 10 | KEEP _id, _index, message, error.message, service.name, container.name, host.name, kubernetes.container.name, kubernetes.node.name, kubernetes.namespace, kubernetes.pod.name) (LIMIT 10000 | STATS COUNT(*) BY CATEGORIZE(message) | SORT `COUNT(*)` DESC | LIMIT 20) (LIMIT 10000 | STATS COUNT(*) BY CATEGORIZE(message) | SORT `COUNT(*)` ASC | LIMIT 20)"
121
- }
122
- ```
123
-
124
- - **fork4** (common): top 20 message patterns by count, from up to 10,000 logs—use to add NOTs for dominant noise.
125
- - **fork5** (rare): bottom 20 message patterns by count—helps spot needles in the haystack.
126
- Count distinct patterns across fork4/fork5 (and the overall categorization) and **continue iterating until fewer than
127
- 20 log patterns remain**.
128
-
129
- Adjust the index pattern (e.g. `logs-*`, `logs-*-*`), time range, and bucket size (e.g. `30s`, `5m`, `1h`). Keep sample
130
- LIMIT small (10–20 by default; cap at 500). Use KEEP so the sample branch returns only summary fields, not full
131
- documents.
132
-
133
- ### Adding a KQL filter
134
-
135
- Narrow results with `KQL("...")`. The KQL expression is a single double-quoted string in ES|QL.
136
-
137
- **Escaping in the request body:** The query is sent inside JSON, so every double quote that is part of the ES|QL string
138
- must be escaped. Use `\"` for the quotes that wrap the KQL expression. If the KQL expression itself contains double
139
- quotes (e.g. a phrase like `message: "GET /health"`), escape those in the JSON as `\\\"` so the KQL parser receives
140
- literal quote characters.
141
-
142
- ```json
143
- POST /_query
144
- {
145
- "query": "FROM logs-* METADATA _id, _index | WHERE @timestamp >= TO_DATETIME(\"2025-03-06T10:00:00.000Z\") AND @timestamp <= TO_DATETIME(\"2025-03-06T11:00:00.000Z\") | WHERE KQL(\"service.name: checkout AND log.level: error\") | FORK (STATS count = COUNT(*) BY bucket = BUCKET(@timestamp, 1m) | SORT bucket) (STATS total = COUNT(*)) (SORT @timestamp DESC | LIMIT 10 | KEEP _id, _index, message, error.message, service.name, host.name, kubernetes.pod.name) (LIMIT 10000 | STATS COUNT(*) BY CATEGORIZE(message) | SORT `COUNT(*)` DESC | LIMIT 20) (LIMIT 10000 | STATS COUNT(*) BY CATEGORIZE(message) | SORT `COUNT(*)` ASC | LIMIT 20)"
146
- }
147
- ```
148
-
149
- ### Excluding noise with NOT
150
-
151
- Build the funnel by excluding known noise. In the request body, wrap the KQL string in `\"...\"` and escape any quotes
152
- inside the KQL expression as `\\\"`:
153
-
154
- ```json
155
- "query": "... | WHERE KQL(\"NOT message: \\\"GET /health\\\" AND NOT kubernetes.namespace: \\\"kube-system\\\"\") | ..."
156
- ```
157
-
158
- ```json
159
- "query": "... | WHERE KQL(\"error.message: * AND NOT message: \\\"Known benign warning\\\"\") | ..."
160
- ```
161
-
162
- ### Histogram grouped by a dimension
163
-
164
- Break down the trend by a second dimension (e.g. `log.level`, `service.name`) to see which level or entity drives the
165
- spike:
166
-
167
- ```text
168
- STATS count = COUNT(*) BY bucket = BUCKET(@timestamp, 1m), log.level
169
- ```
170
-
171
- Use a limited set of group values in the response to avoid explosion (e.g. top N by count, rest as `_other`).
172
-
173
- ## Examples
174
-
175
- ### Last hour of logs for a service
176
-
177
- ```json
178
- POST /_query
179
- {
180
- "query": "FROM logs-* METADATA _id, _index | WHERE @timestamp >= NOW() - 1 hour AND @timestamp <= NOW() | WHERE KQL(\"service.name: api-gateway\") | SORT @timestamp DESC | LIMIT 20"
181
- }
182
- ```
183
-
184
- ### Error logs with trend and samples
185
-
186
- ```json
187
- POST /_query
188
- {
189
- "query": "FROM logs-* METADATA _id, _index | WHERE @timestamp >= NOW() - 2 hours AND @timestamp <= NOW() | WHERE KQL(\"log.level: error\") | FORK (STATS count = COUNT(*) BY bucket = BUCKET(@timestamp, 5m) | SORT bucket) (STATS total = COUNT(*)) (SORT @timestamp DESC | LIMIT 15)"
190
- }
191
- ```
192
-
193
- ### Iterative funnel: NOT and NOT and NOT until the interesting bits
194
-
195
- Do not stop after one exclusion. Each round, add more NOTs for the current top noise, then run again.
196
-
197
- **Round 1:** `KQL("service.name: advertService")` → e.g. 55k logs; samples show "Returning N ads", "WARNING:
198
- request...", "received ad request".
199
-
200
- **Round 2:** Exclude the biggest noise:
201
- `KQL("service.name: advertService AND NOT message: *Returning* AND NOT message: *WARNING*")` → re-run, check new total
202
- and samples.
203
-
204
- **Round 3:** Exclude next noise (e.g. request/cache chatter):
205
- `KQL("service.name: advertService AND NOT message: *Returning* AND NOT message: *WARNING* AND NOT message: *received ad request* AND NOT message: *Adding* AND NOT message: *Cache miss*")` →
206
- re-run.
207
-
208
- **Round 4+:** Keep adding NOTs for whatever still dominates the samples (use fork4/fork5 categorization to see
209
- patterns). Continue until **fewer than 20 log patterns remain**; then what remains is the signal to report (e.g. "error
210
- fetching ads", encoding issues).
211
-
212
- Escaping: wrap the KQL string in `\"...\"` in the JSON; for quoted phrases inside KQL use `\\\"`.
213
-
214
- ## Guidelines
215
-
216
- - **Funnel: iterate with NOT.** Do not report findings after a single broad query. Add NOT clauses for dominant noise,
217
- re-run with the **full** filter (keep all previous NOTs), and repeat until **fewer than 20 log patterns remain** (use
218
- categorization fork4/fork5 to count). Stopping early yields noise, not signal.
219
- - **Histogram first:** Use the trend (fork1) to see when spikes or drops occur; narrow the time range around the spike
220
- if needed before adding more NOTs.
221
- - **Context minimization:** KEEP only summary fields in the sample branch; default LIMIT 10–20, cap at 500. Each funnel
222
- step is for deciding the next call; only the final narrowed result is for context and summary.
223
- - **Request body escaping:** The `query` value is JSON. Escape double quotes in the ES|QL string: `\"` for the KQL
224
- wrapper, `\\\"` for quotes inside the KQL expression (e.g. phrase values).
225
- - Use Elasticsearch date math for `start` and `end` (e.g. `now-1h`, `now-15m`) when building queries programmatically.
226
- - Choose bucket size from the time range: aim for roughly 20–50 buckets (e.g. 1h window → `1m` or `2m`).
227
- - Prefer ECS field names. In Observability index templates, OTel fields are aliased to ECS; see
228
- [references/log-search-reference.md](references/log-search-reference.md) for resource metadata field fallbacks
229
- (container, host, cluster, namespace, pod, workload).
230
- - **`log.level`:** Filtering or grouping by it can be OK but is often unreliable when levels are missing or mis-set;
231
- prefer message content or `error.message` for finding failures.
232
- - **Keyword searches:** Searching only for words like "error" or "fail" is often flawed (e.g. "no error", "error code
233
- 0"); prefer scoping by entity and funneling with NOT on real message patterns.
234
-
235
- ## References
236
-
237
- - [references/log-search-reference.md](references/log-search-reference.md) — ECS/OTel field mapping and index patterns
@@ -1,76 +0,0 @@
1
- # Log Search Reference
2
-
3
- Reference for Observability log search: index patterns, ECS/OpenTelemetry field mapping, and resource metadata
4
- fallbacks.
5
-
6
- ## Log index patterns
7
-
8
- - `logs-*-*,logs-*,filebeat-*` — common pattern for log data streams
9
- - `logs.*`- to also include wired streams
10
-
11
- ## ECS and OpenTelemetry
12
-
13
- Observability index templates provide **field aliases** that map OpenTelemetry fields to ECS. Query using **ECS field
14
- names** only; aliases handle the mapping.
15
-
16
- | ECS field | OTel / other notes |
17
- | ---------------------- | ------------------------ |
18
- | `message` | `body.text` (OTel) |
19
- | `log.level` | `severity_text` |
20
- | `trace.id` | `trace_id` |
21
- | `span.id` | `span_id` |
22
- | `service.name` | Service name |
23
- | `service.environment` | `deployment.environment` |
24
- | `host.name` | Host name |
25
- | `kubernetes.pod.name` | `k8s.pod.name` |
26
- | `kubernetes.namespace` | `k8s.namespace.name` |
27
- | `@timestamp` | Event time |
28
-
29
- ## Resource metadata field fallbacks
30
-
31
- For display or grouping, use the first available in each line (ECS then OTel aliases):
32
-
33
- | Resource | Preferred field(s) — try in order |
34
- | --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
35
- | Service | `service.name` |
36
- | Container | `kubernetes.container.name` → `k8s.container.name` → `container.name` |
37
- | Host/Node | `kubernetes.node.name` → `k8s.node.name` → `host.name` |
38
- | Cluster | `orchestrator.cluster.name` → `k8s.cluster.name` |
39
- | Namespace | `kubernetes.namespace` → `k8s.namespace.name` |
40
- | Pod | `kubernetes.pod.name` → `k8s.pod.name` |
41
- | Workload | One of: `kubernetes.deployment.name`, `k8s.deployment.name`, `kubernetes.replicaset.name`, `k8s.replicaset.name`, `kubernetes.statefulset.name`, `k8s.statefulset.name`, `kubernetes.daemonset.name`, `k8s.daemonset.name`, `kubernetes.job.name`, `k8s.job.name`, `kubernetes.cronjob.name`, `k8s.cronjob.name` |
42
-
43
- ## Message field fallback order
44
-
45
- When building a single “message” for display from a log document, use the first non-empty of:
46
-
47
- 1. `body.text` (OTel message)
48
- 2. `message`
49
- 3. `error.message`
50
- 4. `event.original`
51
- 5. `exception.message`
52
- 6. `error.exception.message`
53
- 7. `attributes.exception.message` (OTel)
54
-
55
- ## ES|QL query shape for logs
56
-
57
- Typical shape when using `POST /_query`:
58
-
59
- 1. `FROM <index_pattern> METADATA _id, _index` — include index and id for samples
60
- 2. `WHERE @timestamp >= ... AND @timestamp <= ...` — time range (ISO or date math)
61
- 3. Optional: `| WHERE KQL("...")` — KQL filter (escape `\"` in JSON)
62
- 4. For histogram + total + samples in one call:
63
- `| FORK (STATS ... BY bucket = BUCKET(@timestamp, <size>) ...) (STATS total = COUNT(*)) (SORT @timestamp DESC | LIMIT n | KEEP ...)`
64
-
65
- Bucket size examples: `30s`, `1m`, `5m`, `1h`. Keep sample LIMIT small (10–20 default; cap 500).
66
-
67
- ## Related documentation
68
-
69
- - [ES|QL FORK command](https://www.elastic.co/docs/reference/query-languages/esql/commands/fork) — branch limits,
70
- default LIMIT behavior, preview status
71
- - [ES|QL CATEGORIZE function](https://www.elastic.co/docs/reference/query-languages/esql/functions-operators/grouping-functions/categorize)
72
- — license requirement, grouping constraints
73
- - [ES|QL KQL function](https://www.elastic.co/docs/reference/query-languages/esql/functions-operators/search-functions/kql)
74
- — syntax and options
75
- - [Use the ES|QL REST API](https://www.elastic.co/docs/reference/query-languages/esql/esql-rest) — `POST /_query`
76
- endpoint, async queries, response formats
@@ -1,53 +0,0 @@
1
- ---
2
- name: payload-triage
3
- description: |
4
- Default SOC Copilot skill for analyzing raw payloads, mixed artifacts, suspicious snippets, and generic alerts.
5
- Use when the artifact type is unknown or when no specialized skill is a better fit.
6
- ---
7
-
8
- # Payload Triage
9
-
10
- Default analysis workflow for raw or mixed security artifacts.
11
-
12
- ## When to Use
13
-
14
- - artifact type is unknown
15
- - input is a raw payload, alert body, suspicious snippet, or mixed text
16
- - no specialized playbook is clearly a better match
17
-
18
- ## Workflow
19
-
20
- ### 1. Establish the artifact context
21
-
22
- - identify whether the input looks like log data, command content, email text, URL-heavy content, or generic payload
23
- - note missing context that limits confidence
24
-
25
- ### 2. Extract observable evidence
26
-
27
- Read [`../../references/ioc-extraction.md`](../../references/ioc-extraction.md) when extraction details matter.
28
-
29
- - identify visible IOCs
30
- - capture suspicious strings, commands, or infrastructure hints
31
- - keep direct observations separate from assumptions
32
-
33
- ### 3. Assess likely risk
34
-
35
- Read [`../../references/evidence-rules.md`](../../references/evidence-rules.md) for verdict and confidence guidance.
36
-
37
- - determine whether the artifact is benign, suspicious, malicious, or inconclusive
38
- - justify the verdict with explicit risk reasons
39
-
40
- ### 4. Suggest next analyst actions
41
-
42
- - prioritize low-risk validation steps first
43
- - recommend containment only when supported by evidence
44
-
45
- ## Output Contract
46
-
47
- Read [`../../references/output-contract.md`](../../references/output-contract.md).
48
-
49
- ## Guardrails
50
-
51
- - Do not force ATT&CK mapping for weak evidence.
52
- - Prefer `inconclusivo` over false precision.
53
- - Keep the result operational and concise.
@@ -1,51 +0,0 @@
1
- ---
2
- name: phishing-analysis
3
- description: |
4
- Specialized SOC Copilot skill for analyzing suspicious emails, headers, senders, embedded links, and attachment clues.
5
- Use when the user wants phishing triage or the artifact clearly contains email context.
6
- ---
7
-
8
- # Phishing Analysis
9
-
10
- Focused workflow for phishing and email-borne social engineering analysis.
11
-
12
- ## When to Use
13
-
14
- - input contains sender, recipient, headers, subject, message body, or attachment details
15
- - user asks whether a message is phishing
16
- - links inside an email are central to the analysis
17
-
18
- ## Workflow
19
-
20
- ### 1. Identify email evidence
21
-
22
- - extract sender, reply-to, recipient, domains, URLs, attachment names, and mail-routing clues when present
23
- - distinguish message content from transport metadata
24
-
25
- ### 2. Evaluate phishing indicators
26
-
27
- - look for sender and domain mismatch
28
- - inspect suspicious links, urgent requests, spoofing language, and attachment risk
29
- - weigh header or link mismatch more heavily than tone alone
30
-
31
- ### 3. Assess likely abuse pattern
32
-
33
- Read [`../../references/evidence-rules.md`](../../references/evidence-rules.md) for verdict discipline.
34
-
35
- - determine whether the artifact supports phishing, impersonation, malware delivery, or remains inconclusive
36
- - explain why the evidence supports that conclusion
37
-
38
- ### 4. Recommend safe handling
39
-
40
- - propose safe validation steps
41
- - suggest reporting, isolation, or user outreach when justified
42
-
43
- ## Output Contract
44
-
45
- Read [`../../references/output-contract.md`](../../references/output-contract.md).
46
-
47
- ## Guardrails
48
-
49
- - Do not mark an email as phishing based only on generic urgency language.
50
- - Separate visible evidence from reputation-dependent conclusions.
51
- - If no header evidence exists, state that limitation clearly.
@@ -1,143 +0,0 @@
1
- ---
2
- name: prd
3
- description: 'Generate high-quality Product Requirements Documents (PRDs) for software systems and AI-powered features. Includes executive summaries, user stories, technical specifications, and risk analysis.'
4
- license: MIT
5
- ---
6
-
7
- # Product Requirements Document (PRD)
8
-
9
- ## Overview
10
-
11
- Design comprehensive, production-grade Product Requirements Documents (PRDs) that bridge the gap between business vision and technical execution. This skill works for modern software systems, ensuring that requirements are clearly defined.
12
-
13
- ## When to Use
14
-
15
- Use this skill when:
16
-
17
- - Starting a new product or feature development cycle
18
- - Translating a vague idea into a concrete technical specification
19
- - Defining requirements for AI-powered features
20
- - Stakeholders need a unified "source of truth" for project scope
21
- - User asks to "write a PRD", "document requirements", or "plan a feature"
22
-
23
- ---
24
-
25
- ## Operational Workflow
26
-
27
- ### Phase 1: Discovery (The Interview)
28
-
29
- Before writing a single line of the PRD, you **MUST** interrogate the user to fill knowledge gaps. Do not assume context.
30
-
31
- **Ask about:**
32
-
33
- - **The Core Problem**: Why are we building this now?
34
- - **Success Metrics**: How do we know it worked?
35
- - **Constraints**: Budget, tech stack, or deadline?
36
-
37
- ### Phase 2: Analysis & Scoping
38
-
39
- Synthesize the user's input. Identify dependencies and hidden complexities.
40
-
41
- - Map out the **User Flow**.
42
- - Define **Non-Goals** to protect the timeline.
43
-
44
- ### Phase 3: Technical Drafting
45
-
46
- Generate the document using the **Strict PRD Schema** below.
47
-
48
- ---
49
-
50
- ## PRD Quality Standards
51
-
52
- ### Requirements Quality
53
-
54
- Use concrete, measurable criteria. Avoid "fast", "easy", or "intuitive".
55
-
56
- ```diff
57
- # Vague (BAD)
58
- - The search should be fast and return relevant results.
59
- - The UI must look modern and be easy to use.
60
-
61
- # Concrete (GOOD)
62
- + The search must return results within 200ms for a 10k record dataset.
63
- + The search algorithm must achieve >= 85% Precision@10 in benchmark evals.
64
- + The UI must follow the 'Vercel/Next.js' design system and achieve 100% Lighthouse Accessibility score.
65
- ```
66
-
67
- ---
68
-
69
- ## Strict PRD Schema
70
-
71
- You **MUST** follow this exact structure for the output:
72
-
73
- ### 1. Executive Summary
74
-
75
- - **Problem Statement**: 1-2 sentences on the pain point.
76
- - **Proposed Solution**: 1-2 sentences on the fix.
77
- - **Success Criteria**: 3-5 measurable KPIs.
78
-
79
- ### 2. User Experience & Functionality
80
-
81
- - **User Personas**: Who is this for?
82
- - **User Stories**: `As a [user], I want to [action] so that [benefit].`
83
- - **Acceptance Criteria**: Bulleted list of "Done" definitions for each story.
84
- - **Non-Goals**: What are we NOT building?
85
-
86
- ### 3. AI System Requirements (If Applicable)
87
-
88
- - **Tool Requirements**: What tools and APIs are needed?
89
- - **Evaluation Strategy**: How to measure output quality and accuracy.
90
-
91
- ### 4. Technical Specifications
92
-
93
- - **Architecture Overview**: Data flow and component interaction.
94
- - **Integration Points**: APIs, DBs, and Auth.
95
- - **Security & Privacy**: Data handling and compliance.
96
-
97
- ### 5. Risks & Roadmap
98
-
99
- - **Phased Rollout**: MVP -> v1.1 -> v2.0.
100
- - **Technical Risks**: Latency, cost, or dependency failures.
101
-
102
- ---
103
-
104
- ## Implementation Guidelines
105
-
106
- ### DO (Always)
107
-
108
- - **Define Testing**: For AI systems, specify how to test and validate output quality.
109
- - **Iterate**: Present a draft and ask for feedback on specific sections.
110
-
111
- ### DON'T (Avoid)
112
-
113
- - **Skip Discovery**: Never write a PRD without asking at least 2 clarifying questions first.
114
- - **Hallucinate Constraints**: If the user didn't specify a tech stack, ask or label it as `TBD`.
115
-
116
- ---
117
-
118
- ## Example: Intelligent Search System
119
-
120
- ### 1. Executive Summary
121
-
122
- **Problem**: Users struggle to find specific documentation snippets in massive repositories.
123
- **Solution**: An intelligent search system that provides direct answers with source citations.
124
- **Success**:
125
-
126
- - Reduce search time by 50%.
127
- - Citation accuracy >= 95%.
128
-
129
- ### 2. User Stories
130
-
131
- - **Story**: As a developer, I want to ask natural language questions so I don't have to guess keywords.
132
- - **AC**:
133
- - Supports multi-turn clarification.
134
- - Returns code blocks with "Copy" button.
135
-
136
- ### 3. AI System Architecture
137
-
138
- - **Tools Required**: `codesearch`, `grep`, `webfetch`.
139
-
140
- ### 4. Evaluation
141
-
142
- - **Benchmark**: Test with 50 common developer questions.
143
- - **Pass Rate**: 90% must match expected citations.