crowdsec-local-mcp 0.1.0__py3-none-any.whl → 0.7.0.post1.dev0__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- crowdsec_local_mcp/__init__.py +6 -1
- crowdsec_local_mcp/__main__.py +1 -1
- crowdsec_local_mcp/_version.py +1 -0
- crowdsec_local_mcp/compose/scenario-test/.gitignore +1 -0
- crowdsec_local_mcp/compose/scenario-test/docker-compose.yml +19 -0
- crowdsec_local_mcp/compose/scenario-test/scenarios/.gitkeep +0 -0
- crowdsec_local_mcp/compose/waf-test/docker-compose.yml +5 -6
- crowdsec_local_mcp/compose/waf-test/nginx/Dockerfile +3 -2
- crowdsec_local_mcp/mcp_core.py +114 -19
- crowdsec_local_mcp/mcp_scenarios.py +579 -23
- crowdsec_local_mcp/mcp_waf.py +567 -337
- crowdsec_local_mcp/prompts/prompt-expr-helpers.txt +514 -0
- crowdsec_local_mcp/prompts/prompt-scenario-deploy.txt +70 -21
- crowdsec_local_mcp/prompts/prompt-scenario.txt +26 -2
- crowdsec_local_mcp/prompts/prompt-waf-tests.txt +101 -0
- crowdsec_local_mcp/prompts/prompt-waf-top-level.txt +31 -0
- crowdsec_local_mcp/prompts/prompt-waf.txt +0 -26
- crowdsec_local_mcp/setup_cli.py +375 -0
- crowdsec_local_mcp-0.7.0.post1.dev0.dist-info/METADATA +114 -0
- crowdsec_local_mcp-0.7.0.post1.dev0.dist-info/RECORD +38 -0
- {crowdsec_local_mcp-0.1.0.dist-info → crowdsec_local_mcp-0.7.0.post1.dev0.dist-info}/entry_points.txt +1 -0
- crowdsec_local_mcp-0.1.0.dist-info/METADATA +0 -93
- crowdsec_local_mcp-0.1.0.dist-info/RECORD +0 -30
- {crowdsec_local_mcp-0.1.0.dist-info → crowdsec_local_mcp-0.7.0.post1.dev0.dist-info}/WHEEL +0 -0
- {crowdsec_local_mcp-0.1.0.dist-info → crowdsec_local_mcp-0.7.0.post1.dev0.dist-info}/licenses/LICENSE +0 -0
- {crowdsec_local_mcp-0.1.0.dist-info → crowdsec_local_mcp-0.7.0.post1.dev0.dist-info}/top_level.txt +0 -0
|
@@ -0,0 +1,514 @@
|
|
|
1
|
+
# Expr helpers list
|
|
2
|
+
|
|
3
|
+
You can use those helpers whenever you write an expression (scenario filter, groupby, distinct, ...)
|
|
4
|
+
|
|
5
|
+
## IP Helpers
|
|
6
|
+
|
|
7
|
+
|
|
8
|
+
### `IpInRange(IPStr, RangeStr) bool`
|
|
9
|
+
|
|
10
|
+
Returns true if the IP `IPStr` is contained in the IP range `RangeStr` (uses `net.ParseCIDR`)
|
|
11
|
+
|
|
12
|
+
> `IpInRange("1.2.3.4", "1.2.3.0/24")`
|
|
13
|
+
|
|
14
|
+
### `IpToRange(IPStr, MaskStr) IpStr`
|
|
15
|
+
|
|
16
|
+
Returns the subnet of the IP with the request cidr size.
|
|
17
|
+
It is intended for scenarios taking actions against the range of an IP, not the IP itself :
|
|
18
|
+
|
|
19
|
+
```yaml
|
|
20
|
+
type: leaky
|
|
21
|
+
...
|
|
22
|
+
scope:
|
|
23
|
+
type: Range
|
|
24
|
+
expression: IpToRange(evt.Meta.source_ip, "/16")
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
> `IpToRange("192.168.0.1", "24")` returns `192.168.0.0/24`
|
|
28
|
+
|
|
29
|
+
> `IpToRange("192.168.42.1", "16")` returns `192.168.0.0/16`
|
|
30
|
+
|
|
31
|
+
|
|
32
|
+
### `IsIP(ip string) bool`
|
|
33
|
+
|
|
34
|
+
Returns true if it's a valid IP (v4 or v6).
|
|
35
|
+
|
|
36
|
+
> `IsIP("2001:0db8:85a3:0000:0000:8a2e:0370:7334")`
|
|
37
|
+
|
|
38
|
+
> `IsIP("1.2.3.4")`
|
|
39
|
+
|
|
40
|
+
> `IsIP(Alert.GetValue())`
|
|
41
|
+
|
|
42
|
+
|
|
43
|
+
### `IsIPV4(ip string) bool`
|
|
44
|
+
|
|
45
|
+
Returns true if it's a valid IPv4.
|
|
46
|
+
|
|
47
|
+
> `IsIPV4("1.2.3.4")`
|
|
48
|
+
|
|
49
|
+
> `IsIPV4(Alert.GetValue())`
|
|
50
|
+
|
|
51
|
+
### `IsIPV6(ip string) bool`
|
|
52
|
+
|
|
53
|
+
Returns true if it's a valid IPv6.
|
|
54
|
+
|
|
55
|
+
> `IsIPV6("2001:0db8:85a3:0000:0000:8a2e:0370:7334")`
|
|
56
|
+
|
|
57
|
+
> `IsIPV6(Alert.GetValue())`
|
|
58
|
+
|
|
59
|
+
### `LookupHost(host string) []string`
|
|
60
|
+
|
|
61
|
+
:::warning
|
|
62
|
+
* Only use this function within postoverflows as it is can be very slow
|
|
63
|
+
* Note if you whitelist a domain behind a CDN provider, all domains using the same CDN provider will also be whitelisted
|
|
64
|
+
* Do not use variables within the function as this can be untrusted user input
|
|
65
|
+
:::
|
|
66
|
+
Returns []string ip addresses that resolvable to the hostname EG: `LookupHost('mydomain.tld') => ['1.2.3.4', '5.6.7.8']`
|
|
67
|
+
```yaml
|
|
68
|
+
name: me/my_cool_whitelist
|
|
69
|
+
description: lets whitelist our own IP
|
|
70
|
+
whitelist:
|
|
71
|
+
reason: dont ban my IP
|
|
72
|
+
expression:
|
|
73
|
+
- evt.Overflow.Alert.Source.IP in LookupHost('mydomain.tld')
|
|
74
|
+
# This can be useful when you have a dynamic ip and use dynamic DNS providers
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
### `GeoIPEnrich(ip string) *geoip2.City`
|
|
78
|
+
|
|
79
|
+
Performs a geo lookup for IP and returns the associated [geoip2.City](https://pkg.go.dev/github.com/oschwald/geoip2-golang#City) object.
|
|
80
|
+
|
|
81
|
+
|
|
82
|
+
### `GeoIPASNEnrich(ip string) *geoip2.ASN`
|
|
83
|
+
|
|
84
|
+
Performs a geo lookup for IP and returns the associated [geoip2.ASN](https://pkg.go.dev/github.com/oschwald/geoip2-golang#ASN) object.
|
|
85
|
+
|
|
86
|
+
### `GeoIPRangeEnrich(ip string) net.IPNet`
|
|
87
|
+
|
|
88
|
+
Returns the `net.IPNet` object associated to the IP if possible.
|
|
89
|
+
|
|
90
|
+
## URI Related helpers
|
|
91
|
+
|
|
92
|
+
### `ParseUri(string) map[string][]string`
|
|
93
|
+
|
|
94
|
+
Parses an URI into a map of string list.
|
|
95
|
+
|
|
96
|
+
`ParseURI("/foo?a=1&b=2")` would return :
|
|
97
|
+
|
|
98
|
+
```
|
|
99
|
+
{
|
|
100
|
+
"a": []string{"1"},
|
|
101
|
+
"b": []string{"2"}
|
|
102
|
+
}
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
### `PathUnescape(string) string`
|
|
106
|
+
|
|
107
|
+
`PathUnescape` does the inverse transformation of PathEscape, converting each 3-byte encoded substring of the form "%AB" into the hex-decoded byte 0xAB. It returns an error if any % is not followed by two hexadecimal digits.
|
|
108
|
+
|
|
109
|
+
### `PathEscape(string) string`
|
|
110
|
+
|
|
111
|
+
`PathEscape` escapes the string so it can be safely placed inside a URL path segment, replacing special characters (including /) with %XX sequences as needed.
|
|
112
|
+
|
|
113
|
+
### `QueryUnescape(string) string`
|
|
114
|
+
|
|
115
|
+
`QueryUnescape` does the inverse transformation of QueryEscape, converting each 3-byte encoded substring of the form "%AB" into the hex-decoded byte 0xAB. It returns an error if any % is not followed by two hexadecimal digits.
|
|
116
|
+
|
|
117
|
+
### `QueryEscape(string) string`
|
|
118
|
+
|
|
119
|
+
`QueryEscape` escapes the string so it can be safely placed inside a URL query.
|
|
120
|
+
|
|
121
|
+
### `ExtractQueryParam(query string, param string) []string`
|
|
122
|
+
|
|
123
|
+
`ExtractQueryParam` extract the `param` parameter value from the URL query `query` and returns the list of values.
|
|
124
|
+
|
|
125
|
+
> `any(ExtractQueryParam("/foo?id=1&b=2", "id"), { # == "1" })` returns true if at least one of the `id` parameter value is equal to `1`
|
|
126
|
+
|
|
127
|
+
## JSON Helpers
|
|
128
|
+
|
|
129
|
+
### `UnmarshalJSON(jsonBlob string, out map[string]interface{}, targetKey string)`
|
|
130
|
+
|
|
131
|
+
`UnmarshalJSON` allows to unmarshal a full json object into the `out` map, under the `targetKey` key.
|
|
132
|
+
|
|
133
|
+
In most situation, the `evt.Unmarshaled` field will be used to store the unmarshaled json object.
|
|
134
|
+
|
|
135
|
+
```yaml
|
|
136
|
+
filter: |
|
|
137
|
+
evt.Parsed.program == "my-program"
|
|
138
|
+
statics:
|
|
139
|
+
- parsed: json_parsed
|
|
140
|
+
expression: UnmarshalJSON(evt.Line.Raw, evt.Unmarshaled, "message")
|
|
141
|
+
- meta: user
|
|
142
|
+
expression: evt.Unmarshaled.message.user
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
|
|
146
|
+
### `JsonExtract(JsonBlob, FieldName) string`
|
|
147
|
+
|
|
148
|
+
Extract the `FieldName` from the `JsonBlob` and returns it as a string. (binding on [jsonparser](https://github.com/buger/jsonparser/))
|
|
149
|
+
|
|
150
|
+
> `JsonExtract(evt.Parsed.some_json_blob, "foo.bar[0].one_item")`
|
|
151
|
+
|
|
152
|
+
### `JsonExtractSlice(JsonBlob, FieldName) []interface{}`
|
|
153
|
+
|
|
154
|
+
Extract the JSON array in `FieldName` from `JsonBlob` and returns it as a go slice.
|
|
155
|
+
|
|
156
|
+
Returns nil if the field does not exist or is not an array.
|
|
157
|
+
|
|
158
|
+
> `JsonExtractSlice(evt.Parsed.message, "params")[0]['value']['login']`
|
|
159
|
+
|
|
160
|
+
> `any(JsonExtractSlice(evt.Parsed.message, "params"), {.key == 'user' && .value.login != ''})`
|
|
161
|
+
|
|
162
|
+
### `JsonExtractObject(JsonBlob, FieldName) map[string]interface{}`
|
|
163
|
+
|
|
164
|
+
Extract the JSON object in `FieldName` from `JsonBlob` and returns it as a go map.
|
|
165
|
+
|
|
166
|
+
Returns `nil` if the field does not exist or does is not an object.
|
|
167
|
+
|
|
168
|
+
> `JsonExtractObject(evt.Parsed.message, "params.user")["login"]`
|
|
169
|
+
|
|
170
|
+
### `ToJsonString(Obj) string`
|
|
171
|
+
|
|
172
|
+
Returns the JSON representation of `obj`
|
|
173
|
+
|
|
174
|
+
Returns an empty string if `obj` cannot be serialized to JSON.
|
|
175
|
+
|
|
176
|
+
> `ToJsonString(JsonExtractSlice(evt.Parsed.message, "params"))`
|
|
177
|
+
|
|
178
|
+
|
|
179
|
+
## XML Helpers
|
|
180
|
+
|
|
181
|
+
|
|
182
|
+
### `XMLGetAttributeValue(xmlString string, path string, attributeName string) string`
|
|
183
|
+
|
|
184
|
+
Returns the value of `attribute` in the XML node identified by the XPath query `path`.
|
|
185
|
+
|
|
186
|
+
> `XMLGetAttributeValue(evt.Line.Raw, "/Event/System[1]/Provider", "Name")`
|
|
187
|
+
|
|
188
|
+
### `XMLGetNodeValue(xmlString string, path string) string`
|
|
189
|
+
|
|
190
|
+
Returns the content of the XML node identified by the XPath query `path`.
|
|
191
|
+
|
|
192
|
+
> `XMLGetNodeValue(evt.Line.Raw, "/Event/System[1]/EventID")`
|
|
193
|
+
|
|
194
|
+
|
|
195
|
+
## Key-Value Helpers
|
|
196
|
+
|
|
197
|
+
### `ParseKV(kvString string, out map[string]interface{}, targetKey string)`
|
|
198
|
+
|
|
199
|
+
Parse a key-value string (such as `key=value foo=bar fu="a string"` ) into the `out` map, under the `targetKey` key.
|
|
200
|
+
|
|
201
|
+
In most situation, the `evt.Unmarshaled` field will be used to store the object.
|
|
202
|
+
|
|
203
|
+
```yaml
|
|
204
|
+
filter: |
|
|
205
|
+
evt.Parsed.program == "my-program"
|
|
206
|
+
statics:
|
|
207
|
+
- parsed: kv_parsed
|
|
208
|
+
expression: ParseKV(evt.Line.Raw, evt.Unmarshaled, "message")
|
|
209
|
+
- meta: user
|
|
210
|
+
expression: evt.Unmarshaled.message.user
|
|
211
|
+
```
|
|
212
|
+
|
|
213
|
+
## Strings
|
|
214
|
+
|
|
215
|
+
### `Atof(string) float64`
|
|
216
|
+
|
|
217
|
+
Parses a string representation of a float number to an actual float number (binding on `strconv.ParseFloat`)
|
|
218
|
+
|
|
219
|
+
> `Atof(evt.Parsed.tcp_port)`
|
|
220
|
+
|
|
221
|
+
### `Upper(string) string`
|
|
222
|
+
|
|
223
|
+
Returns the uppercase version of the string
|
|
224
|
+
|
|
225
|
+
> `Upper("yop")`
|
|
226
|
+
|
|
227
|
+
### `Lower(string) string`
|
|
228
|
+
|
|
229
|
+
Returns the lowercase version of the string
|
|
230
|
+
|
|
231
|
+
> `Lower("YOP")`
|
|
232
|
+
|
|
233
|
+
### `Sprintf(format string, a ...interface{}) string`
|
|
234
|
+
|
|
235
|
+
[Official doc](https://pkg.go.dev/fmt#Sprintf) : Sprintf formats according to a format specifier and returns the resulting string.
|
|
236
|
+
|
|
237
|
+
> `Sprintf('%dh', 1)` returns `1h`
|
|
238
|
+
|
|
239
|
+
### `Match(pattern string, object string) bool`
|
|
240
|
+
|
|
241
|
+
`Match` returns true if the object string matches the pattern. Pattern only supports wildcard :
|
|
242
|
+
- `*` multi-character wildcard (including zero-length)
|
|
243
|
+
- `?` single character wildcard
|
|
244
|
+
|
|
245
|
+
> `Match('to?o*', 'totoooooo')` returns `true`
|
|
246
|
+
|
|
247
|
+
### `Fields(s string) []string`
|
|
248
|
+
|
|
249
|
+
`Fields` splits the string s around each instance of one or more consecutive white space characters, as defined by unicode.IsSpace, returning a slice of substrings of s or an empty slice if s contains only white space.
|
|
250
|
+
|
|
251
|
+
### `Index(s string, substr string) int`
|
|
252
|
+
|
|
253
|
+
Index returns the index of the first instance of substr in s, or -1 if substr is not present in s.
|
|
254
|
+
|
|
255
|
+
### `IndexAny(s string, chars string) int`
|
|
256
|
+
|
|
257
|
+
IndexAny returns the index of the first instance of any Unicode code point from chars in s, or -1 if no Unicode code point from chars is present in s.
|
|
258
|
+
|
|
259
|
+
### `Join(elems []string, sep string) string`
|
|
260
|
+
|
|
261
|
+
Join concatenates the elements of its first argument to create a single string. The separator string sep is placed between elements in the resulting string.
|
|
262
|
+
|
|
263
|
+
### `Split(s string, sep string) []string`
|
|
264
|
+
|
|
265
|
+
Split slices s into all substrings separated by sep and returns a slice of the substrings between those separators.
|
|
266
|
+
|
|
267
|
+
If s does not contain sep and sep is not empty, Split returns a slice of length 1 whose only element is s.
|
|
268
|
+
|
|
269
|
+
If sep is empty, Split splits after each UTF-8 sequence. If both s and sep are empty, Split returns an empty slice.
|
|
270
|
+
|
|
271
|
+
It is equivalent to SplitN with a count of -1.
|
|
272
|
+
|
|
273
|
+
To split around the first instance of a separator, see Cut.
|
|
274
|
+
|
|
275
|
+
### `SplitAfter(s string, sep string) []string`
|
|
276
|
+
|
|
277
|
+
SplitAfter slices s into all substrings after each instance of sep and returns a slice of those substrings.
|
|
278
|
+
|
|
279
|
+
If s does not contain sep and sep is not empty, SplitAfter returns a slice of length 1 whose only element is s.
|
|
280
|
+
|
|
281
|
+
If sep is empty, SplitAfter splits after each UTF-8 sequence. If both s and sep are empty, SplitAfter returns an empty slice.
|
|
282
|
+
|
|
283
|
+
It is equivalent to SplitAfterN with a count of -1.
|
|
284
|
+
|
|
285
|
+
### `SplitAfterN(s string, sep string, n int) []string `
|
|
286
|
+
|
|
287
|
+
SplitAfterN slices s into substrings after each instance of sep and returns a slice of those substrings.
|
|
288
|
+
|
|
289
|
+
The count determines the number of substrings to return:
|
|
290
|
+
|
|
291
|
+
```
|
|
292
|
+
n > 0: at most n substrings; the last substring will be the unsplit remainder.
|
|
293
|
+
n == 0: the result is nil (zero substrings)
|
|
294
|
+
n < 0: all substrings
|
|
295
|
+
```
|
|
296
|
+
|
|
297
|
+
Edge cases for s and sep (for example, empty strings) are handled as described in the documentation for SplitAfter.
|
|
298
|
+
|
|
299
|
+
### `SplitN(s string, sep string, n int) []string`
|
|
300
|
+
|
|
301
|
+
|
|
302
|
+
SplitN slices s into substrings separated by sep and returns a slice of the substrings between those separators.
|
|
303
|
+
|
|
304
|
+
The count determines the number of substrings to return:
|
|
305
|
+
|
|
306
|
+
```
|
|
307
|
+
n > 0: at most n substrings; the last substring will be the unsplit remainder.
|
|
308
|
+
n == 0: the result is nil (zero substrings)
|
|
309
|
+
n < 0: all substrings
|
|
310
|
+
```
|
|
311
|
+
|
|
312
|
+
Edge cases for s and sep (for example, empty strings) are handled as described in the documentation for Split.
|
|
313
|
+
|
|
314
|
+
To split around the first instance of a separator, see Cut.
|
|
315
|
+
|
|
316
|
+
### `Replace(s string, old string, new string, n int) string`
|
|
317
|
+
|
|
318
|
+
Replace returns a copy of the string s with the first n non-overlapping instances of old replaced by new. If old is empty, it matches at the beginning of the string and after each UTF-8 sequence, yielding up to k+1 replacements for a k-rune string. If n < 0, there is no limit on the number of replacements.
|
|
319
|
+
|
|
320
|
+
### `ReplaceAll(s string, old string, new string) string`
|
|
321
|
+
|
|
322
|
+
ReplaceAll returns a copy of the string s with all non-overlapping instances of old replaced by new. If old is empty, it matches at the beginning of the string and after each UTF-8 sequence, yielding up to k+1 replacements for a k-rune string.
|
|
323
|
+
|
|
324
|
+
### `Trim(s string, cutset string) string`
|
|
325
|
+
|
|
326
|
+
Trim returns a slice of the string s with all leading and trailing Unicode code points contained in cutset removed.
|
|
327
|
+
|
|
328
|
+
### `TrimLeft(s string, cutset string) string`
|
|
329
|
+
|
|
330
|
+
TrimLeft returns a slice of the string s with all leading Unicode code points contained in cutset removed.
|
|
331
|
+
|
|
332
|
+
To remove a prefix, use TrimPrefix instead.
|
|
333
|
+
|
|
334
|
+
### `TrimRight(s string, cutset string) string`
|
|
335
|
+
|
|
336
|
+
TrimRight returns a slice of the string s, with all trailing Unicode code points contained in cutset removed.
|
|
337
|
+
|
|
338
|
+
To remove a suffix, use TrimSuffix instead.
|
|
339
|
+
|
|
340
|
+
### `TrimSpace(s string) string`
|
|
341
|
+
|
|
342
|
+
TrimSpace returns a slice of the string s, with all leading and trailing white space removed, as defined by Unicode.
|
|
343
|
+
|
|
344
|
+
### `TrimPrefix(s string, prefix string) string`
|
|
345
|
+
|
|
346
|
+
TrimPrefix returns s without the provided leading prefix string. If s doesn't start with prefix, s is returned unchanged.
|
|
347
|
+
|
|
348
|
+
### `TrimSuffix(s string, suffix string) string`
|
|
349
|
+
|
|
350
|
+
TrimSuffix returns s without the provided trailing suffix string. If s doesn't end with suffix, s is returned unchanged.
|
|
351
|
+
|
|
352
|
+
### `ToString(s) string`
|
|
353
|
+
|
|
354
|
+
Returns the string representation of s, if available (does a `s.(sttring)`).
|
|
355
|
+
|
|
356
|
+
### `LogInfo(format string, ...)`
|
|
357
|
+
|
|
358
|
+
Performs a logging call with the provided parameters, see [logrus reference](https://pkg.go.dev/github.com/sirupsen/logrus#Infof) for formatting info.
|
|
359
|
+
|
|
360
|
+
## LibInjection Helpers
|
|
361
|
+
|
|
362
|
+
### `LibInjectionIsSQLI(str) bool`
|
|
363
|
+
|
|
364
|
+
Use [libinjection](https://github.com/libinjection/libinjection) to detect SQL injection in `str`.
|
|
365
|
+
|
|
366
|
+
> `LibInjectionIsSQLI(evt.Parsed.http_args)`
|
|
367
|
+
|
|
368
|
+
### `LibInjectionIsXSS(str) bool`
|
|
369
|
+
|
|
370
|
+
Use [libinjection](https://github.com/libinjection/libinjection) to detect XSS in `str`.
|
|
371
|
+
|
|
372
|
+
> `LibInjectionIsXSS(evt.Parsed.http_args)`
|
|
373
|
+
|
|
374
|
+
## Time Helpers
|
|
375
|
+
|
|
376
|
+
### `TimeNow() string`
|
|
377
|
+
|
|
378
|
+
Return RFC3339 formatted time
|
|
379
|
+
|
|
380
|
+
> `TimeNow()`
|
|
381
|
+
|
|
382
|
+
### `ParseUnix(unix string) string`
|
|
383
|
+
```
|
|
384
|
+
ParseUnix("1672239773.3590894") -> "2022-12-28T15:02:53Z"
|
|
385
|
+
ParseUnix("1672239773") -> "2022-12-28T15:02:53Z"
|
|
386
|
+
ParseUnix("notatimestamp") -> ""
|
|
387
|
+
```
|
|
388
|
+
Parses unix timestamp string and returns RFC3339 formatted time
|
|
389
|
+
|
|
390
|
+
## Stash Helpers
|
|
391
|
+
|
|
392
|
+
### `GetFromStash(cache string, key string)`
|
|
393
|
+
|
|
394
|
+
`GetFromStash` retrieves the value for `key` in the named `cache`.
|
|
395
|
+
The cache are usually populated by [parser's stash section](/log_processor/parsers/format.md#stash).
|
|
396
|
+
An empty string if the key doesn't exist (or has been evicted), and error is raised if the `cache` doesn't exist.
|
|
397
|
+
|
|
398
|
+
## Others
|
|
399
|
+
|
|
400
|
+
### `IsIPV4(ip string) bool`
|
|
401
|
+
|
|
402
|
+
Returns true if it's a valid IPv4.
|
|
403
|
+
|
|
404
|
+
> `IsIPV4("192.168.1.1")`
|
|
405
|
+
|
|
406
|
+
> `IsIPV4(Alert.GetValue())`
|
|
407
|
+
|
|
408
|
+
### `IsIP(ip string) bool`
|
|
409
|
+
|
|
410
|
+
Returns true if it's a valid IP (v4 or v6).
|
|
411
|
+
|
|
412
|
+
> `IsIP("2001:0db8:85a3:0000:0000:8a2e:0370:7334")`
|
|
413
|
+
|
|
414
|
+
> `IsIP("192.168.1.1")`
|
|
415
|
+
|
|
416
|
+
> `IsIP(Alert.GetValue())`
|
|
417
|
+
|
|
418
|
+
### `GetDecisionsCount(value string) int`
|
|
419
|
+
|
|
420
|
+
Returns the number of existing decisions in the database with the same value.
|
|
421
|
+
This can return expired decisions if they have not been flushed yet.
|
|
422
|
+
|
|
423
|
+
> `GetDecisionsCount("192.168.1.1")`
|
|
424
|
+
|
|
425
|
+
> `GetDecisionsCount(Alert.GetValue())`
|
|
426
|
+
|
|
427
|
+
### `GetDecisionsSinceCount(value string, since string) int`
|
|
428
|
+
|
|
429
|
+
Returns the number of existing decisions in the database with the same value since duration string (valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h".).
|
|
430
|
+
This can return expired decisions if they have not been flushed yet.
|
|
431
|
+
|
|
432
|
+
> `GetDecisionsSinceCount("192.168.1.1", "7h")`
|
|
433
|
+
|
|
434
|
+
> `GetDecisionsSinceCount(Alert.GetValue(), "30min")`
|
|
435
|
+
|
|
436
|
+
### `GetActiveDecisionsCount(value string) int`
|
|
437
|
+
|
|
438
|
+
Returns the number of active decisions in the database with the same value.
|
|
439
|
+
|
|
440
|
+
> `GetActiveDecisionsCount(Alert.GetValue())`
|
|
441
|
+
|
|
442
|
+
|
|
443
|
+
### `GetActiveDecisionsTimeLeft(value string) time.Duration`
|
|
444
|
+
|
|
445
|
+
Returns the time left for the longest decision associated with the value.
|
|
446
|
+
|
|
447
|
+
The returned value type is `time.Duration`, so you can use all the [time.Duration methods](https://pkg.go.dev/time#Duration).
|
|
448
|
+
|
|
449
|
+
> `GetActiveDecisionsTimeLeft(Alert.GetValue())`
|
|
450
|
+
|
|
451
|
+
> `GetActiveDecisionsTimeLeft(Alert.GetValue()).Hours() > 1"
|
|
452
|
+
|
|
453
|
+
### `KeyExists(key string, map map[string]interface{}) bool`
|
|
454
|
+
|
|
455
|
+
Return true if the `key` exists in the map.
|
|
456
|
+
|
|
457
|
+
### `Get(arr []string, index int) string`
|
|
458
|
+
|
|
459
|
+
Returns the index'th entry of arr, or `""`.
|
|
460
|
+
|
|
461
|
+
|
|
462
|
+
### `Distance(lat1 string, long1 string, lat2 string, long2 string) float64`
|
|
463
|
+
|
|
464
|
+
Computes the distance in kilometers between the set of coordinates represented by lat1/long1 and lat2/long2.
|
|
465
|
+
Designed to implement impossible travel and similar scenarios:
|
|
466
|
+
|
|
467
|
+
```yaml
|
|
468
|
+
type: conditional
|
|
469
|
+
name: demo/impossible-travel
|
|
470
|
+
description: "test"
|
|
471
|
+
filter: "evt.Meta.log_type == 'fake_ok'"
|
|
472
|
+
groupby: evt.Meta.user
|
|
473
|
+
capacity: -1
|
|
474
|
+
condition: |
|
|
475
|
+
len(queue.Queue) >= 2
|
|
476
|
+
and Distance(queue.Queue[-1].Enriched.Latitude, queue.Queue[-1].Enriched.Longitude,
|
|
477
|
+
queue.Queue[-2].Enriched.Latitude, queue.Queue[-2].Enriched.Longitude) > 100
|
|
478
|
+
leakspeed: 3h
|
|
479
|
+
labels:
|
|
480
|
+
type: fraud
|
|
481
|
+
```
|
|
482
|
+
Notes:
|
|
483
|
+
- Will return `0` if either set of coordinates is nil (ie. IP couldn't be geoloc)
|
|
484
|
+
- Assumes that the earth is spherical and uses the haversine formula.
|
|
485
|
+
|
|
486
|
+
### `Hostname() string`
|
|
487
|
+
|
|
488
|
+
Returns the hostname of the machine.
|
|
489
|
+
|
|
490
|
+
## Alert specific helpers
|
|
491
|
+
|
|
492
|
+
### `Alert.Remediation bool`
|
|
493
|
+
|
|
494
|
+
Is `true` if the alert asks for a remediation. Will be true for alerts from scenarios with `remediation: true` flag. Will be false for alerts from manual `cscli decisions add` commands (as they come with their own decision).
|
|
495
|
+
|
|
496
|
+
### `Alert.GetScenario() string`
|
|
497
|
+
|
|
498
|
+
Returns the name of the scenario that triggered the alert.
|
|
499
|
+
|
|
500
|
+
### `Alert.GetScope() string`
|
|
501
|
+
|
|
502
|
+
Returns the scope of an alert. Most common value is `Ip`. `Country` and `As` are generally used for more distributed attacks detection/remediation.
|
|
503
|
+
|
|
504
|
+
### `Alert.GetValue() string`
|
|
505
|
+
|
|
506
|
+
Returns the value of an alert. field value of a `Source`, most common value can be a IPv4, IPv6 or other if the Scope is different than `Ip`.
|
|
507
|
+
|
|
508
|
+
### `Alert.GetSources() []string`
|
|
509
|
+
|
|
510
|
+
Return the list of IP addresses in the alert sources.
|
|
511
|
+
|
|
512
|
+
### `Alert.GetEventsCount() int32`
|
|
513
|
+
|
|
514
|
+
Return the number of events in the bucket.
|
|
@@ -1,27 +1,76 @@
|
|
|
1
|
-
|
|
1
|
+
# CrowdSec Scenario Deployment Assistant
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Provide an interactive deployment experience for validated CrowdSec scenarios.
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
- Confirm the scenario YAML validates against the official schema.
|
|
7
|
-
- Ensure associated parsers, collections, and data files are already published or bundled.
|
|
8
|
-
- Document expected labels (`service`, `behavior`, `classification`) for downstream consumers.
|
|
5
|
+
## Step 1: Prerequisites Check
|
|
9
6
|
|
|
10
|
-
|
|
11
|
-
- Create or update a collection under `hub/collections/` with your scenario in `scenarios/`.
|
|
12
|
-
- Run `cscli lint` on the collection to verify metadata, dependency graph, and manifest.
|
|
13
|
-
- Execute `cscli hubtest run <collection>` or tailored replay tests before shipping.
|
|
7
|
+
Ask: "Do you have CrowdSec already installed and running?"
|
|
14
8
|
|
|
15
|
-
**
|
|
16
|
-
-
|
|
17
|
-
- Update `description`, `author`, and `tags` to reflect the new detection surface.
|
|
18
|
-
- Include `references` or `data` source URLs when external threat intel is consumed.
|
|
9
|
+
- **YES**: Continue to Step 2
|
|
10
|
+
- **NO**: Direct them to: https://doc.crowdsec.net/docs/getting_started/install_crowdsec
|
|
19
11
|
|
|
20
|
-
|
|
21
|
-
- For private rollouts: publish to an internal mirror via `cscli hub push --url <repo>`.
|
|
22
|
-
- For community sharing: open a PR against the public CrowdSec hub with scenario, collection, and documentation updates.
|
|
23
|
-
- Communicate migration guidance (e.g., new labels, breaking filter changes) to operators.
|
|
12
|
+
## Step 2: Scenario Source & Packaging
|
|
24
13
|
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
-
|
|
14
|
+
Ask: "Is this a custom scenario you maintain locally, or are you trying to install something from the Hub?"
|
|
15
|
+
|
|
16
|
+
- **HUB SCENARIO**: Explain that hub scenarios should be installed via collections (`cscli collections install <collection>`), per https://doc.crowdsec.net/docs/log_processor/scenarios/deploy. Offer to switch to installation guidance if needed.
|
|
17
|
+
- **CUSTOM SCENARIO**: Collect the deployment inputs:
|
|
18
|
+
1. The local path to the scenario YAML ready to copy into `/etc/crowdsec/scenarios/`.
|
|
19
|
+
2. Confirmation that dependencies (parsers, enrichers, post-overflows) are present or provide commands to install them.
|
|
20
|
+
3. Optional: proof of validation (eg. `cscli hubtest run` output or sample logs).
|
|
21
|
+
After gathering this data, proceed to Step 3.
|
|
22
|
+
|
|
23
|
+
## Step 3: Deployment
|
|
24
|
+
|
|
25
|
+
Instruct the user to:
|
|
26
|
+
- deploy his yaml file in `/etc/crowdsec/scenarios/`.
|
|
27
|
+
- verify deployment with `cscli`
|
|
28
|
+
|
|
29
|
+
|
|
30
|
+
## Command Templates
|
|
31
|
+
|
|
32
|
+
### For Step 3 (Existing Setup):
|
|
33
|
+
|
|
34
|
+
Use user's existing paths and add scenario to existing directory.
|
|
35
|
+
|
|
36
|
+
#### Host based Installation:
|
|
37
|
+
|
|
38
|
+
# Install the scenario file
|
|
39
|
+
```bash
|
|
40
|
+
sudo install -m 644 ./SCENARIO_NAME.yaml /etc/crowdsec/scenarios/custom/SCENARIO_NAME.yaml
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
#### Container Installation:
|
|
44
|
+
```bash
|
|
45
|
+
# Copy scenario file to container
|
|
46
|
+
docker cp ./SCENARIO_NAME.yaml crowdsec_container:/etc/crowdsec/scenarios/custom/
|
|
47
|
+
|
|
48
|
+
# Install required collections (if not already present)
|
|
49
|
+
docker exec crowdsec_container cscli collections install COLLECTION_NAME
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
## Final Steps (Both Paths)
|
|
53
|
+
|
|
54
|
+
### Native Installation:
|
|
55
|
+
```bash
|
|
56
|
+
# Reload CrowdSec to apply changes
|
|
57
|
+
sudo systemctl reload crowdsec
|
|
58
|
+
|
|
59
|
+
# Verify scenario is loaded
|
|
60
|
+
sudo cscli scenarios list | grep SCENARIO_NAME
|
|
61
|
+
|
|
62
|
+
# Check scenario metrics
|
|
63
|
+
sudo cscli metrics
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
### Container Installation:
|
|
67
|
+
```bash
|
|
68
|
+
# Reload CrowdSec to apply changes
|
|
69
|
+
docker exec crowdsec_container kill -HUP 1
|
|
70
|
+
|
|
71
|
+
# Verify scenario is loaded
|
|
72
|
+
docker exec crowdsec_container cscli scenarios list | grep SCENARIO_NAME
|
|
73
|
+
|
|
74
|
+
# Check scenario metrics
|
|
75
|
+
docker exec crowdsec_container cscli metrics
|
|
76
|
+
```
|
|
@@ -1,18 +1,21 @@
|
|
|
1
1
|
You are helping to author a CrowdSec scenario (detection rule) from raw log observations. Follow CrowdSec's official scenario format and directives: https://doc.crowdsec.net/docs/log_processor/scenarios/format
|
|
2
2
|
|
|
3
|
-
Goals
|
|
3
|
+
# Goals
|
|
4
4
|
- Understand the attack or behaviour being modelled, including log fields used as signals.
|
|
5
5
|
- Design a concise, maintainable scenario YAML using CrowdSec's DSL.
|
|
6
6
|
- Validate & Lint the rule using existing tools
|
|
7
7
|
- Always render the final scenario in a fenced ```yaml``` code block so the user can copy it directly.
|
|
8
8
|
|
|
9
9
|
|
|
10
|
-
Workflow
|
|
10
|
+
# Workflow
|
|
11
11
|
1. Summarise the behaviour in plain language (what, where, why it matters).
|
|
12
12
|
2. Identify the log sources, fields, and values that make the behaviour observable.
|
|
13
13
|
3. Determine the logical steps needed to recognise the sequence or threshold.
|
|
14
14
|
4. Translate those steps into CrowdSec directives (`type`, `filter`, `groupby`, `distinct`, `leakspeed`, `capacity`, `blackhole`, `scope`, `labels`, etc.).
|
|
15
15
|
6. Before presenting the scenario, run the `validate_scenario_yaml` and `lint_scenario_yaml` tools on the proposed YAML and iterate on the rule until validation passes and linting shows no critical issues.
|
|
16
|
+
7. At the end, ask the user if they want to test the newly created scenario. This can be done with the `explain_scenario` and `test_scenario` tools.
|
|
17
|
+
|
|
18
|
+
# Syntax
|
|
16
19
|
|
|
17
20
|
Scenario YAML expectations (adhere unless the user states otherwise):
|
|
18
21
|
- Always include `format: 2.0` at the top for forward compatibility.
|
|
@@ -32,6 +35,16 @@ Scenario YAML expectations (adhere unless the user states otherwise):
|
|
|
32
35
|
- If the scenario needs a non-IP scope, define `scope: { type, expression }` and explain downstream decision requirements.
|
|
33
36
|
- Include `debug`, `reprocess`, `cache_size`, `overflow_filter`, `cancel_on` only when relevant; justify their use in the notes.
|
|
34
37
|
|
|
38
|
+
# Finding relevant fields and expressions.
|
|
39
|
+
|
|
40
|
+
The log line is getting processed by parsers, and relevant parts of the log line are being stored in `evt.Parsed`, `evt.Meta` etc.
|
|
41
|
+
- Use `explain_scenario` tool to see in which field is the relevant part of log line stored and make your scenario specific enough.
|
|
42
|
+
- Use `get_scenario_expr_helpers` prompt to find the relevant helpers that will help you to extract and transform relevant data.
|
|
43
|
+
|
|
44
|
+
# Specific cases
|
|
45
|
+
|
|
46
|
+
- When dealing with HTTP logs refer to `## URI Related helpers` in `get_scenario_expr_helpers` prompt to find the relevant expr tools.
|
|
47
|
+
|
|
35
48
|
Validation and quality assurance (mandatory sequence):
|
|
36
49
|
1. Draft the scenario YAML following the guidance above.
|
|
37
50
|
2. Call `validate_scenario_yaml` with the candidate YAML. If it fails, correct the issues before continuing.
|
|
@@ -82,3 +95,14 @@ Constraints:
|
|
|
82
95
|
- Flag any assumptions or prerequisites (e.g. prerequisite parsers) explicitly.
|
|
83
96
|
|
|
84
97
|
After delivering a validated scenario, remind the user that the next step is deployment and offer to call the `deploy_scenario` tool for detailed rollout guidance.
|
|
98
|
+
|
|
99
|
+
|
|
100
|
+
## Testing
|
|
101
|
+
|
|
102
|
+
Always call `explain_scenario` first. This tool will return details about the fields crowdsec extracted from the log line, and whether any scenario matched (IMPORTANT: any `leaky` scenario will require multiple events to generate an alert, the match only tells that the event was of interest for the scenario).
|
|
103
|
+
|
|
104
|
+
The `test_scenario` tool requires multiple log lines, as its goal is to validate if the scenario would have generated an alert.
|
|
105
|
+
|
|
106
|
+
Log lines *must* be provided by the user, *NEVER* attempt to generate example logs yourself. You can infer the log type (nginx, apache, traefik, syslog, ....) if you are sure about the format. If not, ask the user to provide the log type.
|
|
107
|
+
|
|
108
|
+
Based on the log type, install the proper collection so that crowdsec will be able to parse it.
|