detkit-cli 0.1.0__tar.gz → 0.1.3__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- detkit_cli-0.1.3/.github/workflows/ci.yml +21 -0
- detkit_cli-0.1.3/.github/workflows/publish.yml +34 -0
- detkit_cli-0.1.3/.gitignore +9 -0
- {detkit_cli-0.1.0 → detkit_cli-0.1.3}/LICENSE +0 -0
- detkit_cli-0.1.3/PKG-INFO +114 -0
- detkit_cli-0.1.3/README.md +92 -0
- detkit_cli-0.1.3/action.yml +27 -0
- detkit_cli-0.1.3/detkit/__init__.py +6 -0
- {detkit_cli-0.1.0 → detkit_cli-0.1.3}/detkit/__main__.py +0 -0
- {detkit_cli-0.1.0 → detkit_cli-0.1.3}/detkit/cli.py +4 -1
- {detkit_cli-0.1.0 → detkit_cli-0.1.3}/detkit/evaluator.py +0 -0
- detkit_cli-0.1.3/detkit-demo.gif +0 -0
- detkit_cli-0.1.3/detkit-demo.tape +51 -0
- detkit_cli-0.1.3/detkit_cli.egg-info/PKG-INFO +114 -0
- {detkit_cli-0.1.0 → detkit_cli-0.1.3}/detkit_cli.egg-info/SOURCES.txt +10 -0
- {detkit_cli-0.1.0 → detkit_cli-0.1.3}/detkit_cli.egg-info/dependency_links.txt +0 -0
- {detkit_cli-0.1.0 → detkit_cli-0.1.3}/detkit_cli.egg-info/entry_points.txt +0 -0
- {detkit_cli-0.1.0 → detkit_cli-0.1.3}/detkit_cli.egg-info/requires.txt +0 -0
- detkit_cli-0.1.3/detkit_cli.egg-info/scm_file_list.json +20 -0
- detkit_cli-0.1.3/detkit_cli.egg-info/scm_version.json +8 -0
- {detkit_cli-0.1.0 → detkit_cli-0.1.3}/detkit_cli.egg-info/top_level.txt +0 -0
- detkit_cli-0.1.3/examples/rules/whoami_execution.test.yml +23 -0
- detkit_cli-0.1.3/examples/rules/whoami_execution.yml +15 -0
- {detkit_cli-0.1.0 → detkit_cli-0.1.3}/pyproject.toml +38 -38
- {detkit_cli-0.1.0 → detkit_cli-0.1.3}/setup.cfg +0 -0
- {detkit_cli-0.1.0 → detkit_cli-0.1.3}/tests/test_evaluator.py +0 -0
- detkit_cli-0.1.0/PKG-INFO +0 -129
- detkit_cli-0.1.0/README.md +0 -107
- detkit_cli-0.1.0/detkit/__init__.py +0 -1
- detkit_cli-0.1.0/detkit_cli.egg-info/PKG-INFO +0 -129
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
name: CI
|
|
2
|
+
on: [push, pull_request]
|
|
3
|
+
|
|
4
|
+
jobs:
|
|
5
|
+
test:
|
|
6
|
+
runs-on: ubuntu-latest
|
|
7
|
+
strategy:
|
|
8
|
+
matrix:
|
|
9
|
+
python-version: ['3.9', '3.12']
|
|
10
|
+
steps:
|
|
11
|
+
- uses: actions/checkout@v4
|
|
12
|
+
with:
|
|
13
|
+
fetch-depth: 0
|
|
14
|
+
- uses: actions/setup-python@v5
|
|
15
|
+
with:
|
|
16
|
+
python-version: ${{ matrix.python-version }}
|
|
17
|
+
- run: pip install -e .
|
|
18
|
+
- name: Unit checks (evaluator core)
|
|
19
|
+
run: python tests/test_evaluator.py
|
|
20
|
+
- name: Example rules must pass their tests
|
|
21
|
+
run: detkit test examples/rules
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
name: Publish to PyPI
|
|
2
|
+
|
|
3
|
+
|
|
4
|
+
on:
|
|
5
|
+
release:
|
|
6
|
+
types: [published]
|
|
7
|
+
|
|
8
|
+
jobs:
|
|
9
|
+
publish:
|
|
10
|
+
runs-on: ubuntu-latest
|
|
11
|
+
environment: pypi
|
|
12
|
+
permissions:
|
|
13
|
+
id-token: write
|
|
14
|
+
steps:
|
|
15
|
+
- uses: actions/checkout@v4
|
|
16
|
+
with:
|
|
17
|
+
fetch-depth: 0
|
|
18
|
+
- uses: actions/setup-python@v5
|
|
19
|
+
with:
|
|
20
|
+
python-version: '3.x'
|
|
21
|
+
|
|
22
|
+
- name: Build
|
|
23
|
+
run: |
|
|
24
|
+
pip install build
|
|
25
|
+
python -m build
|
|
26
|
+
|
|
27
|
+
- name: Don't ship a broken build
|
|
28
|
+
run: |
|
|
29
|
+
pip install dist/*.whl
|
|
30
|
+
python tests/test_evaluator.py
|
|
31
|
+
detkit test examples/rules
|
|
32
|
+
|
|
33
|
+
- name: Publish to PyPI
|
|
34
|
+
uses: pypa/gh-action-pypi-publish@release/v1
|
|
File without changes
|
|
@@ -0,0 +1,114 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: detkit-cli
|
|
3
|
+
Version: 0.1.3
|
|
4
|
+
Summary: dbt for detections — test, validate, and CI-gate Sigma detection rules as code.
|
|
5
|
+
Author: detkit
|
|
6
|
+
License: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/ELSATOAH/detkit
|
|
8
|
+
Project-URL: Repository, https://github.com/ELSATOAH/detkit
|
|
9
|
+
Project-URL: Issues, https://github.com/ELSATOAH/detkit/issues
|
|
10
|
+
Keywords: sigma,detection-engineering,detection-as-code,siem,security,detections,blue-team,cybersecurity
|
|
11
|
+
Classifier: Development Status :: 3 - Alpha
|
|
12
|
+
Classifier: Intended Audience :: Information Technology
|
|
13
|
+
Classifier: Topic :: Security
|
|
14
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
15
|
+
Classifier: Programming Language :: Python :: 3
|
|
16
|
+
Classifier: Operating System :: OS Independent
|
|
17
|
+
Requires-Python: >=3.9
|
|
18
|
+
Description-Content-Type: text/markdown
|
|
19
|
+
License-File: LICENSE
|
|
20
|
+
Requires-Dist: pyyaml>=6
|
|
21
|
+
Dynamic: license-file
|
|
22
|
+
|
|
23
|
+
# detkit
|
|
24
|
+
|
|
25
|
+
[](https://github.com/ELSATOAH/detkit/actions/workflows/ci.yml) [](https://pypi.org/project/detkit-cli/) [](https://github.com/ELSATOAH/detkit/blob/main/LICENSE)
|
|
26
|
+
|
|
27
|
+
Unit tests for your Sigma detection rules. Write a rule, add a couple of example log events, and detkit tells you whether it fires on the ones that should trip it and stays quiet on the ones that shouldn't. Wire it into CI and a broken detection fails the pull request instead of failing silently in production.
|
|
28
|
+
|
|
29
|
+
Think `dbt test`, but for detections.
|
|
30
|
+
|
|
31
|
+

|
|
32
|
+
|
|
33
|
+
## Install
|
|
34
|
+
|
|
35
|
+
```bash
|
|
36
|
+
pipx install detkit-cli # installs the `detkit` command
|
|
37
|
+
# or the latest from source:
|
|
38
|
+
pipx install git+https://github.com/ELSATOAH/detkit.git
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
Then:
|
|
42
|
+
|
|
43
|
+
```bash
|
|
44
|
+
detkit init # drops a starter rule, its test, and a CI workflow
|
|
45
|
+
detkit test rules # green on the first run
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
## How it works
|
|
49
|
+
|
|
50
|
+
Your rule is plain [Sigma](https://sigmahq.io/). Next to it you keep a `<rule>.test.yml` listing sample events and what you expect:
|
|
51
|
+
|
|
52
|
+
```yaml
|
|
53
|
+
# whoami_execution.test.yml
|
|
54
|
+
tests:
|
|
55
|
+
- name: fires on whoami
|
|
56
|
+
event: { EventID: 4688, CommandLine: "cmd /c whoami /all", User: "alice" }
|
|
57
|
+
expect: match
|
|
58
|
+
- name: not for SYSTEM
|
|
59
|
+
event: { EventID: 4688, CommandLine: "whoami", User: "SYSTEM" }
|
|
60
|
+
expect: no_match
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
`detkit test` runs every rule against its events and exits non-zero if any expectation is wrong. That's the whole idea:
|
|
64
|
+
|
|
65
|
+
```console
|
|
66
|
+
$ detkit test rules
|
|
67
|
+
. process_exec.yml :: fires on the encoded command
|
|
68
|
+
x process_exec.yml :: ignores the admin allowlist (rule fired, expected no_match)
|
|
69
|
+
|
|
70
|
+
1 passed, 1 failed, 0 rule(s) without tests
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
`detkit validate` does a quicker structural check: the required fields are there, and the condition only references identifiers that actually exist.
|
|
74
|
+
|
|
75
|
+
## In CI
|
|
76
|
+
|
|
77
|
+
Drop this in your rules repo and any PR that breaks a detection gets blocked before it merges:
|
|
78
|
+
|
|
79
|
+
```yaml
|
|
80
|
+
# .github/workflows/detections.yml
|
|
81
|
+
name: Detections
|
|
82
|
+
on: [pull_request]
|
|
83
|
+
jobs:
|
|
84
|
+
detkit:
|
|
85
|
+
runs-on: ubuntu-latest
|
|
86
|
+
steps:
|
|
87
|
+
- uses: actions/checkout@v4
|
|
88
|
+
- uses: ELSATOAH/detkit@v0
|
|
89
|
+
with:
|
|
90
|
+
path: rules
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
## What it handles
|
|
94
|
+
|
|
95
|
+
detkit runs your rules locally. No SIEM, no credentials, nothing leaves your machine or CI runner. I ran it against every rule in the SigmaHQ repo, and about 91% use only features it understands today: `contains`, `startswith`, `endswith`, `re`, value wildcards (`*` and `?`), `|cidr`, keyword lists, and `X of` / `all of` conditions.
|
|
96
|
+
|
|
97
|
+
It won't guess at the rest. If a rule uses something detkit can't evaluate yet, like nested fields (`DeviceDetail.deviceId`) or `base64` modifiers, `test` and `validate` print a WARN naming the feature instead of returning an answer that might be wrong. A detection tool that's quietly wrong is worse than no tool.
|
|
98
|
+
|
|
99
|
+
## Why
|
|
100
|
+
|
|
101
|
+
Detections live in Git now, but the testing habit that comes with the rest of software never followed them there. dbt sorted this out for data models years ago; Sigma rules deserve the same. And it runs locally on purpose, because security logs aren't something you can hand to someone else's cloud just to check a rule.
|
|
102
|
+
|
|
103
|
+
## Roadmap
|
|
104
|
+
|
|
105
|
+
- Nested/dotted fields and `base64` modifiers (the two things it warns on today), probably via [pySigma](https://github.com/SigmaHQ/pySigma).
|
|
106
|
+
- `detkit generate`: draft a rule and its tests from a plain-English description. The tests come with it, not as an afterthought.
|
|
107
|
+
- Field mapping, so one rule can be checked against more than one log schema.
|
|
108
|
+
- A hosted option down the line for teams that want shared runs and history. The CLI stays free.
|
|
109
|
+
|
|
110
|
+
## Status
|
|
111
|
+
|
|
112
|
+
Early, but the core works and has tests (`python3 tests/test_evaluator.py`). Rough edges are marked with `# ponytail:` comments in the source. Issues and PRs welcome.
|
|
113
|
+
|
|
114
|
+
MIT.
|
|
@@ -0,0 +1,92 @@
|
|
|
1
|
+
# detkit
|
|
2
|
+
|
|
3
|
+
[](https://github.com/ELSATOAH/detkit/actions/workflows/ci.yml) [](https://pypi.org/project/detkit-cli/) [](https://github.com/ELSATOAH/detkit/blob/main/LICENSE)
|
|
4
|
+
|
|
5
|
+
Unit tests for your Sigma detection rules. Write a rule, add a couple of example log events, and detkit tells you whether it fires on the ones that should trip it and stays quiet on the ones that shouldn't. Wire it into CI and a broken detection fails the pull request instead of failing silently in production.
|
|
6
|
+
|
|
7
|
+
Think `dbt test`, but for detections.
|
|
8
|
+
|
|
9
|
+

|
|
10
|
+
|
|
11
|
+
## Install
|
|
12
|
+
|
|
13
|
+
```bash
|
|
14
|
+
pipx install detkit-cli # installs the `detkit` command
|
|
15
|
+
# or the latest from source:
|
|
16
|
+
pipx install git+https://github.com/ELSATOAH/detkit.git
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
Then:
|
|
20
|
+
|
|
21
|
+
```bash
|
|
22
|
+
detkit init # drops a starter rule, its test, and a CI workflow
|
|
23
|
+
detkit test rules # green on the first run
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
## How it works
|
|
27
|
+
|
|
28
|
+
Your rule is plain [Sigma](https://sigmahq.io/). Next to it you keep a `<rule>.test.yml` listing sample events and what you expect:
|
|
29
|
+
|
|
30
|
+
```yaml
|
|
31
|
+
# whoami_execution.test.yml
|
|
32
|
+
tests:
|
|
33
|
+
- name: fires on whoami
|
|
34
|
+
event: { EventID: 4688, CommandLine: "cmd /c whoami /all", User: "alice" }
|
|
35
|
+
expect: match
|
|
36
|
+
- name: not for SYSTEM
|
|
37
|
+
event: { EventID: 4688, CommandLine: "whoami", User: "SYSTEM" }
|
|
38
|
+
expect: no_match
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
`detkit test` runs every rule against its events and exits non-zero if any expectation is wrong. That's the whole idea:
|
|
42
|
+
|
|
43
|
+
```console
|
|
44
|
+
$ detkit test rules
|
|
45
|
+
. process_exec.yml :: fires on the encoded command
|
|
46
|
+
x process_exec.yml :: ignores the admin allowlist (rule fired, expected no_match)
|
|
47
|
+
|
|
48
|
+
1 passed, 1 failed, 0 rule(s) without tests
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
`detkit validate` does a quicker structural check: the required fields are there, and the condition only references identifiers that actually exist.
|
|
52
|
+
|
|
53
|
+
## In CI
|
|
54
|
+
|
|
55
|
+
Drop this in your rules repo and any PR that breaks a detection gets blocked before it merges:
|
|
56
|
+
|
|
57
|
+
```yaml
|
|
58
|
+
# .github/workflows/detections.yml
|
|
59
|
+
name: Detections
|
|
60
|
+
on: [pull_request]
|
|
61
|
+
jobs:
|
|
62
|
+
detkit:
|
|
63
|
+
runs-on: ubuntu-latest
|
|
64
|
+
steps:
|
|
65
|
+
- uses: actions/checkout@v4
|
|
66
|
+
- uses: ELSATOAH/detkit@v0
|
|
67
|
+
with:
|
|
68
|
+
path: rules
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
## What it handles
|
|
72
|
+
|
|
73
|
+
detkit runs your rules locally. No SIEM, no credentials, nothing leaves your machine or CI runner. I ran it against every rule in the SigmaHQ repo, and about 91% use only features it understands today: `contains`, `startswith`, `endswith`, `re`, value wildcards (`*` and `?`), `|cidr`, keyword lists, and `X of` / `all of` conditions.
|
|
74
|
+
|
|
75
|
+
It won't guess at the rest. If a rule uses something detkit can't evaluate yet, like nested fields (`DeviceDetail.deviceId`) or `base64` modifiers, `test` and `validate` print a WARN naming the feature instead of returning an answer that might be wrong. A detection tool that's quietly wrong is worse than no tool.
|
|
76
|
+
|
|
77
|
+
## Why
|
|
78
|
+
|
|
79
|
+
Detections live in Git now, but the testing habit that comes with the rest of software never followed them there. dbt sorted this out for data models years ago; Sigma rules deserve the same. And it runs locally on purpose, because security logs aren't something you can hand to someone else's cloud just to check a rule.
|
|
80
|
+
|
|
81
|
+
## Roadmap
|
|
82
|
+
|
|
83
|
+
- Nested/dotted fields and `base64` modifiers (the two things it warns on today), probably via [pySigma](https://github.com/SigmaHQ/pySigma).
|
|
84
|
+
- `detkit generate`: draft a rule and its tests from a plain-English description. The tests come with it, not as an afterthought.
|
|
85
|
+
- Field mapping, so one rule can be checked against more than one log schema.
|
|
86
|
+
- A hosted option down the line for teams that want shared runs and history. The CLI stays free.
|
|
87
|
+
|
|
88
|
+
## Status
|
|
89
|
+
|
|
90
|
+
Early, but the core works and has tests (`python3 tests/test_evaluator.py`). Rough edges are marked with `# ponytail:` comments in the source. Issues and PRs welcome.
|
|
91
|
+
|
|
92
|
+
MIT.
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
name: 'detkit — test detections'
|
|
2
|
+
description: 'Run detkit against your Sigma detection rules to gate pull requests in CI.'
|
|
3
|
+
branding:
|
|
4
|
+
icon: 'shield'
|
|
5
|
+
color: 'purple'
|
|
6
|
+
inputs:
|
|
7
|
+
path:
|
|
8
|
+
description: 'Directory containing your Sigma rules and their *.test.yml files.'
|
|
9
|
+
required: false
|
|
10
|
+
default: '.'
|
|
11
|
+
command:
|
|
12
|
+
description: 'Which detkit command to run: test or validate.'
|
|
13
|
+
required: false
|
|
14
|
+
default: 'test'
|
|
15
|
+
runs:
|
|
16
|
+
using: 'composite'
|
|
17
|
+
steps:
|
|
18
|
+
- name: Set up Python
|
|
19
|
+
uses: actions/setup-python@v5
|
|
20
|
+
with:
|
|
21
|
+
python-version: '3.11'
|
|
22
|
+
- name: Install detkit
|
|
23
|
+
shell: bash
|
|
24
|
+
run: pip install "${{ github.action_path }}"
|
|
25
|
+
- name: Run detkit
|
|
26
|
+
shell: bash
|
|
27
|
+
run: detkit "${{ inputs.command }}" "${{ inputs.path }}"
|
|
File without changes
|
|
@@ -69,7 +69,10 @@ def cmd_test(args):
|
|
|
69
69
|
failed += 1
|
|
70
70
|
verb = "fired" if got else "did not fire"
|
|
71
71
|
print(f" x {rule_path} :: {case.get('name', '?')} (rule {verb}, expected {case['expect']})")
|
|
72
|
-
|
|
72
|
+
tested = len(rules) - untested
|
|
73
|
+
pct = round(100 * tested / len(rules)) if rules else 0
|
|
74
|
+
print(f"\n{passed} passed, {failed} failed")
|
|
75
|
+
print(f"{tested} of {len(rules)} rules have tests ({pct}% coverage)")
|
|
73
76
|
return 1 if failed else 0
|
|
74
77
|
|
|
75
78
|
|
|
File without changes
|
|
Binary file
|
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
# detkit demo — generate the GIF with: vhs detkit-demo.tape
|
|
2
|
+
# Prereqs: `go install github.com/charmbracelet/vhs@latest` (or brew install vhs),
|
|
3
|
+
# run from the detkit repo root, and `pip install -e .` first.
|
|
4
|
+
# Uses GNU sed (Linux/WSL). On macOS, change `sed -i` to `sed -i ''`.
|
|
5
|
+
|
|
6
|
+
Output detkit-demo.gif
|
|
7
|
+
|
|
8
|
+
Require detkit
|
|
9
|
+
|
|
10
|
+
Set Shell bash
|
|
11
|
+
Set Theme "Dracula"
|
|
12
|
+
Set FontSize 18
|
|
13
|
+
Set Width 1200
|
|
14
|
+
Set Height 680
|
|
15
|
+
Set Padding 24
|
|
16
|
+
Set TypingSpeed 45ms
|
|
17
|
+
|
|
18
|
+
Hide
|
|
19
|
+
Type "clear"
|
|
20
|
+
Enter
|
|
21
|
+
Show
|
|
22
|
+
|
|
23
|
+
Type "# A Sigma rule ships with sample events saying what it should catch:"
|
|
24
|
+
Enter
|
|
25
|
+
Sleep 1s
|
|
26
|
+
Type "detkit test examples/rules"
|
|
27
|
+
Enter
|
|
28
|
+
Sleep 3s
|
|
29
|
+
|
|
30
|
+
Type "# A teammate 'tidies up' the rule and quietly breaks the detection..."
|
|
31
|
+
Enter
|
|
32
|
+
Sleep 1s
|
|
33
|
+
Type "sed -i 's/whoami/whoamiXYZ/' examples/rules/whoami_execution.yml"
|
|
34
|
+
Enter
|
|
35
|
+
Sleep 1200ms
|
|
36
|
+
|
|
37
|
+
Type "# detkit catches it. Non-zero exit means CI blocks the pull request:"
|
|
38
|
+
Enter
|
|
39
|
+
Sleep 800ms
|
|
40
|
+
Type "detkit test examples/rules; echo exit=$?"
|
|
41
|
+
Enter
|
|
42
|
+
Sleep 4s
|
|
43
|
+
|
|
44
|
+
Type "# Revert -> green again."
|
|
45
|
+
Enter
|
|
46
|
+
Sleep 800ms
|
|
47
|
+
Type "sed -i 's/whoamiXYZ/whoami/' examples/rules/whoami_execution.yml"
|
|
48
|
+
Enter
|
|
49
|
+
Type "detkit test examples/rules; echo exit=$?"
|
|
50
|
+
Enter
|
|
51
|
+
Sleep 3500ms
|
|
@@ -0,0 +1,114 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: detkit-cli
|
|
3
|
+
Version: 0.1.3
|
|
4
|
+
Summary: dbt for detections — test, validate, and CI-gate Sigma detection rules as code.
|
|
5
|
+
Author: detkit
|
|
6
|
+
License: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/ELSATOAH/detkit
|
|
8
|
+
Project-URL: Repository, https://github.com/ELSATOAH/detkit
|
|
9
|
+
Project-URL: Issues, https://github.com/ELSATOAH/detkit/issues
|
|
10
|
+
Keywords: sigma,detection-engineering,detection-as-code,siem,security,detections,blue-team,cybersecurity
|
|
11
|
+
Classifier: Development Status :: 3 - Alpha
|
|
12
|
+
Classifier: Intended Audience :: Information Technology
|
|
13
|
+
Classifier: Topic :: Security
|
|
14
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
15
|
+
Classifier: Programming Language :: Python :: 3
|
|
16
|
+
Classifier: Operating System :: OS Independent
|
|
17
|
+
Requires-Python: >=3.9
|
|
18
|
+
Description-Content-Type: text/markdown
|
|
19
|
+
License-File: LICENSE
|
|
20
|
+
Requires-Dist: pyyaml>=6
|
|
21
|
+
Dynamic: license-file
|
|
22
|
+
|
|
23
|
+
# detkit
|
|
24
|
+
|
|
25
|
+
[](https://github.com/ELSATOAH/detkit/actions/workflows/ci.yml) [](https://pypi.org/project/detkit-cli/) [](https://github.com/ELSATOAH/detkit/blob/main/LICENSE)
|
|
26
|
+
|
|
27
|
+
Unit tests for your Sigma detection rules. Write a rule, add a couple of example log events, and detkit tells you whether it fires on the ones that should trip it and stays quiet on the ones that shouldn't. Wire it into CI and a broken detection fails the pull request instead of failing silently in production.
|
|
28
|
+
|
|
29
|
+
Think `dbt test`, but for detections.
|
|
30
|
+
|
|
31
|
+

|
|
32
|
+
|
|
33
|
+
## Install
|
|
34
|
+
|
|
35
|
+
```bash
|
|
36
|
+
pipx install detkit-cli # installs the `detkit` command
|
|
37
|
+
# or the latest from source:
|
|
38
|
+
pipx install git+https://github.com/ELSATOAH/detkit.git
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
Then:
|
|
42
|
+
|
|
43
|
+
```bash
|
|
44
|
+
detkit init # drops a starter rule, its test, and a CI workflow
|
|
45
|
+
detkit test rules # green on the first run
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
## How it works
|
|
49
|
+
|
|
50
|
+
Your rule is plain [Sigma](https://sigmahq.io/). Next to it you keep a `<rule>.test.yml` listing sample events and what you expect:
|
|
51
|
+
|
|
52
|
+
```yaml
|
|
53
|
+
# whoami_execution.test.yml
|
|
54
|
+
tests:
|
|
55
|
+
- name: fires on whoami
|
|
56
|
+
event: { EventID: 4688, CommandLine: "cmd /c whoami /all", User: "alice" }
|
|
57
|
+
expect: match
|
|
58
|
+
- name: not for SYSTEM
|
|
59
|
+
event: { EventID: 4688, CommandLine: "whoami", User: "SYSTEM" }
|
|
60
|
+
expect: no_match
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
`detkit test` runs every rule against its events and exits non-zero if any expectation is wrong. That's the whole idea:
|
|
64
|
+
|
|
65
|
+
```console
|
|
66
|
+
$ detkit test rules
|
|
67
|
+
. process_exec.yml :: fires on the encoded command
|
|
68
|
+
x process_exec.yml :: ignores the admin allowlist (rule fired, expected no_match)
|
|
69
|
+
|
|
70
|
+
1 passed, 1 failed, 0 rule(s) without tests
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
`detkit validate` does a quicker structural check: the required fields are there, and the condition only references identifiers that actually exist.
|
|
74
|
+
|
|
75
|
+
## In CI
|
|
76
|
+
|
|
77
|
+
Drop this in your rules repo and any PR that breaks a detection gets blocked before it merges:
|
|
78
|
+
|
|
79
|
+
```yaml
|
|
80
|
+
# .github/workflows/detections.yml
|
|
81
|
+
name: Detections
|
|
82
|
+
on: [pull_request]
|
|
83
|
+
jobs:
|
|
84
|
+
detkit:
|
|
85
|
+
runs-on: ubuntu-latest
|
|
86
|
+
steps:
|
|
87
|
+
- uses: actions/checkout@v4
|
|
88
|
+
- uses: ELSATOAH/detkit@v0
|
|
89
|
+
with:
|
|
90
|
+
path: rules
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
## What it handles
|
|
94
|
+
|
|
95
|
+
detkit runs your rules locally. No SIEM, no credentials, nothing leaves your machine or CI runner. I ran it against every rule in the SigmaHQ repo, and about 91% use only features it understands today: `contains`, `startswith`, `endswith`, `re`, value wildcards (`*` and `?`), `|cidr`, keyword lists, and `X of` / `all of` conditions.
|
|
96
|
+
|
|
97
|
+
It won't guess at the rest. If a rule uses something detkit can't evaluate yet, like nested fields (`DeviceDetail.deviceId`) or `base64` modifiers, `test` and `validate` print a WARN naming the feature instead of returning an answer that might be wrong. A detection tool that's quietly wrong is worse than no tool.
|
|
98
|
+
|
|
99
|
+
## Why
|
|
100
|
+
|
|
101
|
+
Detections live in Git now, but the testing habit that comes with the rest of software never followed them there. dbt sorted this out for data models years ago; Sigma rules deserve the same. And it runs locally on purpose, because security logs aren't something you can hand to someone else's cloud just to check a rule.
|
|
102
|
+
|
|
103
|
+
## Roadmap
|
|
104
|
+
|
|
105
|
+
- Nested/dotted fields and `base64` modifiers (the two things it warns on today), probably via [pySigma](https://github.com/SigmaHQ/pySigma).
|
|
106
|
+
- `detkit generate`: draft a rule and its tests from a plain-English description. The tests come with it, not as an afterthought.
|
|
107
|
+
- Field mapping, so one rule can be checked against more than one log schema.
|
|
108
|
+
- A hosted option down the line for teams that want shared runs and history. The CLI stays free.
|
|
109
|
+
|
|
110
|
+
## Status
|
|
111
|
+
|
|
112
|
+
Early, but the core works and has tests (`python3 tests/test_evaluator.py`). Rough edges are marked with `# ponytail:` comments in the source. Issues and PRs welcome.
|
|
113
|
+
|
|
114
|
+
MIT.
|
|
@@ -1,6 +1,12 @@
|
|
|
1
|
+
.gitignore
|
|
1
2
|
LICENSE
|
|
2
3
|
README.md
|
|
4
|
+
action.yml
|
|
5
|
+
detkit-demo.gif
|
|
6
|
+
detkit-demo.tape
|
|
3
7
|
pyproject.toml
|
|
8
|
+
.github/workflows/ci.yml
|
|
9
|
+
.github/workflows/publish.yml
|
|
4
10
|
detkit/__init__.py
|
|
5
11
|
detkit/__main__.py
|
|
6
12
|
detkit/cli.py
|
|
@@ -10,5 +16,9 @@ detkit_cli.egg-info/SOURCES.txt
|
|
|
10
16
|
detkit_cli.egg-info/dependency_links.txt
|
|
11
17
|
detkit_cli.egg-info/entry_points.txt
|
|
12
18
|
detkit_cli.egg-info/requires.txt
|
|
19
|
+
detkit_cli.egg-info/scm_file_list.json
|
|
20
|
+
detkit_cli.egg-info/scm_version.json
|
|
13
21
|
detkit_cli.egg-info/top_level.txt
|
|
22
|
+
examples/rules/whoami_execution.test.yml
|
|
23
|
+
examples/rules/whoami_execution.yml
|
|
14
24
|
tests/test_evaluator.py
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
{
|
|
2
|
+
"files": [
|
|
3
|
+
".gitignore",
|
|
4
|
+
"detkit-demo.tape",
|
|
5
|
+
"README.md",
|
|
6
|
+
"action.yml",
|
|
7
|
+
"pyproject.toml",
|
|
8
|
+
"detkit-demo.gif",
|
|
9
|
+
"LICENSE",
|
|
10
|
+
".github/workflows/publish.yml",
|
|
11
|
+
".github/workflows/ci.yml",
|
|
12
|
+
"examples/rules/whoami_execution.yml",
|
|
13
|
+
"examples/rules/whoami_execution.test.yml",
|
|
14
|
+
"detkit/evaluator.py",
|
|
15
|
+
"detkit/__main__.py",
|
|
16
|
+
"detkit/cli.py",
|
|
17
|
+
"detkit/__init__.py",
|
|
18
|
+
"tests/test_evaluator.py"
|
|
19
|
+
]
|
|
20
|
+
}
|
|
File without changes
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
# Unit tests for whoami_execution.yml
|
|
2
|
+
# every detection ships with sample events proving it fires (and doesn't).
|
|
3
|
+
tests:
|
|
4
|
+
- name: fires on whoami run by a normal user
|
|
5
|
+
event:
|
|
6
|
+
EventID: 4688
|
|
7
|
+
CommandLine: "cmd /c whoami /all"
|
|
8
|
+
User: "alice"
|
|
9
|
+
expect: match
|
|
10
|
+
|
|
11
|
+
- name: suppressed for SYSTEM (the filter)
|
|
12
|
+
event:
|
|
13
|
+
EventID: 4688
|
|
14
|
+
CommandLine: "whoami"
|
|
15
|
+
User: "SYSTEM"
|
|
16
|
+
expect: no_match
|
|
17
|
+
|
|
18
|
+
- name: no match on an unrelated command
|
|
19
|
+
event:
|
|
20
|
+
EventID: 4688
|
|
21
|
+
CommandLine: "ipconfig /all"
|
|
22
|
+
User: "alice"
|
|
23
|
+
expect: no_match
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
title: Whoami Execution
|
|
2
|
+
id: 6f0c3f2e-2a5e-4a1e-9b3a-0d5f1a2b3c4d
|
|
3
|
+
status: experimental
|
|
4
|
+
description: Detects execution of whoami.exe, common in post-exploitation recon.
|
|
5
|
+
logsource:
|
|
6
|
+
category: process_creation
|
|
7
|
+
product: windows
|
|
8
|
+
detection:
|
|
9
|
+
selection:
|
|
10
|
+
EventID: 4688
|
|
11
|
+
CommandLine|contains: 'whoami'
|
|
12
|
+
filter:
|
|
13
|
+
User: 'SYSTEM'
|
|
14
|
+
condition: selection and not filter
|
|
15
|
+
level: low
|
|
@@ -1,38 +1,38 @@
|
|
|
1
|
-
[project]
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
"
|
|
17
|
-
"
|
|
18
|
-
"
|
|
19
|
-
"
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
1
|
+
[project]
|
|
2
|
+
name = "detkit-cli"
|
|
3
|
+
dynamic = ["version"]
|
|
4
|
+
description = "dbt for detections — test, validate, and CI-gate Sigma detection rules as code."
|
|
5
|
+
readme = "README.md"
|
|
6
|
+
requires-python = ">=3.9"
|
|
7
|
+
license = { text = "MIT" }
|
|
8
|
+
authors = [{ name = "detkit" }]
|
|
9
|
+
keywords = [
|
|
10
|
+
"sigma", "detection-engineering", "detection-as-code",
|
|
11
|
+
"siem", "security", "detections", "blue-team", "cybersecurity",
|
|
12
|
+
]
|
|
13
|
+
classifiers = [
|
|
14
|
+
"Development Status :: 3 - Alpha",
|
|
15
|
+
"Intended Audience :: Information Technology",
|
|
16
|
+
"Topic :: Security",
|
|
17
|
+
"License :: OSI Approved :: MIT License",
|
|
18
|
+
"Programming Language :: Python :: 3",
|
|
19
|
+
"Operating System :: OS Independent",
|
|
20
|
+
]
|
|
21
|
+
dependencies = ["pyyaml>=6"]
|
|
22
|
+
|
|
23
|
+
[project.urls]
|
|
24
|
+
Homepage = "https://github.com/ELSATOAH/detkit"
|
|
25
|
+
Repository = "https://github.com/ELSATOAH/detkit"
|
|
26
|
+
Issues = "https://github.com/ELSATOAH/detkit/issues"
|
|
27
|
+
|
|
28
|
+
[project.scripts]
|
|
29
|
+
detkit = "detkit.cli:main"
|
|
30
|
+
|
|
31
|
+
[build-system]
|
|
32
|
+
requires = ["setuptools>=61", "setuptools-scm>=8"]
|
|
33
|
+
build-backend = "setuptools.build_meta"
|
|
34
|
+
|
|
35
|
+
[tool.setuptools.packages.find]
|
|
36
|
+
include = ["detkit*"]
|
|
37
|
+
|
|
38
|
+
[tool.setuptools_scm]
|
|
File without changes
|
|
File without changes
|
detkit_cli-0.1.0/PKG-INFO
DELETED
|
@@ -1,129 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.4
|
|
2
|
-
Name: detkit-cli
|
|
3
|
-
Version: 0.1.0
|
|
4
|
-
Summary: dbt for detections — test, validate, and CI-gate Sigma detection rules as code.
|
|
5
|
-
Author: detkit
|
|
6
|
-
License: MIT
|
|
7
|
-
Project-URL: Homepage, https://github.com/ELSATOAH/detkit
|
|
8
|
-
Project-URL: Repository, https://github.com/ELSATOAH/detkit
|
|
9
|
-
Project-URL: Issues, https://github.com/ELSATOAH/detkit/issues
|
|
10
|
-
Keywords: sigma,detection-engineering,detection-as-code,siem,security,detections,blue-team,cybersecurity
|
|
11
|
-
Classifier: Development Status :: 3 - Alpha
|
|
12
|
-
Classifier: Intended Audience :: Information Technology
|
|
13
|
-
Classifier: Topic :: Security
|
|
14
|
-
Classifier: License :: OSI Approved :: MIT License
|
|
15
|
-
Classifier: Programming Language :: Python :: 3
|
|
16
|
-
Classifier: Operating System :: OS Independent
|
|
17
|
-
Requires-Python: >=3.9
|
|
18
|
-
Description-Content-Type: text/markdown
|
|
19
|
-
License-File: LICENSE
|
|
20
|
-
Requires-Dist: pyyaml>=6
|
|
21
|
-
Dynamic: license-file
|
|
22
|
-
|
|
23
|
-
# detkit — dbt for detections
|
|
24
|
-
|
|
25
|
-
[](https://github.com/ELSATOAH/detkit/actions/workflows/ci.yml) [](LICENSE)
|
|
26
|
-
|
|
27
|
-
Test, validate, and CI-gate your Sigma detection rules **as code**, before they
|
|
28
|
-
ever reach production.
|
|
29
|
-
|
|
30
|
-

|
|
31
|
-
|
|
32
|
-
Detection engineers write rules in [Sigma](https://sigmahq.io/), commit them to
|
|
33
|
-
Git, and ship them to a SIEM — but there's no standard way to *unit-test* a rule
|
|
34
|
-
locally: does it actually fire on the attack it targets, and stay quiet on benign
|
|
35
|
-
traffic? Today that's checked by hand, or in prod. detkit closes that loop.
|
|
36
|
-
|
|
37
|
-
```
|
|
38
|
-
detkit test ./rules # run every rule against its sample events
|
|
39
|
-
detkit validate ./rules # structural + condition-reference checks
|
|
40
|
-
```
|
|
41
|
-
|
|
42
|
-
detkit is **open source (MIT)**, **self-hostable**, and **vendor-neutral** — it
|
|
43
|
-
sits *alongside* your SIEM (Wazuh, Elastic, Splunk, Sentinel), not in front of
|
|
44
|
-
it. Your logs and rules never leave your machine or CI runner.
|
|
45
|
-
|
|
46
|
-
## Quickstart
|
|
47
|
-
|
|
48
|
-
```bash
|
|
49
|
-
pipx install git+https://github.com/ELSATOAH/detkit.git # PyPI: `pipx install detkit-cli` (coming soon)
|
|
50
|
-
|
|
51
|
-
detkit init # scaffold a starter rule + test + a CI workflow
|
|
52
|
-
detkit test rules # -> green on the first run
|
|
53
|
-
```
|
|
54
|
-
|
|
55
|
-
Each rule gets a sibling `*.test.yml` describing sample events and whether the
|
|
56
|
-
rule should match:
|
|
57
|
-
|
|
58
|
-
```yaml
|
|
59
|
-
# whoami_execution.test.yml
|
|
60
|
-
tests:
|
|
61
|
-
- name: fires on whoami run by a normal user
|
|
62
|
-
event: { EventID: 4688, CommandLine: "cmd /c whoami /all", User: "alice" }
|
|
63
|
-
expect: match
|
|
64
|
-
- name: suppressed for SYSTEM
|
|
65
|
-
event: { EventID: 4688, CommandLine: "whoami", User: "SYSTEM" }
|
|
66
|
-
expect: no_match
|
|
67
|
-
```
|
|
68
|
-
|
|
69
|
-
`detkit test` exits non-zero on any failure, so you can drop it straight into CI
|
|
70
|
-
and block a pull request that breaks a detection.
|
|
71
|
-
|
|
72
|
-
## Use it in CI (GitHub Actions)
|
|
73
|
-
|
|
74
|
-
Drop this into your **rules** repo to gate every pull request:
|
|
75
|
-
|
|
76
|
-
```yaml
|
|
77
|
-
# .github/workflows/detections.yml
|
|
78
|
-
name: Detections
|
|
79
|
-
on: [pull_request]
|
|
80
|
-
jobs:
|
|
81
|
-
detkit:
|
|
82
|
-
runs-on: ubuntu-latest
|
|
83
|
-
steps:
|
|
84
|
-
- uses: actions/checkout@v4
|
|
85
|
-
- uses: ELSATOAH/detkit@v0 # this repo's composite action
|
|
86
|
-
with:
|
|
87
|
-
path: rules # where your Sigma rules live
|
|
88
|
-
```
|
|
89
|
-
|
|
90
|
-
A rule whose `*.test.yml` no longer passes now fails the check and blocks the
|
|
91
|
-
merge — the same safety net dbt gives analytics engineers.
|
|
92
|
-
|
|
93
|
-
## Why this, why now
|
|
94
|
-
|
|
95
|
-
- **Detection-as-code is mainstream** (SigmaHQ, Elastic detection-rules, Splunk
|
|
96
|
-
ESCU) but the *test* step is missing — the same gap dbt filled for analytics.
|
|
97
|
-
- **Self-host is a hard requirement, not a preference:** security telemetry can't
|
|
98
|
-
be shipped to someone else's cloud. That's the wedge closed SaaS SOC tools
|
|
99
|
-
(Dropzone, Prophet, Intezer) structurally can't serve.
|
|
100
|
-
- **The community distributes it:** good detection tooling spreads bottom-up on
|
|
101
|
-
GitHub. detkit is built to be forked, extended, and shared.
|
|
102
|
-
|
|
103
|
-
## Roadmap
|
|
104
|
-
|
|
105
|
-
- `detkit generate` — AI-draft a rule **and its tests** from a natural-language
|
|
106
|
-
threat description (tests are mandatory, never optional).
|
|
107
|
-
- More log schemas / field-mapping so one rule tests against multiple log sources.
|
|
108
|
-
- Close the remaining gaps detkit currently warns on: nested/dotted field access
|
|
109
|
-
(~3% of rules) and `base64` modifiers — likely via
|
|
110
|
-
[pySigma](https://github.com/SigmaHQ/pySigma) for full-spec parsing.
|
|
111
|
-
- Managed cloud (hosted runs, SSO, shared rule/test libraries) — the paid tier.
|
|
112
|
-
The tool stays free forever.
|
|
113
|
-
|
|
114
|
-
## Status
|
|
115
|
-
|
|
116
|
-
Early, but the core is real. detkit evaluates a rule's `detection`/`condition`
|
|
117
|
-
against log events and runs tests around it — covering the features used by the
|
|
118
|
-
**large majority of published SigmaHQ rules**: `contains`/`startswith`/`endswith`/
|
|
119
|
-
`re` modifiers, value wildcards (`*`/`?`), `|cidr`, list-as-OR, keyword lists, and
|
|
120
|
-
`X of` / `all of` conditions.
|
|
121
|
-
|
|
122
|
-
What it can't yet evaluate (nested/dotted fields, `base64` modifiers) it **flags
|
|
123
|
-
loudly** — `detkit validate` and `detkit test` print a `WARN` rather than return a
|
|
124
|
-
confident wrong answer. A detection tool that's silently wrong is worse than none.
|
|
125
|
-
|
|
126
|
-
Run the checks: `python tests/test_evaluator.py`. Limits are marked with
|
|
127
|
-
`# ponytail:` comments in `detkit/evaluator.py`.
|
|
128
|
-
|
|
129
|
-
MIT licensed. Contributions welcome.
|
detkit_cli-0.1.0/README.md
DELETED
|
@@ -1,107 +0,0 @@
|
|
|
1
|
-
# detkit — dbt for detections
|
|
2
|
-
|
|
3
|
-
[](https://github.com/ELSATOAH/detkit/actions/workflows/ci.yml) [](LICENSE)
|
|
4
|
-
|
|
5
|
-
Test, validate, and CI-gate your Sigma detection rules **as code**, before they
|
|
6
|
-
ever reach production.
|
|
7
|
-
|
|
8
|
-

|
|
9
|
-
|
|
10
|
-
Detection engineers write rules in [Sigma](https://sigmahq.io/), commit them to
|
|
11
|
-
Git, and ship them to a SIEM — but there's no standard way to *unit-test* a rule
|
|
12
|
-
locally: does it actually fire on the attack it targets, and stay quiet on benign
|
|
13
|
-
traffic? Today that's checked by hand, or in prod. detkit closes that loop.
|
|
14
|
-
|
|
15
|
-
```
|
|
16
|
-
detkit test ./rules # run every rule against its sample events
|
|
17
|
-
detkit validate ./rules # structural + condition-reference checks
|
|
18
|
-
```
|
|
19
|
-
|
|
20
|
-
detkit is **open source (MIT)**, **self-hostable**, and **vendor-neutral** — it
|
|
21
|
-
sits *alongside* your SIEM (Wazuh, Elastic, Splunk, Sentinel), not in front of
|
|
22
|
-
it. Your logs and rules never leave your machine or CI runner.
|
|
23
|
-
|
|
24
|
-
## Quickstart
|
|
25
|
-
|
|
26
|
-
```bash
|
|
27
|
-
pipx install git+https://github.com/ELSATOAH/detkit.git # PyPI: `pipx install detkit-cli` (coming soon)
|
|
28
|
-
|
|
29
|
-
detkit init # scaffold a starter rule + test + a CI workflow
|
|
30
|
-
detkit test rules # -> green on the first run
|
|
31
|
-
```
|
|
32
|
-
|
|
33
|
-
Each rule gets a sibling `*.test.yml` describing sample events and whether the
|
|
34
|
-
rule should match:
|
|
35
|
-
|
|
36
|
-
```yaml
|
|
37
|
-
# whoami_execution.test.yml
|
|
38
|
-
tests:
|
|
39
|
-
- name: fires on whoami run by a normal user
|
|
40
|
-
event: { EventID: 4688, CommandLine: "cmd /c whoami /all", User: "alice" }
|
|
41
|
-
expect: match
|
|
42
|
-
- name: suppressed for SYSTEM
|
|
43
|
-
event: { EventID: 4688, CommandLine: "whoami", User: "SYSTEM" }
|
|
44
|
-
expect: no_match
|
|
45
|
-
```
|
|
46
|
-
|
|
47
|
-
`detkit test` exits non-zero on any failure, so you can drop it straight into CI
|
|
48
|
-
and block a pull request that breaks a detection.
|
|
49
|
-
|
|
50
|
-
## Use it in CI (GitHub Actions)
|
|
51
|
-
|
|
52
|
-
Drop this into your **rules** repo to gate every pull request:
|
|
53
|
-
|
|
54
|
-
```yaml
|
|
55
|
-
# .github/workflows/detections.yml
|
|
56
|
-
name: Detections
|
|
57
|
-
on: [pull_request]
|
|
58
|
-
jobs:
|
|
59
|
-
detkit:
|
|
60
|
-
runs-on: ubuntu-latest
|
|
61
|
-
steps:
|
|
62
|
-
- uses: actions/checkout@v4
|
|
63
|
-
- uses: ELSATOAH/detkit@v0 # this repo's composite action
|
|
64
|
-
with:
|
|
65
|
-
path: rules # where your Sigma rules live
|
|
66
|
-
```
|
|
67
|
-
|
|
68
|
-
A rule whose `*.test.yml` no longer passes now fails the check and blocks the
|
|
69
|
-
merge — the same safety net dbt gives analytics engineers.
|
|
70
|
-
|
|
71
|
-
## Why this, why now
|
|
72
|
-
|
|
73
|
-
- **Detection-as-code is mainstream** (SigmaHQ, Elastic detection-rules, Splunk
|
|
74
|
-
ESCU) but the *test* step is missing — the same gap dbt filled for analytics.
|
|
75
|
-
- **Self-host is a hard requirement, not a preference:** security telemetry can't
|
|
76
|
-
be shipped to someone else's cloud. That's the wedge closed SaaS SOC tools
|
|
77
|
-
(Dropzone, Prophet, Intezer) structurally can't serve.
|
|
78
|
-
- **The community distributes it:** good detection tooling spreads bottom-up on
|
|
79
|
-
GitHub. detkit is built to be forked, extended, and shared.
|
|
80
|
-
|
|
81
|
-
## Roadmap
|
|
82
|
-
|
|
83
|
-
- `detkit generate` — AI-draft a rule **and its tests** from a natural-language
|
|
84
|
-
threat description (tests are mandatory, never optional).
|
|
85
|
-
- More log schemas / field-mapping so one rule tests against multiple log sources.
|
|
86
|
-
- Close the remaining gaps detkit currently warns on: nested/dotted field access
|
|
87
|
-
(~3% of rules) and `base64` modifiers — likely via
|
|
88
|
-
[pySigma](https://github.com/SigmaHQ/pySigma) for full-spec parsing.
|
|
89
|
-
- Managed cloud (hosted runs, SSO, shared rule/test libraries) — the paid tier.
|
|
90
|
-
The tool stays free forever.
|
|
91
|
-
|
|
92
|
-
## Status
|
|
93
|
-
|
|
94
|
-
Early, but the core is real. detkit evaluates a rule's `detection`/`condition`
|
|
95
|
-
against log events and runs tests around it — covering the features used by the
|
|
96
|
-
**large majority of published SigmaHQ rules**: `contains`/`startswith`/`endswith`/
|
|
97
|
-
`re` modifiers, value wildcards (`*`/`?`), `|cidr`, list-as-OR, keyword lists, and
|
|
98
|
-
`X of` / `all of` conditions.
|
|
99
|
-
|
|
100
|
-
What it can't yet evaluate (nested/dotted fields, `base64` modifiers) it **flags
|
|
101
|
-
loudly** — `detkit validate` and `detkit test` print a `WARN` rather than return a
|
|
102
|
-
confident wrong answer. A detection tool that's silently wrong is worse than none.
|
|
103
|
-
|
|
104
|
-
Run the checks: `python tests/test_evaluator.py`. Limits are marked with
|
|
105
|
-
`# ponytail:` comments in `detkit/evaluator.py`.
|
|
106
|
-
|
|
107
|
-
MIT licensed. Contributions welcome.
|
|
@@ -1 +0,0 @@
|
|
|
1
|
-
__version__ = "0.1.0"
|
|
@@ -1,129 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.4
|
|
2
|
-
Name: detkit-cli
|
|
3
|
-
Version: 0.1.0
|
|
4
|
-
Summary: dbt for detections — test, validate, and CI-gate Sigma detection rules as code.
|
|
5
|
-
Author: detkit
|
|
6
|
-
License: MIT
|
|
7
|
-
Project-URL: Homepage, https://github.com/ELSATOAH/detkit
|
|
8
|
-
Project-URL: Repository, https://github.com/ELSATOAH/detkit
|
|
9
|
-
Project-URL: Issues, https://github.com/ELSATOAH/detkit/issues
|
|
10
|
-
Keywords: sigma,detection-engineering,detection-as-code,siem,security,detections,blue-team,cybersecurity
|
|
11
|
-
Classifier: Development Status :: 3 - Alpha
|
|
12
|
-
Classifier: Intended Audience :: Information Technology
|
|
13
|
-
Classifier: Topic :: Security
|
|
14
|
-
Classifier: License :: OSI Approved :: MIT License
|
|
15
|
-
Classifier: Programming Language :: Python :: 3
|
|
16
|
-
Classifier: Operating System :: OS Independent
|
|
17
|
-
Requires-Python: >=3.9
|
|
18
|
-
Description-Content-Type: text/markdown
|
|
19
|
-
License-File: LICENSE
|
|
20
|
-
Requires-Dist: pyyaml>=6
|
|
21
|
-
Dynamic: license-file
|
|
22
|
-
|
|
23
|
-
# detkit — dbt for detections
|
|
24
|
-
|
|
25
|
-
[](https://github.com/ELSATOAH/detkit/actions/workflows/ci.yml) [](LICENSE)
|
|
26
|
-
|
|
27
|
-
Test, validate, and CI-gate your Sigma detection rules **as code**, before they
|
|
28
|
-
ever reach production.
|
|
29
|
-
|
|
30
|
-

|
|
31
|
-
|
|
32
|
-
Detection engineers write rules in [Sigma](https://sigmahq.io/), commit them to
|
|
33
|
-
Git, and ship them to a SIEM — but there's no standard way to *unit-test* a rule
|
|
34
|
-
locally: does it actually fire on the attack it targets, and stay quiet on benign
|
|
35
|
-
traffic? Today that's checked by hand, or in prod. detkit closes that loop.
|
|
36
|
-
|
|
37
|
-
```
|
|
38
|
-
detkit test ./rules # run every rule against its sample events
|
|
39
|
-
detkit validate ./rules # structural + condition-reference checks
|
|
40
|
-
```
|
|
41
|
-
|
|
42
|
-
detkit is **open source (MIT)**, **self-hostable**, and **vendor-neutral** — it
|
|
43
|
-
sits *alongside* your SIEM (Wazuh, Elastic, Splunk, Sentinel), not in front of
|
|
44
|
-
it. Your logs and rules never leave your machine or CI runner.
|
|
45
|
-
|
|
46
|
-
## Quickstart
|
|
47
|
-
|
|
48
|
-
```bash
|
|
49
|
-
pipx install git+https://github.com/ELSATOAH/detkit.git # PyPI: `pipx install detkit-cli` (coming soon)
|
|
50
|
-
|
|
51
|
-
detkit init # scaffold a starter rule + test + a CI workflow
|
|
52
|
-
detkit test rules # -> green on the first run
|
|
53
|
-
```
|
|
54
|
-
|
|
55
|
-
Each rule gets a sibling `*.test.yml` describing sample events and whether the
|
|
56
|
-
rule should match:
|
|
57
|
-
|
|
58
|
-
```yaml
|
|
59
|
-
# whoami_execution.test.yml
|
|
60
|
-
tests:
|
|
61
|
-
- name: fires on whoami run by a normal user
|
|
62
|
-
event: { EventID: 4688, CommandLine: "cmd /c whoami /all", User: "alice" }
|
|
63
|
-
expect: match
|
|
64
|
-
- name: suppressed for SYSTEM
|
|
65
|
-
event: { EventID: 4688, CommandLine: "whoami", User: "SYSTEM" }
|
|
66
|
-
expect: no_match
|
|
67
|
-
```
|
|
68
|
-
|
|
69
|
-
`detkit test` exits non-zero on any failure, so you can drop it straight into CI
|
|
70
|
-
and block a pull request that breaks a detection.
|
|
71
|
-
|
|
72
|
-
## Use it in CI (GitHub Actions)
|
|
73
|
-
|
|
74
|
-
Drop this into your **rules** repo to gate every pull request:
|
|
75
|
-
|
|
76
|
-
```yaml
|
|
77
|
-
# .github/workflows/detections.yml
|
|
78
|
-
name: Detections
|
|
79
|
-
on: [pull_request]
|
|
80
|
-
jobs:
|
|
81
|
-
detkit:
|
|
82
|
-
runs-on: ubuntu-latest
|
|
83
|
-
steps:
|
|
84
|
-
- uses: actions/checkout@v4
|
|
85
|
-
- uses: ELSATOAH/detkit@v0 # this repo's composite action
|
|
86
|
-
with:
|
|
87
|
-
path: rules # where your Sigma rules live
|
|
88
|
-
```
|
|
89
|
-
|
|
90
|
-
A rule whose `*.test.yml` no longer passes now fails the check and blocks the
|
|
91
|
-
merge — the same safety net dbt gives analytics engineers.
|
|
92
|
-
|
|
93
|
-
## Why this, why now
|
|
94
|
-
|
|
95
|
-
- **Detection-as-code is mainstream** (SigmaHQ, Elastic detection-rules, Splunk
|
|
96
|
-
ESCU) but the *test* step is missing — the same gap dbt filled for analytics.
|
|
97
|
-
- **Self-host is a hard requirement, not a preference:** security telemetry can't
|
|
98
|
-
be shipped to someone else's cloud. That's the wedge closed SaaS SOC tools
|
|
99
|
-
(Dropzone, Prophet, Intezer) structurally can't serve.
|
|
100
|
-
- **The community distributes it:** good detection tooling spreads bottom-up on
|
|
101
|
-
GitHub. detkit is built to be forked, extended, and shared.
|
|
102
|
-
|
|
103
|
-
## Roadmap
|
|
104
|
-
|
|
105
|
-
- `detkit generate` — AI-draft a rule **and its tests** from a natural-language
|
|
106
|
-
threat description (tests are mandatory, never optional).
|
|
107
|
-
- More log schemas / field-mapping so one rule tests against multiple log sources.
|
|
108
|
-
- Close the remaining gaps detkit currently warns on: nested/dotted field access
|
|
109
|
-
(~3% of rules) and `base64` modifiers — likely via
|
|
110
|
-
[pySigma](https://github.com/SigmaHQ/pySigma) for full-spec parsing.
|
|
111
|
-
- Managed cloud (hosted runs, SSO, shared rule/test libraries) — the paid tier.
|
|
112
|
-
The tool stays free forever.
|
|
113
|
-
|
|
114
|
-
## Status
|
|
115
|
-
|
|
116
|
-
Early, but the core is real. detkit evaluates a rule's `detection`/`condition`
|
|
117
|
-
against log events and runs tests around it — covering the features used by the
|
|
118
|
-
**large majority of published SigmaHQ rules**: `contains`/`startswith`/`endswith`/
|
|
119
|
-
`re` modifiers, value wildcards (`*`/`?`), `|cidr`, list-as-OR, keyword lists, and
|
|
120
|
-
`X of` / `all of` conditions.
|
|
121
|
-
|
|
122
|
-
What it can't yet evaluate (nested/dotted fields, `base64` modifiers) it **flags
|
|
123
|
-
loudly** — `detkit validate` and `detkit test` print a `WARN` rather than return a
|
|
124
|
-
confident wrong answer. A detection tool that's silently wrong is worse than none.
|
|
125
|
-
|
|
126
|
-
Run the checks: `python tests/test_evaluator.py`. Limits are marked with
|
|
127
|
-
`# ponytail:` comments in `detkit/evaluator.py`.
|
|
128
|
-
|
|
129
|
-
MIT licensed. Contributions welcome.
|