ocsf-toolkit 0.1.0a1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- ocsf_toolkit-0.1.0a1/.gitignore +20 -0
- ocsf_toolkit-0.1.0a1/.importlinter +50 -0
- ocsf_toolkit-0.1.0a1/CHANGELOG.md +23 -0
- ocsf_toolkit-0.1.0a1/CONTRIBUTING.md +50 -0
- ocsf_toolkit-0.1.0a1/LICENSE +202 -0
- ocsf_toolkit-0.1.0a1/Makefile +91 -0
- ocsf_toolkit-0.1.0a1/PKG-INFO +205 -0
- ocsf_toolkit-0.1.0a1/README.md +183 -0
- ocsf_toolkit-0.1.0a1/SECURITY.md +22 -0
- ocsf_toolkit-0.1.0a1/docs/cli.md +79 -0
- ocsf_toolkit-0.1.0a1/docs/diff.md +55 -0
- ocsf_toolkit-0.1.0a1/docs/events.md +166 -0
- ocsf_toolkit-0.1.0a1/docs/models.md +99 -0
- ocsf_toolkit-0.1.0a1/docs/observables.md +74 -0
- ocsf_toolkit-0.1.0a1/docs/schema.md +133 -0
- ocsf_toolkit-0.1.0a1/ocsf.lock +115 -0
- ocsf_toolkit-0.1.0a1/packaging/version_dist/README.md.tmpl +18 -0
- ocsf_toolkit-0.1.0a1/packaging/version_dist/dist.py +247 -0
- ocsf_toolkit-0.1.0a1/packaging/version_dist/pyproject.toml.tmpl +27 -0
- ocsf_toolkit-0.1.0a1/pyproject.toml +118 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/__init__.py +55 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/_version.py +24 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/cli/__init__.py +113 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/cli/_common.py +19 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/cli/_upstream.py +147 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/cli/check.py +77 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/cli/compile.py +39 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/cli/diff.py +65 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/cli/generate.py +90 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/cli/ir.py +113 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/cli/lock.py +331 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/cli/models.py +88 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/cli/trees.py +27 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/codegen/__init__.py +10 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/codegen/emit.py +239 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/codegen/generate.py +337 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/codegen/layout.py +74 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/codegen/naming.py +65 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/codegen/types.py +27 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/conform.py +396 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/data/ir-1.0.0.json +28073 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/data/ir-1.1.0.json +41579 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/data/ir-1.2.0.json +50321 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/data/ir-1.3.0.json +57198 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/data/ir-1.4.0.json +85424 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/data/ir-1.5.0.json +91969 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/data/ir-1.6.0.json +94639 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/data/ir-1.7.0.json +96525 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/data/ir-1.8.0.json +98267 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/data/ir-1.9.0.json +108067 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/derive.py +288 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/diff.py +123 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/engine.py +243 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/export.py +460 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/ir.py +335 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/loader.py +290 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/models.py +177 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/observables.py +196 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/py.typed +0 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/schema.py +284 -0
- ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/walk.py +189 -0
- ocsf_toolkit-0.1.0a1/tests/__init__.py +0 -0
- ocsf_toolkit-0.1.0a1/tests/architecture/__init__.py +0 -0
- ocsf_toolkit-0.1.0a1/tests/architecture/test_docs.py +45 -0
- ocsf_toolkit-0.1.0a1/tests/architecture/test_imports.py +111 -0
- ocsf_toolkit-0.1.0a1/tests/architecture/test_scope.py +29 -0
- ocsf_toolkit-0.1.0a1/tests/bench/__init__.py +0 -0
- ocsf_toolkit-0.1.0a1/tests/bench/engine.py +151 -0
- ocsf_toolkit-0.1.0a1/tests/conftest.py +39 -0
- ocsf_toolkit-0.1.0a1/tests/fixtures/__init__.py +0 -0
- ocsf_toolkit-0.1.0a1/tests/fixtures/compiled.py +182 -0
- ocsf_toolkit-0.1.0a1/tests/fixtures/events.py +119 -0
- ocsf_toolkit-0.1.0a1/tests/fixtures/notations.py +10 -0
- ocsf_toolkit-0.1.0a1/tests/fixtures/packages.py +42 -0
- ocsf_toolkit-0.1.0a1/tests/generated/__init__.py +0 -0
- ocsf_toolkit-0.1.0a1/tests/generated/conftest.py +45 -0
- ocsf_toolkit-0.1.0a1/tests/generated/samples/usage.py +43 -0
- ocsf_toolkit-0.1.0a1/tests/generated/test_import.py +67 -0
- ocsf_toolkit-0.1.0a1/tests/generated/test_readme.py +22 -0
- ocsf_toolkit-0.1.0a1/tests/generated/test_snapshot.py +19 -0
- ocsf_toolkit-0.1.0a1/tests/generated/test_static.py +54 -0
- ocsf_toolkit-0.1.0a1/tests/golden/detection_finding_1_9_0.py +178 -0
- ocsf_toolkit-0.1.0a1/tests/property/__init__.py +0 -0
- ocsf_toolkit-0.1.0a1/tests/property/strategies.py +216 -0
- ocsf_toolkit-0.1.0a1/tests/property/test_repair.py +154 -0
- ocsf_toolkit-0.1.0a1/tests/property/test_roundtrip.py +75 -0
- ocsf_toolkit-0.1.0a1/tests/property/test_strategies.py +35 -0
- ocsf_toolkit-0.1.0a1/tests/unit/__init__.py +0 -0
- ocsf_toolkit-0.1.0a1/tests/unit/cli/__init__.py +0 -0
- ocsf_toolkit-0.1.0a1/tests/unit/cli/conftest.py +48 -0
- ocsf_toolkit-0.1.0a1/tests/unit/cli/support.py +58 -0
- ocsf_toolkit-0.1.0a1/tests/unit/cli/test_check.py +96 -0
- ocsf_toolkit-0.1.0a1/tests/unit/cli/test_compile.py +181 -0
- ocsf_toolkit-0.1.0a1/tests/unit/cli/test_diff.py +87 -0
- ocsf_toolkit-0.1.0a1/tests/unit/cli/test_generate.py +103 -0
- ocsf_toolkit-0.1.0a1/tests/unit/cli/test_ir.py +124 -0
- ocsf_toolkit-0.1.0a1/tests/unit/cli/test_lock.py +243 -0
- ocsf_toolkit-0.1.0a1/tests/unit/cli/test_models.py +72 -0
- ocsf_toolkit-0.1.0a1/tests/unit/cli/test_parser.py +72 -0
- ocsf_toolkit-0.1.0a1/tests/unit/codegen/__init__.py +0 -0
- ocsf_toolkit-0.1.0a1/tests/unit/codegen/test_emit.py +84 -0
- ocsf_toolkit-0.1.0a1/tests/unit/codegen/test_generate.py +169 -0
- ocsf_toolkit-0.1.0a1/tests/unit/codegen/test_naming.py +91 -0
- ocsf_toolkit-0.1.0a1/tests/unit/codegen/test_types_layout.py +57 -0
- ocsf_toolkit-0.1.0a1/tests/unit/conftest.py +41 -0
- ocsf_toolkit-0.1.0a1/tests/unit/test_conform.py +195 -0
- ocsf_toolkit-0.1.0a1/tests/unit/test_derive.py +273 -0
- ocsf_toolkit-0.1.0a1/tests/unit/test_diff.py +176 -0
- ocsf_toolkit-0.1.0a1/tests/unit/test_engine.py +339 -0
- ocsf_toolkit-0.1.0a1/tests/unit/test_export.py +170 -0
- ocsf_toolkit-0.1.0a1/tests/unit/test_ir_loader.py +284 -0
- ocsf_toolkit-0.1.0a1/tests/unit/test_models.py +223 -0
- ocsf_toolkit-0.1.0a1/tests/unit/test_observables.py +236 -0
- ocsf_toolkit-0.1.0a1/tests/unit/test_packaging.py +130 -0
- ocsf_toolkit-0.1.0a1/tests/unit/test_public_api.py +95 -0
- ocsf_toolkit-0.1.0a1/tests/unit/test_schema.py +383 -0
- ocsf_toolkit-0.1.0a1/tests/unit/test_walk.py +137 -0
- ocsf_toolkit-0.1.0a1/uv.lock +849 -0
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
# build outputs: compiled schemas, schema checkouts, generated dists, wheels
|
|
2
|
+
build/
|
|
3
|
+
dist/
|
|
4
|
+
*.egg-info/
|
|
5
|
+
|
|
6
|
+
# written by hatch-vcs at build time
|
|
7
|
+
src/ocsf_toolkit/_version.py
|
|
8
|
+
|
|
9
|
+
# environments and caches
|
|
10
|
+
.venv/
|
|
11
|
+
__pycache__/
|
|
12
|
+
*.py[cod]
|
|
13
|
+
.pytest_cache/
|
|
14
|
+
.ruff_cache/
|
|
15
|
+
.mypy_cache/
|
|
16
|
+
.import_linter_cache/
|
|
17
|
+
.coverage
|
|
18
|
+
.coverage.*
|
|
19
|
+
htmlcov/
|
|
20
|
+
.DS_Store
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# Import contracts: the core layering, the msgspec boundary, and no imports of generated packages.
|
|
2
|
+
# Modules arrive milestone by milestone (M1–M6); until then these contracts are vacuous.
|
|
3
|
+
# Layers in parentheses are optional: absent today, enforced as soon as the module exists.
|
|
4
|
+
# (A `forbidden` contract errors on a missing source module, so per-module rules are expressed
|
|
5
|
+
# as layers or anchored on the root package.)
|
|
6
|
+
|
|
7
|
+
[importlinter]
|
|
8
|
+
root_package = ocsf_toolkit
|
|
9
|
+
include_external_packages = True
|
|
10
|
+
|
|
11
|
+
# ir <- loader/export <- schema <- walk <- derive/conform/observables <- engine <- models,
|
|
12
|
+
# with codegen and diff beside walk: they may use schema and below, nothing above, not each other.
|
|
13
|
+
# cli is the outermost layer; nothing in core imports it.
|
|
14
|
+
[importlinter:contract:layers]
|
|
15
|
+
name = Core layering (codegen and diff depend on schema only)
|
|
16
|
+
type = layers
|
|
17
|
+
containers = ocsf_toolkit
|
|
18
|
+
layers =
|
|
19
|
+
(cli)
|
|
20
|
+
(models)
|
|
21
|
+
(engine)
|
|
22
|
+
(derive) | (conform) | (observables)
|
|
23
|
+
(walk) | (codegen) | (diff)
|
|
24
|
+
(schema)
|
|
25
|
+
(loader) | (export)
|
|
26
|
+
(ir)
|
|
27
|
+
|
|
28
|
+
[importlinter:contract:msgspec-boundary]
|
|
29
|
+
name = Only models and codegen import msgspec
|
|
30
|
+
type = forbidden
|
|
31
|
+
source_modules =
|
|
32
|
+
ocsf_toolkit
|
|
33
|
+
forbidden_modules =
|
|
34
|
+
msgspec
|
|
35
|
+
allow_indirect_imports = True
|
|
36
|
+
ignore_imports =
|
|
37
|
+
ocsf_toolkit.models -> msgspec
|
|
38
|
+
ocsf_toolkit.codegen -> msgspec
|
|
39
|
+
ocsf_toolkit.codegen.** -> msgspec
|
|
40
|
+
unmatched_ignore_imports_alerting = none
|
|
41
|
+
|
|
42
|
+
[importlinter:contract:no-generated-packages]
|
|
43
|
+
name = Nothing in ocsf_toolkit imports a generated version package (ocsf_toolkit_v*)
|
|
44
|
+
type = forbidden
|
|
45
|
+
source_modules =
|
|
46
|
+
ocsf_toolkit
|
|
47
|
+
forbidden_modules =
|
|
48
|
+
ocsf_toolkit_v1_7_0
|
|
49
|
+
ocsf_toolkit_v1_8_0
|
|
50
|
+
ocsf_toolkit_v1_9_0
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
The library follows SemVer on its own API. OCSF schema versions are data, so support for a new OCSF
|
|
4
|
+
version ships as a minor release. One line per merged PR, taken from the PR title.
|
|
5
|
+
|
|
6
|
+
## Unreleased
|
|
7
|
+
|
|
8
|
+
## 0.1.0a1
|
|
9
|
+
|
|
10
|
+
The first alpha. Model packages `ocsf-toolkit-v1-7-0` and `ocsf-toolkit-v1-8-0` publish with it;
|
|
11
|
+
`ocsf-toolkit-v1-9-0` follows with the next release.
|
|
12
|
+
|
|
13
|
+
- Repository skeleton, `ocsf.lock` pinning OCSF 1.9.0, `ocsf-toolkit compile` and `ocsf-toolkit lock --add`.
|
|
14
|
+
- Schema IR (`SchemaIR`), compiler-JSON loader with key accounting, canonical IR JSON (`ir_version` 1.0) and `load_ir`; `ocsf-toolkit ir export` / `ir check`; IR for OCSF 1.9.0 shipped in the package; `ocsf.lock` pins OCSF 1.0.0–1.9.0.
|
|
15
|
+
- `load()` / `resolve()` → `ResolvedSchema`: profile- and extension-aware attribute sets, class identity, sibling pairs, `type_uid` registry and observable markers, reachable without a model.
|
|
16
|
+
- `build()`, `parse()` and `repair()` on event dicts under the `STRICT`, `LENIENT` and `REPAIR` policies: one traversal (`walk()`) that derives what the schema computes and reports every deviation as a `Finding` of a `Kind` in a `Report`; only `build()` raises, and only `ConformanceError`. `ocsf-toolkit check` reports on JSON Lines files.
|
|
17
|
+
- Observables: `extract()` and `merge()` → `Observable`, with value validation.
|
|
18
|
+
- `diff()` → `SchemaDiff`: a structural diff of two IRs with classes matched by uid, so a renamed class shows as a rename; `ocsf-toolkit diff`, with `--json` and `--semver` (the release category for a re-pinned IR). IRs for OCSF 1.1.0, 1.3.0 and 1.7.0 shipped.
|
|
19
|
+
- `generate()` and `ocsf-toolkit generate`: an OCSF version or IR projected into a typed package of msgspec Structs, superset by default or exact for a `--profiles` set, with `--ext` for custom extensions; `build()` and `parse()` accept and return the generated classes.
|
|
20
|
+
- Model dists `ocsf-toolkit-v1-7-0`, `-v1-8-0` and `-v1-9-0`, each bundling its IR and requiring a range of core versions, so a core release republishes a dist only when its generated tree changes. `ocsf-toolkit models check` / `record` pin each tree in `ocsf.lock`; `lock --add` keeps models for the newest three versions. IRs for every OCSF release from 1.0.0 to 1.9.0 ship in core.
|
|
21
|
+
- Release workflow: reproducible sdists and wheels, PyPI trusted publishing, and a GitHub release with `ocsf.lock` and `SHA256SUMS`.
|
|
22
|
+
- Property tests (Hypothesis, strategies drawn from the IR): dict and typed round trips for every core class of the model-window versions, and repair undoing unambiguous damage idempotently. Fixes they found: a typed `build()` reads profiles from a `Metadata` Struct, and `repair()` finds the class from a `class_uid`/`type_uid` it coerces. `make bench` measures engine throughput; the engine walks at about 2 µs per attribute (1.55× faster than its first cut).
|
|
23
|
+
- README with installation and a worked example, and API docs in `docs/` (schema, events, observables, diff, typed models, CLI); every example runs as a doctest.
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# Contributing
|
|
2
|
+
|
|
3
|
+
## Setup
|
|
4
|
+
|
|
5
|
+
Requires [uv](https://docs.astral.sh/uv/) and git. The library runs on Python ≥ 3.12. Compiling an OCSF
|
|
6
|
+
schema needs Python 3.14 (because of `ocsf-schema-compiler`); `uvx` fetches it for that step, so you
|
|
7
|
+
do not need to install it yourself.
|
|
8
|
+
|
|
9
|
+
```shell
|
|
10
|
+
make setup # uv sync: dev environment in .venv
|
|
11
|
+
make lint # ruff format --check, ruff check, lint-imports
|
|
12
|
+
make type # pyright --strict (blocking), mypy --strict (informational)
|
|
13
|
+
make test # pytest
|
|
14
|
+
make schema # ocsf-toolkit compile: rebuild build/compiled/ and check the lockfile hashes
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
`uvx pre-commit install` runs the lint and type checks on every commit (`.pre-commit-config.yaml`).
|
|
18
|
+
|
|
19
|
+
## Rules that bind code
|
|
20
|
+
|
|
21
|
+
These architecture rules are binding: one event walker; import direction enforced by
|
|
22
|
+
`.importlinter`; one exception type (`ConformanceError`); no `__getattr__` in core; thin entry points;
|
|
23
|
+
a 13-module core budget; a query surface of frozen mappings. A PR that needs to break one of
|
|
24
|
+
them argues for it in the description.
|
|
25
|
+
|
|
26
|
+
Conventions:
|
|
27
|
+
|
|
28
|
+
- No runtime dependencies besides `msgspec`. The CLI uses `argparse`.
|
|
29
|
+
- Each module's docstring states its single responsibility in one sentence.
|
|
30
|
+
- Lines are at most 120 characters; ruff format sets the layout.
|
|
31
|
+
- Every public module, class and function has a docstring whose first line is an imperative
|
|
32
|
+
summary (pydocstyle, pep257 convention). Tests are exempt.
|
|
33
|
+
|
|
34
|
+
## Pull requests
|
|
35
|
+
|
|
36
|
+
- `main` is protected; PRs are squash-merged and the PR title becomes the changelog line.
|
|
37
|
+
- Plain commit messages; no conventional-commits prefixes required.
|
|
38
|
+
- Changing `ocsf.lock` triggers the schema workflow.
|
|
39
|
+
- A compiler bump (`[toolchain]` in `ocsf.lock`) is expected to change every `compiled_sha256`; run
|
|
40
|
+
`make schema` and update them in the same PR.
|
|
41
|
+
|
|
42
|
+
## Adding an OCSF version
|
|
43
|
+
|
|
44
|
+
```shell
|
|
45
|
+
make lock VERSION=x.y.z # ocsf-toolkit lock --add x.y.z
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
This resolves the upstream tag (`x.y.z`, then `vx.y.z`), pins its commit, compiles it, and records
|
|
49
|
+
`compiled_sha256`. The compile fails if the compiled schema's `version` does not equal `x.y.z`. Tag
|
|
50
|
+
`1.5.0` upstream is one such case; pass `--ref` to pin a branch instead.
|
|
@@ -0,0 +1,202 @@
|
|
|
1
|
+
|
|
2
|
+
Apache License
|
|
3
|
+
Version 2.0, January 2004
|
|
4
|
+
http://www.apache.org/licenses/
|
|
5
|
+
|
|
6
|
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
|
7
|
+
|
|
8
|
+
1. Definitions.
|
|
9
|
+
|
|
10
|
+
"License" shall mean the terms and conditions for use, reproduction,
|
|
11
|
+
and distribution as defined by Sections 1 through 9 of this document.
|
|
12
|
+
|
|
13
|
+
"Licensor" shall mean the copyright owner or entity authorized by
|
|
14
|
+
the copyright owner that is granting the License.
|
|
15
|
+
|
|
16
|
+
"Legal Entity" shall mean the union of the acting entity and all
|
|
17
|
+
other entities that control, are controlled by, or are under common
|
|
18
|
+
control with that entity. For the purposes of this definition,
|
|
19
|
+
"control" means (i) the power, direct or indirect, to cause the
|
|
20
|
+
direction or management of such entity, whether by contract or
|
|
21
|
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
|
22
|
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
|
23
|
+
|
|
24
|
+
"You" (or "Your") shall mean an individual or Legal Entity
|
|
25
|
+
exercising permissions granted by this License.
|
|
26
|
+
|
|
27
|
+
"Source" form shall mean the preferred form for making modifications,
|
|
28
|
+
including but not limited to software source code, documentation
|
|
29
|
+
source, and configuration files.
|
|
30
|
+
|
|
31
|
+
"Object" form shall mean any form resulting from mechanical
|
|
32
|
+
transformation or translation of a Source form, including but
|
|
33
|
+
not limited to compiled object code, generated documentation,
|
|
34
|
+
and conversions to other media types.
|
|
35
|
+
|
|
36
|
+
"Work" shall mean the work of authorship, whether in Source or
|
|
37
|
+
Object form, made available under the License, as indicated by a
|
|
38
|
+
copyright notice that is included in or attached to the work
|
|
39
|
+
(an example is provided in the Appendix below).
|
|
40
|
+
|
|
41
|
+
"Derivative Works" shall mean any work, whether in Source or Object
|
|
42
|
+
form, that is based on (or derived from) the Work and for which the
|
|
43
|
+
editorial revisions, annotations, elaborations, or other modifications
|
|
44
|
+
represent, as a whole, an original work of authorship. For the purposes
|
|
45
|
+
of this License, Derivative Works shall not include works that remain
|
|
46
|
+
separable from, or merely link (or bind by name) to the interfaces of,
|
|
47
|
+
the Work and Derivative Works thereof.
|
|
48
|
+
|
|
49
|
+
"Contribution" shall mean any work of authorship, including
|
|
50
|
+
the original version of the Work and any modifications or additions
|
|
51
|
+
to that Work or Derivative Works thereof, that is intentionally
|
|
52
|
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
|
53
|
+
or by an individual or Legal Entity authorized to submit on behalf of
|
|
54
|
+
the copyright owner. For the purposes of this definition, "submitted"
|
|
55
|
+
means any form of electronic, verbal, or written communication sent
|
|
56
|
+
to the Licensor or its representatives, including but not limited to
|
|
57
|
+
communication on electronic mailing lists, source code control systems,
|
|
58
|
+
and issue tracking systems that are managed by, or on behalf of, the
|
|
59
|
+
Licensor for the purpose of discussing and improving the Work, but
|
|
60
|
+
excluding communication that is conspicuously marked or otherwise
|
|
61
|
+
designated in writing by the copyright owner as "Not a Contribution."
|
|
62
|
+
|
|
63
|
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
|
64
|
+
on behalf of whom a Contribution has been received by Licensor and
|
|
65
|
+
subsequently incorporated within the Work.
|
|
66
|
+
|
|
67
|
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
|
68
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
69
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
70
|
+
copyright license to reproduce, prepare Derivative Works of,
|
|
71
|
+
publicly display, publicly perform, sublicense, and distribute the
|
|
72
|
+
Work and such Derivative Works in Source or Object form.
|
|
73
|
+
|
|
74
|
+
3. Grant of Patent License. Subject to the terms and conditions of
|
|
75
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
76
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
77
|
+
(except as stated in this section) patent license to make, have made,
|
|
78
|
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
|
79
|
+
where such license applies only to those patent claims licensable
|
|
80
|
+
by such Contributor that are necessarily infringed by their
|
|
81
|
+
Contribution(s) alone or by combination of their Contribution(s)
|
|
82
|
+
with the Work to which such Contribution(s) was submitted. If You
|
|
83
|
+
institute patent litigation against any entity (including a
|
|
84
|
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
|
85
|
+
or a Contribution incorporated within the Work constitutes direct
|
|
86
|
+
or contributory patent infringement, then any patent licenses
|
|
87
|
+
granted to You under this License for that Work shall terminate
|
|
88
|
+
as of the date such litigation is filed.
|
|
89
|
+
|
|
90
|
+
4. Redistribution. You may reproduce and distribute copies of the
|
|
91
|
+
Work or Derivative Works thereof in any medium, with or without
|
|
92
|
+
modifications, and in Source or Object form, provided that You
|
|
93
|
+
meet the following conditions:
|
|
94
|
+
|
|
95
|
+
(a) You must give any other recipients of the Work or
|
|
96
|
+
Derivative Works a copy of this License; and
|
|
97
|
+
|
|
98
|
+
(b) You must cause any modified files to carry prominent notices
|
|
99
|
+
stating that You changed the files; and
|
|
100
|
+
|
|
101
|
+
(c) You must retain, in the Source form of any Derivative Works
|
|
102
|
+
that You distribute, all copyright, patent, trademark, and
|
|
103
|
+
attribution notices from the Source form of the Work,
|
|
104
|
+
excluding those notices that do not pertain to any part of
|
|
105
|
+
the Derivative Works; and
|
|
106
|
+
|
|
107
|
+
(d) If the Work includes a "NOTICE" text file as part of its
|
|
108
|
+
distribution, then any Derivative Works that You distribute must
|
|
109
|
+
include a readable copy of the attribution notices contained
|
|
110
|
+
within such NOTICE file, excluding those notices that do not
|
|
111
|
+
pertain to any part of the Derivative Works, in at least one
|
|
112
|
+
of the following places: within a NOTICE text file distributed
|
|
113
|
+
as part of the Derivative Works; within the Source form or
|
|
114
|
+
documentation, if provided along with the Derivative Works; or,
|
|
115
|
+
within a display generated by the Derivative Works, if and
|
|
116
|
+
wherever such third-party notices normally appear. The contents
|
|
117
|
+
of the NOTICE file are for informational purposes only and
|
|
118
|
+
do not modify the License. You may add Your own attribution
|
|
119
|
+
notices within Derivative Works that You distribute, alongside
|
|
120
|
+
or as an addendum to the NOTICE text from the Work, provided
|
|
121
|
+
that such additional attribution notices cannot be construed
|
|
122
|
+
as modifying the License.
|
|
123
|
+
|
|
124
|
+
You may add Your own copyright statement to Your modifications and
|
|
125
|
+
may provide additional or different license terms and conditions
|
|
126
|
+
for use, reproduction, or distribution of Your modifications, or
|
|
127
|
+
for any such Derivative Works as a whole, provided Your use,
|
|
128
|
+
reproduction, and distribution of the Work otherwise complies with
|
|
129
|
+
the conditions stated in this License.
|
|
130
|
+
|
|
131
|
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
|
132
|
+
any Contribution intentionally submitted for inclusion in the Work
|
|
133
|
+
by You to the Licensor shall be under the terms and conditions of
|
|
134
|
+
this License, without any additional terms or conditions.
|
|
135
|
+
Notwithstanding the above, nothing herein shall supersede or modify
|
|
136
|
+
the terms of any separate license agreement you may have executed
|
|
137
|
+
with Licensor regarding such Contributions.
|
|
138
|
+
|
|
139
|
+
6. Trademarks. This License does not grant permission to use the trade
|
|
140
|
+
names, trademarks, service marks, or product names of the Licensor,
|
|
141
|
+
except as required for reasonable and customary use in describing the
|
|
142
|
+
origin of the Work and reproducing the content of the NOTICE file.
|
|
143
|
+
|
|
144
|
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
|
145
|
+
agreed to in writing, Licensor provides the Work (and each
|
|
146
|
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
|
147
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
148
|
+
implied, including, without limitation, any warranties or conditions
|
|
149
|
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
|
150
|
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
|
151
|
+
appropriateness of using or redistributing the Work and assume any
|
|
152
|
+
risks associated with Your exercise of permissions under this License.
|
|
153
|
+
|
|
154
|
+
8. Limitation of Liability. In no event and under no legal theory,
|
|
155
|
+
whether in tort (including negligence), contract, or otherwise,
|
|
156
|
+
unless required by applicable law (such as deliberate and grossly
|
|
157
|
+
negligent acts) or agreed to in writing, shall any Contributor be
|
|
158
|
+
liable to You for damages, including any direct, indirect, special,
|
|
159
|
+
incidental, or consequential damages of any character arising as a
|
|
160
|
+
result of this License or out of the use or inability to use the
|
|
161
|
+
Work (including but not limited to damages for loss of goodwill,
|
|
162
|
+
work stoppage, computer failure or malfunction, or any and all
|
|
163
|
+
other commercial damages or losses), even if such Contributor
|
|
164
|
+
has been advised of the possibility of such damages.
|
|
165
|
+
|
|
166
|
+
9. Accepting Warranty or Additional Liability. While redistributing
|
|
167
|
+
the Work or Derivative Works thereof, You may choose to offer,
|
|
168
|
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
|
169
|
+
or other liability obligations and/or rights consistent with this
|
|
170
|
+
License. However, in accepting such obligations, You may act only
|
|
171
|
+
on Your own behalf and on Your sole responsibility, not on behalf
|
|
172
|
+
of any other Contributor, and only if You agree to indemnify,
|
|
173
|
+
defend, and hold each Contributor harmless for any liability
|
|
174
|
+
incurred by, or claims asserted against, such Contributor by reason
|
|
175
|
+
of your accepting any such warranty or additional liability.
|
|
176
|
+
|
|
177
|
+
END OF TERMS AND CONDITIONS
|
|
178
|
+
|
|
179
|
+
APPENDIX: How to apply the Apache License to your work.
|
|
180
|
+
|
|
181
|
+
To apply the Apache License to your work, attach the following
|
|
182
|
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
|
183
|
+
replaced with your own identifying information. (Don't include
|
|
184
|
+
the brackets!) The text should be enclosed in the appropriate
|
|
185
|
+
comment syntax for the file format. We also recommend that a
|
|
186
|
+
file or class name and description of purpose be included on the
|
|
187
|
+
same "printed page" as the copyright notice for easier
|
|
188
|
+
identification within third-party archives.
|
|
189
|
+
|
|
190
|
+
Copyright [yyyy] [name of copyright owner]
|
|
191
|
+
|
|
192
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
|
193
|
+
you may not use this file except in compliance with the License.
|
|
194
|
+
You may obtain a copy of the License at
|
|
195
|
+
|
|
196
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
|
197
|
+
|
|
198
|
+
Unless required by applicable law or agreed to in writing, software
|
|
199
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
|
200
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
201
|
+
See the License for the specific language governing permissions and
|
|
202
|
+
limitations under the License.
|
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
.PHONY: setup format lint type test bench schema ir lock generate dists wheels compat test-generated clean
|
|
2
|
+
|
|
3
|
+
UV ?= uv
|
|
4
|
+
|
|
5
|
+
setup:
|
|
6
|
+
$(UV) sync
|
|
7
|
+
|
|
8
|
+
format:
|
|
9
|
+
$(UV) run ruff format .
|
|
10
|
+
$(UV) run ruff check --fix .
|
|
11
|
+
|
|
12
|
+
lint:
|
|
13
|
+
$(UV) run ruff format --check .
|
|
14
|
+
$(UV) run ruff check .
|
|
15
|
+
$(UV) run lint-imports
|
|
16
|
+
|
|
17
|
+
# pyright strict is the blocking gate; mypy strict is informational.
|
|
18
|
+
type:
|
|
19
|
+
$(UV) run pyright
|
|
20
|
+
-$(UV) run mypy
|
|
21
|
+
|
|
22
|
+
test:
|
|
23
|
+
$(UV) run pytest
|
|
24
|
+
|
|
25
|
+
# S1 throughput at 2-20 KB per event, on this machine; slow, so not part of `test` or CI
|
|
26
|
+
bench:
|
|
27
|
+
$(UV) run python -m tests.bench.engine
|
|
28
|
+
|
|
29
|
+
# gates 1 and 2: compile every lockfile schema, then check every committed IR against it
|
|
30
|
+
schema:
|
|
31
|
+
$(UV) run ocsf-toolkit compile
|
|
32
|
+
$(UV) run ocsf-toolkit ir check
|
|
33
|
+
|
|
34
|
+
# rewrite data/ir-<ver>.json for VERSION (default: every entry with an ir_sha256); RECORD=1 accepts a new IR
|
|
35
|
+
ir:
|
|
36
|
+
$(UV) run ocsf-toolkit ir export $(VERSION) $(if $(RECORD),--record)
|
|
37
|
+
|
|
38
|
+
# pin a new OCSF version, ship its IR, move the model window and record the window's trees
|
|
39
|
+
lock:
|
|
40
|
+
@test -n "$(VERSION)" || { echo "usage: make lock VERSION=x.y.z"; exit 2; }
|
|
41
|
+
$(UV) run ocsf-toolkit lock --add $(VERSION)
|
|
42
|
+
$(UV) run ocsf-toolkit ir export $(VERSION) --record
|
|
43
|
+
$(UV) run ocsf-toolkit models record
|
|
44
|
+
|
|
45
|
+
# The model dist for one OCSF version as an installable project in build/dists/<package>/.
|
|
46
|
+
# Nothing here deletes: `generate` replaces <package>/ only if every file in it is generated,
|
|
47
|
+
# and the rendered project files are overwritten by name. The dist requires the core version
|
|
48
|
+
# building it, up to the next generated-code contract break.
|
|
49
|
+
OCSF ?= 1.9.0
|
|
50
|
+
PACKAGE = ocsf_toolkit_v$(subst .,_,$(OCSF))
|
|
51
|
+
DIST = build/dists/$(PACKAGE)
|
|
52
|
+
WINDOW = $(shell $(UV) run python -c 'import tomllib; print(*(e["version"] for e in \
|
|
53
|
+
tomllib.load(open("ocsf.lock", "rb"))["schemas"] if e["ship_models"]))')
|
|
54
|
+
WHEELS = build/wheels
|
|
55
|
+
# sdists and wheels carry this timestamp, so rebuilding one commit gives identical bytes
|
|
56
|
+
export SOURCE_DATE_EPOCH ?= $(shell git log -1 --format=%ct)
|
|
57
|
+
|
|
58
|
+
generate:
|
|
59
|
+
@echo "$(OCSF)" | grep -Eq '^[0-9]+\.[0-9]+\.[0-9]+$$' || { echo "OCSF must be x.y.z"; exit 2; }
|
|
60
|
+
mkdir -p $(DIST)
|
|
61
|
+
$(UV) run ocsf-toolkit generate $(OCSF) --package $(PACKAGE) --out $(DIST)
|
|
62
|
+
version=$$($(UV)x --quiet --with hatch-vcs hatchling version) && \
|
|
63
|
+
$(UV) run python packaging/version_dist/dist.py render --ocsf $(OCSF) --version "$$version" \
|
|
64
|
+
--out $(DIST)
|
|
65
|
+
|
|
66
|
+
# every model-window dist, then regeneration gate 3 over the trees on disk
|
|
67
|
+
dists:
|
|
68
|
+
for v in $(WINDOW); do $(MAKE) --no-print-directory generate OCSF=$$v || exit 1; done
|
|
69
|
+
$(UV) run ocsf-toolkit models check
|
|
70
|
+
|
|
71
|
+
# core and every model-window dist as sdist and wheel in build/wheels/, each wheel checked
|
|
72
|
+
wheels: dists
|
|
73
|
+
$(UV) build --quiet --out-dir $(WHEELS)
|
|
74
|
+
for v in $(WINDOW); do \
|
|
75
|
+
$(UV) build --quiet --out-dir $(WHEELS) build/dists/ocsf_toolkit_v$$(echo $$v | tr . _) \
|
|
76
|
+
|| exit 1; \
|
|
77
|
+
done
|
|
78
|
+
$(UV) run python packaging/version_dist/dist.py check-wheels $(WHEELS)/*.whl
|
|
79
|
+
|
|
80
|
+
# the released model dists (LOCAL=1: the wheels in build/wheels) run on the core in this checkout
|
|
81
|
+
compat:
|
|
82
|
+
version=$$($(UV)x --quiet --with hatch-vcs hatchling version) && \
|
|
83
|
+
$(UV) run python packaging/version_dist/dist.py compat --version "$$version" \
|
|
84
|
+
$(if $(LOCAL),--wheels $(WHEELS))
|
|
85
|
+
|
|
86
|
+
# static analysis, snapshot, laziness and a clean-environment install of the generated dist
|
|
87
|
+
test-generated:
|
|
88
|
+
$(UV) run pytest tests/generated
|
|
89
|
+
|
|
90
|
+
clean:
|
|
91
|
+
rm -rf build dist .pytest_cache .ruff_cache .mypy_cache .import_linter_cache
|
|
@@ -0,0 +1,205 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: ocsf-toolkit
|
|
3
|
+
Version: 0.1.0a1
|
|
4
|
+
Summary: Version-pinned OCSF schema semantics: substrate, derivation, conformance, observables, diff, and generated msgspec models.
|
|
5
|
+
Project-URL: Repository, https://github.com/k4otix/ocsf-toolkit
|
|
6
|
+
Project-URL: Changelog, https://github.com/k4otix/ocsf-toolkit/blob/main/CHANGELOG.md
|
|
7
|
+
Author: Eddie Allan
|
|
8
|
+
License-Expression: Apache-2.0
|
|
9
|
+
License-File: LICENSE
|
|
10
|
+
Keywords: msgspec,ocsf,schema,security
|
|
11
|
+
Classifier: Development Status :: 2 - Pre-Alpha
|
|
12
|
+
Classifier: Intended Audience :: Developers
|
|
13
|
+
Classifier: Programming Language :: Python :: 3
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
17
|
+
Classifier: Topic :: Security
|
|
18
|
+
Classifier: Typing :: Typed
|
|
19
|
+
Requires-Python: >=3.12
|
|
20
|
+
Requires-Dist: msgspec<1,>=0.22
|
|
21
|
+
Description-Content-Type: text/markdown
|
|
22
|
+
|
|
23
|
+
# ocsf-toolkit
|
|
24
|
+
|
|
25
|
+
Version-pinned [OCSF](https://schema.ocsf.io/) semantics for Python. Each OCSF release is compiled
|
|
26
|
+
from upstream `ocsf/ocsf-schema` at a pinned commit into a schema you can query as data. On top of
|
|
27
|
+
that schema, the library builds events and derives everything the schema computes, checks events and
|
|
28
|
+
reports every deviation it finds, extracts observables, and diffs two versions. Typed
|
|
29
|
+
[msgspec](https://github.com/msgspec/msgspec) models are generated from the same schema.
|
|
30
|
+
|
|
31
|
+
**Status: alpha.** The API may change before 0.1.0.
|
|
32
|
+
|
|
33
|
+
## Install
|
|
34
|
+
|
|
35
|
+
```shell
|
|
36
|
+
pip install --pre ocsf-toolkit # core: every OCSF release from 1.0.0 to 1.9.0, as data
|
|
37
|
+
pip install --pre ocsf-toolkit-v1-8-0 # optional: typed models for OCSF 1.8.0 (also -v1-7-0)
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
Typed models for OCSF 1.9.0 (`ocsf-toolkit-v1-9-0`) arrive on PyPI with the next release. Until then,
|
|
41
|
+
`ocsf-toolkit generate` builds the same package locally (see [Command line](docs/cli.md)).
|
|
42
|
+
|
|
43
|
+
Python 3.12 or newer. The only runtime dependency is msgspec.
|
|
44
|
+
|
|
45
|
+
## Build and emit one event
|
|
46
|
+
|
|
47
|
+
Pick an OCSF version and a class, give the fields you know, and `build` fills in what the schema
|
|
48
|
+
derives: class and category identity, `type_uid`, the label beside every `_id` enum, and
|
|
49
|
+
`metadata.version`.
|
|
50
|
+
|
|
51
|
+
```pycon
|
|
52
|
+
>>> import json
|
|
53
|
+
>>> from ocsf_toolkit import build, load
|
|
54
|
+
>>> schema = load("1.9.0")
|
|
55
|
+
>>> event = build(
|
|
56
|
+
... (schema, "detection_finding"),
|
|
57
|
+
... activity_id=1,
|
|
58
|
+
... severity_id=4,
|
|
59
|
+
... time=1_700_000_000_000,
|
|
60
|
+
... metadata={"product": {"name": "Sensor", "vendor_name": "Acme"}},
|
|
61
|
+
... finding_info={"uid": "f-1", "title": "Suspicious login"},
|
|
62
|
+
... )
|
|
63
|
+
>>> print(json.dumps(event, indent=2, sort_keys=True))
|
|
64
|
+
{
|
|
65
|
+
"activity_id": 1,
|
|
66
|
+
"activity_name": "Create",
|
|
67
|
+
"category_name": "Findings",
|
|
68
|
+
"category_uid": 2,
|
|
69
|
+
"class_name": "Detection Finding",
|
|
70
|
+
"class_uid": 2004,
|
|
71
|
+
"finding_info": {
|
|
72
|
+
"title": "Suspicious login",
|
|
73
|
+
"uid": "f-1"
|
|
74
|
+
},
|
|
75
|
+
"metadata": {
|
|
76
|
+
"product": {
|
|
77
|
+
"name": "Sensor",
|
|
78
|
+
"vendor_name": "Acme"
|
|
79
|
+
},
|
|
80
|
+
"version": "1.9.0"
|
|
81
|
+
},
|
|
82
|
+
"severity": "High",
|
|
83
|
+
"severity_id": 4,
|
|
84
|
+
"time": 1700000000000,
|
|
85
|
+
"type_name": "Detection Finding: Create",
|
|
86
|
+
"type_uid": 200401
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
`build` raises `ConformanceError` if the event can't conform, and the error carries every problem
|
|
92
|
+
it found:
|
|
93
|
+
|
|
94
|
+
```pycon
|
|
95
|
+
>>> from ocsf_toolkit import ConformanceError
|
|
96
|
+
>>> try:
|
|
97
|
+
... build((schema, "detection_finding"), severity_id=4, time=1)
|
|
98
|
+
... except ConformanceError as error:
|
|
99
|
+
... for finding in error.report.errors():
|
|
100
|
+
... print(finding.message)
|
|
101
|
+
activity_id is required and absent
|
|
102
|
+
finding_info is required and absent
|
|
103
|
+
metadata is required and absent
|
|
104
|
+
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
## Check an event you received
|
|
108
|
+
|
|
109
|
+
`parse` returns the event with a report of findings, each with a kind, an indexed path and a
|
|
110
|
+
severity. You decide which findings are fatal.
|
|
111
|
+
|
|
112
|
+
```pycon
|
|
113
|
+
>>> from ocsf_toolkit import parse
|
|
114
|
+
>>> received = dict(event, severity_id=9, status_id="x")
|
|
115
|
+
>>> _, report = parse(received)
|
|
116
|
+
>>> report.ok()
|
|
117
|
+
False
|
|
118
|
+
>>> for finding in report.errors():
|
|
119
|
+
... print(finding.kind.value, finding.path, finding.message)
|
|
120
|
+
enum_out_of_range ('severity_id',) severity_id is 9, not one of ['0', '1', '2', '3', '4', '5', '6', '99']
|
|
121
|
+
type_mismatch ('status_id',) status_id is str 'x'; expected integer_t
|
|
122
|
+
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
The version and profiles come from the event's own `metadata`. `repair` returns a normalized copy
|
|
126
|
+
and records every change it made. The same check runs from the shell over JSON Lines files:
|
|
127
|
+
`ocsf-toolkit check --version 1.9.0 events.jsonl`.
|
|
128
|
+
|
|
129
|
+
## Typed models
|
|
130
|
+
|
|
131
|
+
With `ocsf-toolkit-v1-9-0` installed, every class and object is a msgspec Struct, every enum is
|
|
132
|
+
referenceable by its caption, and `build` and `parse` accept and return the typed classes:
|
|
133
|
+
|
|
134
|
+
```python
|
|
135
|
+
import msgspec
|
|
136
|
+
|
|
137
|
+
from ocsf_toolkit import build, parse
|
|
138
|
+
from ocsf_toolkit_v1_9_0 import DetectionFinding
|
|
139
|
+
from ocsf_toolkit_v1_9_0.classes.detection_finding import DetectionFindingActivityId
|
|
140
|
+
from ocsf_toolkit_v1_9_0.enums import SeverityId
|
|
141
|
+
from ocsf_toolkit_v1_9_0.objects import FindingInfo, Metadata, Product
|
|
142
|
+
|
|
143
|
+
finding = build(
|
|
144
|
+
DetectionFinding,
|
|
145
|
+
activity_id=DetectionFindingActivityId.CREATE,
|
|
146
|
+
severity_id=SeverityId.HIGH,
|
|
147
|
+
time=1_700_000_000_000,
|
|
148
|
+
metadata=Metadata(product=Product(name="Sensor", vendor_name="Acme")),
|
|
149
|
+
finding_info=FindingInfo(uid="f-1", title="Suspicious login"),
|
|
150
|
+
)
|
|
151
|
+
assert finding.type_uid == 200401
|
|
152
|
+
wire = msgspec.json.encode(finding) # the event as JSON bytes
|
|
153
|
+
|
|
154
|
+
same, report = parse(msgspec.json.decode(wire), as_=DetectionFinding)
|
|
155
|
+
assert same == finding and report.ok()
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
The models pass pyright in strict mode, and importing one class loads only that class's module.
|
|
159
|
+
|
|
160
|
+
## More
|
|
161
|
+
|
|
162
|
+
- [Schema as data](docs/schema.md): required attributes, enums and captions, siblings, profiles,
|
|
163
|
+
class identity, all without building an event.
|
|
164
|
+
- [Building and checking events](docs/events.md): `build`, `parse`, `repair`, policies and reports.
|
|
165
|
+
- [Observables](docs/observables.md): extracting and merging the observables an event carries.
|
|
166
|
+
- [Schema diff](docs/diff.md): what changed between two OCSF versions, renames included.
|
|
167
|
+
- [Typed models](docs/models.md): the model packages, and generating your own (custom extensions
|
|
168
|
+
and exact profile sets).
|
|
169
|
+
- [Command line](docs/cli.md): `check`, `diff`, `generate` and the schema pipeline.
|
|
170
|
+
|
|
171
|
+
## How versions work
|
|
172
|
+
|
|
173
|
+
Every OCSF release is pinned by upstream commit in [`ocsf.lock`](ocsf.lock) and compiled with the
|
|
174
|
+
official `ocsf-schema-compiler`. CI recompiles each one and checks it byte for byte against the
|
|
175
|
+
lockfile. Core ships the schema of every release, so you can check, repair and diff events of any of
|
|
176
|
+
them and extract their observables. Typed models ship for the newest three releases, one package
|
|
177
|
+
each, so you install only the versions you produce.
|
|
178
|
+
|
|
179
|
+
The library's own version follows SemVer on its API. Support for a new OCSF release ships as a minor
|
|
180
|
+
release of the library.
|
|
181
|
+
|
|
182
|
+
## Why not an existing library?
|
|
183
|
+
|
|
184
|
+
We installed and ran each Python OCSF package we could find. These results are from 2026-11-02,
|
|
185
|
+
against the versions named:
|
|
186
|
+
|
|
187
|
+
- **pydantic-ocsf** (3.0.0) is the closest relative and the one we learned the most from: multiple
|
|
188
|
+
OCSF versions side by side, sibling-label checks and computed `type_uid`. It builds its models
|
|
189
|
+
from the schema's *source* files, which leave profiles unresolved, so profile attributes such as
|
|
190
|
+
`actor` or `action_id` are missing or untyped. Against the official compiler's output for
|
|
191
|
+
OCSF 1.7.0, about a third of its integer enums differ; for example, `device.type_id=2`
|
|
192
|
+
("Desktop") is rejected.
|
|
193
|
+
- **py-ocsf-models** (0.10.0, Prowler) is a hand-written model set for three finding classes at
|
|
194
|
+
OCSF 1.5.0. It silently drops attributes it doesn't model.
|
|
195
|
+
- **ocsf-lib** (0.10.4) is useful schema tooling (compile, diff, compatibility checks), but it has
|
|
196
|
+
no event models or validation. It can't read the official `ocsf-schema-compiler` output, and its
|
|
197
|
+
own compiler disagrees with upstream on profile gating.
|
|
198
|
+
|
|
199
|
+
This library exists for four things those packages lack: OCSF knowledge as data you can query
|
|
200
|
+
without building a model, a conformance report that lists every problem in an event, observables
|
|
201
|
+
derived from the schema, and profiles treated as part of an event's identity.
|
|
202
|
+
|
|
203
|
+
## License
|
|
204
|
+
|
|
205
|
+
Apache-2.0.
|