ocsf-toolkit 0.1.0a1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (118) hide show
  1. ocsf_toolkit-0.1.0a1/.gitignore +20 -0
  2. ocsf_toolkit-0.1.0a1/.importlinter +50 -0
  3. ocsf_toolkit-0.1.0a1/CHANGELOG.md +23 -0
  4. ocsf_toolkit-0.1.0a1/CONTRIBUTING.md +50 -0
  5. ocsf_toolkit-0.1.0a1/LICENSE +202 -0
  6. ocsf_toolkit-0.1.0a1/Makefile +91 -0
  7. ocsf_toolkit-0.1.0a1/PKG-INFO +205 -0
  8. ocsf_toolkit-0.1.0a1/README.md +183 -0
  9. ocsf_toolkit-0.1.0a1/SECURITY.md +22 -0
  10. ocsf_toolkit-0.1.0a1/docs/cli.md +79 -0
  11. ocsf_toolkit-0.1.0a1/docs/diff.md +55 -0
  12. ocsf_toolkit-0.1.0a1/docs/events.md +166 -0
  13. ocsf_toolkit-0.1.0a1/docs/models.md +99 -0
  14. ocsf_toolkit-0.1.0a1/docs/observables.md +74 -0
  15. ocsf_toolkit-0.1.0a1/docs/schema.md +133 -0
  16. ocsf_toolkit-0.1.0a1/ocsf.lock +115 -0
  17. ocsf_toolkit-0.1.0a1/packaging/version_dist/README.md.tmpl +18 -0
  18. ocsf_toolkit-0.1.0a1/packaging/version_dist/dist.py +247 -0
  19. ocsf_toolkit-0.1.0a1/packaging/version_dist/pyproject.toml.tmpl +27 -0
  20. ocsf_toolkit-0.1.0a1/pyproject.toml +118 -0
  21. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/__init__.py +55 -0
  22. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/_version.py +24 -0
  23. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/cli/__init__.py +113 -0
  24. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/cli/_common.py +19 -0
  25. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/cli/_upstream.py +147 -0
  26. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/cli/check.py +77 -0
  27. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/cli/compile.py +39 -0
  28. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/cli/diff.py +65 -0
  29. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/cli/generate.py +90 -0
  30. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/cli/ir.py +113 -0
  31. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/cli/lock.py +331 -0
  32. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/cli/models.py +88 -0
  33. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/cli/trees.py +27 -0
  34. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/codegen/__init__.py +10 -0
  35. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/codegen/emit.py +239 -0
  36. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/codegen/generate.py +337 -0
  37. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/codegen/layout.py +74 -0
  38. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/codegen/naming.py +65 -0
  39. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/codegen/types.py +27 -0
  40. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/conform.py +396 -0
  41. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/data/ir-1.0.0.json +28073 -0
  42. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/data/ir-1.1.0.json +41579 -0
  43. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/data/ir-1.2.0.json +50321 -0
  44. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/data/ir-1.3.0.json +57198 -0
  45. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/data/ir-1.4.0.json +85424 -0
  46. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/data/ir-1.5.0.json +91969 -0
  47. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/data/ir-1.6.0.json +94639 -0
  48. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/data/ir-1.7.0.json +96525 -0
  49. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/data/ir-1.8.0.json +98267 -0
  50. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/data/ir-1.9.0.json +108067 -0
  51. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/derive.py +288 -0
  52. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/diff.py +123 -0
  53. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/engine.py +243 -0
  54. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/export.py +460 -0
  55. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/ir.py +335 -0
  56. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/loader.py +290 -0
  57. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/models.py +177 -0
  58. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/observables.py +196 -0
  59. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/py.typed +0 -0
  60. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/schema.py +284 -0
  61. ocsf_toolkit-0.1.0a1/src/ocsf_toolkit/walk.py +189 -0
  62. ocsf_toolkit-0.1.0a1/tests/__init__.py +0 -0
  63. ocsf_toolkit-0.1.0a1/tests/architecture/__init__.py +0 -0
  64. ocsf_toolkit-0.1.0a1/tests/architecture/test_docs.py +45 -0
  65. ocsf_toolkit-0.1.0a1/tests/architecture/test_imports.py +111 -0
  66. ocsf_toolkit-0.1.0a1/tests/architecture/test_scope.py +29 -0
  67. ocsf_toolkit-0.1.0a1/tests/bench/__init__.py +0 -0
  68. ocsf_toolkit-0.1.0a1/tests/bench/engine.py +151 -0
  69. ocsf_toolkit-0.1.0a1/tests/conftest.py +39 -0
  70. ocsf_toolkit-0.1.0a1/tests/fixtures/__init__.py +0 -0
  71. ocsf_toolkit-0.1.0a1/tests/fixtures/compiled.py +182 -0
  72. ocsf_toolkit-0.1.0a1/tests/fixtures/events.py +119 -0
  73. ocsf_toolkit-0.1.0a1/tests/fixtures/notations.py +10 -0
  74. ocsf_toolkit-0.1.0a1/tests/fixtures/packages.py +42 -0
  75. ocsf_toolkit-0.1.0a1/tests/generated/__init__.py +0 -0
  76. ocsf_toolkit-0.1.0a1/tests/generated/conftest.py +45 -0
  77. ocsf_toolkit-0.1.0a1/tests/generated/samples/usage.py +43 -0
  78. ocsf_toolkit-0.1.0a1/tests/generated/test_import.py +67 -0
  79. ocsf_toolkit-0.1.0a1/tests/generated/test_readme.py +22 -0
  80. ocsf_toolkit-0.1.0a1/tests/generated/test_snapshot.py +19 -0
  81. ocsf_toolkit-0.1.0a1/tests/generated/test_static.py +54 -0
  82. ocsf_toolkit-0.1.0a1/tests/golden/detection_finding_1_9_0.py +178 -0
  83. ocsf_toolkit-0.1.0a1/tests/property/__init__.py +0 -0
  84. ocsf_toolkit-0.1.0a1/tests/property/strategies.py +216 -0
  85. ocsf_toolkit-0.1.0a1/tests/property/test_repair.py +154 -0
  86. ocsf_toolkit-0.1.0a1/tests/property/test_roundtrip.py +75 -0
  87. ocsf_toolkit-0.1.0a1/tests/property/test_strategies.py +35 -0
  88. ocsf_toolkit-0.1.0a1/tests/unit/__init__.py +0 -0
  89. ocsf_toolkit-0.1.0a1/tests/unit/cli/__init__.py +0 -0
  90. ocsf_toolkit-0.1.0a1/tests/unit/cli/conftest.py +48 -0
  91. ocsf_toolkit-0.1.0a1/tests/unit/cli/support.py +58 -0
  92. ocsf_toolkit-0.1.0a1/tests/unit/cli/test_check.py +96 -0
  93. ocsf_toolkit-0.1.0a1/tests/unit/cli/test_compile.py +181 -0
  94. ocsf_toolkit-0.1.0a1/tests/unit/cli/test_diff.py +87 -0
  95. ocsf_toolkit-0.1.0a1/tests/unit/cli/test_generate.py +103 -0
  96. ocsf_toolkit-0.1.0a1/tests/unit/cli/test_ir.py +124 -0
  97. ocsf_toolkit-0.1.0a1/tests/unit/cli/test_lock.py +243 -0
  98. ocsf_toolkit-0.1.0a1/tests/unit/cli/test_models.py +72 -0
  99. ocsf_toolkit-0.1.0a1/tests/unit/cli/test_parser.py +72 -0
  100. ocsf_toolkit-0.1.0a1/tests/unit/codegen/__init__.py +0 -0
  101. ocsf_toolkit-0.1.0a1/tests/unit/codegen/test_emit.py +84 -0
  102. ocsf_toolkit-0.1.0a1/tests/unit/codegen/test_generate.py +169 -0
  103. ocsf_toolkit-0.1.0a1/tests/unit/codegen/test_naming.py +91 -0
  104. ocsf_toolkit-0.1.0a1/tests/unit/codegen/test_types_layout.py +57 -0
  105. ocsf_toolkit-0.1.0a1/tests/unit/conftest.py +41 -0
  106. ocsf_toolkit-0.1.0a1/tests/unit/test_conform.py +195 -0
  107. ocsf_toolkit-0.1.0a1/tests/unit/test_derive.py +273 -0
  108. ocsf_toolkit-0.1.0a1/tests/unit/test_diff.py +176 -0
  109. ocsf_toolkit-0.1.0a1/tests/unit/test_engine.py +339 -0
  110. ocsf_toolkit-0.1.0a1/tests/unit/test_export.py +170 -0
  111. ocsf_toolkit-0.1.0a1/tests/unit/test_ir_loader.py +284 -0
  112. ocsf_toolkit-0.1.0a1/tests/unit/test_models.py +223 -0
  113. ocsf_toolkit-0.1.0a1/tests/unit/test_observables.py +236 -0
  114. ocsf_toolkit-0.1.0a1/tests/unit/test_packaging.py +130 -0
  115. ocsf_toolkit-0.1.0a1/tests/unit/test_public_api.py +95 -0
  116. ocsf_toolkit-0.1.0a1/tests/unit/test_schema.py +383 -0
  117. ocsf_toolkit-0.1.0a1/tests/unit/test_walk.py +137 -0
  118. ocsf_toolkit-0.1.0a1/uv.lock +849 -0
@@ -0,0 +1,20 @@
1
+ # build outputs: compiled schemas, schema checkouts, generated dists, wheels
2
+ build/
3
+ dist/
4
+ *.egg-info/
5
+
6
+ # written by hatch-vcs at build time
7
+ src/ocsf_toolkit/_version.py
8
+
9
+ # environments and caches
10
+ .venv/
11
+ __pycache__/
12
+ *.py[cod]
13
+ .pytest_cache/
14
+ .ruff_cache/
15
+ .mypy_cache/
16
+ .import_linter_cache/
17
+ .coverage
18
+ .coverage.*
19
+ htmlcov/
20
+ .DS_Store
@@ -0,0 +1,50 @@
1
+ # Import contracts: the core layering, the msgspec boundary, and no imports of generated packages.
2
+ # Modules arrive milestone by milestone (M1–M6); until then these contracts are vacuous.
3
+ # Layers in parentheses are optional: absent today, enforced as soon as the module exists.
4
+ # (A `forbidden` contract errors on a missing source module, so per-module rules are expressed
5
+ # as layers or anchored on the root package.)
6
+
7
+ [importlinter]
8
+ root_package = ocsf_toolkit
9
+ include_external_packages = True
10
+
11
+ # ir <- loader/export <- schema <- walk <- derive/conform/observables <- engine <- models,
12
+ # with codegen and diff beside walk: they may use schema and below, nothing above, not each other.
13
+ # cli is the outermost layer; nothing in core imports it.
14
+ [importlinter:contract:layers]
15
+ name = Core layering (codegen and diff depend on schema only)
16
+ type = layers
17
+ containers = ocsf_toolkit
18
+ layers =
19
+ (cli)
20
+ (models)
21
+ (engine)
22
+ (derive) | (conform) | (observables)
23
+ (walk) | (codegen) | (diff)
24
+ (schema)
25
+ (loader) | (export)
26
+ (ir)
27
+
28
+ [importlinter:contract:msgspec-boundary]
29
+ name = Only models and codegen import msgspec
30
+ type = forbidden
31
+ source_modules =
32
+ ocsf_toolkit
33
+ forbidden_modules =
34
+ msgspec
35
+ allow_indirect_imports = True
36
+ ignore_imports =
37
+ ocsf_toolkit.models -> msgspec
38
+ ocsf_toolkit.codegen -> msgspec
39
+ ocsf_toolkit.codegen.** -> msgspec
40
+ unmatched_ignore_imports_alerting = none
41
+
42
+ [importlinter:contract:no-generated-packages]
43
+ name = Nothing in ocsf_toolkit imports a generated version package (ocsf_toolkit_v*)
44
+ type = forbidden
45
+ source_modules =
46
+ ocsf_toolkit
47
+ forbidden_modules =
48
+ ocsf_toolkit_v1_7_0
49
+ ocsf_toolkit_v1_8_0
50
+ ocsf_toolkit_v1_9_0
@@ -0,0 +1,23 @@
1
+ # Changelog
2
+
3
+ The library follows SemVer on its own API. OCSF schema versions are data, so support for a new OCSF
4
+ version ships as a minor release. One line per merged PR, taken from the PR title.
5
+
6
+ ## Unreleased
7
+
8
+ ## 0.1.0a1
9
+
10
+ The first alpha. Model packages `ocsf-toolkit-v1-7-0` and `ocsf-toolkit-v1-8-0` publish with it;
11
+ `ocsf-toolkit-v1-9-0` follows with the next release.
12
+
13
+ - Repository skeleton, `ocsf.lock` pinning OCSF 1.9.0, `ocsf-toolkit compile` and `ocsf-toolkit lock --add`.
14
+ - Schema IR (`SchemaIR`), compiler-JSON loader with key accounting, canonical IR JSON (`ir_version` 1.0) and `load_ir`; `ocsf-toolkit ir export` / `ir check`; IR for OCSF 1.9.0 shipped in the package; `ocsf.lock` pins OCSF 1.0.0–1.9.0.
15
+ - `load()` / `resolve()` → `ResolvedSchema`: profile- and extension-aware attribute sets, class identity, sibling pairs, `type_uid` registry and observable markers, reachable without a model.
16
+ - `build()`, `parse()` and `repair()` on event dicts under the `STRICT`, `LENIENT` and `REPAIR` policies: one traversal (`walk()`) that derives what the schema computes and reports every deviation as a `Finding` of a `Kind` in a `Report`; only `build()` raises, and only `ConformanceError`. `ocsf-toolkit check` reports on JSON Lines files.
17
+ - Observables: `extract()` and `merge()` → `Observable`, with value validation.
18
+ - `diff()` → `SchemaDiff`: a structural diff of two IRs with classes matched by uid, so a renamed class shows as a rename; `ocsf-toolkit diff`, with `--json` and `--semver` (the release category for a re-pinned IR). IRs for OCSF 1.1.0, 1.3.0 and 1.7.0 shipped.
19
+ - `generate()` and `ocsf-toolkit generate`: an OCSF version or IR projected into a typed package of msgspec Structs, superset by default or exact for a `--profiles` set, with `--ext` for custom extensions; `build()` and `parse()` accept and return the generated classes.
20
+ - Model dists `ocsf-toolkit-v1-7-0`, `-v1-8-0` and `-v1-9-0`, each bundling its IR and requiring a range of core versions, so a core release republishes a dist only when its generated tree changes. `ocsf-toolkit models check` / `record` pin each tree in `ocsf.lock`; `lock --add` keeps models for the newest three versions. IRs for every OCSF release from 1.0.0 to 1.9.0 ship in core.
21
+ - Release workflow: reproducible sdists and wheels, PyPI trusted publishing, and a GitHub release with `ocsf.lock` and `SHA256SUMS`.
22
+ - Property tests (Hypothesis, strategies drawn from the IR): dict and typed round trips for every core class of the model-window versions, and repair undoing unambiguous damage idempotently. Fixes they found: a typed `build()` reads profiles from a `Metadata` Struct, and `repair()` finds the class from a `class_uid`/`type_uid` it coerces. `make bench` measures engine throughput; the engine walks at about 2 µs per attribute (1.55× faster than its first cut).
23
+ - README with installation and a worked example, and API docs in `docs/` (schema, events, observables, diff, typed models, CLI); every example runs as a doctest.
@@ -0,0 +1,50 @@
1
+ # Contributing
2
+
3
+ ## Setup
4
+
5
+ Requires [uv](https://docs.astral.sh/uv/) and git. The library runs on Python ≥ 3.12. Compiling an OCSF
6
+ schema needs Python 3.14 (because of `ocsf-schema-compiler`); `uvx` fetches it for that step, so you
7
+ do not need to install it yourself.
8
+
9
+ ```shell
10
+ make setup # uv sync: dev environment in .venv
11
+ make lint # ruff format --check, ruff check, lint-imports
12
+ make type # pyright --strict (blocking), mypy --strict (informational)
13
+ make test # pytest
14
+ make schema # ocsf-toolkit compile: rebuild build/compiled/ and check the lockfile hashes
15
+ ```
16
+
17
+ `uvx pre-commit install` runs the lint and type checks on every commit (`.pre-commit-config.yaml`).
18
+
19
+ ## Rules that bind code
20
+
21
+ These architecture rules are binding: one event walker; import direction enforced by
22
+ `.importlinter`; one exception type (`ConformanceError`); no `__getattr__` in core; thin entry points;
23
+ a 13-module core budget; a query surface of frozen mappings. A PR that needs to break one of
24
+ them argues for it in the description.
25
+
26
+ Conventions:
27
+
28
+ - No runtime dependencies besides `msgspec`. The CLI uses `argparse`.
29
+ - Each module's docstring states its single responsibility in one sentence.
30
+ - Lines are at most 120 characters; ruff format sets the layout.
31
+ - Every public module, class and function has a docstring whose first line is an imperative
32
+ summary (pydocstyle, pep257 convention). Tests are exempt.
33
+
34
+ ## Pull requests
35
+
36
+ - `main` is protected; PRs are squash-merged and the PR title becomes the changelog line.
37
+ - Plain commit messages; no conventional-commits prefixes required.
38
+ - Changing `ocsf.lock` triggers the schema workflow.
39
+ - A compiler bump (`[toolchain]` in `ocsf.lock`) is expected to change every `compiled_sha256`; run
40
+ `make schema` and update them in the same PR.
41
+
42
+ ## Adding an OCSF version
43
+
44
+ ```shell
45
+ make lock VERSION=x.y.z # ocsf-toolkit lock --add x.y.z
46
+ ```
47
+
48
+ This resolves the upstream tag (`x.y.z`, then `vx.y.z`), pins its commit, compiles it, and records
49
+ `compiled_sha256`. The compile fails if the compiled schema's `version` does not equal `x.y.z`. Tag
50
+ `1.5.0` upstream is one such case; pass `--ref` to pin a branch instead.
@@ -0,0 +1,202 @@
1
+
2
+ Apache License
3
+ Version 2.0, January 2004
4
+ http://www.apache.org/licenses/
5
+
6
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
7
+
8
+ 1. Definitions.
9
+
10
+ "License" shall mean the terms and conditions for use, reproduction,
11
+ and distribution as defined by Sections 1 through 9 of this document.
12
+
13
+ "Licensor" shall mean the copyright owner or entity authorized by
14
+ the copyright owner that is granting the License.
15
+
16
+ "Legal Entity" shall mean the union of the acting entity and all
17
+ other entities that control, are controlled by, or are under common
18
+ control with that entity. For the purposes of this definition,
19
+ "control" means (i) the power, direct or indirect, to cause the
20
+ direction or management of such entity, whether by contract or
21
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
22
+ outstanding shares, or (iii) beneficial ownership of such entity.
23
+
24
+ "You" (or "Your") shall mean an individual or Legal Entity
25
+ exercising permissions granted by this License.
26
+
27
+ "Source" form shall mean the preferred form for making modifications,
28
+ including but not limited to software source code, documentation
29
+ source, and configuration files.
30
+
31
+ "Object" form shall mean any form resulting from mechanical
32
+ transformation or translation of a Source form, including but
33
+ not limited to compiled object code, generated documentation,
34
+ and conversions to other media types.
35
+
36
+ "Work" shall mean the work of authorship, whether in Source or
37
+ Object form, made available under the License, as indicated by a
38
+ copyright notice that is included in or attached to the work
39
+ (an example is provided in the Appendix below).
40
+
41
+ "Derivative Works" shall mean any work, whether in Source or Object
42
+ form, that is based on (or derived from) the Work and for which the
43
+ editorial revisions, annotations, elaborations, or other modifications
44
+ represent, as a whole, an original work of authorship. For the purposes
45
+ of this License, Derivative Works shall not include works that remain
46
+ separable from, or merely link (or bind by name) to the interfaces of,
47
+ the Work and Derivative Works thereof.
48
+
49
+ "Contribution" shall mean any work of authorship, including
50
+ the original version of the Work and any modifications or additions
51
+ to that Work or Derivative Works thereof, that is intentionally
52
+ submitted to Licensor for inclusion in the Work by the copyright owner
53
+ or by an individual or Legal Entity authorized to submit on behalf of
54
+ the copyright owner. For the purposes of this definition, "submitted"
55
+ means any form of electronic, verbal, or written communication sent
56
+ to the Licensor or its representatives, including but not limited to
57
+ communication on electronic mailing lists, source code control systems,
58
+ and issue tracking systems that are managed by, or on behalf of, the
59
+ Licensor for the purpose of discussing and improving the Work, but
60
+ excluding communication that is conspicuously marked or otherwise
61
+ designated in writing by the copyright owner as "Not a Contribution."
62
+
63
+ "Contributor" shall mean Licensor and any individual or Legal Entity
64
+ on behalf of whom a Contribution has been received by Licensor and
65
+ subsequently incorporated within the Work.
66
+
67
+ 2. Grant of Copyright License. Subject to the terms and conditions of
68
+ this License, each Contributor hereby grants to You a perpetual,
69
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
70
+ copyright license to reproduce, prepare Derivative Works of,
71
+ publicly display, publicly perform, sublicense, and distribute the
72
+ Work and such Derivative Works in Source or Object form.
73
+
74
+ 3. Grant of Patent License. Subject to the terms and conditions of
75
+ this License, each Contributor hereby grants to You a perpetual,
76
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
77
+ (except as stated in this section) patent license to make, have made,
78
+ use, offer to sell, sell, import, and otherwise transfer the Work,
79
+ where such license applies only to those patent claims licensable
80
+ by such Contributor that are necessarily infringed by their
81
+ Contribution(s) alone or by combination of their Contribution(s)
82
+ with the Work to which such Contribution(s) was submitted. If You
83
+ institute patent litigation against any entity (including a
84
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
85
+ or a Contribution incorporated within the Work constitutes direct
86
+ or contributory patent infringement, then any patent licenses
87
+ granted to You under this License for that Work shall terminate
88
+ as of the date such litigation is filed.
89
+
90
+ 4. Redistribution. You may reproduce and distribute copies of the
91
+ Work or Derivative Works thereof in any medium, with or without
92
+ modifications, and in Source or Object form, provided that You
93
+ meet the following conditions:
94
+
95
+ (a) You must give any other recipients of the Work or
96
+ Derivative Works a copy of this License; and
97
+
98
+ (b) You must cause any modified files to carry prominent notices
99
+ stating that You changed the files; and
100
+
101
+ (c) You must retain, in the Source form of any Derivative Works
102
+ that You distribute, all copyright, patent, trademark, and
103
+ attribution notices from the Source form of the Work,
104
+ excluding those notices that do not pertain to any part of
105
+ the Derivative Works; and
106
+
107
+ (d) If the Work includes a "NOTICE" text file as part of its
108
+ distribution, then any Derivative Works that You distribute must
109
+ include a readable copy of the attribution notices contained
110
+ within such NOTICE file, excluding those notices that do not
111
+ pertain to any part of the Derivative Works, in at least one
112
+ of the following places: within a NOTICE text file distributed
113
+ as part of the Derivative Works; within the Source form or
114
+ documentation, if provided along with the Derivative Works; or,
115
+ within a display generated by the Derivative Works, if and
116
+ wherever such third-party notices normally appear. The contents
117
+ of the NOTICE file are for informational purposes only and
118
+ do not modify the License. You may add Your own attribution
119
+ notices within Derivative Works that You distribute, alongside
120
+ or as an addendum to the NOTICE text from the Work, provided
121
+ that such additional attribution notices cannot be construed
122
+ as modifying the License.
123
+
124
+ You may add Your own copyright statement to Your modifications and
125
+ may provide additional or different license terms and conditions
126
+ for use, reproduction, or distribution of Your modifications, or
127
+ for any such Derivative Works as a whole, provided Your use,
128
+ reproduction, and distribution of the Work otherwise complies with
129
+ the conditions stated in this License.
130
+
131
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
132
+ any Contribution intentionally submitted for inclusion in the Work
133
+ by You to the Licensor shall be under the terms and conditions of
134
+ this License, without any additional terms or conditions.
135
+ Notwithstanding the above, nothing herein shall supersede or modify
136
+ the terms of any separate license agreement you may have executed
137
+ with Licensor regarding such Contributions.
138
+
139
+ 6. Trademarks. This License does not grant permission to use the trade
140
+ names, trademarks, service marks, or product names of the Licensor,
141
+ except as required for reasonable and customary use in describing the
142
+ origin of the Work and reproducing the content of the NOTICE file.
143
+
144
+ 7. Disclaimer of Warranty. Unless required by applicable law or
145
+ agreed to in writing, Licensor provides the Work (and each
146
+ Contributor provides its Contributions) on an "AS IS" BASIS,
147
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
148
+ implied, including, without limitation, any warranties or conditions
149
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
150
+ PARTICULAR PURPOSE. You are solely responsible for determining the
151
+ appropriateness of using or redistributing the Work and assume any
152
+ risks associated with Your exercise of permissions under this License.
153
+
154
+ 8. Limitation of Liability. In no event and under no legal theory,
155
+ whether in tort (including negligence), contract, or otherwise,
156
+ unless required by applicable law (such as deliberate and grossly
157
+ negligent acts) or agreed to in writing, shall any Contributor be
158
+ liable to You for damages, including any direct, indirect, special,
159
+ incidental, or consequential damages of any character arising as a
160
+ result of this License or out of the use or inability to use the
161
+ Work (including but not limited to damages for loss of goodwill,
162
+ work stoppage, computer failure or malfunction, or any and all
163
+ other commercial damages or losses), even if such Contributor
164
+ has been advised of the possibility of such damages.
165
+
166
+ 9. Accepting Warranty or Additional Liability. While redistributing
167
+ the Work or Derivative Works thereof, You may choose to offer,
168
+ and charge a fee for, acceptance of support, warranty, indemnity,
169
+ or other liability obligations and/or rights consistent with this
170
+ License. However, in accepting such obligations, You may act only
171
+ on Your own behalf and on Your sole responsibility, not on behalf
172
+ of any other Contributor, and only if You agree to indemnify,
173
+ defend, and hold each Contributor harmless for any liability
174
+ incurred by, or claims asserted against, such Contributor by reason
175
+ of your accepting any such warranty or additional liability.
176
+
177
+ END OF TERMS AND CONDITIONS
178
+
179
+ APPENDIX: How to apply the Apache License to your work.
180
+
181
+ To apply the Apache License to your work, attach the following
182
+ boilerplate notice, with the fields enclosed by brackets "[]"
183
+ replaced with your own identifying information. (Don't include
184
+ the brackets!) The text should be enclosed in the appropriate
185
+ comment syntax for the file format. We also recommend that a
186
+ file or class name and description of purpose be included on the
187
+ same "printed page" as the copyright notice for easier
188
+ identification within third-party archives.
189
+
190
+ Copyright [yyyy] [name of copyright owner]
191
+
192
+ Licensed under the Apache License, Version 2.0 (the "License");
193
+ you may not use this file except in compliance with the License.
194
+ You may obtain a copy of the License at
195
+
196
+ http://www.apache.org/licenses/LICENSE-2.0
197
+
198
+ Unless required by applicable law or agreed to in writing, software
199
+ distributed under the License is distributed on an "AS IS" BASIS,
200
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
201
+ See the License for the specific language governing permissions and
202
+ limitations under the License.
@@ -0,0 +1,91 @@
1
+ .PHONY: setup format lint type test bench schema ir lock generate dists wheels compat test-generated clean
2
+
3
+ UV ?= uv
4
+
5
+ setup:
6
+ $(UV) sync
7
+
8
+ format:
9
+ $(UV) run ruff format .
10
+ $(UV) run ruff check --fix .
11
+
12
+ lint:
13
+ $(UV) run ruff format --check .
14
+ $(UV) run ruff check .
15
+ $(UV) run lint-imports
16
+
17
+ # pyright strict is the blocking gate; mypy strict is informational.
18
+ type:
19
+ $(UV) run pyright
20
+ -$(UV) run mypy
21
+
22
+ test:
23
+ $(UV) run pytest
24
+
25
+ # S1 throughput at 2-20 KB per event, on this machine; slow, so not part of `test` or CI
26
+ bench:
27
+ $(UV) run python -m tests.bench.engine
28
+
29
+ # gates 1 and 2: compile every lockfile schema, then check every committed IR against it
30
+ schema:
31
+ $(UV) run ocsf-toolkit compile
32
+ $(UV) run ocsf-toolkit ir check
33
+
34
+ # rewrite data/ir-<ver>.json for VERSION (default: every entry with an ir_sha256); RECORD=1 accepts a new IR
35
+ ir:
36
+ $(UV) run ocsf-toolkit ir export $(VERSION) $(if $(RECORD),--record)
37
+
38
+ # pin a new OCSF version, ship its IR, move the model window and record the window's trees
39
+ lock:
40
+ @test -n "$(VERSION)" || { echo "usage: make lock VERSION=x.y.z"; exit 2; }
41
+ $(UV) run ocsf-toolkit lock --add $(VERSION)
42
+ $(UV) run ocsf-toolkit ir export $(VERSION) --record
43
+ $(UV) run ocsf-toolkit models record
44
+
45
+ # The model dist for one OCSF version as an installable project in build/dists/<package>/.
46
+ # Nothing here deletes: `generate` replaces <package>/ only if every file in it is generated,
47
+ # and the rendered project files are overwritten by name. The dist requires the core version
48
+ # building it, up to the next generated-code contract break.
49
+ OCSF ?= 1.9.0
50
+ PACKAGE = ocsf_toolkit_v$(subst .,_,$(OCSF))
51
+ DIST = build/dists/$(PACKAGE)
52
+ WINDOW = $(shell $(UV) run python -c 'import tomllib; print(*(e["version"] for e in \
53
+ tomllib.load(open("ocsf.lock", "rb"))["schemas"] if e["ship_models"]))')
54
+ WHEELS = build/wheels
55
+ # sdists and wheels carry this timestamp, so rebuilding one commit gives identical bytes
56
+ export SOURCE_DATE_EPOCH ?= $(shell git log -1 --format=%ct)
57
+
58
+ generate:
59
+ @echo "$(OCSF)" | grep -Eq '^[0-9]+\.[0-9]+\.[0-9]+$$' || { echo "OCSF must be x.y.z"; exit 2; }
60
+ mkdir -p $(DIST)
61
+ $(UV) run ocsf-toolkit generate $(OCSF) --package $(PACKAGE) --out $(DIST)
62
+ version=$$($(UV)x --quiet --with hatch-vcs hatchling version) && \
63
+ $(UV) run python packaging/version_dist/dist.py render --ocsf $(OCSF) --version "$$version" \
64
+ --out $(DIST)
65
+
66
+ # every model-window dist, then regeneration gate 3 over the trees on disk
67
+ dists:
68
+ for v in $(WINDOW); do $(MAKE) --no-print-directory generate OCSF=$$v || exit 1; done
69
+ $(UV) run ocsf-toolkit models check
70
+
71
+ # core and every model-window dist as sdist and wheel in build/wheels/, each wheel checked
72
+ wheels: dists
73
+ $(UV) build --quiet --out-dir $(WHEELS)
74
+ for v in $(WINDOW); do \
75
+ $(UV) build --quiet --out-dir $(WHEELS) build/dists/ocsf_toolkit_v$$(echo $$v | tr . _) \
76
+ || exit 1; \
77
+ done
78
+ $(UV) run python packaging/version_dist/dist.py check-wheels $(WHEELS)/*.whl
79
+
80
+ # the released model dists (LOCAL=1: the wheels in build/wheels) run on the core in this checkout
81
+ compat:
82
+ version=$$($(UV)x --quiet --with hatch-vcs hatchling version) && \
83
+ $(UV) run python packaging/version_dist/dist.py compat --version "$$version" \
84
+ $(if $(LOCAL),--wheels $(WHEELS))
85
+
86
+ # static analysis, snapshot, laziness and a clean-environment install of the generated dist
87
+ test-generated:
88
+ $(UV) run pytest tests/generated
89
+
90
+ clean:
91
+ rm -rf build dist .pytest_cache .ruff_cache .mypy_cache .import_linter_cache
@@ -0,0 +1,205 @@
1
+ Metadata-Version: 2.5
2
+ Name: ocsf-toolkit
3
+ Version: 0.1.0a1
4
+ Summary: Version-pinned OCSF schema semantics: substrate, derivation, conformance, observables, diff, and generated msgspec models.
5
+ Project-URL: Repository, https://github.com/k4otix/ocsf-toolkit
6
+ Project-URL: Changelog, https://github.com/k4otix/ocsf-toolkit/blob/main/CHANGELOG.md
7
+ Author: Eddie Allan
8
+ License-Expression: Apache-2.0
9
+ License-File: LICENSE
10
+ Keywords: msgspec,ocsf,schema,security
11
+ Classifier: Development Status :: 2 - Pre-Alpha
12
+ Classifier: Intended Audience :: Developers
13
+ Classifier: Programming Language :: Python :: 3
14
+ Classifier: Programming Language :: Python :: 3.12
15
+ Classifier: Programming Language :: Python :: 3.13
16
+ Classifier: Programming Language :: Python :: 3.14
17
+ Classifier: Topic :: Security
18
+ Classifier: Typing :: Typed
19
+ Requires-Python: >=3.12
20
+ Requires-Dist: msgspec<1,>=0.22
21
+ Description-Content-Type: text/markdown
22
+
23
+ # ocsf-toolkit
24
+
25
+ Version-pinned [OCSF](https://schema.ocsf.io/) semantics for Python. Each OCSF release is compiled
26
+ from upstream `ocsf/ocsf-schema` at a pinned commit into a schema you can query as data. On top of
27
+ that schema, the library builds events and derives everything the schema computes, checks events and
28
+ reports every deviation it finds, extracts observables, and diffs two versions. Typed
29
+ [msgspec](https://github.com/msgspec/msgspec) models are generated from the same schema.
30
+
31
+ **Status: alpha.** The API may change before 0.1.0.
32
+
33
+ ## Install
34
+
35
+ ```shell
36
+ pip install --pre ocsf-toolkit # core: every OCSF release from 1.0.0 to 1.9.0, as data
37
+ pip install --pre ocsf-toolkit-v1-8-0 # optional: typed models for OCSF 1.8.0 (also -v1-7-0)
38
+ ```
39
+
40
+ Typed models for OCSF 1.9.0 (`ocsf-toolkit-v1-9-0`) arrive on PyPI with the next release. Until then,
41
+ `ocsf-toolkit generate` builds the same package locally (see [Command line](docs/cli.md)).
42
+
43
+ Python 3.12 or newer. The only runtime dependency is msgspec.
44
+
45
+ ## Build and emit one event
46
+
47
+ Pick an OCSF version and a class, give the fields you know, and `build` fills in what the schema
48
+ derives: class and category identity, `type_uid`, the label beside every `_id` enum, and
49
+ `metadata.version`.
50
+
51
+ ```pycon
52
+ >>> import json
53
+ >>> from ocsf_toolkit import build, load
54
+ >>> schema = load("1.9.0")
55
+ >>> event = build(
56
+ ... (schema, "detection_finding"),
57
+ ... activity_id=1,
58
+ ... severity_id=4,
59
+ ... time=1_700_000_000_000,
60
+ ... metadata={"product": {"name": "Sensor", "vendor_name": "Acme"}},
61
+ ... finding_info={"uid": "f-1", "title": "Suspicious login"},
62
+ ... )
63
+ >>> print(json.dumps(event, indent=2, sort_keys=True))
64
+ {
65
+ "activity_id": 1,
66
+ "activity_name": "Create",
67
+ "category_name": "Findings",
68
+ "category_uid": 2,
69
+ "class_name": "Detection Finding",
70
+ "class_uid": 2004,
71
+ "finding_info": {
72
+ "title": "Suspicious login",
73
+ "uid": "f-1"
74
+ },
75
+ "metadata": {
76
+ "product": {
77
+ "name": "Sensor",
78
+ "vendor_name": "Acme"
79
+ },
80
+ "version": "1.9.0"
81
+ },
82
+ "severity": "High",
83
+ "severity_id": 4,
84
+ "time": 1700000000000,
85
+ "type_name": "Detection Finding: Create",
86
+ "type_uid": 200401
87
+ }
88
+
89
+ ```
90
+
91
+ `build` raises `ConformanceError` if the event can't conform, and the error carries every problem
92
+ it found:
93
+
94
+ ```pycon
95
+ >>> from ocsf_toolkit import ConformanceError
96
+ >>> try:
97
+ ... build((schema, "detection_finding"), severity_id=4, time=1)
98
+ ... except ConformanceError as error:
99
+ ... for finding in error.report.errors():
100
+ ... print(finding.message)
101
+ activity_id is required and absent
102
+ finding_info is required and absent
103
+ metadata is required and absent
104
+
105
+ ```
106
+
107
+ ## Check an event you received
108
+
109
+ `parse` returns the event with a report of findings, each with a kind, an indexed path and a
110
+ severity. You decide which findings are fatal.
111
+
112
+ ```pycon
113
+ >>> from ocsf_toolkit import parse
114
+ >>> received = dict(event, severity_id=9, status_id="x")
115
+ >>> _, report = parse(received)
116
+ >>> report.ok()
117
+ False
118
+ >>> for finding in report.errors():
119
+ ... print(finding.kind.value, finding.path, finding.message)
120
+ enum_out_of_range ('severity_id',) severity_id is 9, not one of ['0', '1', '2', '3', '4', '5', '6', '99']
121
+ type_mismatch ('status_id',) status_id is str 'x'; expected integer_t
122
+
123
+ ```
124
+
125
+ The version and profiles come from the event's own `metadata`. `repair` returns a normalized copy
126
+ and records every change it made. The same check runs from the shell over JSON Lines files:
127
+ `ocsf-toolkit check --version 1.9.0 events.jsonl`.
128
+
129
+ ## Typed models
130
+
131
+ With `ocsf-toolkit-v1-9-0` installed, every class and object is a msgspec Struct, every enum is
132
+ referenceable by its caption, and `build` and `parse` accept and return the typed classes:
133
+
134
+ ```python
135
+ import msgspec
136
+
137
+ from ocsf_toolkit import build, parse
138
+ from ocsf_toolkit_v1_9_0 import DetectionFinding
139
+ from ocsf_toolkit_v1_9_0.classes.detection_finding import DetectionFindingActivityId
140
+ from ocsf_toolkit_v1_9_0.enums import SeverityId
141
+ from ocsf_toolkit_v1_9_0.objects import FindingInfo, Metadata, Product
142
+
143
+ finding = build(
144
+ DetectionFinding,
145
+ activity_id=DetectionFindingActivityId.CREATE,
146
+ severity_id=SeverityId.HIGH,
147
+ time=1_700_000_000_000,
148
+ metadata=Metadata(product=Product(name="Sensor", vendor_name="Acme")),
149
+ finding_info=FindingInfo(uid="f-1", title="Suspicious login"),
150
+ )
151
+ assert finding.type_uid == 200401
152
+ wire = msgspec.json.encode(finding) # the event as JSON bytes
153
+
154
+ same, report = parse(msgspec.json.decode(wire), as_=DetectionFinding)
155
+ assert same == finding and report.ok()
156
+ ```
157
+
158
+ The models pass pyright in strict mode, and importing one class loads only that class's module.
159
+
160
+ ## More
161
+
162
+ - [Schema as data](docs/schema.md): required attributes, enums and captions, siblings, profiles,
163
+ class identity, all without building an event.
164
+ - [Building and checking events](docs/events.md): `build`, `parse`, `repair`, policies and reports.
165
+ - [Observables](docs/observables.md): extracting and merging the observables an event carries.
166
+ - [Schema diff](docs/diff.md): what changed between two OCSF versions, renames included.
167
+ - [Typed models](docs/models.md): the model packages, and generating your own (custom extensions
168
+ and exact profile sets).
169
+ - [Command line](docs/cli.md): `check`, `diff`, `generate` and the schema pipeline.
170
+
171
+ ## How versions work
172
+
173
+ Every OCSF release is pinned by upstream commit in [`ocsf.lock`](ocsf.lock) and compiled with the
174
+ official `ocsf-schema-compiler`. CI recompiles each one and checks it byte for byte against the
175
+ lockfile. Core ships the schema of every release, so you can check, repair and diff events of any of
176
+ them and extract their observables. Typed models ship for the newest three releases, one package
177
+ each, so you install only the versions you produce.
178
+
179
+ The library's own version follows SemVer on its API. Support for a new OCSF release ships as a minor
180
+ release of the library.
181
+
182
+ ## Why not an existing library?
183
+
184
+ We installed and ran each Python OCSF package we could find. These results are from 2026-11-02,
185
+ against the versions named:
186
+
187
+ - **pydantic-ocsf** (3.0.0) is the closest relative and the one we learned the most from: multiple
188
+ OCSF versions side by side, sibling-label checks and computed `type_uid`. It builds its models
189
+ from the schema's *source* files, which leave profiles unresolved, so profile attributes such as
190
+ `actor` or `action_id` are missing or untyped. Against the official compiler's output for
191
+ OCSF 1.7.0, about a third of its integer enums differ; for example, `device.type_id=2`
192
+ ("Desktop") is rejected.
193
+ - **py-ocsf-models** (0.10.0, Prowler) is a hand-written model set for three finding classes at
194
+ OCSF 1.5.0. It silently drops attributes it doesn't model.
195
+ - **ocsf-lib** (0.10.4) is useful schema tooling (compile, diff, compatibility checks), but it has
196
+ no event models or validation. It can't read the official `ocsf-schema-compiler` output, and its
197
+ own compiler disagrees with upstream on profile gating.
198
+
199
+ This library exists for four things those packages lack: OCSF knowledge as data you can query
200
+ without building a model, a conformance report that lists every problem in an event, observables
201
+ derived from the schema, and profiles treated as part of an event's identity.
202
+
203
+ ## License
204
+
205
+ Apache-2.0.