reg2es 2.2.0__tar.gz → 2.2.1__tar.gz

reg2es-2.2.1/PKG-INFO

@@ -0,0 +1,277 @@
+Metadata-Version: 2.5
+Name: reg2es
+Version: 2.2.1
+Summary: A fast library for parsing and importing Windows NT Registry(REGF) into Elasticsearch.
+Author-email: sumeshi <j15322sn@gmail.com>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: elasticsearch,json,registry
+Requires-Python: >=3.13
+Requires-Dist: elasticsearch>=9.4.1
+Requires-Dist: orjson>=3.11.9
+Requires-Dist: python-registry>=1.3.1
+Requires-Dist: tqdm>=4.67.3
+Requires-Dist: urllib3>=2.6.3
+Description-Content-Type: text/markdown
+
+# reg2es
+
+[![MIT License](http://img.shields.io/badge/license-MIT-blue.svg?style=flat)](LICENSE)
+[![PyPI Version](https://img.shields.io/pypi/v/reg2es)](https://pypi.org/project/reg2es/)
+
+![reg2es logo](https://gist.githubusercontent.com/sumeshi/c2f430d352ae763273faadf9616a29e5/raw/bd51b2539d8bb639d4f630ef13639706bed1f905/reg2es.svg)
+
+A command-line tool and Python library for extracting forensic artifacts from
+Windows NT Registry (REGF) hives and importing them into Elasticsearch.
+
+The original plugin set is based on
+[airbus-cert/regrippy](https://github.com/airbus-cert/regrippy); the package
+also includes reg2es-specific additions. The project also draws inspiration
+from [RegRipper](https://github.com/keydet89/RegRipper3.0). `reg2es` runs
+standalone and does not require the `regrippy` package at runtime. Both
+`reg2es` and `reg2json` use the same plugin runner and emit the same
+ECS-oriented documents.
+
+
+## Usage
+
+**reg2es** can be used as a standalone command-line tool or integrated directly into your Python scripts.
+
+```bash
+reg2es SYSTEM SOFTWARE SAM
+reg2json NTUSER.DAT -o artifacts.json
+```
+
+```python
+from reg2es import reg2es
+
+reg2es(["SYSTEM", "SOFTWARE", "SAM"])
+```
+
+
+### Arguments
+
+Multiple inputs passed in one invocation form one registry dataset. This lets
+plugins such as `localgroups` enrich SAM results using SOFTWARE data, regardless
+of the order of input paths.
+
+```bash
+reg2es SAM SOFTWARE
+```
+
+reg2es can recursively process all registry files under a specified directory:
+
+```bash
+tree .
+regfiles/
+  ├── NTUSER.DAT
+  ├── NTUSER.MAN
+  ├── SAM
+  └── subdirectory/
+    ├── SOFTWARE
+    └── subsubdirectory/
+      ├── SYSTEM
+      └── UsrClass.dat
+
+reg2es /regfiles/ # Recursively collects hives as one dataset.
+```
+
+Directory scans process REGF hive files only. Registry transaction logs and
+unrelated files are not treated as standalone hives.
+
+
+### Common options
+
+- `--plugin NAME`: run one plugin; repeat to select several. By default,
+  compatible, default-enabled plugins run. The exhaustive `regtime` plugin is
+  opt-in.
+- `--list-plugins`: print the 41 bundled plugins and exit.
+- `--size N`: set the generation and indexing chunk size (default: 500).
+- `--tags tag1,tag2`: add tags to every document.
+- `--quiet`: suppress progress output.
+
+`reg2es` also accepts Elasticsearch connection options including
+`--host`, `--port`, `--index`, `--scheme`, `--pipeline`, `--login`, `--pwd`,
+`--ca-certs`, and `--no-verify-certs`. TLS verification is enabled by default;
+use `--ca-certs /path/to/ca.pem` for a private CA bundle. Run `reg2es --help`
+or `reg2json --help` for the full current interface.
+
+
+### Examples
+
+When using from the command line:
+
+```bash
+reg2es SYSTEM --plugin services --host localhost --index registry-artifacts
+```
+
+When using from a Python script:
+
+```py
+reg2es(
+    ["SYSTEM", "SOFTWARE"],
+    host="localhost",
+    index="registry-artifacts",
+    plugin_names=["services", "systeminfo"],
+    additional_tags=["host-01", "case-42"],
+)
+```
+
+With Elasticsearch authentication:
+
+```bash
+reg2es SYSTEM --login elastic --pwd '******'
+```
+
+
+## Appendix
+
+### reg2json
+
+**reg2es** also includes `reg2json`, a command-line tool for converting Windows NT Registry into JSON files. :sushi: :sushi: :sushi:
+
+```bash
+reg2json NTUSER.DAT --plugin userassist -o artifacts.json
+```
+
+`reg2json` also supports line-delimited output. `--format jsonl` (or `ndjson`)
+writes one record per line without holding the entire dataset in memory. When
+no output path is specified, the default extension is `.jsonl`:
+
+```bash
+reg2json NTUSER.DAT --plugin userassist --format jsonl -o artifacts.jsonl
+```
+
+Use `--split` to write one file per plugin. With `--split`, `-o` names the
+output directory:
+
+```bash
+reg2json collected-hives/ --split -o artifacts/
+```
+
+The exhaustive `regtime` timeline is excluded from the default plugin set
+because it emits one record for every registry key. Run it explicitly when
+needed:
+
+```bash
+reg2json collected-hives/ --plugin regtime -o regtime.json
+```
+
+You can also convert registry files directly into a Python `list[dict]`:
+
+```python
+from reg2es import reg2json
+
+result: list[dict] = reg2json(
+    ["SOFTWARE", "SAM"],
+    plugin_names=["localgroups"],
+    additional_tags=["host-01"],
+)
+```
+
+
+## Output Format Example
+
+Each plugin result becomes one document with `ecs.version` set to `8.17.0`.
+Standard `event`,
+`registry`, `log.file`, `tags`, and `@timestamp` fields describe the artifact.
+Lossless plugin-specific data and the original offline-hive location are kept
+under `reg2es`.
+
+When a plugin has a reliable artifact time, it uses that time for `@timestamp`.
+Otherwise it uses the registry key's LastWrite; if no valid time exists,
+`@timestamp` is omitted. Check `reg2es.timestamp.source` and
+`reg2es.timestamp.meaning` before treating a result as an activity time.
+Original values and fallback reasons remain in the document.
+
+The current ECS output changes `registry.data.bytes` from a byte count to
+Base64, uses `REG_*` names in `registry.data.type`, and includes the value
+name in `registry.path`. Existing Elasticsearch mappings may conflict with
+these fields; use a new index for the updated output.
+
+```json
+{
+  "ecs": {"version": "8.17.0"},
+  "@timestamp": "2015-10-30T07:24:57.814133+00:00",
+  "event": {
+    "kind": "event",
+    "category": ["registry"],
+    "type": ["info"],
+    "action": "compname"
+  },
+  "registry": {
+    "hive": "HKLM",
+    "key": "SYSTEM\\ControlSet001\\Control\\ComputerName\\ComputerName",
+    "path": "HKLM\\SYSTEM\\ControlSet001\\Control\\ComputerName\\ComputerName\\ComputerName",
+      "value": "ComputerName",
+      "data": {
+      "type": "REG_SZ",
+      "strings": ["DESKTOP-EXAMPLE"]
+    }
+  },
+  "log": {
+    "file": {"path": "/evidence/SYSTEM"}
+  },
+  "tags": ["registry", "host-01"],
+  "reg2es": {
+    "plugin": {"name": "compname"},
+    "source": {
+      "hive": "SYSTEM",
+      "key_path": "ROOT\\ControlSet001\\Control\\ComputerName\\ComputerName"
+    },
+    "value_data": "DESKTOP-EXAMPLE",
+    "value_type": "RegSZ"
+  }
+}
+```
+
+
+## Installation
+
+### From PyPI
+
+```bash
+$ pip install reg2es
+```
+
+### With uv
+
+```bash
+$ uv add reg2es
+```
+
+### From GitHub Releases
+
+Standalone binaries built with Nuitka are available from GitHub Releases for systems without a Python environment.
+
+```bash
+$ chmod +x ./reg2es
+$ ./reg2es {{options...}}
+```
+
+```powershell
+> reg2es.exe {{options...}}
+```
+
+
+## Contributing
+
+The source code for **reg2es** is hosted on GitHub: https://github.com/sumeshi/reg2es.
+Please report issues and feature requests. :sushi: :sushi: :sushi:
+
+
+## License
+
+Standalone release ZIPs include `LICENSES.txt` with the project, bundled plugin,
+runtime dependency and build Python license notices. Keep it with the executables
+when redistributing them.
+
+**reg2es** is released under the [MIT](LICENSE) License.
+
+### Third-Party Notices
+
+This product includes code derived from
+[regrippy v2.0.3](https://github.com/airbus-cert/regrippy) by Airbus CERT,
+licensed under Apache License 2.0. The included ShimCache parser carries
+the original copyright notice of Andrew Davis, Mandiant (2012). See the
+[Apache 2.0 license text](LICENSES/Apache-2.0.txt) and the source-file notices.

reg2es-2.2.1/README.md

@@ -0,0 +1,261 @@
+# reg2es
+
+[![MIT License](http://img.shields.io/badge/license-MIT-blue.svg?style=flat)](LICENSE)
+[![PyPI Version](https://img.shields.io/pypi/v/reg2es)](https://pypi.org/project/reg2es/)
+
+![reg2es logo](https://gist.githubusercontent.com/sumeshi/c2f430d352ae763273faadf9616a29e5/raw/bd51b2539d8bb639d4f630ef13639706bed1f905/reg2es.svg)
+
+A command-line tool and Python library for extracting forensic artifacts from
+Windows NT Registry (REGF) hives and importing them into Elasticsearch.
+
+The original plugin set is based on
+[airbus-cert/regrippy](https://github.com/airbus-cert/regrippy); the package
+also includes reg2es-specific additions. The project also draws inspiration
+from [RegRipper](https://github.com/keydet89/RegRipper3.0). `reg2es` runs
+standalone and does not require the `regrippy` package at runtime. Both
+`reg2es` and `reg2json` use the same plugin runner and emit the same
+ECS-oriented documents.
+
+
+## Usage
+
+**reg2es** can be used as a standalone command-line tool or integrated directly into your Python scripts.
+
+```bash
+reg2es SYSTEM SOFTWARE SAM
+reg2json NTUSER.DAT -o artifacts.json
+```
+
+```python
+from reg2es import reg2es
+
+reg2es(["SYSTEM", "SOFTWARE", "SAM"])
+```
+
+
+### Arguments
+
+Multiple inputs passed in one invocation form one registry dataset. This lets
+plugins such as `localgroups` enrich SAM results using SOFTWARE data, regardless
+of the order of input paths.
+
+```bash
+reg2es SAM SOFTWARE
+```
+
+reg2es can recursively process all registry files under a specified directory:
+
+```bash
+tree .
+regfiles/
+  ├── NTUSER.DAT
+  ├── NTUSER.MAN
+  ├── SAM
+  └── subdirectory/
+    ├── SOFTWARE
+    └── subsubdirectory/
+      ├── SYSTEM
+      └── UsrClass.dat
+
+reg2es /regfiles/ # Recursively collects hives as one dataset.
+```
+
+Directory scans process REGF hive files only. Registry transaction logs and
+unrelated files are not treated as standalone hives.
+
+
+### Common options
+
+- `--plugin NAME`: run one plugin; repeat to select several. By default,
+  compatible, default-enabled plugins run. The exhaustive `regtime` plugin is
+  opt-in.
+- `--list-plugins`: print the 41 bundled plugins and exit.
+- `--size N`: set the generation and indexing chunk size (default: 500).
+- `--tags tag1,tag2`: add tags to every document.
+- `--quiet`: suppress progress output.
+
+`reg2es` also accepts Elasticsearch connection options including
+`--host`, `--port`, `--index`, `--scheme`, `--pipeline`, `--login`, `--pwd`,
+`--ca-certs`, and `--no-verify-certs`. TLS verification is enabled by default;
+use `--ca-certs /path/to/ca.pem` for a private CA bundle. Run `reg2es --help`
+or `reg2json --help` for the full current interface.
+
+
+### Examples
+
+When using from the command line:
+
+```bash
+reg2es SYSTEM --plugin services --host localhost --index registry-artifacts
+```
+
+When using from a Python script:
+
+```py
+reg2es(
+    ["SYSTEM", "SOFTWARE"],
+    host="localhost",
+    index="registry-artifacts",
+    plugin_names=["services", "systeminfo"],
+    additional_tags=["host-01", "case-42"],
+)
+```
+
+With Elasticsearch authentication:
+
+```bash
+reg2es SYSTEM --login elastic --pwd '******'
+```
+
+
+## Appendix
+
+### reg2json
+
+**reg2es** also includes `reg2json`, a command-line tool for converting Windows NT Registry into JSON files. :sushi: :sushi: :sushi:
+
+```bash
+reg2json NTUSER.DAT --plugin userassist -o artifacts.json
+```
+
+`reg2json` also supports line-delimited output. `--format jsonl` (or `ndjson`)
+writes one record per line without holding the entire dataset in memory. When
+no output path is specified, the default extension is `.jsonl`:
+
+```bash
+reg2json NTUSER.DAT --plugin userassist --format jsonl -o artifacts.jsonl
+```
+
+Use `--split` to write one file per plugin. With `--split`, `-o` names the
+output directory:
+
+```bash
+reg2json collected-hives/ --split -o artifacts/
+```
+
+The exhaustive `regtime` timeline is excluded from the default plugin set
+because it emits one record for every registry key. Run it explicitly when
+needed:
+
+```bash
+reg2json collected-hives/ --plugin regtime -o regtime.json
+```
+
+You can also convert registry files directly into a Python `list[dict]`:
+
+```python
+from reg2es import reg2json
+
+result: list[dict] = reg2json(
+    ["SOFTWARE", "SAM"],
+    plugin_names=["localgroups"],
+    additional_tags=["host-01"],
+)
+```
+
+
+## Output Format Example
+
+Each plugin result becomes one document with `ecs.version` set to `8.17.0`.
+Standard `event`,
+`registry`, `log.file`, `tags`, and `@timestamp` fields describe the artifact.
+Lossless plugin-specific data and the original offline-hive location are kept
+under `reg2es`.
+
+When a plugin has a reliable artifact time, it uses that time for `@timestamp`.
+Otherwise it uses the registry key's LastWrite; if no valid time exists,
+`@timestamp` is omitted. Check `reg2es.timestamp.source` and
+`reg2es.timestamp.meaning` before treating a result as an activity time.
+Original values and fallback reasons remain in the document.
+
+The current ECS output changes `registry.data.bytes` from a byte count to
+Base64, uses `REG_*` names in `registry.data.type`, and includes the value
+name in `registry.path`. Existing Elasticsearch mappings may conflict with
+these fields; use a new index for the updated output.
+
+```json
+{
+  "ecs": {"version": "8.17.0"},
+  "@timestamp": "2015-10-30T07:24:57.814133+00:00",
+  "event": {
+    "kind": "event",
+    "category": ["registry"],
+    "type": ["info"],
+    "action": "compname"
+  },
+  "registry": {
+    "hive": "HKLM",
+    "key": "SYSTEM\\ControlSet001\\Control\\ComputerName\\ComputerName",
+    "path": "HKLM\\SYSTEM\\ControlSet001\\Control\\ComputerName\\ComputerName\\ComputerName",
+      "value": "ComputerName",
+      "data": {
+      "type": "REG_SZ",
+      "strings": ["DESKTOP-EXAMPLE"]
+    }
+  },
+  "log": {
+    "file": {"path": "/evidence/SYSTEM"}
+  },
+  "tags": ["registry", "host-01"],
+  "reg2es": {
+    "plugin": {"name": "compname"},
+    "source": {
+      "hive": "SYSTEM",
+      "key_path": "ROOT\\ControlSet001\\Control\\ComputerName\\ComputerName"
+    },
+    "value_data": "DESKTOP-EXAMPLE",
+    "value_type": "RegSZ"
+  }
+}
+```
+
+
+## Installation
+
+### From PyPI
+
+```bash
+$ pip install reg2es
+```
+
+### With uv
+
+```bash
+$ uv add reg2es
+```
+
+### From GitHub Releases
+
+Standalone binaries built with Nuitka are available from GitHub Releases for systems without a Python environment.
+
+```bash
+$ chmod +x ./reg2es
+$ ./reg2es {{options...}}
+```
+
+```powershell
+> reg2es.exe {{options...}}
+```
+
+
+## Contributing
+
+The source code for **reg2es** is hosted on GitHub: https://github.com/sumeshi/reg2es.
+Please report issues and feature requests. :sushi: :sushi: :sushi:
+
+
+## License
+
+Standalone release ZIPs include `LICENSES.txt` with the project, bundled plugin,
+runtime dependency and build Python license notices. Keep it with the executables
+when redistributing them.
+
+**reg2es** is released under the [MIT](LICENSE) License.
+
+### Third-Party Notices
+
+This product includes code derived from
+[regrippy v2.0.3](https://github.com/airbus-cert/regrippy) by Airbus CERT,
+licensed under Apache License 2.0. The included ShimCache parser carries
+the original copyright notice of Andrew Davis, Mandiant (2012). See the
+[Apache 2.0 license text](LICENSES/Apache-2.0.txt) and the source-file notices.

reg2es-2.2.1/src/reg2es/__about__.py

@@ -0,0 +1 @@
+__version__ = "2.2.1"

{reg2es-2.2.0 → reg2es-2.2.1}/src/reg2es/models/Reg2es.py

@@ -20,6 +20,7 @@ import re
 import shutil
 import tempfile
 import math
+import base64
 from copy import deepcopy
 from datetime import datetime, timezone
 from enum import Enum
@@ -451,6 +452,26 @@ _ECS_HKLM_HIVES = {
     "SYSTEM",
 }
 
+# ECS 8.17.0 registry.data.type uses the Windows REG_* names. Keep this
+# explicit: python-registry's names are useful provenance, but are not ECS
+# values and unknown names must not be guessed into a known type.
+_ECS_REGISTRY_TYPES = {
+    "RegNone": "REG_NONE",
+    "RegSZ": "REG_SZ",
+    "RegExpandSZ": "REG_EXPAND_SZ",
+    "RegBin": "REG_BINARY",
+    "RegDWord": "REG_DWORD",
+    "RegBigEndian": "REG_DWORD_BIG_ENDIAN",
+    "RegLink": "REG_LINK",
+    "RegMultiSZ": "REG_MULTI_SZ",
+    "RegResourceList": "REG_RESOURCE_LIST",
+    "RegFullResourceDescriptor": "REG_FULL_RESOURCE_DESCRIPTOR",
+    "RegResourceRequirementsList": "REG_RESOURCE_REQUIREMENTS_LIST",
+    "RegQWord": "REG_QWORD",
+}
+
+_ECS_VERSION = "8.17.0"
+
 
 def _join_registry_path(*parts: Optional[str]) -> str:
     return "\\".join(part.strip("\\") for part in parts if part)
@@ -548,7 +569,10 @@ def plugin_result_to_document(
     registry_hive, registry_key, registry_path = _ecs_registry_location(
         hive_name, result.path
     )
+    if result.value_name is not None:
+        registry_path = _join_registry_path(registry_path, result.value_name)
     doc: dict = {
+        "ecs": {"version": _ECS_VERSION},
         "event": {
             "kind": "event",
             "category": ["registry"],
@@ -592,16 +616,30 @@ def plugin_result_to_document(
     if result.value_name is not None:
         normalized_data = _normalize_value(result.value_data)
         doc["registry"]["value"] = result.value_name
-        registry_data: dict = {"type": result.value_type}
-        if isinstance(result.value_data, bytes):
-            registry_data["bytes"] = len(result.value_data)
+        registry_data: dict = {}
+        ecs_type = _ECS_REGISTRY_TYPES.get(result.value_type)
+        if ecs_type is not None:
+            registry_data["type"] = ecs_type
+        if result.value_type is not None:
+            doc["reg2es"]["value_type"] = result.value_type
+        if isinstance(result.value_data, (bytes, bytearray, memoryview)):
+            raw_bytes = bytes(result.value_data)
+            registry_data["bytes"] = base64.b64encode(raw_bytes).decode("ascii")
+            doc["reg2es"]["value_size"] = len(raw_bytes)
         elif isinstance(normalized_data, str):
             registry_data["strings"] = [normalized_data]
         elif isinstance(normalized_data, list) and all(
             isinstance(item, str) for item in normalized_data
         ):
             registry_data["strings"] = normalized_data
-        doc["registry"]["data"] = registry_data
+        elif (
+            ecs_type in {"REG_DWORD", "REG_DWORD_BIG_ENDIAN", "REG_QWORD"}
+            and isinstance(result.value_data, int)
+            and not isinstance(result.value_data, bool)
+        ):
+            registry_data["strings"] = [str(result.value_data)]
+        if registry_data:
+            doc["registry"]["data"] = registry_data
         doc["reg2es"]["value_data"] = normalized_data
 
     # Custom fields (lossless).

reg2es-2.2.1/src/reg2es/plugins/mounteddevices.py

@@ -0,0 +1,16 @@
+"""Extract raw drive-letter and volume mappings from SYSTEM\\MountedDevices."""
+
+from reg2es.plugins import BasePlugin, PluginResult
+
+
+class Plugin(BasePlugin):
+    """Emit each MountedDevices value without guessing its binary encoding."""
+
+    __REGHIVE__ = "SYSTEM"
+
+    def run(self):
+        key = self.open_key("MountedDevices")
+        if not key:
+            return
+        for value in key.values():
+            yield PluginResult(key=key, value=value)

reg2es-2.2.1/src/reg2es/plugins/usbstor.py

@@ -0,0 +1,28 @@
+"""Extract USB mass-storage device instances recorded by the SYSTEM hive."""
+
+from reg2es.plugins import BasePlugin, PluginResult
+
+
+class Plugin(BasePlugin):
+    """Emit each USBSTOR hardware instance with its raw registry values."""
+
+    __REGHIVE__ = "SYSTEM"
+
+    def run(self):
+        control_set = self.get_currentcontrolset_path()
+        if not control_set:
+            return
+        root = self.open_key(f"{control_set}\\Enum\\USBSTOR")
+        if not root:
+            return
+
+        for device_type in root.subkeys():
+            for instance in device_type.subkeys():
+                result = PluginResult(key=instance)
+                result.custom["device_type"] = device_type.name()
+                result.custom["instance_id"] = instance.name()
+                values = {}
+                for value in instance.values():
+                    values[value.name()] = value.value()
+                result.custom["registry_values"] = values
+                yield result