cyvest 1.0.1__tar.gz → 3.0.0__tar.gz

{cyvest-1.0.1 → cyvest-3.0.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: cyvest
-Version: 1.0.1
+Version: 3.0.0
 Summary: Cybersecurity investigation model
 Keywords: cybersecurity,investigation,threat-intel,security-analysis
 Author: PakitoSec
@@ -16,7 +16,9 @@ Classifier: Programming Language :: Python :: 3.12
 Classifier: Topic :: Security
 Requires-Dist: click>=8
 Requires-Dist: logurich[click]>=0.1
+Requires-Dist: pydantic>=2.12.5
 Requires-Dist: rich>=13
+Requires-Dist: typing-extensions>=4.15
 Requires-Dist: pyvis>=0.3.2 ; extra == 'visualization'
 Requires-Python: >=3.10
 Project-URL: Homepage, https://github.com/PakitoSec/cyvest
@@ -37,8 +39,8 @@ Description-Content-Type: text/markdown
 - 🔍 **Structured Investigation Modeling**: Model investigations with observables, checks, threat intelligence, and enrichments
 - 📊 **Automatic Scoring**: Dynamic score calculation and propagation through investigation hierarchy
 - 🎯 **Level Classification**: Automatic security level assignment (TRUSTED, INFO, SAFE, NOTABLE, SUSPICIOUS, MALICIOUS)
-- 🔗 **Relationship Tracking**: STIX2-compliant relationship modeling between observables
-- 🏷️ **STIX2 Type Support**: Built-in enums for STIX2 Observable and Relationship types with autocomplete
+- 🔗 **Relationship Tracking**: Lightweight relationship modeling between observables
+- 🏷️ **Typed Helpers**: Built-in enums for observable types and relationships with autocomplete
 - 📈 **Real-time Statistics**: Live metrics and aggregations throughout the investigation
 - 🔄 **Investigation Merging**: Combine investigations from multiple threads or processes
 - 🧵 **Multi-Threading Support**: Advanced thread-safe shared context available via `cyvest.investigation` module
@@ -82,7 +84,7 @@ from cyvest import Cyvest, Level, ObservableType, RelationshipType
 
 # Create an investigation
 with Cyvest(data={"type": "email"}) as cv:
-    # Create observables with STIX2 types
+    # Create observables
     url = (
         cv.observable(ObservableType.URL, "https://phishing-site.com", internal=False)
         .with_ti("virustotal", score=Decimal("8.5"), level=Level.MALICIOUS)
@@ -126,108 +128,30 @@ Dictionary fields merge by default; pass `merge_extra=False` (or `merge_data=Fal
 
 ### Observables
 
-Observables represent cyber artifacts (URLs, IPs, domains, hashes, files, etc.). Use STIX2-compliant types for standardization:
+Observables represent cyber artifacts (URLs, IPs, domains, hashes, files, etc.).
 
 ```python
-from cyvest import ObservableType, RelationshipType
+from cyvest import ObservableType, RelationshipType, RelationshipDirection
 
-# Create observable with STIX2 type enum
 url_obs = cv.observable_create(
     ObservableType.URL,
     "https://malicious.com",
     internal=False
 )
 
-# Or use strings (backward compatible)
 ip_obs = cv.observable_create("ipv4-addr", "192.0.2.1", internal=False)
 
-# Add threat intelligence
-cv.observable_add_threat_intel(
-    url_obs.key,
-    source="virustotal",
-    score=Decimal("9.0"),
-    level=Level.MALICIOUS,
-    comment="Detected as malware distribution site"
-)
-
-# Create relationships with STIX2 relationship types
-# Accepts observable proxies or string keys
 cv.observable_add_relationship(
     url_obs,  # Can pass ObservableProxy directly
     ip_obs,   # Or use .key for string keys
-    RelationshipType.RESOLVES_TO
+    RelationshipType.RELATED_TO,
+    RelationshipDirection.BIDIRECTIONAL,
 )
 ```
 
-**Available STIX2 Observable Types:**
-
-- Network: `IPV4_ADDR`, `IPV6_ADDR`, `DOMAIN_NAME`, `URL`, `MAC_ADDR`, `NETWORK_TRAFFIC`
-- Email: `EMAIL_ADDR`, `EMAIL_MESSAGE`, `EMAIL_MIME_PART`
-- File: `FILE`, `DIRECTORY`, `ARTIFACT`
-- System: `PROCESS`, `SOFTWARE`, `USER_ACCOUNT`, `WINDOWS_REGISTRY_KEY`
-- Other: `AUTONOMOUS_SYSTEM`, `MUTEX`, `X509_CERTIFICATE`
-
-**Available STIX2 Relationship Types:**
-
-- Network: `RESOLVES_TO`, `BELONGS_TO`, `COMMUNICATES_WITH`
-- File: `CONTAINS`, `DOWNLOADED`, `DROPPED`
-- Email: `FROM`, `SENDER`, `TO`, `CC`, `BCC`
-- Process: `CREATED`, `OPENED`, `PARENT`, `CHILD`
-- General: `RELATED_TO`, `DERIVED_FROM`, `DUPLICATE_OF`
-
-**Relationship Direction:**
-
-Relationships support directional semantics with **automatic semantic defaults** that determine **hierarchical score propagation**:
-
-```python
-from cyvest import RelationshipType, RelationshipDirection
-
-# Automatically gets OUTBOUND (domain → IP)
-# IP is a child of domain, IP's score propagates UP to domain
-# Accepts observable proxies directly
-cv.observable_add_relationship(domain, ip, RelationshipType.RESOLVES_TO)
-
-# Automatically gets INBOUND (file ← URL)
-# URL is parent of file, file's score propagates UP to URL
-cv.observable_add_relationship(malware, url, RelationshipType.DOWNLOADED)
-
-# Automatically gets BIDIRECTIONAL (host ↔ host)
-# No hierarchical propagation - scores remain independent
-cv.observable_add_relationship(host1, host2, RelationshipType.COMMUNICATES_WITH)
-
-# Can override semantic defaults if needed
-cv.observable_add_relationship(
-    domain, ip,  # Accepts observable proxies
-    RelationshipType.RESOLVES_TO,
-    RelationshipDirection.INBOUND  # explicit override
-)
-```
-
-**Direction-Based Hierarchical Scoring:**
-
-Relationship directions define parent-child hierarchies for score propagation:
-
-- **OUTBOUND (→)**: `source → target` — Target is a **child** of source
-  - Source's score includes child's score: `score = max(TI_scores, child_scores)`
-  - Example: `domain → IP` means IP score flows up to domain
-
-- **INBOUND (←)**: `source ← target` — Target is a **parent** of source
-  - Target's score includes source's score
-  - Example: `file ← URL` means file score flows up to URL
-
-- **BIDIRECTIONAL (↔)**: `source ↔ target` — **No hierarchy**
-  - Scores do NOT propagate between observables
-  - Each maintains independent score from its own threat intel
-  - Example: `host1 ↔ host2` keeps separate scores
-
-**Semantic Default Directions:**
-
-Each relationship type has an intuitive default direction:
-- **OUTBOUND (→)**: `RESOLVES_TO`, `BELONGS_TO`, `CONTAINS`, `TO`, `CC`, `BCC`, `CREATED`, `OPENED`, `PARENT`, `VALUES`
-- **INBOUND (←)**: `DOWNLOADED`, `DROPPED`, `FROM`, `SENDER`, `CHILD`, `DERIVED_FROM`
-- **BIDIRECTIONAL (↔)**: `COMMUNICATES_WITH`, `RELATED_TO`, `DUPLICATE_OF`
-
-Direction symbols appear in visualizations, markdown exports, and determine score flow.
+Cyvest ships enums for the most common observable types; you can still pass strings for custom types.
+Relationships are intentionally simple for now: use `RelationshipType.RELATED_TO` to link observables
+and optionally choose a direction (`OUTBOUND`, `INBOUND`, or `BIDIRECTIONAL`) to control score propagation.
 
 ### Checks
 
@@ -430,6 +354,9 @@ cyvest merge inv1.json inv2.json -o merged.json -f rich --stats
 
 # Generate an interactive visualization (requires visualization extra)
 cyvest visualize investigation.json --min-level SUSPICIOUS --group-by-type
+
+# Output the JSON Schema describing serialized investigations and generate types
+uv run cyvest schema -o ./schema/cyvest.schema.json && pnpm -C js/packages/cyvest-js run generate:types
 ```
 
 ## Development
@@ -484,29 +411,15 @@ mkdocs serve
 mkdocs build
 ```
 
-## Project Structure
+## JavaScript packages
 
-```
-cyvest/
-├── src/cyvest/
-│   ├── __init__.py          # Package initialization
-│   ├── cyvest.py            # High-level API facade
-│   ├── investigation.py     # Core state management with merge-on-create
-│   ├── proxies.py           # Read-only proxies + fluent helper methods
-│   ├── levels.py            # Level enum and scoring logic
-│   ├── keys.py              # Key generation utilities
-│   ├── model.py             # Core data models
-│   ├── score.py             # Scoring and propagation engine
-│   ├── stats.py             # Statistics and aggregations
-│   ├── io_serialization.py  # JSON and Markdown export
-│   ├── io_rich.py           # Rich console output
-│   └── cli.py               # CLI interface
-├── examples/                # Example scripts
-├── tests/                   # Test suite
-├── docs/                    # Documentation
-├── pyproject.toml           # Project configuration
-└── README.md                # This file
-```
+The repo includes a PNPM workspace under `js/` with three packages:
+
+- `@cyvest/cyvest-js`: TypeScript types, schema validation, and helpers for Cyvest investigations.
+- `@cyvest/cyvest-vis`: React components for graph visualization (depends on `@cyvest/cyvest-js`).
+- `@cyvest/cyvest-app`: Vite demo that bundles the JS packages with sample investigations.
+
+See `docs/js-packages.md` for workspace commands and usage snippets.
 
 ## Contributing
 
@@ -539,7 +452,6 @@ Cyvest is designed for:
 - **Deterministic Keys**: Same objects always generate same keys for merging
 - **Score Propagation**: Automatic hierarchical score calculation
 - **Flexible Export**: JSON for storage, Markdown for LLM analysis
-- **STIX2 Relationships**: Industry-standard relationship modeling
 - **Audit Trail**: Score change history for debugging
 
 ## Future Enhancements

{cyvest-1.0.1 → cyvest-3.0.0}/README.md

@@ -10,8 +10,8 @@
 - 🔍 **Structured Investigation Modeling**: Model investigations with observables, checks, threat intelligence, and enrichments
 - 📊 **Automatic Scoring**: Dynamic score calculation and propagation through investigation hierarchy
 - 🎯 **Level Classification**: Automatic security level assignment (TRUSTED, INFO, SAFE, NOTABLE, SUSPICIOUS, MALICIOUS)
-- 🔗 **Relationship Tracking**: STIX2-compliant relationship modeling between observables
-- 🏷️ **STIX2 Type Support**: Built-in enums for STIX2 Observable and Relationship types with autocomplete
+- 🔗 **Relationship Tracking**: Lightweight relationship modeling between observables
+- 🏷️ **Typed Helpers**: Built-in enums for observable types and relationships with autocomplete
 - 📈 **Real-time Statistics**: Live metrics and aggregations throughout the investigation
 - 🔄 **Investigation Merging**: Combine investigations from multiple threads or processes
 - 🧵 **Multi-Threading Support**: Advanced thread-safe shared context available via `cyvest.investigation` module
@@ -55,7 +55,7 @@ from cyvest import Cyvest, Level, ObservableType, RelationshipType
 
 # Create an investigation
 with Cyvest(data={"type": "email"}) as cv:
-    # Create observables with STIX2 types
+    # Create observables
     url = (
         cv.observable(ObservableType.URL, "https://phishing-site.com", internal=False)
         .with_ti("virustotal", score=Decimal("8.5"), level=Level.MALICIOUS)
@@ -99,108 +99,30 @@ Dictionary fields merge by default; pass `merge_extra=False` (or `merge_data=Fal
 
 ### Observables
 
-Observables represent cyber artifacts (URLs, IPs, domains, hashes, files, etc.). Use STIX2-compliant types for standardization:
+Observables represent cyber artifacts (URLs, IPs, domains, hashes, files, etc.).
 
 ```python
-from cyvest import ObservableType, RelationshipType
+from cyvest import ObservableType, RelationshipType, RelationshipDirection
 
-# Create observable with STIX2 type enum
 url_obs = cv.observable_create(
     ObservableType.URL,
     "https://malicious.com",
     internal=False
 )
 
-# Or use strings (backward compatible)
 ip_obs = cv.observable_create("ipv4-addr", "192.0.2.1", internal=False)
 
-# Add threat intelligence
-cv.observable_add_threat_intel(
-    url_obs.key,
-    source="virustotal",
-    score=Decimal("9.0"),
-    level=Level.MALICIOUS,
-    comment="Detected as malware distribution site"
-)
-
-# Create relationships with STIX2 relationship types
-# Accepts observable proxies or string keys
 cv.observable_add_relationship(
     url_obs,  # Can pass ObservableProxy directly
     ip_obs,   # Or use .key for string keys
-    RelationshipType.RESOLVES_TO
+    RelationshipType.RELATED_TO,
+    RelationshipDirection.BIDIRECTIONAL,
 )
 ```
 
-**Available STIX2 Observable Types:**
-
-- Network: `IPV4_ADDR`, `IPV6_ADDR`, `DOMAIN_NAME`, `URL`, `MAC_ADDR`, `NETWORK_TRAFFIC`
-- Email: `EMAIL_ADDR`, `EMAIL_MESSAGE`, `EMAIL_MIME_PART`
-- File: `FILE`, `DIRECTORY`, `ARTIFACT`
-- System: `PROCESS`, `SOFTWARE`, `USER_ACCOUNT`, `WINDOWS_REGISTRY_KEY`
-- Other: `AUTONOMOUS_SYSTEM`, `MUTEX`, `X509_CERTIFICATE`
-
-**Available STIX2 Relationship Types:**
-
-- Network: `RESOLVES_TO`, `BELONGS_TO`, `COMMUNICATES_WITH`
-- File: `CONTAINS`, `DOWNLOADED`, `DROPPED`
-- Email: `FROM`, `SENDER`, `TO`, `CC`, `BCC`
-- Process: `CREATED`, `OPENED`, `PARENT`, `CHILD`
-- General: `RELATED_TO`, `DERIVED_FROM`, `DUPLICATE_OF`
-
-**Relationship Direction:**
-
-Relationships support directional semantics with **automatic semantic defaults** that determine **hierarchical score propagation**:
-
-```python
-from cyvest import RelationshipType, RelationshipDirection
-
-# Automatically gets OUTBOUND (domain → IP)
-# IP is a child of domain, IP's score propagates UP to domain
-# Accepts observable proxies directly
-cv.observable_add_relationship(domain, ip, RelationshipType.RESOLVES_TO)
-
-# Automatically gets INBOUND (file ← URL)
-# URL is parent of file, file's score propagates UP to URL
-cv.observable_add_relationship(malware, url, RelationshipType.DOWNLOADED)
-
-# Automatically gets BIDIRECTIONAL (host ↔ host)
-# No hierarchical propagation - scores remain independent
-cv.observable_add_relationship(host1, host2, RelationshipType.COMMUNICATES_WITH)
-
-# Can override semantic defaults if needed
-cv.observable_add_relationship(
-    domain, ip,  # Accepts observable proxies
-    RelationshipType.RESOLVES_TO,
-    RelationshipDirection.INBOUND  # explicit override
-)
-```
-
-**Direction-Based Hierarchical Scoring:**
-
-Relationship directions define parent-child hierarchies for score propagation:
-
-- **OUTBOUND (→)**: `source → target` — Target is a **child** of source
-  - Source's score includes child's score: `score = max(TI_scores, child_scores)`
-  - Example: `domain → IP` means IP score flows up to domain
-
-- **INBOUND (←)**: `source ← target` — Target is a **parent** of source
-  - Target's score includes source's score
-  - Example: `file ← URL` means file score flows up to URL
-
-- **BIDIRECTIONAL (↔)**: `source ↔ target` — **No hierarchy**
-  - Scores do NOT propagate between observables
-  - Each maintains independent score from its own threat intel
-  - Example: `host1 ↔ host2` keeps separate scores
-
-**Semantic Default Directions:**
-
-Each relationship type has an intuitive default direction:
-- **OUTBOUND (→)**: `RESOLVES_TO`, `BELONGS_TO`, `CONTAINS`, `TO`, `CC`, `BCC`, `CREATED`, `OPENED`, `PARENT`, `VALUES`
-- **INBOUND (←)**: `DOWNLOADED`, `DROPPED`, `FROM`, `SENDER`, `CHILD`, `DERIVED_FROM`
-- **BIDIRECTIONAL (↔)**: `COMMUNICATES_WITH`, `RELATED_TO`, `DUPLICATE_OF`
-
-Direction symbols appear in visualizations, markdown exports, and determine score flow.
+Cyvest ships enums for the most common observable types; you can still pass strings for custom types.
+Relationships are intentionally simple for now: use `RelationshipType.RELATED_TO` to link observables
+and optionally choose a direction (`OUTBOUND`, `INBOUND`, or `BIDIRECTIONAL`) to control score propagation.
 
 ### Checks
 
@@ -403,6 +325,9 @@ cyvest merge inv1.json inv2.json -o merged.json -f rich --stats
 
 # Generate an interactive visualization (requires visualization extra)
 cyvest visualize investigation.json --min-level SUSPICIOUS --group-by-type
+
+# Output the JSON Schema describing serialized investigations and generate types
+uv run cyvest schema -o ./schema/cyvest.schema.json && pnpm -C js/packages/cyvest-js run generate:types
 ```
 
 ## Development
@@ -457,29 +382,15 @@ mkdocs serve
 mkdocs build
 ```
 
-## Project Structure
+## JavaScript packages
 
-```
-cyvest/
-├── src/cyvest/
-│   ├── __init__.py          # Package initialization
-│   ├── cyvest.py            # High-level API facade
-│   ├── investigation.py     # Core state management with merge-on-create
-│   ├── proxies.py           # Read-only proxies + fluent helper methods
-│   ├── levels.py            # Level enum and scoring logic
-│   ├── keys.py              # Key generation utilities
-│   ├── model.py             # Core data models
-│   ├── score.py             # Scoring and propagation engine
-│   ├── stats.py             # Statistics and aggregations
-│   ├── io_serialization.py  # JSON and Markdown export
-│   ├── io_rich.py           # Rich console output
-│   └── cli.py               # CLI interface
-├── examples/                # Example scripts
-├── tests/                   # Test suite
-├── docs/                    # Documentation
-├── pyproject.toml           # Project configuration
-└── README.md                # This file
-```
+The repo includes a PNPM workspace under `js/` with three packages:
+
+- `@cyvest/cyvest-js`: TypeScript types, schema validation, and helpers for Cyvest investigations.
+- `@cyvest/cyvest-vis`: React components for graph visualization (depends on `@cyvest/cyvest-js`).
+- `@cyvest/cyvest-app`: Vite demo that bundles the JS packages with sample investigations.
+
+See `docs/js-packages.md` for workspace commands and usage snippets.
 
 ## Contributing
 
@@ -512,7 +423,6 @@ Cyvest is designed for:
 - **Deterministic Keys**: Same objects always generate same keys for merging
 - **Score Propagation**: Automatic hierarchical score calculation
 - **Flexible Export**: JSON for storage, Markdown for LLM analysis
-- **STIX2 Relationships**: Industry-standard relationship modeling
 - **Audit Trail**: Score change history for debugging
 
 ## Future Enhancements

{cyvest-1.0.1 → cyvest-3.0.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "cyvest"
-version = "1.0.1"
+version = "3.0.0"
 description = "Cybersecurity investigation model"
 readme = {file = "README.md", content-type = "text/markdown"}
 requires-python = ">=3.10"
@@ -11,7 +11,9 @@ authors = [
 dependencies = [
     "click>=8",
     "logurich[click]>=0.1",
+    "pydantic>=2.12.5",
     "rich>=13",
+    "typing-extensions>=4.15",
 ]
 keywords = ["cybersecurity", "investigation", "threat-intel", "security-analysis"]
 classifiers = [
@@ -40,6 +42,7 @@ build-backend = "uv_build"
 [dependency-groups]
 dev = [
     "debugpy>=1.8.17",
+    "jsonschema>=4.23.0",
     "pre-commit>=4.4.0",
     "pytest>=8.4.2",
     "pytest-cov>=6.0.0",

{cyvest-1.0.1 → cyvest-3.0.0}/src/cyvest/__init__.py

@@ -8,12 +8,17 @@ programmatically with automatic scoring, level calculation, and rich reporting c
 from logurich import logger
 
 from cyvest.cyvest import Cyvest
-from cyvest.investigation import InvestigationWhitelist
 from cyvest.levels import Level
-from cyvest.model import CheckScorePolicy, ObservableType, RelationshipDirection, RelationshipType
+from cyvest.model import (
+    CheckScorePolicy,
+    InvestigationWhitelist,
+    ObservableType,
+    RelationshipDirection,
+    RelationshipType,
+)
 from cyvest.proxies import CheckProxy, ContainerProxy, EnrichmentProxy, ObservableProxy, ThreatIntelProxy
 
-__version__ = "1.0.1"
+__version__ = "3.0.0"
 
 logger.disable("cyvest")
 

{cyvest-1.0.1 → cyvest-3.0.0}/src/cyvest/cli.py

@@ -12,11 +12,12 @@ from pathlib import Path
 from typing import Any
 
 import click
-from logurich import init_logger, logger
+from logurich import logger
 from logurich.opt_click import click_logger_params
 from rich.console import Console
 
 from cyvest import __version__
+from cyvest.io_schema import get_investigation_schema
 from cyvest.io_serialization import load_investigation_json
 from cyvest.io_visualization import VisualizationDependencyMissingError
 
@@ -69,7 +70,6 @@ def _write_markdown(data: dict[str, Any], output_path: Path) -> None:
 @click.version_option(__version__, prog_name="Cyvest")
 def cli() -> None:
     """Cyvest - Cybersecurity Investigation Framework."""
-    init_logger("INFO")
     logger.enable("cyvest")
     logger.info("> [green bold]CYVEST[/green bold]")
 
@@ -166,10 +166,10 @@ def merge(inputs: tuple[Path, ...], output: Path, output_format: str, stats: boo
     if stats:
         logger.info("[bold]Merged Investigation Statistics:[/bold]")
         investigation_stats = main_investigation.get_statistics()
-        logger.info(f"  Total Observables: {investigation_stats.get('total_observables', 0)}")
-        logger.info(f"  Total Checks: {investigation_stats.get('total_checks', 0)}")
-        logger.info(f"  Total Threat Intel: {investigation_stats.get('total_threat_intel', 0)}")
-        logger.info(f"  Total Containers: {investigation_stats.get('total_containers', 0)}")
+        logger.info(f"  Total Observables: {investigation_stats.total_observables}")
+        logger.info(f"  Total Checks: {investigation_stats.total_checks}")
+        logger.info(f"  Total Threat Intel: {investigation_stats.total_threat_intel}")
+        logger.info(f"  Total Containers: {investigation_stats.total_containers}")
         logger.info(f"  Global Score: {main_investigation.get_global_score()}")
         logger.info(f"  Global Level: {main_investigation.get_global_level()}\n")
 
@@ -221,6 +221,28 @@ def export(input: Path, output: Path, export_format: str) -> None:
     logger.info(f"[green]Exported to Markdown: {output_path}[/green]")
 
 
+@cli.command(name="schema")
+@click.option(
+    "-o",
+    "--output",
+    type=click.Path(dir_okay=False, path_type=Path),
+    help="Write the JSON Schema to a file instead of stdout.",
+)
+def schema_cmd(output: Path | None) -> None:
+    """
+    Emit the JSON Schema describing serialized investigations.
+    """
+    schema = get_investigation_schema()
+    if output:
+        output_path = output.resolve()
+        output_path.parent.mkdir(parents=True, exist_ok=True)
+        output_path.write_text(json.dumps(schema, indent=2) + "\n", encoding="utf-8")
+        logger.info(f"[green]Schema written to: {output_path}[/green]")
+        return
+
+    logger.rich("INFO", json.dumps(schema, indent=2), prefix=False)
+
+
 @cli.command()
 @click.argument("input", type=click.Path(exists=True, dir_okay=False, path_type=Path))
 @click.option(