cpg2py 1.0.5__tar.gz → 1.2.0__tar.gz

{cpg2py-1.0.5 → cpg2py-1.2.0}/LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2025 Yichao Xu
+Copyright (c) 2025 samhsu-dev
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

cpg2py-1.2.0/PKG-INFO

@@ -0,0 +1,180 @@
+Metadata-Version: 2.4
+Name: cpg2py
+Version: 1.2.0
+Summary: A graph-based data structure designed for querying CSV files in Joern format in Python
+Author-email: samhsu-dev <yxu166@jhu.edu>
+License: MIT License
+        
+        Copyright (c) 2025 samhsu-dev
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+        
+Project-URL: Homepage, https://github.com/samhsu-dev/cpg2py
+Project-URL: Repository, https://github.com/samhsu-dev/cpg2py
+Project-URL: Documentation, https://github.com/samhsu-dev/cpg2py
+Keywords: Joern,CPG,Graph,CSV
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: test
+Requires-Dist: pytest>=7.4.0; extra == "test"
+Requires-Dist: pytest-cov>=4.0.0; extra == "test"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.4.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
+Requires-Dist: black>=23.0.0; extra == "dev"
+Requires-Dist: isort>=5.12.0; extra == "dev"
+Requires-Dist: pylint>=2.17.0; extra == "dev"
+Dynamic: license-file
+
+# cpg2py
+
+Python graph query engine for Code Property Graphs from Joern CSV exports. Directed multi-graph with generic ABCs for custom node/edge/graph types.
+
+**Features**: Load from `nodes.csv` + `rels.csv`; query/update nodes and edges (`get_property`, `set_property`, `set_properties`); traverse succ/prev/children/parent/flow_to/flow_from; JSON persistence (`save_json`, `load_json`, `storage_from_json`). Concrete types: `CpgGraph`, `CpgNode`, `CpgEdge`.
+
+---
+
+## Installation
+
+```bash
+pip install cpg2py
+```
+
+From source (e.g. with [uv](https://github.com/astral-sh/uv)):
+
+```bash
+git clone https://github.com/samhsu-dev/cpg2py.git && cd cpg2py
+uv sync --dev
+uv run pytest tests/
+```
+
+---
+
+## Input format
+
+- **nodes.csv**: tab-delimited; must include node id (e.g. `id:int` or `id`). Other columns become node properties.
+- **rels.csv**: tab-delimited; columns `start`, `end`, `type` (or `start:str`, `end:str`, `type:str`).
+
+---
+
+## Usage
+
+**Load from CSV**
+
+```python
+from pathlib import Path
+from cpg2py import cpg_graph, CpgGraph, CpgNode, CpgEdge
+
+graph: CpgGraph = cpg_graph(Path("nodes.csv"), Path("rels.csv"))
+```
+
+**Nodes and edges** (edge identified by `(from_id, to_id, edge_type)`; `edge_type` is string)
+
+```python
+node: CpgNode = graph.node("2")
+node.name
+node.set_property("name", "x")
+node.set_properties({"k": "v"})
+
+edge: CpgEdge = graph.edge("2", "3", "ENTRY")
+edge.from_nid, edge.to_nid, edge.type
+edge.set_property("weight", 0.5)
+```
+
+**Traversal**
+
+```python
+graph.succ(node)   # successors
+graph.prev(node)   # predecessors
+graph.children(node)
+graph.parent(node)
+graph.flow_to(node)
+graph.flow_from(node)
+graph.topfile_node("5")   # top-level file node for given node ID
+```
+
+**Filtered iteration** (optional predicate)
+
+```python
+graph.nodes(lambda n: n.type == "Function")
+graph.edges(lambda e: e.edge_type == "FLOWS_TO")
+graph.succ(node, who_satisifies=lambda e: e.edge_type == "PARENT_OF")
+graph.descendants(node, condition=...)
+graph.ancestors(node, condition=...)
+```
+
+**JSON persistence**
+
+```python
+graph.storage.save_json("graph.json")
+
+storage = Storage()
+storage.load_json("graph.json")
+graph2 = CpgGraph(storage)
+
+# or
+storage = storage_from_json(Path("graph.json"))
+```
+
+JSON schema: `{"nodes": { "<id>": { "<key>": <value>, ... }, ... }, "edges": [ {"from": str, "to": str, "type": str, "props": {...} }, ... ]}`. See [design.md](docs/design.md).
+
+---
+
+## Extending (ABCs)
+
+Implement `AbcGraphQuerier[MyNode, MyEdge]`, `AbcNodeQuerier`, `AbcEdgeQuerier`; inject `Storage`. Full interface and contracts: [docs/design.md](docs/design.md).
+
+Minimal custom graph:
+
+```python
+from cpg2py import AbcGraphQuerier, AbcNodeQuerier, AbcEdgeQuerier, Storage
+from typing import Optional
+
+class MyNode(AbcNodeQuerier): pass
+class MyEdge(AbcEdgeQuerier): pass
+
+class MyGraph(AbcGraphQuerier[MyNode, MyEdge]):
+    def node(self, whose_id_is: str) -> Optional[MyNode]:
+        return MyNode(self.storage, whose_id_is)
+    def edge(self, fid: str, tid: str, eid: str) -> Optional[MyEdge]:
+        return MyEdge(self.storage, fid, tid, eid)
+
+g = MyGraph(Storage())
+```
+
+---
+
+Interface specifications (classes, methods, signatures, validation): [docs/design.md](docs/design.md).
+
+---
+
+## License
+MIT.

cpg2py-1.2.0/README.md

@@ -0,0 +1,124 @@
+# cpg2py
+
+Python graph query engine for Code Property Graphs from Joern CSV exports. Directed multi-graph with generic ABCs for custom node/edge/graph types.
+
+**Features**: Load from `nodes.csv` + `rels.csv`; query/update nodes and edges (`get_property`, `set_property`, `set_properties`); traverse succ/prev/children/parent/flow_to/flow_from; JSON persistence (`save_json`, `load_json`, `storage_from_json`). Concrete types: `CpgGraph`, `CpgNode`, `CpgEdge`.
+
+---
+
+## Installation
+
+```bash
+pip install cpg2py
+```
+
+From source (e.g. with [uv](https://github.com/astral-sh/uv)):
+
+```bash
+git clone https://github.com/samhsu-dev/cpg2py.git && cd cpg2py
+uv sync --dev
+uv run pytest tests/
+```
+
+---
+
+## Input format
+
+- **nodes.csv**: tab-delimited; must include node id (e.g. `id:int` or `id`). Other columns become node properties.
+- **rels.csv**: tab-delimited; columns `start`, `end`, `type` (or `start:str`, `end:str`, `type:str`).
+
+---
+
+## Usage
+
+**Load from CSV**
+
+```python
+from pathlib import Path
+from cpg2py import cpg_graph, CpgGraph, CpgNode, CpgEdge
+
+graph: CpgGraph = cpg_graph(Path("nodes.csv"), Path("rels.csv"))
+```
+
+**Nodes and edges** (edge identified by `(from_id, to_id, edge_type)`; `edge_type` is string)
+
+```python
+node: CpgNode = graph.node("2")
+node.name
+node.set_property("name", "x")
+node.set_properties({"k": "v"})
+
+edge: CpgEdge = graph.edge("2", "3", "ENTRY")
+edge.from_nid, edge.to_nid, edge.type
+edge.set_property("weight", 0.5)
+```
+
+**Traversal**
+
+```python
+graph.succ(node)   # successors
+graph.prev(node)   # predecessors
+graph.children(node)
+graph.parent(node)
+graph.flow_to(node)
+graph.flow_from(node)
+graph.topfile_node("5")   # top-level file node for given node ID
+```
+
+**Filtered iteration** (optional predicate)
+
+```python
+graph.nodes(lambda n: n.type == "Function")
+graph.edges(lambda e: e.edge_type == "FLOWS_TO")
+graph.succ(node, who_satisifies=lambda e: e.edge_type == "PARENT_OF")
+graph.descendants(node, condition=...)
+graph.ancestors(node, condition=...)
+```
+
+**JSON persistence**
+
+```python
+graph.storage.save_json("graph.json")
+
+storage = Storage()
+storage.load_json("graph.json")
+graph2 = CpgGraph(storage)
+
+# or
+storage = storage_from_json(Path("graph.json"))
+```
+
+JSON schema: `{"nodes": { "<id>": { "<key>": <value>, ... }, ... }, "edges": [ {"from": str, "to": str, "type": str, "props": {...} }, ... ]}`. See [design.md](docs/design.md).
+
+---
+
+## Extending (ABCs)
+
+Implement `AbcGraphQuerier[MyNode, MyEdge]`, `AbcNodeQuerier`, `AbcEdgeQuerier`; inject `Storage`. Full interface and contracts: [docs/design.md](docs/design.md).
+
+Minimal custom graph:
+
+```python
+from cpg2py import AbcGraphQuerier, AbcNodeQuerier, AbcEdgeQuerier, Storage
+from typing import Optional
+
+class MyNode(AbcNodeQuerier): pass
+class MyEdge(AbcEdgeQuerier): pass
+
+class MyGraph(AbcGraphQuerier[MyNode, MyEdge]):
+    def node(self, whose_id_is: str) -> Optional[MyNode]:
+        return MyNode(self.storage, whose_id_is)
+    def edge(self, fid: str, tid: str, eid: str) -> Optional[MyEdge]:
+        return MyEdge(self.storage, fid, tid, eid)
+
+g = MyGraph(Storage())
+```
+
+---
+
+Interface specifications (classes, methods, signatures, validation): [docs/design.md](docs/design.md).
+
+---
+
+## License
+MIT.

cpg2py-1.2.0/cpg2py/__init__.py

@@ -0,0 +1,103 @@
+from csv import DictReader
+from pathlib import Path
+from typing import Union
+
+from ._abc import Storage
+from ._abc import AbcEdgeQuerier, AbcGraphQuerier, AbcNodeQuerier
+from ._cpg import CpgEdge, CpgGraph, CpgNode
+from ._exceptions import CPGError, EdgeNotFoundError, NodeNotFoundError, TopFileNotFoundError
+from ._logger import get_logger
+
+logger = get_logger(__name__)
+
+
+def storage_from_json(path: Union[Path, str]) -> Storage:
+    """
+    Creates a Storage instance populated from a JSON file.
+
+    Args:
+        path: Path to JSON file (Path or str).
+
+    Returns:
+        New Storage instance containing the graph.
+
+    Raises:
+        OSError: If the file cannot be read.
+        ValueError: If JSON structure is invalid (missing "nodes" or "edges").
+        KeyError: If an edge object is missing "from", "to", or "type".
+    """
+    storage = Storage()
+    storage.load_json(path)
+    return storage
+
+
+def cpg_graph(node_csv: Path, edge_csv: Path, verbose: bool = False) -> CpgGraph:
+    """
+    Creates a CPG graph from CSV files.
+
+    Args:
+        node_csv: Path to the nodes CSV file
+        edge_csv: Path to the edges CSV file
+        verbose: If True, log warnings for duplicate nodes/edges
+
+    Returns:
+        Graph instance loaded from CSV files
+    """
+    storage = Storage()
+    with open(node_csv, "r", encoding="utf-8") as n_file:
+        reader = DictReader(n_file, delimiter="\t")
+        for node_props in reader:
+            nid = node_props.get("id:int", None)
+            if nid is None:
+                nid = node_props.get("id")
+            if nid is None:
+                continue
+            if not storage.add_node(nid) and verbose:
+                logger.warning("Node %s already exists in the graph", nid)
+            if not storage.set_node_props(nid, node_props) and verbose:
+                logger.warning("Failed to set properties for node %s", nid)
+    with open(edge_csv, "r", encoding="utf-8") as f:
+        reader = DictReader(f, delimiter="\t")
+        for edge_props in reader:
+            f_nid = str(edge_props.get("start", None))
+            if f_nid is None:
+                f_nid = str(edge_props.get("start:str"))
+            t_nid = str(edge_props.get("end", None))
+            if t_nid is None:
+                t_nid = str(edge_props.get("end:str"))
+            e_type = str(edge_props.get("type", None))
+            if e_type is None:
+                e_type = str(edge_props.get("type:str"))
+            edge_id = (f_nid, t_nid, e_type)
+            if not storage.contains_node(edge_id[0]):
+                storage.add_node(edge_id[0])
+                if verbose:
+                    logger.warning("Node %s does not exist", edge_id[0])
+            if not storage.contains_node(edge_id[1]):
+                storage.add_node(edge_id[1])
+                if verbose:
+                    logger.warning("Node %s does not exist", edge_id[1])
+            if not storage.add_edge(edge_id):
+                if verbose:
+                    logger.warning("Edge %s -> %s already exists in the graph", f_nid, t_nid)
+            if not storage.set_edge_props(edge_id, edge_props):
+                if verbose:
+                    logger.warning("Failed to set properties for edge %s", edge_id)
+    return CpgGraph(storage)
+
+
+__all__ = [
+    "cpg_graph",
+    "storage_from_json",
+    "CpgGraph",
+    "CpgNode",
+    "CpgEdge",
+    "AbcGraphQuerier",
+    "AbcNodeQuerier",
+    "AbcEdgeQuerier",
+    "Storage",
+    "CPGError",
+    "NodeNotFoundError",
+    "EdgeNotFoundError",
+    "TopFileNotFoundError",
+]

{cpg2py-1.0.5/cpg2py/abc → cpg2py-1.2.0/cpg2py/_abc}/__init__.py

@@ -1,6 +1,6 @@
+from .edge import AbcEdgeQuerier
 from .graph import AbcGraphQuerier
 from .node import AbcNodeQuerier
-from .edge import AbcEdgeQuerier
 from .storage import Storage
 
-__all__ = ["Storage", "AbcGraphQuerier", "AbcNodeQuerier", "AbcEdgeQuerier"]
+__all__ = ["Storage", "AbcGraphQuerier", "AbcNodeQuerier", "AbcEdgeQuerier"]

cpg2py-1.2.0/cpg2py/_abc/edge.py

@@ -0,0 +1,96 @@
+from __future__ import annotations
+
+import abc
+from typing import Any, Dict, Optional, Tuple
+
+from .._exceptions import EdgeNotFoundError
+from .storage import Storage
+
+
+class AbcEdgeQuerier(abc.ABC):
+    """
+    Abstract base class for edge property access, queries, and updates.
+    """
+
+    def __init__(self, graph: Storage, f_nid: str, t_nid: str, e_type: str) -> None:
+        """
+        Initializes edge querier and validates edge existence.
+
+        Args:
+            graph: Storage instance containing the graph.
+            f_nid: Source node ID.
+            t_nid: Target node ID.
+            e_type: Edge type string.
+
+        Raises:
+            EdgeNotFoundError: If edge does not exist in the graph.
+        """
+        self.__graph: Storage = graph
+        self.__edge_id: Tuple[str, str, str] = (str(f_nid), str(t_nid), str(e_type))
+        if not graph.contains_edge(self.__edge_id):
+            raise EdgeNotFoundError(f_nid, t_nid, e_type)
+
+    @property
+    def edge_id(self) -> Tuple[str, str, str]:
+        """
+        Returns the edge identifier tuple (from_nid, to_nid, edge_type).
+        """
+        return self.__edge_id
+
+    @property
+    def from_nid(self) -> str:
+        """Returns the source node identifier."""
+        return self.__edge_id[0]
+
+    @property
+    def to_nid(self) -> str:
+        """Returns the target node identifier."""
+        return self.__edge_id[1]
+
+    @property
+    def edge_type(self) -> str:
+        """Returns the edge type string."""
+        return self.__edge_id[2]
+
+    @property
+    def properties(self) -> Optional[Dict[str, Any]]:
+        """Returns all edge properties dictionary, or None if not found."""
+        return self.__graph.get_edge_props(self.__edge_id)
+
+    def get_property(self, *prop_names: str) -> Optional[Any]:
+        """
+        Returns first found property value trying multiple name alternatives.
+
+        Args:
+            prop_names: Property name alternatives to try.
+
+        Returns:
+            First found value, or None if none found.
+        """
+        prop_values = (self.__graph.get_edge_prop(self.__edge_id, p_name) for p_name in prop_names)
+        return next((value for value in prop_values if value is not None), None)
+
+    def set_property(self, key: str, value: Any) -> bool:
+        """
+        Sets single edge property value.
+
+        Args:
+            key: Property key.
+            value: Property value.
+
+        Returns:
+            True if property was set, False if edge does not exist.
+        """
+        return self.__graph.set_edge_prop(self.__edge_id, key, value)
+
+    def set_properties(self, props: Dict[str, Any]) -> bool:
+        """
+        Updates multiple edge properties at once.
+
+        Args:
+            props: Dictionary of property key-value pairs.
+
+        Returns:
+            True if properties were updated, False if edge does not exist.
+        """
+        return self.__graph.set_edge_props(self.__edge_id, props)