graphrag-codeproperty-graph 0.1.0__tar.gz

graphrag_codeproperty_graph-0.1.0/.github/workflows/publish.yml

@@ -0,0 +1,18 @@
+name: Publish to PyPI
+on:
+  push:
+    tags: ['v*']
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'
+      - run: pip install build pytest
+      - run: pytest tests/ -v || true
+      - run: python -m build
+      - uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          password: ${{ secrets.PYPI_API_TOKEN }}

graphrag_codeproperty_graph-0.1.0/.gitignore

@@ -0,0 +1,5 @@
+.venv/
+dist/
+*.egg-info/
+__pycache__/
+.pytest_cache/

graphrag_codeproperty_graph-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Evan Erwee
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

graphrag_codeproperty_graph-0.1.0/NOTICE

@@ -0,0 +1,27 @@
+NOTICE
+
+Code Property Graph
+Copyright (c) 2026 Evan Erwee
+
+This project is licensed under the MIT License.
+
+---
+
+Third-Party Components
+======================
+
+This project builds upon and integrates with the AWS GraphRAG Toolkit
+(lexical-graph), which is licensed under the Apache License, Version 2.0.
+
+  AWS GraphRAG Toolkit (graphrag-toolkit-lexical-graph)
+  Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+  Licensed under the Apache License, Version 2.0
+  https://github.com/awslabs/graphrag-toolkit
+
+You may obtain a copy of the Apache License at:
+  http://www.apache.org/licenses/LICENSE-2.0
+
+---
+
+Our original additions and modifications in this repository are released
+under the MIT License. See the LICENSE file for the full MIT License text.

graphrag_codeproperty_graph-0.1.0/PKG-INFO

@@ -0,0 +1,168 @@
+Metadata-Version: 2.4
+Name: graphrag-codeproperty-graph
+Version: 0.1.0
+Summary: Code Property Graphs — Joern/Semgrep extraction, AI vulnerability enrichment, risk scoring
+Project-URL: Repository, https://github.com/RW-Lab/codeproperty-graph
+Author-email: Evan Erwee <evan@erwee.com>
+License-Expression: MIT
+License-File: LICENSE
+License-File: NOTICE
+Keywords: code-analysis,cpg,graph,joern,neptune,semgrep,vulnerability
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Security
+Classifier: Topic :: Software Development :: Quality Assurance
+Requires-Python: >=3.11
+Requires-Dist: boto3>=1.26.0
+Requires-Dist: document-graph>=0.1.0
+Provides-Extra: dev
+Requires-Dist: pytest-asyncio>=0.21.0; extra == 'dev'
+Requires-Dist: pytest-cov>=4.0; extra == 'dev'
+Requires-Dist: pytest>=7.0; extra == 'dev'
+Provides-Extra: graphrag
+Requires-Dist: graphrag-toolkit-lexical-graph>=3.18.0; extra == 'graphrag'
+Description-Content-Type: text/markdown
+
+# Code Property Graph
+
+Code Property Graphs — Joern/Semgrep extraction, AI vulnerability enrichment, risk scoring.
+
+> **This package depends on AWS GraphRAG Toolkit (graphrag-toolkit-lexical-graph) for graph storage, vector indexing, and retrieval.**
+
+## Quick Start
+
+### Install
+
+```bash
+pip install codeproperty-graph
+```
+
+### Python API Example
+
+```python
+from codeproperty_graph import DeltaIngestor, CPGNode, CPGEdge, GraphDiff
+
+# Delta ingestion — only writes to Neptune when code actually changed
+ingestor = DeltaIngestor(bucket="graphrag-artifacts-705909755305")
+
+result = await ingestor.ingest(
+    repo="my-service",
+    job_id="build-456",
+    tenant_id="tenant_abc123",
+    nodes_data=joern_nodes,       # Joern CPG export
+    edges_data=joern_edges,       # Joern CPG export
+    nodes_path="s3://bucket/cpg-exports/my-service/build-456/nodes.json",
+    edges_path="s3://bucket/cpg-exports/my-service/build-456/edges.json",
+    graph_store=neptune_store,
+    write_fn=batch_write_function,
+)
+
+# result: {"status": "SKIPPED"} or {"status": "INGESTED", "delta": "+5 -2 ~3 =150"}
+```
+
+### Graph Diff — Compare CPG States
+
+```python
+from codeproperty_graph import GraphDiff, CPGNode
+
+# Compare current vs previous code analysis
+diff = GraphDiff.compare(
+    current_nodes=current_cpg_nodes,
+    previous_nodes=previous_cpg_nodes,
+)
+
+print(f"Added: {len(diff.added)}")
+print(f"Removed: {len(diff.removed)}")
+print(f"Modified: {len(diff.modified)}")
+print(f"Unchanged: {len(diff.unchanged)}")
+```
+
+### Manifest Management
+
+```python
+from codeproperty_graph import ManifestManager
+
+# Track CPG state per repository in S3
+manager = ManifestManager(bucket="graphrag-artifacts-705909755305")
+
+# Save manifest after successful ingestion
+await manager.save(repo="my-service", job_id="build-456", signatures=method_signatures)
+
+# Load previous manifest for diff comparison
+previous = await manager.load(repo="my-service")
+```
+
+## Package Structure
+
+```
+src/codeproperty_graph/
+├── __init__.py           # Public API: CPGNode, CPGEdge, DeltaIngestor, etc.
+├── models.py             # CPGNode, CPGEdge, Manifest — Joern-specific types
+├── graph_diff.py         # Compare two CPG states by method signature
+├── manifest_manager.py   # S3-backed state tracking per repository
+├── delta_ingestor.py     # Skip-or-replace orchestration with tenant purge
+└── tenant_ops.py         # Clean lifecycle management (delete_tenant)
+```
+
+## Integration
+
+### Architecture Stack
+
+```
+┌─────────────────────────────────────────────────────┐
+│         codeproperty-graph (this package)            │
+│  Joern/Semgrep CPG, delta ingestion, risk scoring   │
+├─────────────────────────────────────────────────────┤
+│              document-graph (infra)                  │
+│  Node, Edge, CypherBuilder, PipelineExecutor        │
+│  Multi-tenancy, batch operations                    │
+├─────────────────────────────────────────────────────┤
+│     graphrag-toolkit-lexical-graph (foundation)     │
+│  GraphStore, Neptune writer, AOSS writer            │
+│  Lexical indexing, entity resolution, retrieval     │
+└─────────────────────────────────────────────────────┘
+```
+
+### Delta Logic
+
+1. Joern exports CPG → `nodes.json` + `edges.json`
+2. Extract METHOD node signatures: `{full_name: hash}`
+3. Compare against previous manifest in S3
+4. If identical → **SKIP** (no Neptune writes, saves cost)
+5. If changed → **INGEST** full graph under new tenant, purge old tenant, update manifest
+
+### With Document Graph
+
+Code Property Graph uses document-graph for typed property graph primitives:
+
+```python
+# document-graph provides the graph write infrastructure
+from graphrag_toolkit.document_graph.graph_build.cypher_builder import CypherBuilder
+from graphrag_toolkit.document_graph import Node, Edge
+
+# codeproperty-graph adds CPG-specific semantics on top
+from codeproperty_graph import CPGNode, CPGEdge
+```
+
+### AI Vulnerability Enrichment
+
+After CPG ingestion, enrich with AI-powered vulnerability analysis:
+
+```python
+# Query Neptune for high-risk patterns
+# Score methods by complexity, dependency depth, and known CVE proximity
+# Annotate graph with risk scores for downstream consumption
+```
+
+## Requirements
+
+- Python >= 3.11
+- `document-graph >= 0.1.0`
+- `boto3 >= 1.26.0`
+- Optional: `graphrag-toolkit-lexical-graph >= 3.18.0`
+
+## License
+
+MIT — see [LICENSE](LICENSE) for details.
+
+See [NOTICE](NOTICE) for third-party acknowledgments.

graphrag_codeproperty_graph-0.1.0/README.md

@@ -0,0 +1,143 @@
+# Code Property Graph
+
+Code Property Graphs — Joern/Semgrep extraction, AI vulnerability enrichment, risk scoring.
+
+> **This package depends on AWS GraphRAG Toolkit (graphrag-toolkit-lexical-graph) for graph storage, vector indexing, and retrieval.**
+
+## Quick Start
+
+### Install
+
+```bash
+pip install codeproperty-graph
+```
+
+### Python API Example
+
+```python
+from codeproperty_graph import DeltaIngestor, CPGNode, CPGEdge, GraphDiff
+
+# Delta ingestion — only writes to Neptune when code actually changed
+ingestor = DeltaIngestor(bucket="graphrag-artifacts-705909755305")
+
+result = await ingestor.ingest(
+    repo="my-service",
+    job_id="build-456",
+    tenant_id="tenant_abc123",
+    nodes_data=joern_nodes,       # Joern CPG export
+    edges_data=joern_edges,       # Joern CPG export
+    nodes_path="s3://bucket/cpg-exports/my-service/build-456/nodes.json",
+    edges_path="s3://bucket/cpg-exports/my-service/build-456/edges.json",
+    graph_store=neptune_store,
+    write_fn=batch_write_function,
+)
+
+# result: {"status": "SKIPPED"} or {"status": "INGESTED", "delta": "+5 -2 ~3 =150"}
+```
+
+### Graph Diff — Compare CPG States
+
+```python
+from codeproperty_graph import GraphDiff, CPGNode
+
+# Compare current vs previous code analysis
+diff = GraphDiff.compare(
+    current_nodes=current_cpg_nodes,
+    previous_nodes=previous_cpg_nodes,
+)
+
+print(f"Added: {len(diff.added)}")
+print(f"Removed: {len(diff.removed)}")
+print(f"Modified: {len(diff.modified)}")
+print(f"Unchanged: {len(diff.unchanged)}")
+```
+
+### Manifest Management
+
+```python
+from codeproperty_graph import ManifestManager
+
+# Track CPG state per repository in S3
+manager = ManifestManager(bucket="graphrag-artifacts-705909755305")
+
+# Save manifest after successful ingestion
+await manager.save(repo="my-service", job_id="build-456", signatures=method_signatures)
+
+# Load previous manifest for diff comparison
+previous = await manager.load(repo="my-service")
+```
+
+## Package Structure
+
+```
+src/codeproperty_graph/
+├── __init__.py           # Public API: CPGNode, CPGEdge, DeltaIngestor, etc.
+├── models.py             # CPGNode, CPGEdge, Manifest — Joern-specific types
+├── graph_diff.py         # Compare two CPG states by method signature
+├── manifest_manager.py   # S3-backed state tracking per repository
+├── delta_ingestor.py     # Skip-or-replace orchestration with tenant purge
+└── tenant_ops.py         # Clean lifecycle management (delete_tenant)
+```
+
+## Integration
+
+### Architecture Stack
+
+```
+┌─────────────────────────────────────────────────────┐
+│         codeproperty-graph (this package)            │
+│  Joern/Semgrep CPG, delta ingestion, risk scoring   │
+├─────────────────────────────────────────────────────┤
+│              document-graph (infra)                  │
+│  Node, Edge, CypherBuilder, PipelineExecutor        │
+│  Multi-tenancy, batch operations                    │
+├─────────────────────────────────────────────────────┤
+│     graphrag-toolkit-lexical-graph (foundation)     │
+│  GraphStore, Neptune writer, AOSS writer            │
+│  Lexical indexing, entity resolution, retrieval     │
+└─────────────────────────────────────────────────────┘
+```
+
+### Delta Logic
+
+1. Joern exports CPG → `nodes.json` + `edges.json`
+2. Extract METHOD node signatures: `{full_name: hash}`
+3. Compare against previous manifest in S3
+4. If identical → **SKIP** (no Neptune writes, saves cost)
+5. If changed → **INGEST** full graph under new tenant, purge old tenant, update manifest
+
+### With Document Graph
+
+Code Property Graph uses document-graph for typed property graph primitives:
+
+```python
+# document-graph provides the graph write infrastructure
+from graphrag_toolkit.document_graph.graph_build.cypher_builder import CypherBuilder
+from graphrag_toolkit.document_graph import Node, Edge
+
+# codeproperty-graph adds CPG-specific semantics on top
+from codeproperty_graph import CPGNode, CPGEdge
+```
+
+### AI Vulnerability Enrichment
+
+After CPG ingestion, enrich with AI-powered vulnerability analysis:
+
+```python
+# Query Neptune for high-risk patterns
+# Score methods by complexity, dependency depth, and known CVE proximity
+# Annotate graph with risk scores for downstream consumption
+```
+
+## Requirements
+
+- Python >= 3.11
+- `document-graph >= 0.1.0`
+- `boto3 >= 1.26.0`
+- Optional: `graphrag-toolkit-lexical-graph >= 3.18.0`
+
+## License
+
+MIT — see [LICENSE](LICENSE) for details.
+
+See [NOTICE](NOTICE) for third-party acknowledgments.

graphrag_codeproperty_graph-0.1.0/docs/ARCHITECTURE.md

@@ -0,0 +1,232 @@
+# codeproperty-graph — Dependency Architecture
+
+## Library Stack
+
+```mermaid
+graph TD
+    CPG[codeproperty-graph 0.1.0] --> DG[document-graph 3.0.7]
+    DG --> GRT[graphrag-toolkit / lexical-graph 3.18.5]
+    GRT --> Neptune[(Amazon Neptune)]
+    GRT --> AOSS[(OpenSearch Serverless)]
+    CPG --> S3[(Amazon S3)]
+    CPG --> Boto3[boto3]
+    DG --> Boto3
+    DG --> Pydantic
+    DG --> NetworkX
+    DG --> Pandas
+```
+
+## HELPER Services That Use This Stack
+
+```mermaid
+graph LR
+    subgraph "ECS Fargate — ccms-cluster"
+        GH[GraphRAG Helper<br/>graphrag.ccms.local:8080]
+        JH[Joern CPG Helper<br/>joern-cpg-sidecar]
+    end
+
+    subgraph "Libraries (installed in containers)"
+        CPG[codeproperty-graph]
+        DG[document-graph]
+        LG[lexical-graph]
+    end
+
+    subgraph "Storage"
+        Neptune[(Neptune)]
+        S3[(S3: graphrag-artifacts)]
+    end
+
+    GH -->|imports| CPG
+    GH -->|imports| DG
+    GH -->|imports| LG
+    JH -->|imports| CPG
+
+    GH -->|writes nodes/edges| Neptune
+    GH -->|reads/writes manifests| S3
+    JH -->|writes CPG JSON| S3
+```
+
+## A2A Agents (AgentCore) and Their Dependencies
+
+```mermaid
+graph TD
+    subgraph "AgentCore Runtime (A2A Protocol)"
+        CPA[cpg-agent<br/>cpgagent_cpg_agent]
+        NIA[neptune-ingest-agent<br/>neptuneingestagent]
+        GQA[graph-query-agent<br/>graphqueryagent]
+        GIA[graph-index-agent<br/>graphindexagent]
+    end
+
+    subgraph "HELPER Services (backends)"
+        JH[Joern CPG Helper]
+        GH[GraphRAG Helper]
+    end
+
+    subgraph "Libraries"
+        CPG[codeproperty-graph<br/><i>DeltaIngestor, ManifestManager<br/>GraphDiff, tenant_ops</i>]
+        DG[document-graph<br/><i>Node, Edge, batch_*_unwind<br/>CypherBuilder</i>]
+        LG[lexical-graph<br/><i>GraphStore, NeptuneWriter<br/>LexicalGraphIndex</i>]
+    end
+
+    subgraph "Storage"
+        S3[(S3<br/>cpg-exports/REPO/manifest.json<br/>nodes.json, edges.json)]
+        Neptune[(Neptune<br/>CPG nodes + edges<br/>tenant-scoped)]
+    end
+
+    CPA -->|delegates to| JH
+    NIA -->|delegates to| GH
+    GQA -->|delegates to| GH
+    GIA -->|delegates to| GH
+
+    JH -->|uses| CPG
+    JH -->|writes| S3
+    GH -->|uses| CPG
+    GH -->|uses| DG
+    GH -->|uses| LG
+    GH -->|writes| Neptune
+    GH -->|reads/writes| S3
+
+    CPG -.->|depends on| DG
+    DG -.->|depends on| LG
+```
+
+## Delta Ingest Flow (P4)
+
+```mermaid
+sequenceDiagram
+    participant User
+    participant RUN as AI-RUN (workflow engine)
+    participant CPA as cpg-agent (A2A)
+    participant JH as Joern Helper
+    participant S3
+    participant NIA as neptune-ingest-agent (A2A)
+    participant GH as GraphRAG Helper
+    participant Neptune
+
+    User->>RUN: Run cpg-pipeline (repo=amigo-core.zip)
+    RUN->>CPA: analyze_repo(repo_s3_uri)
+    CPA->>JH: /export
+    JH->>JH: joern-parse → joern-export → parse CSVs
+    JH->>S3: Write nodes.json + edges.json
+    JH->>JH: Compute method signatures (FULL_NAME + HASH)
+    JH->>S3: Write manifest.json (signature, job_id, method_sigs)
+    JH-->>CPA: output: s3://bucket/cpg-exports/amigo-core/JOB_ID/nodes.json
+    CPA-->>RUN: step result
+
+    RUN->>NIA: ingest_cpg(source_s3_uri)
+    NIA->>GH: /extract/structured
+    GH->>S3: Read nodes.json
+    GH->>S3: Read manifest.json (previous)
+    
+    alt Signature matches (no code changes)
+        GH-->>NIA: SKIPPED - no_changes
+    else Signature differs (code changed)
+        GH->>Neptune: Write new nodes + edges (new tenant_id)
+        GH->>Neptune: DETACH DELETE old tenant
+        GH->>S3: Update manifest.json
+        GH-->>NIA: INGESTED - delta +2 -1 ~3 =150
+    end
+    NIA-->>RUN: step result
+```
+
+## codeproperty-graph Module Structure
+
+```mermaid
+classDiagram
+    class CPGNode {
+        +id: str
+        +node_type: str
+        +full_name: str
+        +hash: str
+        +filename: str
+        +signature: str
+        +stable_id: str
+    }
+
+    class CPGEdge {
+        +source_id: str
+        +target_id: str
+        +edge_type: str
+        +properties: dict
+        +key: str
+    }
+
+    class Manifest {
+        +repo: str
+        +signature: str
+        +job_id: str
+        +tenant_id: str
+        +method_signatures: dict
+    }
+
+    class GraphDiff {
+        +compare(previous, current): DiffResult
+    }
+
+    class DiffResult {
+        +added: dict
+        +removed: dict
+        +modified: dict
+        +unchanged: int
+        +has_changes: bool
+        +summary: str
+    }
+
+    class ManifestManager {
+        +compute_signature(method_sigs): str
+        +get(repo): Manifest
+        +put(manifest): void
+        +has_changes(repo, sigs): tuple
+    }
+
+    class DeltaIngestor {
+        +ingest(repo, job_id, nodes, edges, ...): dict
+    }
+
+    DeltaIngestor --> ManifestManager
+    DeltaIngestor --> GraphDiff
+    DeltaIngestor --> Manifest
+    GraphDiff --> DiffResult
+    ManifestManager --> Manifest
+
+    note for CPGNode "Identity: full_name\nChange detection: hash\nFrom Joern CPG spec"
+    note for CPGEdge "Types: AST, CFG, CDG,\nREACHING_DEF, CALL,\nARGUMENT, DOMINATE"
+```
+
+## Relationship to Platform
+
+```mermaid
+graph TB
+    subgraph "AI-RUN (cyber-agentic.com)"
+        Builder[Builder Tab]
+        Workflows[Workflows Tab]
+        Agents[Agent Registry]
+    end
+
+    subgraph "AI-GOV (cyber-ai-governance.com)"
+        Assets[AI Assets]
+        Trace[AIGP Trace]
+    end
+
+    subgraph "A2A Agents"
+        CPA[cpg-agent]
+        NIA[neptune-ingest-agent]
+    end
+
+    subgraph "Libraries"
+        CPG[codeproperty-graph]
+    end
+
+    Builder -->|creates workflows using| CPA
+    Builder -->|creates workflows using| NIA
+    Workflows -->|executes| CPA
+    Workflows -->|executes| NIA
+    Agents -->|registers + names| CPA
+    Agents -->|registers + names| NIA
+    CPA -->|uses internally| CPG
+    NIA -->|uses internally| CPG
+    Assets -->|discovers| CPA
+    Assets -->|discovers| NIA
+    Trace -->|traces invocations of| CPA
+    Trace -->|traces invocations of| NIA
+```