semanticembed 0.1.0__tar.gz → 0.2.1__tar.gz

semanticembed-0.2.1/.claude/skills/analyze/SKILL.md

@@ -0,0 +1,95 @@
+---
+name: analyze
+description: Scan a repo for infrastructure files (docker-compose, k8s, terraform, GitHub Actions), extract edges, and run 6D structural risk analysis via SemanticEmbed.
+allowed-tools: Bash Read Glob
+---
+
+# /analyze — Structural Risk Analysis
+
+Scan the current repository (or a given path) for infrastructure files, extract service dependencies, encode them with SemanticEmbed, and report structural risks.
+
+## Steps
+
+1. **Detect infrastructure and code files** in the repo. Look for:
+   - `docker-compose.yml` / `docker-compose.yaml` / `compose.yml` / `compose.yaml`
+   - Kubernetes YAML in `k8s/`, `kubernetes/`, `manifests/`, `deploy/`, or any directory with Service/Deployment resources
+   - `.github/workflows/*.yml` (GitHub Actions)
+   - `*.tf` files (Terraform)
+   - `*.py` files (Python module imports)
+   - `package.json` files (Node.js monorepo workspace dependencies)
+
+2. **Run the extract + encode script** using the Python snippet below. If the user provided a specific path as an argument, pass it to `from_directory()`. Otherwise use the current directory.
+
+3. **Present the results** clearly:
+   - How many edges were found and from which sources
+   - The full structural risk report
+   - The top 5 most critical nodes with their 6D vectors
+   - Specific actionable recommendations based on the risks found
+
+4. **If no infrastructure files are found**, tell the user they can:
+   - Point to a specific file: `/analyze path/to/docker-compose.yml`
+   - Pass edges manually: `se.encode([("A","B"), ("B","C")])`
+   - Use one of the example notebooks
+
+## Python Script
+
+```python
+import semanticembed as se
+
+# Auto-detect and extract edges
+edges, sources = se.extract.from_directory("TARGET_PATH")
+
+if not edges:
+    print("No infrastructure files found.")
+else:
+    print(f"Found {len(edges)} edges from: {sources}")
+    print()
+
+    # Encode
+    result = se.encode(edges)
+
+    # Risk report
+    print(result.table)
+    print()
+    print(se.report(result))
+```
+
+Replace `TARGET_PATH` with the user's argument or `"."` for current directory.
+
+If `se.extract.from_directory` finds nothing but the user pointed to a specific file or directory, use the appropriate parser directly:
+- `.yml`/`.yaml` with `services:` key → `se.extract.from_docker_compose(path)`
+- `.yml`/`.yaml` with `kind:` key → `se.extract.from_kubernetes(path)`
+- `.yml`/`.yaml` with `jobs:` key → `se.extract.from_github_actions(path)`
+- `.tf` files → `se.extract.from_terraform(path)`
+- Python directory → `se.extract.from_python_imports(path)`
+- `package.json` → `se.extract.from_package_json(path)` or `se.extract.from_package_json_workspaces(path)`
+
+## Requirements
+
+- `pip install git+https://github.com/jmurray10/semanticembed-sdk.git pyyaml`
+- Or if published: `pip install semanticembed pyyaml`
+- Free tier: up to 50 nodes (covers most single-service repos)
+
+## Example Output
+
+```
+Found 8 edges from: {'docker-compose': 8}
+
+Node          Depth  Indep   Hier   Thru   Crit    Fan
+------------------------------------------------------
+api           0.333  0.500  0.500  0.500  0.083  0.750
+auth          0.667  0.000  0.500  0.375  0.056  0.333
+frontend      0.000  0.000  0.500  0.250  0.000  1.000
+postgres      1.000  0.500  0.500  0.375  0.000  0.000
+redis         0.667  1.000  1.000  0.250  0.000  0.000
+worker        0.000  1.000  1.000  0.250  0.000  1.000
+
+STRUCTURAL RISK REPORT
+======================
+SINGLE POINT OF FAILURE:
+  - api    | criticality=0.083, independence=0.50
+  - auth   | criticality=0.056, independence=0.00
+
+CONVERGENCE SINK:
+  - postgres | 3 upstream services, no downstream
+```

semanticembed-0.1.0/README.md → semanticembed-0.2.1/PKG-INFO

@@ -1,3 +1,39 @@
+Metadata-Version: 2.4
+Name: semanticembed
+Version: 0.2.1
+Summary: 6D structural intelligence for directed graphs. Six numbers per node. Sub-millisecond.
+Project-URL: Homepage, https://github.com/jmurray10/semanticembed-sdk
+Project-URL: Documentation, https://github.com/jmurray10/semanticembed-sdk
+Project-URL: Repository, https://github.com/jmurray10/semanticembed-sdk
+Author-email: Jeff Murray <jeffmurr@seas.upenn.edu>
+License: Proprietary
+License-File: LICENSE
+Keywords: graph,microservices,observability,risk-detection,structural-analysis
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Libraries
+Requires-Python: >=3.9
+Requires-Dist: httpx>=0.24
+Provides-Extra: agent-claude
+Requires-Dist: claude-agent-sdk; extra == 'agent-claude'
+Requires-Dist: pyyaml>=6.0; extra == 'agent-claude'
+Provides-Extra: agent-gemini
+Requires-Dist: google-genai; extra == 'agent-gemini'
+Requires-Dist: pyyaml>=6.0; extra == 'agent-gemini'
+Provides-Extra: all
+Requires-Dist: claude-agent-sdk; extra == 'all'
+Requires-Dist: google-genai; extra == 'all'
+Requires-Dist: pyyaml>=6.0; extra == 'all'
+Provides-Extra: extract
+Requires-Dist: pyyaml>=6.0; extra == 'extract'
+Description-Content-Type: text/markdown
+
 # SemanticEmbed SDK
 
 **Structural intelligence for directed graphs. Six numbers per node. Sub-millisecond.**
@@ -15,7 +51,7 @@ Observability tools tell you **what broke**. SemanticEmbed tells you **what will
 - **No agents, no instrumentation** -- just an edge list
 - **Sub-millisecond** -- encodes 100+ node graphs in <1ms
 - **Works on any directed graph** -- microservices, AI agent pipelines, data workflows, CI/CD
-- **Mathematically independent axes** -- six dimensions, zero redundancy, each captures structural information no other metric provides
+- **Complementary structural axes** -- six dimensions, each captures risk signals the others cannot
 
 ---
 
@@ -104,7 +140,7 @@ Every node gets six independent structural measurements:
 | **Criticality** | Fraction of end-to-end paths depending on this node | High criticality = SPOF |
 | **Fanout** | Broadcaster (1.0) vs aggregator (0.0) | High fanout = amplification risk |
 
-These six properties are mathematically independent -- knowing any five tells you nothing about the sixth.
+These six properties capture complementary structural information. Each dimension provides risk signals the others cannot.
 
 See [docs/dimensions.md](docs/dimensions.md) for the full reference.
 
@@ -135,6 +171,104 @@ Step-by-step Colab notebooks. Click to open, run in your browser.
 | [05 - AI Agent Pipelines](https://colab.research.google.com/github/jmurray10/semanticembed-sdk/blob/main/notebooks/05_ai_agent_pipelines.ipynb) | AI/LLM agents | Vendor concentration, gateway bottlenecks, guardrail SPOFs |
 | [06 - CI/CD & Data Pipelines](https://colab.research.google.com/github/jmurray10/semanticembed-sdk/blob/main/notebooks/06_cicd_pipelines.ipynb) | CI/CD & ETL | Build graph fragility, pipeline bottlenecks, drift gates |
 | [07 - OpenTelemetry](https://colab.research.google.com/github/jmurray10/semanticembed-sdk/blob/main/notebooks/07_opentelemetry.ipynb) | OTel traces | Extract edges from traces, structural analysis, CI/CD gates |
+| [08 - Qwen Compression](https://colab.research.google.com/github/jmurray10/semanticembed-sdk/blob/main/notebooks/08_qwen_compression.ipynb) | LLM compression | Structural pruning of Qwen2.5-7B, 10% speedup at Grade A |
+
+---
+
+## Extract Edges from Infrastructure
+
+Don't have an edge list? The `extract` module parses common infrastructure files automatically.
+
+```python
+import semanticembed as se
+
+# From Docker Compose
+edges = se.extract.from_docker_compose("docker-compose.yml")
+
+# From Kubernetes manifests
+edges = se.extract.from_kubernetes("k8s/")
+
+# From GitHub Actions workflows
+edges = se.extract.from_github_actions(".github/workflows")
+
+# From Terraform
+edges = se.extract.from_terraform("infra/")
+
+# From Python imports (module dependency graph)
+edges = se.extract.from_python_imports("src/")
+
+# From Node.js monorepo (inter-package dependencies)
+edges = se.extract.from_package_json_workspaces(".")
+
+# Auto-detect everything in a directory
+edges, sources = se.extract.from_directory(".")
+print(f"Found {len(edges)} edges from {sources}")
+
+# Then encode as usual
+result = se.encode(edges)
+print(result.table)
+```
+
+Requires `pyyaml` for YAML parsing: `pip install pyyaml`
+
+---
+
+## LLM-Powered Analysis
+
+Get plain-language explanations and actionable recommendations using your own LLM key.
+
+```python
+import semanticembed as se
+
+result = se.encode(edges)
+
+# One-shot analysis (OpenAI, Anthropic, or local Ollama)
+print(se.explain(result, model="gpt-4o-mini", api_key="sk-..."))
+print(se.explain(result, model="claude-sonnet-4-5", api_key="sk-ant-..."))
+print(se.explain(result, model="ollama/llama3"))  # local, no key needed
+
+# Follow-up questions
+answer = se.ask(result, "What happens if the database goes down?",
+                model="gpt-4o-mini", api_key="sk-...")
+```
+
+The LLM sees only the encoding output (6D vectors, risk report) -- never the algorithm.
+
+---
+
+## Structural Diff
+
+Compare two graph versions in one call:
+
+```python
+changes = se.encode_diff(edges_v1, edges_v2)
+for node, deltas in changes.items():
+    for dim, info in deltas.items():
+        print(f"{node}.{dim}: {info['before']:.3f} -> {info['after']:.3f}")
+```
+
+---
+
+## Agent
+
+An autonomous agent that scans your repo, extracts edges, encodes, and explains results interactively. Choose your LLM backend:
+
+```bash
+# Claude agent
+pip install claude-agent-sdk
+export ANTHROPIC_API_KEY=sk-ant-...
+python -m agent
+
+# Gemini agent
+pip install google-genai
+export GOOGLE_API_KEY=...
+python -m agent.gemini_agent
+
+# Single question
+python -m agent --ask "What is my biggest SPOF?"
+```
+
+The agent has 7 tools: scan, extract (docker-compose, k8s, Python imports), encode, diff, and simulate architecture changes. See [agent/README.md](agent/README.md) for full docs.
 
 ---
 

semanticembed-0.1.0/PKG-INFO → semanticembed-0.2.1/README.md

@@ -1,27 +1,3 @@
-Metadata-Version: 2.4
-Name: semanticembed
-Version: 0.1.0
-Summary: 6D structural intelligence for directed graphs. Six numbers per node. Sub-millisecond.
-Project-URL: Homepage, https://github.com/jmurray10/semanticembed-sdk
-Project-URL: Documentation, https://github.com/jmurray10/semanticembed-sdk
-Project-URL: Repository, https://github.com/jmurray10/semanticembed-sdk
-Author-email: Jeff Murray <jeffmurr@seas.upenn.edu>
-License: Proprietary
-License-File: LICENSE
-Keywords: graph,microservices,observability,risk-detection,structural-analysis
-Classifier: Development Status :: 4 - Beta
-Classifier: Intended Audience :: Developers
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.9
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Classifier: Programming Language :: Python :: 3.13
-Classifier: Topic :: Software Development :: Libraries
-Requires-Python: >=3.9
-Requires-Dist: httpx>=0.24
-Description-Content-Type: text/markdown
-
 # SemanticEmbed SDK
 
 **Structural intelligence for directed graphs. Six numbers per node. Sub-millisecond.**
@@ -39,7 +15,7 @@ Observability tools tell you **what broke**. SemanticEmbed tells you **what will
 - **No agents, no instrumentation** -- just an edge list
 - **Sub-millisecond** -- encodes 100+ node graphs in <1ms
 - **Works on any directed graph** -- microservices, AI agent pipelines, data workflows, CI/CD
-- **Mathematically independent axes** -- six dimensions, zero redundancy, each captures structural information no other metric provides
+- **Complementary structural axes** -- six dimensions, each captures risk signals the others cannot
 
 ---
 
@@ -128,7 +104,7 @@ Every node gets six independent structural measurements:
 | **Criticality** | Fraction of end-to-end paths depending on this node | High criticality = SPOF |
 | **Fanout** | Broadcaster (1.0) vs aggregator (0.0) | High fanout = amplification risk |
 
-These six properties are mathematically independent -- knowing any five tells you nothing about the sixth.
+These six properties capture complementary structural information. Each dimension provides risk signals the others cannot.
 
 See [docs/dimensions.md](docs/dimensions.md) for the full reference.
 
@@ -159,6 +135,104 @@ Step-by-step Colab notebooks. Click to open, run in your browser.
 | [05 - AI Agent Pipelines](https://colab.research.google.com/github/jmurray10/semanticembed-sdk/blob/main/notebooks/05_ai_agent_pipelines.ipynb) | AI/LLM agents | Vendor concentration, gateway bottlenecks, guardrail SPOFs |
 | [06 - CI/CD & Data Pipelines](https://colab.research.google.com/github/jmurray10/semanticembed-sdk/blob/main/notebooks/06_cicd_pipelines.ipynb) | CI/CD & ETL | Build graph fragility, pipeline bottlenecks, drift gates |
 | [07 - OpenTelemetry](https://colab.research.google.com/github/jmurray10/semanticembed-sdk/blob/main/notebooks/07_opentelemetry.ipynb) | OTel traces | Extract edges from traces, structural analysis, CI/CD gates |
+| [08 - Qwen Compression](https://colab.research.google.com/github/jmurray10/semanticembed-sdk/blob/main/notebooks/08_qwen_compression.ipynb) | LLM compression | Structural pruning of Qwen2.5-7B, 10% speedup at Grade A |
+
+---
+
+## Extract Edges from Infrastructure
+
+Don't have an edge list? The `extract` module parses common infrastructure files automatically.
+
+```python
+import semanticembed as se
+
+# From Docker Compose
+edges = se.extract.from_docker_compose("docker-compose.yml")
+
+# From Kubernetes manifests
+edges = se.extract.from_kubernetes("k8s/")
+
+# From GitHub Actions workflows
+edges = se.extract.from_github_actions(".github/workflows")
+
+# From Terraform
+edges = se.extract.from_terraform("infra/")
+
+# From Python imports (module dependency graph)
+edges = se.extract.from_python_imports("src/")
+
+# From Node.js monorepo (inter-package dependencies)
+edges = se.extract.from_package_json_workspaces(".")
+
+# Auto-detect everything in a directory
+edges, sources = se.extract.from_directory(".")
+print(f"Found {len(edges)} edges from {sources}")
+
+# Then encode as usual
+result = se.encode(edges)
+print(result.table)
+```
+
+Requires `pyyaml` for YAML parsing: `pip install pyyaml`
+
+---
+
+## LLM-Powered Analysis
+
+Get plain-language explanations and actionable recommendations using your own LLM key.
+
+```python
+import semanticembed as se
+
+result = se.encode(edges)
+
+# One-shot analysis (OpenAI, Anthropic, or local Ollama)
+print(se.explain(result, model="gpt-4o-mini", api_key="sk-..."))
+print(se.explain(result, model="claude-sonnet-4-5", api_key="sk-ant-..."))
+print(se.explain(result, model="ollama/llama3"))  # local, no key needed
+
+# Follow-up questions
+answer = se.ask(result, "What happens if the database goes down?",
+                model="gpt-4o-mini", api_key="sk-...")
+```
+
+The LLM sees only the encoding output (6D vectors, risk report) -- never the algorithm.
+
+---
+
+## Structural Diff
+
+Compare two graph versions in one call:
+
+```python
+changes = se.encode_diff(edges_v1, edges_v2)
+for node, deltas in changes.items():
+    for dim, info in deltas.items():
+        print(f"{node}.{dim}: {info['before']:.3f} -> {info['after']:.3f}")
+```
+
+---
+
+## Agent
+
+An autonomous agent that scans your repo, extracts edges, encodes, and explains results interactively. Choose your LLM backend:
+
+```bash
+# Claude agent
+pip install claude-agent-sdk
+export ANTHROPIC_API_KEY=sk-ant-...
+python -m agent
+
+# Gemini agent
+pip install google-genai
+export GOOGLE_API_KEY=...
+python -m agent.gemini_agent
+
+# Single question
+python -m agent --ask "What is my biggest SPOF?"
+```
+
+The agent has 7 tools: scan, extract (docker-compose, k8s, Python imports), encode, diff, and simulate architecture changes. See [agent/README.md](agent/README.md) for full docs.
 
 ---
 

semanticembed-0.2.1/agent/README.md

@@ -0,0 +1,172 @@
+# SemanticEmbed Agent
+
+An LLM-powered structural analysis assistant built on the Claude Agent SDK.
+
+The agent autonomously scans your codebase, extracts service dependencies, runs 6D structural encoding, and explains the results in plain language. Ask follow-up questions, simulate architecture changes, and get actionable recommendations.
+
+## How It Works
+
+```
+Your Code  ──→  Agent scans for infra files (docker-compose, k8s, terraform, Python imports)
+                    │
+                    ▼
+              Extracts edges automatically
+                    │
+                    ▼
+              Encodes via SemanticEmbed API (deterministic, server-side)
+                    │
+                    ▼
+              LLM interprets results (your Anthropic key)
+                    │
+                    ▼
+              Plain language analysis + actionable recommendations
+```
+
+The 6D encoding is deterministic and proprietary (runs server-side). The LLM only sees the output vectors and risk report — never the algorithm.
+
+## Install
+
+Two agent options — pick your LLM provider:
+
+### Claude Agent (Anthropic)
+
+```bash
+pip install semanticembed claude-agent-sdk pyyaml
+export ANTHROPIC_API_KEY=sk-ant-...
+```
+
+### Gemini Agent (Google)
+
+```bash
+pip install semanticembed google-genai pyyaml
+export GOOGLE_API_KEY=...
+```
+
+## Usage
+
+### Claude Agent
+
+```bash
+# Interactive — scans your project, explains risks, answers questions
+python -m agent
+
+# Analyze a specific project
+python -m agent /path/to/project
+
+# Single question
+python -m agent --ask "What is my biggest single point of failure?"
+```
+
+### Gemini Agent
+
+```bash
+# Interactive
+python -m agent.gemini_agent
+
+# Analyze a specific project
+python -m agent.gemini_agent /path/to/project
+
+# Single question
+python -m agent.gemini_agent --ask "What happens if the database goes down?"
+
+# Use a different Gemini model
+python -m agent.gemini_agent --model gemini-2.5-pro
+```
+
+Both agents have the same 7 tools and produce equivalent results — just different LLM backends.
+
+The agent will:
+1. Scan for infrastructure files
+2. Extract edges
+3. Encode the graph
+4. Present a structural risk summary
+5. Wait for your questions
+
+## What the Agent Can Do
+
+| Capability | Example |
+|------------|---------|
+| **Auto-scan** | Finds docker-compose, k8s manifests, terraform, Python imports, package.json |
+| **Encode** | Runs 6D structural analysis on extracted edges |
+| **Explain** | "Your api-gateway is on 89% of all paths — if it goes down, everything fails" |
+| **Simulate** | "What if I add a cache between api and database?" → shows structural impact |
+| **Compare** | "Show me the drift between v1 and v2 of this architecture" |
+| **Recommend** | "Add a circuit breaker upstream of order-service to contain cascading failures" |
+
+## Tools
+
+The agent has 7 tools it calls autonomously:
+
+| Tool | What it does |
+|------|-------------|
+| `scan_directory` | Auto-detect and extract edges from all infrastructure file types |
+| `extract_docker_compose` | Parse a specific docker-compose.yml |
+| `extract_kubernetes` | Parse Kubernetes YAML manifests |
+| `extract_python_imports` | Build module dependency graph from Python imports |
+| `encode_graph` | Run 6D structural encoding on an edge list |
+| `encode_and_diff` | Compare two graph versions and show structural drift |
+| `simulate_change` | Test hypothetical architecture changes (add/remove edges) |
+
+## Example Session
+
+```
+$ python -m agent
+
+SemanticEmbed Agent
+========================================
+Scanning: .
+
+Found 8 edges from docker-compose.yml:
+  frontend -> api, frontend -> auth, api -> postgres, api -> redis,
+  api -> auth, auth -> postgres, worker -> redis, worker -> postgres
+
+STRUCTURAL ANALYSIS:
+Your architecture has 3 critical risks:
+
+1. **api** is your biggest SPOF (criticality=0.083, independence=0.50).
+   It sits on most end-to-end paths and has limited redundancy.
+   → Add a fallback API or load balancer.
+
+2. **auth** is a deep bottleneck (depth=0.67, independence=0.00).
+   It's the only authentication path — no fallback.
+   → Consider an auth cache or backup auth provider.
+
+3. **postgres** is a convergence sink (3 upstream services, 0 downstream).
+   If it goes down, api, auth, and worker all fail.
+   → Add a read replica for read-heavy services.
+
+> What if I add redis as a cache between api and postgres?
+
+SIMULATED CHANGE: +1 edge (api -> redis-cache -> postgres)
+
+The change reduces postgres convergence pressure:
+  - postgres throughput: 0.375 -> 0.286 (reads shifted to cache)
+  - api criticality unchanged (still on most paths)
+  - New convergence sink: redis-cache (monitor for saturation)
+
+Recommendation: Good change. Reduces database blast radius for reads.
+Add a circuit breaker on the redis-cache connection so API falls back
+to direct postgres if cache is unavailable.
+
+> quit
+```
+
+## Using Programmatically
+
+```python
+import asyncio
+from agent import run_agent
+
+# Single query
+asyncio.run(run_agent(
+    "What are the structural risks in this project?",
+    path="/path/to/project"
+))
+```
+
+## Privacy
+
+- The 6D encoding runs server-side (SemanticEmbed API) — your topology edges are sent to the encoding server
+- The LLM (Anthropic) sees only the encoding output (6D vectors, risk report) — not your raw code or infrastructure files
+- The encoding algorithm is proprietary — neither you nor the LLM ever sees how it works
+- Use Ollama for fully local LLM inference if needed (modify the model in agent.py)

semanticembed-0.2.1/agent/__init__.py

@@ -0,0 +1,3 @@
+"""SemanticEmbed Agent — LLM-powered structural analysis assistant."""
+
+from .agent import run_agent, interactive, create_server, main

semanticembed-0.2.1/agent/__main__.py

@@ -0,0 +1,4 @@
+"""Allow running the agent with: python -m agent"""
+from .agent import main
+
+main()