agentforge-graph 0.3.2__tar.gz → 0.3.3__tar.gz

{agentforge_graph-0.3.2 → agentforge_graph-0.3.3}/CHANGELOG.md

@@ -8,6 +8,23 @@ on a schema mismatch is **rebuild** (ADR-0006).
 
 ## [Unreleased]
 
+## [0.3.3] - 2026-06-19
+
+Leaner base install + the refreshed README on PyPI.
+
+### Changed
+
+- **Dropped the unused `fastembed` dependency** from the base install (and the
+  heavy `onnxruntime` wheel it pulled). It was a leftover from the original
+  feat-005 plan to default to local embeddings; the shipped embedders are
+  `bedrock` / `openai` / `fake`, none of which import it. Removing it keeps the
+  promise true: the base install carries only what the engine actually uses, and
+  every model/DB provider is opt-in. (No local-embeddings driver existed, so
+  nothing is lost; a `fastembed` driver behind a `[local]` extra can be added as
+  a deliberate feature later.)
+- README refreshed (out-of-the-box overview, quick start, demo) now renders on
+  the PyPI project page.
+
 ## [0.3.2] - 2026-06-19
 
 Packaging fix — base-install completeness (caught on a TestPyPI dry run).
@@ -412,7 +429,8 @@ questions over the tools, unattended.
   on a real repo (fixed BUG-008, a default-config `ckg query` break); and parse
   coverage holds at scale (`django`, 2922 files, ~100%, no crash).
 
-[Unreleased]: https://github.com/Scaffoldic/agentforge-grpah/compare/v0.3.2...HEAD
+[Unreleased]: https://github.com/Scaffoldic/agentforge-grpah/compare/v0.3.3...HEAD
+[0.3.3]: https://github.com/Scaffoldic/agentforge-grpah/compare/v0.3.2...v0.3.3
 [0.3.2]: https://github.com/Scaffoldic/agentforge-grpah/compare/v0.3.1...v0.3.2
 [0.3.1]: https://github.com/Scaffoldic/agentforge-grpah/compare/v0.3.0...v0.3.1
 [0.3.0]: https://github.com/Scaffoldic/agentforge-grpah/compare/v0.2.0...v0.3.0

{agentforge_graph-0.3.2 → agentforge_graph-0.3.3}/CONTRIBUTING.md

@@ -11,7 +11,7 @@ first for the system map; this doc is the *how to work on it*.
 Tooling is **`uv`, not pip**.
 
 ```bash
-uv sync --extra engine --extra bedrock     # engine + native wheels + boto3
+uv sync --extra dev --extra bedrock        # engine ships in base; dev = test/lint toolchain
 cp .env.example .env                        # optional: AWS creds for live model tests
 ```
 
@@ -77,10 +77,22 @@ the framework, you're in the wrong layer.
 
 ### Add a framework pack (routes/ORM/DI)
 1. `src/agentforge_graph/frameworks/packs/<name>/`: a `FrameworkPack` with
-   `detect()` (dep manifest + import markers) and `extract()` emitting framework
-   nodes/edges. Mirror `packs/fastapi/`.
-2. Register in `frameworks/registry.py`. Facts are **merged into the
-   `FileSubgraph`** so they ride incremental indexing automatically.
+   `dep_names`/`import_markers` (detection) and `extract()` emitting framework
+   nodes/edges. Pick the closest reference pack: **routes** → `fastapi`/`flask`
+   (Python), `express` (JS/TS), `spring`/`nestjs` (decorator style); **ORM** →
+   `sqlalchemy`/`django`.
+2. **Reuse the shared rails** — don't re-parse from scratch:
+   `packs/_python_ast.py` (Python AST helpers: `iter_class_assignments`,
+   `enclosing_class`, `member_descriptor`, …), `packs/_js_ast.py` (JS/TS), and
+   `frameworks/orm.py` (`ModelIndex` + `relations_to_edges` for `RELATES_TO`).
+3. A pack spanning sibling languages (e.g. JS *and* TS) overrides the
+   `FrameworkPack.slugs` property and builds ids with the file's own slug.
+4. Cross-file stitching (FK/relationship targets, route prefixes) goes in the
+   pass-2 `resolve()` hook — see `sqlalchemy`/`django`. It's globally idempotent.
+5. Register in `frameworks/registry.py`. Pass-1 facts are **merged into the
+   `FileSubgraph`** so they ride incremental indexing automatically; add a golden
+   pack test + an end-to-end test asserting the edge lands on the real symbol node
+   (mirror `tests/frameworks/test_<name>_{pack,integration}.py`).
 
 ### Add a storage backend (Neo4j, SurrealDB, pgvector, …)
 1. Implement `GraphStore` and/or `VectorStore` (`core/contracts.py`).

agentforge_graph-0.3.3/PKG-INFO

@@ -0,0 +1,335 @@
+Metadata-Version: 2.4
+Name: agentforge-graph
+Version: 0.3.3
+Summary: Code Knowledge Graph (CKG) engine + agent toolset, built on AgentForge.
+License-Expression: Apache-2.0
+License-File: LICENSE
+License-File: NOTICE
+Keywords: agent,ckg,code-knowledge-graph,code-search,mcp,rag,static-analysis,tree-sitter
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Documentation
+Classifier: Topic :: Software Development :: Libraries
+Classifier: Typing :: Typed
+Requires-Python: >=3.13
+Requires-Dist: agentforge-anthropic[anthropic]<0.3,>=0.2.4
+Requires-Dist: agentforge-mcp[mcp]<0.3,>=0.2.4
+Requires-Dist: agentforge-py<0.3,>=0.2.4
+Requires-Dist: kuzu>=0.6
+Requires-Dist: lancedb>=0.15
+Requires-Dist: networkx>=3.3
+Requires-Dist: python-dotenv>=1.0
+Requires-Dist: pyyaml>=6
+Requires-Dist: tree-sitter-language-pack>=0.9
+Requires-Dist: tree-sitter>=0.23
+Provides-Extra: bedrock
+Requires-Dist: boto3>=1.35; extra == 'bedrock'
+Provides-Extra: dev
+Requires-Dist: mypy>=1.13; extra == 'dev'
+Requires-Dist: pytest-asyncio>=0.24; extra == 'dev'
+Requires-Dist: pytest-cov>=5; extra == 'dev'
+Requires-Dist: pytest>=8; extra == 'dev'
+Requires-Dist: ruff>=0.7; extra == 'dev'
+Requires-Dist: types-pyyaml>=6; extra == 'dev'
+Provides-Extra: engine
+Provides-Extra: neo4j
+Requires-Dist: neo4j>=5; extra == 'neo4j'
+Provides-Extra: openai
+Requires-Dist: openai>=1.40; extra == 'openai'
+Provides-Extra: otel
+Requires-Dist: agentforge-otel; extra == 'otel'
+Provides-Extra: pgvector
+Requires-Dist: asyncpg>=0.30; extra == 'pgvector'
+Requires-Dist: pgvector>=0.3; extra == 'pgvector'
+Provides-Extra: rerank
+Requires-Dist: sentence-transformers>=2.2; extra == 'rerank'
+Provides-Extra: voyage
+Requires-Dist: agentforge-voyage; extra == 'voyage'
+Description-Content-Type: text/markdown
+
+<!-- AGENTFORGE-MANAGED: template:minimal@0.2.4 hash:c474ab3012ec -->
+<!-- NOTE: customized for agentforge-graph (forked from the scaffold README). -->
+# agentforge-graph
+
+[![CI](https://github.com/Scaffoldic/agentforge-grpah/actions/workflows/ci.yml/badge.svg)](https://github.com/Scaffoldic/agentforge-grpah/actions/workflows/ci.yml)
+[![PyPI](https://img.shields.io/pypi/v/agentforge-graph.svg)](https://pypi.org/project/agentforge-graph/)
+[![Python](https://img.shields.io/badge/python-3.13%2B-blue.svg)](https://pypi.org/project/agentforge-graph/)
+[![License](https://img.shields.io/badge/license-Apache--2.0-green.svg)](LICENSE)
+
+> **Turn any repo into a knowledge graph your coding agent can actually reason
+> over.** Symbols, calls, imports, **API routes, ORM models, dependency
+> injection, architecture decisions, git history, LLM summaries** — one typed,
+> provenance-tracked graph, served over **MCP** in a single command.
+
+Plain code-graph tools answer *"what is connected."* Agents also need *"what is
+this **for**, what decision governs it, what's the API surface, which tables does
+this touch, who calls this, what changed."* **agentforge-graph puts parsed
+structure, framework semantics, architecture decisions, git evolution, and LLM
+enrichment in one graph an agent can traverse — every fact carrying its
+provenance.** Built on [AgentForge](https://pypi.org/project/agentforge-py/).
+
+```bash
+pip install agentforge-graph        # ← the engine is in the box; nothing else to run
+ckg index .                         # repo → typed graph in seconds (no creds, no server)
+ckg serve-mcp --repo .              # → 10 read-only tools for your agent
+```
+
+---
+
+## What you get out of the box
+
+- 🧩 **A typed code graph in one command** — `pip install` → `ckg index .` → files,
+  classes, functions, methods with stable SCIP-style ids and `CONTAINS`/`IMPORTS`/
+  `CALLS`/`INHERITS` edges. **Embedded Kuzu + LanceDB under `.ckg/` — no server, no
+  cloud, no config.** 10 languages: Python, TypeScript, JavaScript, Go, Ruby, PHP,
+  Java, C#, C++, Rust.
+- 🌐 **Framework semantics as graph edges** *(the differentiator)* — routes, ORM
+  models, and DI, not just calls. `ckg routes` is your API surface, `ckg models`
+  your data model, `ckg services` your injection map — across **7 packs**: FastAPI,
+  Flask, SQLAlchemy, Django (Python); Express, NestJS (JS/TS); Spring (Java).
+- 🏛️ **Decisions ↔ code** *(the differentiator)* — ingests ADRs/docs and links them
+  to the code they `GOVERN`. A hit on `payments/` surfaces *"ADR-0012 (accepted):
+  idempotency keys must be client-side"* **before** the agent edits.
+- 🕰️ **Git evolution built in** — `ckg history <symbol>`, `ckg changed-since <ref>`,
+  and `--as-of <commit>` reconstruction. Churn and authorship ride the graph.
+- 🔎 **Hybrid retrieval** — vector search **entry** → typed **graph expansion**. Ask
+  in natural language, get *connected* context: the symbol, its callers, *and* its
+  governing decision.
+- ⚡ **Incremental by default** — re-index only the diff. Edit 3 files in a 5k-file
+  repo → seconds, not minutes. Embeddings/enrichment recompute only what changed.
+- 🤖 **Agent-native** — served read-only over **MCP (10 tools)** or as a native
+  AgentForge toolset. Every response carries a staleness envelope.
+- 🧠 **LLM enrichment, budgeted & opt-in** — design-pattern tags (*"this class is a
+  Repository,"* with confidence + rationale) and bottom-up module summaries, all
+  `llm`-provenance. **CI needs no model calls or cloud creds.**
+
+**Status: 0.3.3 — all 12 planned features shipped.** Published on
+[PyPI](https://pypi.org/project/agentforge-graph/). Each language pack validated on
+a real OSS repo with a creds-enabled embed/retrieval/enrich run; a real agent
+answers questions over the tools unattended. See the [`CHANGELOG`](CHANGELOG.md)
+and [`docs/features/TRACKER.md`](docs/features/TRACKER.md).
+
+---
+
+## Quick start
+
+```bash
+pip install agentforge-graph                # engine included (tree-sitter + kuzu + lancedb)
+
+# 1) index a repo into the graph — incremental on every run after the first
+ckg index .                                 # files/classes/functions/calls (+ ADRs, routes, models…)
+
+# explore the graph — no embeddings, no creds, no server
+ckg map --budget 2000                       # centrality-ranked repo orientation
+ckg routes                                  # API surface: METHOD PATH → handler
+ckg models                                  # ORM data models: table, fields, relations
+ckg services                                # dependency-injection map
+ckg decisions --status accepted             # ADRs and what they govern
+ckg history <symbol-id>                     # when/who/churn for a symbol
+ckg status                                  # indexed commit, staleness, node counts
+```
+
+Add **semantic search** with any embedding provider (AWS Bedrock, OpenAI, or a
+local OpenAI-compatible server — see [Models](#models--pick-a-provider-or-bring-your-own)):
+
+```bash
+pip install "agentforge-graph[bedrock]"     # or [openai]
+ckg embed .                                 # AST chunks → vectors
+ckg query "how are auth tokens validated"   # ranked, *connected* context
+ckg query --symbol "<id>" --mode impact     # reverse deps — "who calls this"
+
+# optional: explicit, budgeted LLM enrichment
+ckg enrich . --all --budget-usd 2           # design-pattern tags + module summaries
+ckg tagged Repository                        # symbols tagged with a design pattern
+```
+
+### See it in action
+
+```text
+$ ckg index .
+indexed 1c2f3a4 · 412 files · 5,290 nodes / 9,133 edges · 3.1s
+
+$ ckg routes
+POST  /payments/{pid}/refund   →  refund()    (app/api.py:42)
+GET   /health                  →  health()    (app/api.py:16)
+
+$ ckg models
+users [users]  (app/models.py:7)
+    fields: id, name, email
+    relations: posts→posts (relationship)
+
+$ ckg query "how are auth tokens validated"
+auth/tokens.py:88  TokenValidator.validate            (cosine 0.71)
+  ← called by  api/middleware.py:23  require_auth
+  ⚖ governed by ADR-0007 (accepted): signing keys must rotate every 90 days
+```
+
+That last block is the whole point: a natural-language question returns the
+symbol, **who calls it**, *and* **the decision that governs it** — connected, with
+provenance.
+
+### Serve it to an agent
+
+Read-only over MCP — **10 tools**: `ckg_repo_map`, `ckg_search`, `ckg_symbol`,
+`ckg_impact`, `ckg_neighbors`, `ckg_status`, `ckg_routes`, `ckg_decisions`,
+`ckg_explain`, `ckg_history`:
+
+```bash
+claude mcp add ckg -- ckg serve-mcp --repo .       # stdio (subprocess)
+ckg serve-mcp --repo . --transport http            # or HTTP → http://127.0.0.1:8765/mcp
+```
+
+Over HTTP, point any MCP client at the URL:
+`{"mcpServers": {"ckg": {"url": "http://127.0.0.1:8765/mcp"}}}`.
+
+Or as a native AgentForge toolset:
+
+```python
+from agentforge import Agent
+from agentforge_graph.serve import code_graph_tools
+
+agent = Agent(model="anthropic:claude-sonnet-4-6", tools=code_graph_tools("."))
+```
+
+→ Full guide (tool schemas, client config, guardrails, staleness envelope):
+[`docs/guides/using-over-mcp.md`](docs/guides/using-over-mcp.md).
+
+---
+
+## What's in the graph
+
+| Capability | What you get |
+|---|---|
+| **Typed code graph** | Files, classes, functions, methods with stable SCIP-style ids; `CONTAINS`/`IMPORTS`/`CALLS`/`INHERITS` edges. Conservative, no-guess resolution across 10 language packs. |
+| **Framework awareness** *(differentiator)* | `Route → HANDLED_BY → handler`, `DataModel → HAS_FIELD`/`RELATES_TO`, `Service → INJECTED_INTO` — across FastAPI, Flask, SQLAlchemy, Django, Express, NestJS, Spring. `ckg routes`/`models`/`services`. |
+| **Decisions ↔ code** *(differentiator)* | ADRs/docs ingested and linked to the code they `GOVERN`; doc prose embedded + searchable. |
+| **Temporal / git evolution** | Per-symbol history, churn, authorship; `changed-since`, `as-of` reconstruction. |
+| **Hybrid retrieval** | Vector entry → typed graph expansion. Connected context, not a flat list. |
+| **LLM enrichment** *(differentiator)* | Budgeted design-pattern tags + bottom-up module summaries — `llm`-provenance, opt-out-able. |
+| **Agent-native** | Read-only MCP (10 tools) or native AgentForge toolset; every response carries a staleness envelope. |
+| **Embedded-first** | Local Kuzu graph + LanceDB vectors under `.ckg/`. No server. Storage + models pluggable. |
+
+---
+
+## Storage — what DB, and can I switch it?
+
+**By default, nothing to run.** The graph lives in an embedded **Kuzu** database
+and the vectors in an embedded **LanceDB** index, both under `.ckg/` in your repo
+(ADR-0006). Zero config, no server.
+
+Storage is **pluggable** behind two contracts — `GraphStore` and `VectorStore`
+([`core/contracts.py`](src/agentforge_graph/core/contracts.py)) — resolved by a
+**driver registry** with entry-point groups
+([`store/registry.py`](src/agentforge_graph/store/registry.py)):
+
+```yaml
+# ckg.yaml
+store:
+  graph:   { driver: kuzu }       # built-in
+  vectors: { driver: lancedb }    # built-in
+```
+
+A server backend (Neo4j, pgvector, …) is an **out-of-tree adapter**: implement the
+contract, pass the reusable `GraphStoreConformance` suite, register an entry point —
+then it's `pip install + one config line`, no core change. Neo4j and pgvector ship
+first-party as opt-in extras. → [`docs/guides/storage-backends.md`](docs/guides/storage-backends.md).
+
+## Models — pick a provider, or bring your own
+
+Every model boundary is an **interface** resolved by a provider registry, so
+switching providers is a `ckg.yaml` line — not a code change:
+
+| Interface | Ships first-party | Select with |
+|---|---|---|
+| `Embedder` | `bedrock` (Cohere `embed-v4`) · `openai` (incl. **local** OpenAI-compatible) · `fake` (CI) | `embed.driver` |
+| `PatternJudge` / `Summarizer` | `bedrock` · `anthropic` (direct API) · `scripted` (CI) | `enrich.provider` |
+
+- **On AWS?** Default `bedrock` (Claude + Cohere) uses your AWS credentials.
+- **Not on AWS?** `enrich.provider: anthropic` (set `ANTHROPIC_API_KEY`) + `embed.driver:
+  openai` (set `OPENAI_API_KEY`) give a full live path, no AWS.
+- **Local / self-hosted?** Point `embed.base_url` at any OpenAI-compatible server
+  (Ollama, vLLM, LM Studio) — same `openai` driver.
+- **Fully offline?** `embed.driver: fake` + `enrich.provider: scripted`.
+
+CI uses the deterministic fakes, so **no model calls or cloud creds are needed to
+build or test**. → [`docs/guides/model-providers.md`](docs/guides/model-providers.md).
+
+---
+
+## Architecture
+
+See **[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)** for the full overview (layer
+diagram, data model, every pipeline in ASCII, extension points). In one breath: a
+`core` of contracts + value types, a **deterministic engine** that never imports the
+framework (parse, store, resolve, embed, retrieve, **frameworks**, knowledge), and a
+thin **framework layer** (`serve` = MCP/Tools, `enrich` = LLM with budget rails) on top.
+
+```
+ckg CLI / MCP server / Agent
+        │
+  serve · enrich            (framework layer — may import agentforge)
+        │
+  ingest · store · chunking · embed · retrieve · repomap · frameworks · knowledge · temporal
+        │                   (deterministic engine — no agentforge)
+  core: contracts · models · SymbolID · provenance · kinds
+        │
+  Kuzu (graph) + LanceDB (vectors)   under .ckg/
+```
+
+---
+
+## Configuration & install extras
+
+Two config files, on purpose:
+
+- **`agentforge.yaml`** — the *framework's* config (agent model, budget, MCP). Strict.
+- **`ckg.yaml`** — *this engine's* config: `store`, `ingest`, `chunking`, `embed`,
+  `retrieve`, `repomap`, `serve`, `frameworks`, `knowledge`, `enrich`, `temporal`.
+  Lenient (unknown keys ignored), so a config written for a later feature still loads.
+
+The base `pip install agentforge-graph` includes the deterministic engine
+(tree-sitter, kuzu, lancedb, networkx). Optional extras add providers/backends:
+
+| Install | Adds |
+|---|---|
+| `pip install agentforge-graph` | base: engine + framework runtime + MCP serving |
+| `…[bedrock]` | `boto3` — Bedrock embeddings + Claude enrichment |
+| `…[openai]` | `openai` — OpenAI / local OpenAI-compatible embeddings |
+| `…[neo4j]` / `…[pgvector]` | opt-in server graph / vector backends |
+| `…[rerank]` | sentence-transformers cross-encoder (off by default) |
+
+The Anthropic-API enrichment path (`enrich.provider: anthropic`) needs no extra —
+the `anthropic` SDK ships with the base install.
+
+---
+
+## Contributing & AI-assisted development
+
+This repo is built to be worked on **with** AI agents. Start here:
+
+- **[`CONTRIBUTING.md`](CONTRIBUTING.md)** — setup, the quality gate, the per-feature
+  development pipeline, and step-by-step playbooks (add a language pack, a framework
+  pack, a storage backend, a model adapter, an MCP tool, an enricher).
+- **[`AGENTS.md`](AGENTS.md)** — read by Claude Code, Cursor, Aider, etc.
+  (the [AGENTS.md convention](https://agents.md)); the invariants an AI assistant must respect.
+- **[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)** — the system map.
+
+## Documentation map
+
+| Doc | What it is |
+|---|---|
+| [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | High-level architecture + every pipeline (ASCII) |
+| [`docs/guides/`](docs/guides/) | Consumer guides: [using over MCP](docs/guides/using-over-mcp.md), [model providers](docs/guides/model-providers.md), [storage backends](docs/guides/storage-backends.md) |
+| [`docs/adr/`](docs/adr/) | 9 architecture decision records (the *why*) |
+| [`docs/features/`](docs/features/) + [`TRACKER.md`](docs/features/TRACKER.md) | 12 feature specs + status board |
+| [`docs/design/`](docs/design/) | Per-feature design docs (the *how*, pre-build) |
+| [`docs/open-source-ckg-research.md`](docs/open-source-ckg-research.md) | The survey that motivates the design |
+
+## License
+
+[**Apache-2.0**](LICENSE) — permissive, with an explicit patent grant and
+patent-retaliation clause. See [`LICENSE`](LICENSE) and [`NOTICE`](NOTICE). Aligns
+with AgentForge, which is also Apache-2.0.