know-do-graph 0.1.0__py3-none-any.whl

know_do_graph-0.1.0.dist-info/METADATA

@@ -0,0 +1,765 @@
+Metadata-Version: 2.4
+Name: know-do-graph
+Version: 0.1.0
+Summary: A wiki-native, agent-oriented infrastructure for executable knowledge, operational memory, and capability discovery.
+Requires-Python: >=3.10
+Requires-Dist: duckduckgo-search>=6.0
+Requires-Dist: fastapi>=0.100
+Requires-Dist: httpx>=0.25
+Requires-Dist: networkx>=3.0
+Requires-Dist: openai>=1.0
+Requires-Dist: prompt-toolkit>=3.0
+Requires-Dist: pydantic>=2.0
+Requires-Dist: python-dotenv>=1.0
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: rich>=13.0
+Requires-Dist: sqlalchemy>=2.0
+Requires-Dist: typer[all]
+Requires-Dist: uvicorn[standard]
+Provides-Extra: embeddings
+Requires-Dist: numpy>=1.24; extra == 'embeddings'
+Requires-Dist: sentence-transformers>=2.7; extra == 'embeddings'
+Requires-Dist: sqlite-vec>=0.1.6; extra == 'embeddings'
+Description-Content-Type: text/markdown
+
+# Know-Do Graph
+
+A wiki-native, agent-oriented infrastructure for **executable knowledge**, **operational memory**, and **capability discovery**.
+
+Entries are the primary object — wiki pages that agents can read, traverse, and evolve.  
+The graph emerges naturally from `[[wikilink]]` references between entries.
+
+---
+
+## Quick start
+
+### Install from PyPI
+
+```bash
+pip install know-do-graph
+
+# Create an empty ./data/know_do_graph.db
+know-do-graph init
+
+# Or start from the database bundled with the package
+know-do-graph init --starter
+
+know-do-graph serve
+```
+
+The starter database is copied into the working location; the installed package
+is never used as the writable database. Existing databases are not replaced
+unless `--force` is explicitly provided.
+
+To choose another database path, set `KDG_DB_PATH` in the environment or in a
+`.env` file in the directory where the command is run:
+
+```bash
+KDG_DB_PATH=./my-data/my-memory.db
+```
+
+Relative `KDG_DB_PATH` values are resolved from the current working directory.
+
+### Install from source
+
+```bash
+# 1. Create and activate a virtual environment
+python -m venv .venv
+.venv\Scripts\activate        # Windows
+source .venv/bin/activate     # macOS / Linux
+
+# 2. Install everything (Python deps + Vite frontend build)
+bash install.sh
+
+# 3. Seed example entries (optional)
+python examples/example_entries.py
+
+# 4. Start the API server
+python main.py serve
+# → http://127.0.0.1:8000
+# → http://127.0.0.1:8000/ui   (graph browser)
+# → http://127.0.0.1:8000/docs (interactive Swagger UI)
+```
+
+> **Manual frontend build** (if you prefer not to use `install.sh`):
+> ```bash
+> cd frontend && npm install && npm run build && cd ..
+> ```
+> Re-run whenever you edit files under `frontend/src/` or `frontend/styles/`.
+
+### Frontend development (hot-reload)
+
+```bash
+# Terminal 1 — API backend
+python main.py serve
+
+# Terminal 2 — Vite dev server with API proxy
+cd frontend && npm run dev
+# → http://localhost:5173  (proxies /entries, /graph, etc. to :8000)
+```
+
+---
+
+## CLI reference
+
+Commands are available via `know-do-graph` after a package installation or
+`python main.py` from a source checkout.
+
+### Database initialization
+
+```bash
+# Create an empty database if one does not exist
+know-do-graph init
+
+# Copy the bundled starter database
+know-do-graph init --starter
+
+# Explicitly replace an existing database with the starter
+know-do-graph init --starter --force
+```
+
+### Entry management
+
+```bash
+# Add an entry
+python main.py entry add "My Tool" \
+  --content "Useful for [[ASE Relaxation]]. See https://example.com" \
+  --type tool \
+  --tags "python,simulation" \
+  --source "https://example.com"
+
+# List entries
+python main.py entry list --limit 50
+
+# Show full entry (by ID or slug)
+python main.py entry show mace-calculator
+python main.py entry show 3e3f0272
+
+# Full-text search
+python main.py entry search "relaxation"
+
+# Delete
+python main.py entry delete <entry-id> --yes
+```
+
+### File extraction
+
+```bash
+# Extract from a single Markdown file
+python main.py extract file notes/my_workflow.md --type workflow --tags "ase,phonon"
+
+# Extract all .md/.txt files from a directory
+python main.py extract file docs/ --type capability
+
+# Skip automatic wikilink resolution
+python main.py extract file notes/ --no-resolve
+```
+
+### Graph inspection
+
+```bash
+python main.py graph stats
+python main.py graph neighbors <entry-id> --depth 2
+python main.py graph export --output data/nodes   # writes YAML files
+```
+
+### Memory (Mem-Graph)
+
+```bash
+# Record a trace manually
+python main.py mem add "MACE calculator worked for bulk Fe relaxation" \
+  --session my-session --tags "success,atomistic"
+
+# List traces for a session
+python main.py mem list --session my-session
+
+# Promote a trace into a full KDG entry
+python main.py mem promote <mem-id> --session my-session --type capability
+```
+
+### Start the server
+
+```bash
+python main.py serve                          # default: 127.0.0.1:8000
+python main.py serve --host 0.0.0.0 --port 9000 --reload
+```
+
+The CLI prints clickable URLs on startup:
+
+```
+Know-Do Graph API  →  http://127.0.0.1:8000
+  Graph UI         →  http://127.0.0.1:8000/ui
+  Swagger          →  http://127.0.0.1:8000/docs
+```
+
+---
+
+## Graph Debugger UI
+
+A built-in browser frontend for visualising and debugging the graph is served at **`/ui`**.
+
+| Feature | Details |
+|---------|---------|
+| Force-directed layout | Nodes sized by degree, coloured by entry type |
+| **Hover** | Tooltip with name, type, slug, refinement status, trust score, tags |
+| **Click** | Side panel with full entry detail: content, wikilinks, all metadata, connected edges |
+| Search & filter | Live search by title/slug; filter by entry type |
+| Labels toggle | Show/hide node title labels |
+| Click edge targets | Jump directly to a connected node from the detail panel |
+
+Open it at `http://127.0.0.1:8000/ui` while the server is running.
+
+---
+
+## API reference
+
+Interactive docs at `http://127.0.0.1:8000/docs` once the server is running.
+
+### Entries
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `GET` | `/entries/` | List entries (paginated) |
+| `GET` | `/entries/search?q=...&tags=...&entry_type=...` | Full-text search |
+| `GET` | `/entries/{id}` | Get entry by ID or slug |
+| `POST` | `/entries/` | Create entry |
+| `PUT` | `/entries/{id}` | Update entry |
+| `DELETE` | `/entries/{id}` | Delete entry |
+| `GET` | `/entries/{id}/related?depth=1&relation=...` | Traverse related entries |
+| `GET` | `/entries/{id}/edges` | All edges incident to an entry |
+| `GET` | `/entries/{id}/download` | Download script content (entries with `script_language` set) |
+| `POST` | `/entries/{id}/feedback` | Record verification feedback (works / bugged / …) |
+
+### Graph
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `GET` | `/graph/stats` | Node/edge counts |
+| `GET` | `/graph/full` | All nodes and edges (used by the UI) |
+| `GET` | `/graph/neighbors/{id}?direction=both` | Immediate neighbors |
+| `GET` | `/graph/subgraph/{id}?depth=2` | Ego-subgraph |
+| `GET` | `/graph/path?source=...&target=...` | All simple paths between two entries |
+
+### Memory (Mem-Graph)
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `GET` | `/mem/sessions` | List all session IDs |
+| `GET` | `/mem/{session}` | List traces for a session |
+| `POST` | `/mem/{session}/add` | Add a plain-text trace |
+| `POST` | `/mem/{session}/ingest/openai` | Ingest OpenAI chat messages |
+| `POST` | `/mem/{session}/ingest/langchain` | Ingest LangChain messages |
+| `POST` | `/mem/{session}/ingest/autogen` | Ingest AutoGen conversation |
+| `POST` | `/mem/{session}/ingest/raw` | Ingest arbitrary JSON |
+| `DELETE` | `/mem/{session}/{mem_id}` | Delete a trace |
+| `POST` | `/mem/{session}/{mem_id}/promote` | Promote trace → KDG entry |
+
+### Progressive retrieval (hierarchical memory)
+
+The graph is organised into four orthogonal **skill levels** so planning
+context stays small and operational details are pulled on demand.
+
+| Level | Stored as | Purpose |
+|-------|-----------|---------|
+| **L1 — Capability** | `entry_type` ∈ {`capability`, `workflow`} | Reusable high-level abilities (planner-facing) |
+| **L2 — Procedure**  | `entry_type` = `procedure` | Executable workflow decomposition |
+| **L3 — Heuristic**  | `entry_type` = `heuristic` | Empirical, conditional guidance (cooling rate ⇒ sp2/sp3 ratio, …) |
+| **L4 — Constraint** | `entry_type` = `constraint` | Known failure modes / instability regions |
+
+`EntryMetadata.skill_level` may override the level explicitly. New typed edges
+wire the layers together:
+
+- `decomposes_to` (L1 → L2)
+- `heuristic_for` (L3 → L1/L2)
+- `constraint_on` (L4 → L1/L2)
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `GET` | `/retrieve/plan?goal=…&k=5&include_l2=true` | L1 (+ L2) candidates for a goal — planner context |
+| `GET` | `/retrieve/heuristics?skill=<id\|slug>&k=5` | L3 heuristics attached to a skill (fallback: semantic) |
+| `GET` | `/retrieve/constraints?skill=<id\|slug>&k=5` | L4 constraints / failure modes (fallback: semantic) |
+| `GET` | `/retrieve/expand/{skill}?stages=heuristics,constraints,decomposition` | Bundle used by verifier / debugging loops |
+
+Recommended flow::
+
+    goal → /retrieve/plan
+         → pick skill, execute
+         → on verifier feedback or uncertainty
+         → /retrieve/heuristics  +  /retrieve/constraints
+         → refinement / debugging
+
+GraphAgent exposes the same staging as tools (`retrieve_plan`,
+`retrieve_heuristics`, `retrieve_constraints`) plus `create_heuristic`,
+`create_constraint`, and `decompose_capability` so it can grow the L3/L4
+layer instead of dumping operational knowledge into capability content.
+
+To migrate an existing graph:
+
+```bash
+python scripts/backfill_skill_levels.py --dry-run   # preview
+python scripts/backfill_skill_levels.py             # apply
+```
+
+---
+
+## Node verification & self-evolution
+
+Every entry carries metadata that lets the graph evolve from raw scraped notes
+into a trusted capability library:
+
+| Field | Purpose |
+|-------|---------|
+| `verification_status` | `unverified` (default) → `self_tested` / `peer_reviewed` / `community_tested` / `bugged` / `deprecated` |
+| `feedback_log` | Append-only list of `{timestamp, agent_id, verdict, note, evidence}` |
+| `needs_generalization` | Set automatically when `create_entry` detects an overly specific title (e.g. `Build H2O`) overlapping an existing generic node |
+| `review_count` / `modify_count` | Incremented by `ReviewAgent` |
+| `trust_score` / `usage_count` | Reserved for downstream ranking |
+
+External agents that **execute** a skill should immediately report the outcome
+via `POST /entries/{id}/feedback` (verdict `works` or `bugged`). The
+`MaintenanceAgent` regularly sweeps for `unverified`, `bugged`, and
+`needs_generalization` entries and proposes fixes; the `GraphAgent` exposes
+`submit_feedback`, `list_by_verification`, and `list_needs_generalization`
+tools so an LLM can do the same.
+
+### Abstraction guard
+
+`create_entry` runs a heuristic that flags titles containing concrete
+chemical formulas (`H2O`, `TiO2`, `TiO2/SrTiO3`) and any title that overlaps
+an existing one. The new entry is still created, but with
+`metadata.needs_generalization = True` so it surfaces in maintenance sweeps.
+The agent system prompt gives BAD/GOOD examples — prefer
+**`Build molecule from formula`** over **`Build H2O`**, and
+**`Material interface construction`** over **`TiO2/SrTiO3 Interface`**.
+
+`build_material_interface_workflow` is now deprecated for this reason and
+returns an error explaining the generic alternative.
+
+---
+
+## Remote agent access
+
+The server exposes a dedicated `/remote` interface so that agents running on
+other machines can discover, query, and interact with the graph over plain HTTP
+— no special client library required.
+
+### Discovery: the instruction sheet
+
+When any client hits the server root (or `/remote`), it receives a plain-text
+instruction sheet explaining every available endpoint, request formats, and
+example `curl` commands:
+
+```bash
+curl http://<host>:<port>/
+# or
+curl http://<host>:<port>/remote
+```
+
+### Remote agent endpoints
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `GET` | `/` | Instruction sheet (plain text) |
+| `GET` | `/remote` | Same instruction sheet |
+| `POST` | `/remote/chat` | Chat with the orchestrator agent (read-only; agents and humans) |
+| `GET` | `/remote/search` | Search entries (`?q=&tags=&entry_type=&limit=`) |
+| `GET` | `/remote/graph` | Graph stats + full node/edge dump |
+| `GET` | `/remote/entry/{id}` | Entry by ID, slug, or alias |
+| `GET` | `/remote/entry/{id}/related` | Related entries via BFS (`?depth=1&relation=`) |
+| `POST` | `/remote/feedback` | Free-form feedback trace; optionally also updates an entry's verification (pass `entry_id` + `verdict`) |
+| `POST` | `/entries/{id}/feedback` | Direct per-entry verification feedback |
+| `DELETE` | `/remote/session/{id}` | Clear a session's chat history |
+| `POST` | `/remote/submit` | Deposit raw knowledge into the inbox (agents and humans) |
+| `GET` | `/remote/inbox` | List pending inbox submissions awaiting distillation (humans) |
+| `POST` | `/remote/distill` | Run graph agent to convert inbox into proper nodes (humans) |
+
+### Chat (one-shot)
+
+```bash
+curl -X POST http://<host>:<port>/remote/chat \
+     -H "Content-Type: application/json" \
+     -d '{"message": "What entries exist in the graph?"}'
+```
+
+```json
+{"response": "The graph currently contains ...", "session_id": "a1b2c3..."}
+```
+
+### Chat (multi-turn)
+
+Pass a stable `session_id` to retain conversation history across calls:
+
+```bash
+# Turn 1
+curl -X POST http://<host>:<port>/remote/chat \
+     -H "Content-Type: application/json" \
+     -d '{"message": "List all procedure entries", "session_id": "agent-42"}'
+
+# Turn 2 — the server remembers the context from turn 1
+curl -X POST http://<host>:<port>/remote/chat \
+     -H "Content-Type: application/json" \
+     -d '{"message": "Now show the dependencies of the first one", "session_id": "agent-42"}'
+
+# Clear history when done
+curl -X DELETE http://<host>:<port>/remote/session/agent-42
+```
+
+### Search
+
+```bash
+# Free-text search
+curl "http://<host>:<port>/remote/search?q=relaxation&limit=5"
+
+# Filter by type
+curl "http://<host>:<port>/remote/search?entry_type=tool"
+
+# Combined: text + tags
+curl "http://<host>:<port>/remote/search?q=ase&tags=python,simulation"
+```
+
+### Feedback / observations
+
+There are **two complementary feedback channels**:
+
+**(a) Per-entry verification feedback** — updates the entry's
+`verification_status` (one of `unverified`, `self_tested`, `peer_reviewed`,
+`community_tested`, `bugged`, `deprecated`) and appends to its `feedback_log`.
+This is how the graph self-evolves — a node that an external agent has run
+and confirmed working will be trusted higher next time.
+
+```bash
+# Verdicts: works | peer_works | bugged | deprecated | unclear
+curl -X POST http://<host>:<port>/entries/<id-or-slug>/feedback \
+     -H "Content-Type: application/json" \
+     -d '{
+       "verdict": "works",
+       "note": "Ran on H2O, energy converged in 12 steps",
+       "evidence": "log link or excerpt",
+       "agent_id": "matcreator-runner-1"
+     }'
+```
+
+**(b) Free-form session feedback** — stored as a MemGraph trace; can later be
+promoted to a full entry. Optionally also routes to (a) when you pass
+`entry_id` and `verdict`:
+
+```bash
+curl -X POST http://<host>:<port>/remote/feedback \
+     -H "Content-Type: application/json" \
+     -d '{
+       "session_id": "agent-42",
+       "content": "MACE relaxation diverged on Cu surfaces",
+       "tags": ["feedback", "graph-quality"],
+       "entry_id": "mace-relaxation",
+       "verdict": "bugged",
+       "agent_id": "matcreator-runner-1"
+     }'
+```
+
+The `MaintenanceAgent` exposes `list_unverified()`, `list_bugged()`, and
+`list_needs_generalization()` so it can sweep for entries needing attention.
+
+Promote feedback traces to entries via `POST /mem/{session_id}/{mem_id}/promote`.
+
+### Knowledge inbox (submit → review → distill)
+
+External agents — and humans — can deposit raw knowledge into an **inbox** for
+later review and distillation into proper graph nodes.  Nothing touches the
+graph until you explicitly trigger distillation, so you stay in control of what
+gets added.
+
+**Step 1 — Submit** (agents or humans)
+
+```bash
+# Plain-text summary or context dump
+curl -X POST http://<host>:<port>/remote/submit \
+     -H "Content-Type: application/json" \
+     -d '{
+       "title": "MACE geometry optimisation walkthrough",
+       "content": "We used MACE-MP-0 to relax a bulk Fe structure ...",
+       "tags": ["mace", "relaxation"],
+       "agent_id": "matcreator-01"
+     }'
+
+# OpenAI-style conversation transcript
+curl -X POST http://<host>:<port>/remote/submit \
+     -H "Content-Type: application/json" \
+     -d '{
+       "title": "ASE relaxation session",
+       "format": "openai",
+       "messages": [
+         {"role": "user",      "content": "How do I relax a structure with ASE?"},
+         {"role": "assistant", "content": "Use BFGS with an Atoms object ..."}
+       ],
+       "agent_id": "matcreator-01"
+     }'
+```
+
+The submission is stored as a memory trace tagged `pending-distillation` and
+returns the entry `id` for reference.
+
+**Step 2 — Review the inbox** (humans)
+
+```bash
+curl http://<host>:<port>/remote/inbox
+# → list of pending submissions with a 300-char preview each
+
+# Scope to a specific agent's session
+curl "http://<host>:<port>/remote/inbox?session_id=matcreator-01"
+```
+
+**Step 3 — Distill** (humans, when ready)
+
+```bash
+# Process all pending submissions and create graph nodes
+curl -X POST http://<host>:<port>/remote/distill \
+     -H "Content-Type: application/json" \
+     -d '{}'
+
+# Preview what the agent would receive without touching the graph
+curl -X POST http://<host>:<port>/remote/distill \
+     -H "Content-Type: application/json" \
+     -d '{"dry_run": true}'
+
+# Distil only one agent's submissions
+curl -X POST http://<host>:<port>/remote/distill \
+     -H "Content-Type: application/json" \
+     -d '{"session_id": "matcreator-01"}'
+```
+
+The graph agent reads every pending submission, extracts reusable
+capabilities/procedures/tools (following the abstraction rules), and marks the
+inbox entries as promoted so they are not processed again.
+
+### Starting the server for remote access
+
+```bash
+# Expose on all interfaces so other machines can connect:
+python main.py serve --host 0.0.0.0 --port 8000
+
+# With auto-reload during development:
+python main.py serve --host 0.0.0.0 --port 8000 --reload
+```
+
+Set `OPENAI_API_KEY` (and optionally `OPENAI_API_BASE`) before starting if you
+want the `/remote/chat` endpoint to work.
+
+---
+
+## Connecting agent frameworks
+
+MemGraph accepts session data in whichever format the agent framework already
+produces.  Pick the adapter that matches your stack.
+
+### OpenAI / OpenAI-compatible APIs
+
+```python
+from core.memory.memgraph import MemGraph
+
+response = openai_client.chat.completions.create(...)
+messages = [m.model_dump() for m in response.choices[0].message]  # or your history list
+
+mg = MemGraph("my-session")
+mg.ingest_openai_messages(messages, tags=["openai", "physics-qa"])
+```
+
+Or via the API:
+
+```bash
+curl -X POST http://localhost:8000/mem/my-session/ingest/openai \
+  -H "Content-Type: application/json" \
+  -d '{"messages": [{"role":"user","content":"..."},{"role":"assistant","content":"..."}]}'
+```
+
+### LangChain
+
+```python
+from core.memory.memgraph import MemGraph
+
+# chain.memory.chat_memory.messages → list of HumanMessage / AIMessage objects
+mg = MemGraph("langchain-session")
+mg.ingest_langchain_messages(chain.memory.chat_memory.messages)
+```
+
+Objects only need a `.content` attribute (and optionally `.type` / `.role`).
+
+### AutoGen
+
+```python
+from core.memory.memgraph import MemGraph
+
+# groupchat.messages → list of {"name": "...", "content": "...", "role": "..."}
+mg = MemGraph("autogen-session")
+mg.ingest_autogen_messages(groupchat.messages, tags=["autogen", "multi-agent"])
+```
+
+### JSON session dump
+
+```python
+from pathlib import Path
+from core.memory.memgraph import MemGraph
+
+mg = MemGraph("dump-session")
+mg.ingest_file(Path("session_export.json"))
+```
+
+Accepted JSON shapes:
+- A **JSON array** → treated as an OpenAI/AutoGen message list
+- A **JSON object** with a `messages`, `history`, `conversation`, or `turns` key → that list is extracted
+- Anything else → stored as a single serialised trace
+
+### Plain text / log file
+
+```python
+from pathlib import Path
+from core.memory.memgraph import MemGraph
+
+mg = MemGraph("log-session")
+mg.ingest_text_file(Path("agent.log"), chunk_by="paragraph")
+# chunk_by options: "none" | "line" | "paragraph"
+```
+
+### Direct `add()` (any framework)
+
+```python
+mg = MemGraph("custom-session")
+mg.add(
+    "Summarised finding from the session: ...",
+    tags=["finding", "success"],
+    success=True,
+)
+```
+
+---
+
+## Entry format and wikilinks
+
+Entries are wiki-style documents.  Internal `[[wikilinks]]` automatically
+create graph edges when you call `resolve_wikilinks()` or use the
+`--resolve` flag during extraction.
+
+```markdown
+# ASE Relaxation
+
+Geometry optimisation workflow using [[ASE]].
+
+## Prerequisites
+- [[ASE]]
+- A [[MACE Calculator]] or other calculator
+
+## Related
+- [[Phonon Workflow]]
+```
+
+Supported `entry_type` values: `capability`, `procedure`, `workflow`, `tool`,
+`repository`, `environment`, `dependency`, `data`, `analytical`, `memory`, `generic`.
+
+Supported edge `relation` values: `dependency`, `compatible_with`, `alternative_to`,
+`related_workflow`, `generated_from`, `memory_of`, `refinement_of`, `derived_from`,
+`warning_about`, `cited_by`, `wikilink`, `prerequisite`, `replacement`,
+`execution_pathway`, `transformation`, `provenance`, `compatibility`.
+
+---
+
+## Project structure
+
+```
+core/
+  schemas/        Pydantic models — Entry, EntryMetadata, Edge, enums
+  graph/          KnowDoGraph (networkx wrapper) + app_state singleton
+  storage/        SQLAlchemy/SQLite models, DB session, repositories
+  retrieval/      RetrievalEngine — search, traversal
+  extraction/     Wikilink parser, external-ref extractor
+  memory/         MemGraph — session memory traces + ingestion adapters
+
+agents/
+  extraction_agent/   File/text → entries + wikilink resolution
+  maintenance_agent/  Graph rebuild, dangling-edge cleanup, YAML export, promotion
+
+api/
+  main.py             FastAPI application
+  routes/
+    entries.py        CRUD + search + traversal endpoints
+    graph.py          Stats, subgraph, path-finding endpoints
+    mem.py            Mem-Graph ingestion + management endpoints
+    remote.py         Remote agent access + instruction sheet endpoints
+
+data/
+  know_do_graph.db    Default working SQLite database
+  memory/             Per-session JSON memory files
+  nodes/              YAML entry exports (via `graph export`)
+
+examples/
+  example_entries.py  Seed script with 5 cross-linked atomistic entries
+
+main.py             Typer CLI entry point
+requirements.txt    Python dependencies
+```
+
+---
+
+## Mem-Graph → Know-Do Graph promotion
+
+Memory traces are shallow and mutable.  When a trace represents a stable,
+reusable insight, promote it:
+
+```bash
+# CLI
+python main.py mem promote <mem-id> --session my-session --type capability
+
+# API
+curl -X POST http://localhost:8000/mem/my-session/<mem-id>/promote \
+  -H "Content-Type: application/json" \
+  -d '{"entry_type": "capability", "tags": ["promoted"]}'
+```
+
+The promotion pathway:
+
+```
+raw mem trace  →  linked note  →  refined capability entry  →  validated knowledge
+```
+
+---
+
+## Development notes
+
+- The default SQLite database is `./data/know_do_graph.db`, relative to the
+  directory where the process is started.
+- Set `KDG_DB_PATH` to configure a different filename or path.
+- `init` creates an empty database; `init --starter` copies the bundled starter
+  database to the working path.
+- To package the current development database as the next starter, stop the API
+  server and run:
+
+  ```bash
+  ./scripts/build_starter.sh
+  ```
+
+  The script checkpoints `data/know_do_graph.db`, copies it to the tracked
+  release snapshot at `assets/starter.db`, builds the source distribution and
+  wheel into `dist/`, and verifies that the wheel contains the complete starter
+  database. The live database under `data/` is ignored by Git.
+- The in-memory networkx graph is rebuilt from the database on every server startup (or via `MaintenanceAgent.rebuild_graph()`).
+- All timestamps are UTC.
+- Vector indexing and heavyweight graph databases are intentionally deferred — the architecture supports adding them later without structural changes.
+
+---
+
+## Agent web access
+
+The `GraphAgent` has two complementary web tools:
+
+| Tool | How it works | When to use |
+|------|-------------|-------------|
+| `web_search` | DuckDuckGo search API, returns titles + snippets | Discovering URLs, broad topic research |
+| `fetch_url` | HTTP GET via `httpx` (or stdlib fallback), returns up to 20 000 chars of page text | Reading a specific URL the user provides, scraping docs/READMEs |
+
+**`fetch_url` requires `httpx`** (already in `requirements.txt` if you're using the API server).  
+It falls back to `urllib` automatically if `httpx` is not installed.
+
+Example agent usage:
+
+```
+You: fetch https://ase.readthedocs.io/en/latest/ and create a tool entry for ASE
+Agent: [calls fetch_url → reads page → calls create_entry]
+```