mdbind 0.1.1__tar.gz

mdbind-0.1.1/PKG-INFO

@@ -0,0 +1,9 @@
+Metadata-Version: 2.4
+Name: mdbind
+Version: 0.1.1
+Summary: MdBind — Structured memory in plain Markdown
+Requires-Python: >=3.11
+Requires-Dist: pydantic>=2.0
+Requires-Dist: markdown-it-py>=3.0
+Requires-Dist: typer>=0.12
+Requires-Dist: pyyaml>=6.0

mdbind-0.1.1/README.md

@@ -0,0 +1,212 @@
+<div align="center">
+
+# MdBind
+
+**Structured memory in plain Markdown.**
+
+Transform your Markdown files into a navigable knowledge graph —  
+without databases, embeddings, or proprietary formats.
+
+[![Python](https://img.shields.io/badge/python-3.11%2B-blue?logo=python&logoColor=white)](https://www.python.org/)
+[![Tests](https://img.shields.io/badge/tests-209%20passing-brightgreen?logo=pytest&logoColor=white)](#development)
+[![Version](https://img.shields.io/badge/version-0.1.0-informational)](#installation)
+[![License](https://img.shields.io/badge/license-MIT-lightgrey)](#license)
+
+</div>
+
+---
+
+## What is MdBind?
+
+MdBind turns Markdown files into a **directed knowledge graph** where every section is an addressable node with stable identity, metadata, and explicit relationships.
+
+Your files stay **plain text, Git-friendly, and human-readable** — but gain:
+
+- Graph traversal and dependency resolution
+- Stable URIs that survive reorganization
+- Structured metadata queries
+- AI-oriented context retrieval with bounded token consumption
+
+---
+
+## Why not embeddings?
+
+| Approach | Inspectable | Versionable | Deterministic | Human-readable |
+|---|:---:|:---:|:---:|:---:|
+| Vector databases | ✗ | ✗ | ✗ | ✗ |
+| Proprietary stores | ✗ | partial | partial | ✗ |
+| **MdBind** | ✓ | ✓ | ✓ | ✓ |
+
+Every node, every edge, every relationship is visible in the source file. What an agent reads, a human can audit.
+
+---
+
+## Quick start
+
+```bash
+# 1. Clone and install
+git clone <repo-url> && cd mdbind
+python3 -m venv .venv && source .venv/bin/activate
+pip install -e .
+
+# 2. Point it at your docs
+mdb validate --root docs/
+
+# 3. Query the graph
+mdb get docs/auth.md#auth --json
+```
+
+---
+
+## See it in action
+
+```bash
+# Navigate the dependency tree
+$ mdb tree docs/auth.md#auth --root docs/
+
+auth  [docs/auth.md]
+├── jwt          [include]  docs/security.md#jwt
+└── permissions  [ref]      docs/users.md#permissions
+
+# Compose a unified document by expanding all @include directives
+$ mdb compose docs/auth.md#auth --root docs/ --depth 2
+
+# Find everything that depends on a node (reverse BFS)
+$ mdb impact docs/auth.md#auth --root docs/ --json
+
+# Boolean metadata query
+$ mdb query "tag:api AND NOT status=obsolete" --root docs/ --json
+
+# Bounded context for LLM consumption
+$ mdb context-compose docs/auth.md#auth --root docs/ --depth 2 --token-limit 2000 --json
+```
+
+---
+
+## Syntax
+
+### Declaring a section
+
+A section is a Markdown heading followed immediately by a YAML block with a `section:` field:
+
+````markdown
+## Authentication
+
+```yaml
+section: auth
+title: Authentication
+type: domain
+owner: security-team
+tags: [auth, core]
+```
+
+Authentication is responsible for user identity.
+
+[@include: JWT handling](security.md#jwt)
+
+See also: [@ref: permissions model](users.md#permissions)
+````
+
+- The YAML block must be the **first element** after the heading
+- `section:` is mandatory and must be **unique per repository**
+- Any additional fields are preserved as queryable metadata
+
+### Directives (graph edges)
+
+```markdown
+[@include: label](path/to/file.md#section-id)   <!-- expands inline during compose -->
+[@ref: label](path/to/file.md#section-id)        <!-- records dependency, no expansion -->
+```
+
+Directives are standard Markdown links — they render correctly in any Markdown viewer.
+
+```
+auth
+├── jwt          [include]
+└── permissions  [ref]
+```
+
+---
+
+## Commands
+
+### Quick reference
+
+| Command | Description |
+|---|---|
+| `mdb get <URI>` | Extract a section with full documentary fidelity |
+| `mdb tree <URI>` | Visual dependency hierarchy |
+| `mdb compose <URI>` | Materialize a unified document (expands `@include`) |
+| `mdb validate` | Check integrity: broken refs, cycles, duplicate IDs |
+| `mdb context <URI>` | Metadata + immediate 1-hop neighborhood |
+| `mdb backlinks <URI>` | All sections that reference this URI |
+| `mdb search <predicate>` | Search sections by metadata |
+| `mdb impact <URI>` | All nodes that depend on this URI (reverse BFS) |
+| `mdb neighbors <URI>` | All nodes reachable within N hops |
+| `mdb explain <URI_A> <URI_B>` | All directed paths between two nodes |
+| `mdb diff` | Structural graph diff against a git reference |
+| `mdb query <expression>` | Boolean metadata query (`AND`, `OR`, `NOT`) |
+| `mdb context-compose <URI>` | Bounded materialization for LLM consumption |
+
+All commands accept `--json` for machine-readable output.  
+All outputs are deterministic and JSON-serializable. All URIs are stable across sessions.
+
+### Selected examples
+
+```bash
+# Validate an entire repository
+mdb validate --root docs/ --json
+
+# 1-hop neighborhood of a node
+mdb context docs/auth.md#auth --root docs/ --json
+
+# Find all sections tagged api that are not obsolete
+mdb query "tag:api AND NOT status=obsolete" --root docs/ --json
+
+# Bounded context for LLM — depth 2, max 2000 tokens
+mdb context-compose docs/auth.md#auth --root docs/ --depth 2 --token-limit 2000 --json
+
+# What changed structurally since the last commit?
+mdb diff --root docs/ --since HEAD~1 --json
+```
+
+---
+
+## Philosophy
+
+Five principles behind every design decision:
+
+1. **Markdown is the source of truth** — no proprietary formats, no hidden state
+2. **Knowledge should be inspectable** — every node, every edge, every relationship is visible in the source
+3. **Relationships should be explicit** — `@include` and `@ref` are first-class graph primitives
+4. **Stable identifiers are better than headings** — `file.md#id` survives reorganization
+5. **AI memory should remain human-readable** — what an agent reads, a human can audit
+
+---
+
+## Examples
+
+See the [examples/](examples/) directory for a complete working knowledge base demonstrating sections, directives, composition, and graph traversal.
+
+---
+
+## Development
+
+```bash
+# Install in editable mode
+pip install -e .
+
+# Run the full test suite
+python -m pytest
+
+# Run a specific module
+python -m pytest tests/test_cli_validate.py -v
+```
+
+> 209 tests, 0 failures.
+
+---
+
+## License
+
+MIT

mdbind-0.1.1/pyproject.toml

@@ -0,0 +1,24 @@
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "mdbind"
+version = "0.1.1"
+description = "MdBind — Structured memory in plain Markdown"
+requires-python = ">=3.11"
+dependencies = [
+    "pydantic>=2.0",
+    "markdown-it-py>=3.0",
+    "typer>=0.12",
+    "pyyaml>=6.0",
+]
+
+[project.scripts]
+mdb = "mdbind.cli:app"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

mdbind-0.1.1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

mdbind-0.1.1/src/mdbind/cache.py

@@ -0,0 +1,170 @@
+"""
+Cache persistente do SectionIndex (spec section 7).
+
+Serializa o indice em <root>/.mdgraph/index.json e, em execucoes subsequentes,
+reprocessa apenas os arquivos cujo hash SHA-256 tenha mudado.
+Arquivos removidos tem suas secoes expurgadas automaticamente.
+"""
+from __future__ import annotations
+
+import hashlib
+import json
+from pathlib import Path
+from typing import Dict, Optional
+
+# Versao do esquema do cache; mudar quando o formato mudar de forma incompativel
+_CACHE_VERSION = 1
+_CACHE_DIR = ".mdgraph"
+_CACHE_FILE = "index.json"
+
+
+# ---------------------------------------------------------------------------
+# Hash de arquivo
+# ---------------------------------------------------------------------------
+
+def file_hash(path: Path) -> str:
+    """Retorna o SHA-256 do conteudo do arquivo."""
+    h = hashlib.sha256()
+    h.update(path.read_bytes())
+    return h.hexdigest()
+
+
+# ---------------------------------------------------------------------------
+# Leitura e escrita do cache
+# ---------------------------------------------------------------------------
+
+def _cache_path(root: Path) -> Path:
+    return root / _CACHE_DIR / _CACHE_FILE
+
+
+def load_cache(root: Path) -> Optional[dict]:
+    """
+    Carrega o cache do disco. Retorna None se nao existir ou for invalido.
+    """
+    cp = _cache_path(root)
+    if not cp.exists():
+        return None
+    try:
+        data = json.loads(cp.read_text(encoding="utf-8"))
+        if data.get("version") != _CACHE_VERSION:
+            return None
+        return data
+    except (json.JSONDecodeError, KeyError):
+        return None
+
+
+def save_cache(root: Path, file_hashes: Dict[str, str], sections_data: list) -> None:
+    """
+    Persiste o cache no disco.
+
+    file_hashes: {str(abs_path): sha256}
+    sections_data: lista de dicts serializaveis das ParsedSections
+    """
+    cp = _cache_path(root)
+    cp.parent.mkdir(parents=True, exist_ok=True)
+    payload = {
+        "version": _CACHE_VERSION,
+        "file_hashes": file_hashes,
+        "sections": sections_data,
+    }
+    cp.write_text(json.dumps(payload, ensure_ascii=False, indent=2), encoding="utf-8")
+
+
+# ---------------------------------------------------------------------------
+# Serializacao / desserializacao de ParsedSection
+# ---------------------------------------------------------------------------
+
+def serialize_section(section) -> dict:
+    """Converte ParsedSection em dict JSON-serializavel."""
+    return {
+        "uri": section.uri,
+        "file_path": section.file_path,
+        "metadata": section.metadata,
+        "directives": [{"type": d.type, "target_uri": d.target_uri}
+                       for d in section.directives],
+        "raw": {
+            "heading_level": section.raw.heading_level,
+            "heading_text": section.raw.heading_text,
+            "token_start": section.raw.token_start,
+            "token_end": section.raw.token_end,
+            "source_start_line": section.raw.source_start_line,
+            "source_end_line": section.raw.source_end_line,
+        },
+    }
+
+
+def deserialize_section(data: dict):
+    """Reconstroi ParsedSection a partir de dict do cache."""
+    from mdbind.models import Directive, ParsedSection, RawSection
+
+    raw = RawSection(**data["raw"])
+    directives = [Directive(type=d["type"], target_uri=d["target_uri"])
+                  for d in data.get("directives", [])]
+    return ParsedSection(
+        raw=raw,
+        uri=data["uri"],
+        file_path=data["file_path"],
+        metadata=data["metadata"],
+        directives=directives,
+    )
+
+
+# ---------------------------------------------------------------------------
+# Logica incremental
+# ---------------------------------------------------------------------------
+
+def build_index_with_cache(
+    root: Path,
+    md_files: list[Path],
+    no_cache: bool = False,
+) -> tuple[list, Dict[str, str]]:
+    """
+    Retorna (sections_list, file_hashes) usando cache quando possivel.
+
+    sections_list: lista de ParsedSection prontas para popular o SectionIndex
+    file_hashes: hashes atuais de todos os arquivos processados
+    """
+    from mdbind.parser import parse_file
+
+    current_hashes: Dict[str, str] = {str(f): file_hash(f) for f in md_files}
+
+    # Sem cache ou --no-cache: reprocessar tudo
+    cached = None if no_cache else load_cache(root)
+
+    if cached is None:
+        sections = _parse_all(md_files, parse_file)
+        return sections, current_hashes
+
+    cached_hashes: Dict[str, str] = cached.get("file_hashes", {})
+    cached_sections_data: list = cached.get("sections", [])
+
+    # Agrupar secoes cacheadas por arquivo
+    cached_by_file: Dict[str, list] = {}
+    for s_data in cached_sections_data:
+        fp = s_data["file_path"]
+        cached_by_file.setdefault(fp, []).append(s_data)
+
+    sections: list = []
+    current_file_strs = {str(f) for f in md_files}
+
+    for f in md_files:
+        fs = str(f)
+        if cached_hashes.get(fs) == current_hashes[fs]:
+            # Cache hit: restaurar secoes do disco
+            for s_data in cached_by_file.get(fs, []):
+                sections.append(deserialize_section(s_data))
+        else:
+            # Cache miss: reparsar arquivo modificado
+            sections.extend(parse_file(f))
+
+    # Arquivos removidos: secoes de arquivos que nao existem mais sao ignoradas
+    # (nao adicionamos ao sections, portanto nao aparecem no indice)
+
+    return sections, current_hashes
+
+
+def _parse_all(md_files: list[Path], parse_file) -> list:
+    sections = []
+    for f in md_files:
+        sections.extend(parse_file(f))
+    return sections