@archznn/crewloop-skills 0.1.0 → 0.3.0

package/references/workflow.md

@@ -0,0 +1,64 @@
+# Workflow Reference
+
+Complete workflow for the Loop Engineering Agents team.
+
+---
+
+## Team Roles
+
+| Role | File | Responsibility |
+|------|------|----------------|
+| Orchestrator | `skills/orchestrator/SKILL.md` | Context discovery and routing |
+| Architect | `skills/architect/SKILL.md` | Specs, contracts, architecture |
+| Designer | `skills/designer/SKILL.md` | Visual/UI direction |
+| Engineer | `skills/engineer/SKILL.md` | Implementation and tests |
+| Reviewer | `skills/reviewer/SKILL.md` | Code review and quality gate |
+| Shipper | `skills/shipper/SKILL.md` | Git operations and PR |
+| Security-Guard | `skills/security-guard/SKILL.md` | Deep-dive security review |
+| Accessibility-Auditor | `skills/accessibility-auditor/SKILL.md` | Accessibility and WCAG review |
+
+---
+
+## Flow Diagram
+
+```mermaid
+flowchart TD
+    O["🎯 Orchestrator<br>Discovery & Routing"] --> A["🏗️ Architect<br>Specs & Architecture"]
+    A --> D["🎨 Designer<br>UI/UX Direction"]
+    A --> E["🔧 Engineer<br>Implementation"]
+    D --> E
+    E --> R["🔍 Reviewer<br>Quality Gate"]
+    R --> S["🚀 Shipper<br>Git & PR"]
+    S --> O
+
+    SG["🛡️ Security-Guard<br>Security Review"] -.-> R
+    AA["♿ Accessibility-Auditor<br>Accessibility Review"] -.-> R
+    R -.-> SG
+    R -.-> AA
+    SG --> E
+    AA --> E
+
+    style O fill:#01579b,color:#fff
+    style A fill:#e65100,color:#fff
+    style D fill:#6a1b9a,color:#fff
+    style E fill:#1b5e20,color:#fff
+    style R fill:#b71c1c,color:#fff
+    style S fill:#00695c,color:#fff
+    style SG fill:#b71c1c,color:#fff
+    style AA fill:#6a1b9a,color:#fff
+```
+
+---
+
+## Routing Rules
+
+1. **Orchestrator ALWAYS sends to Architect first** — never directly to Designer or Engineer.
+2. **Architect is the gatekeeper** — creates specs and routes to Designer (UI) or Engineer (code).
+3. **Designer acts BEFORE Engineer** — visual spec before implementation.
+4. **Engineer never does git or review** — routes to Reviewer after BUILD.
+5. **Reviewer is the quality gate** — routes to Shipper if clean, or back to Engineer/Architect if issues are found.
+6. **Security-Guard and Accessibility-Auditor are optional review specialists** — invoked by the Orchestrator or Reviewer when the change involves security-sensitive work or UI accessibility. They report findings back to the Engineer or Reviewer and do not touch git.
+7. **Shipper is the only one who touches git** — commit, branch, push, PR.
+8. **All skills return to Orchestrator** — it is the central hub.
+
+Each skill reads from the second brain at the start of a task and persists outcomes at the end. The Orchestrator initiates memory reads during discovery; the Shipper updates `Journal/loop-engineering-agents.md` with active/archived spec links.

package/servers/obsidian-mcp/README.md

@@ -0,0 +1,82 @@
+# Obsidian MCP Second Brain
+
+Local MCP server that connects the `loop-engineering-agents` skill bundle to an Obsidian vault at `~/.lea`, acting as a second brain / RAG for AI agents.
+
+## Vault Architecture
+
+The vault at `~/.lea` follows a three-layer memory architecture. Agents should read `AGENT.md` on first use and `MEMORY.md` at the start of major tasks. Notes are organized into:
+
+- `memory/` — raw working-memory session logs
+- `Memory/` — durable user profile and preferences
+- `Knowledge/` — long-lived technical guides and decisions
+- `Journal/` — important session logs and dashboards
+- `Notes/` — temporary notes and drafts
+- `_Inbox/` — agent proposals before promotion
+
+See [`references/obsidian-mcp-usage.md`](../../references/obsidian-mcp-usage.md) for the full usage guide.
+
+## Installation
+
+```bash
+cd servers/obsidian-mcp
+python3 -m venv .venv
+source .venv/bin/activate
+pip install -e ".[dev]"
+```
+
+To use local sentence-transformer embeddings (heavier), install:
+
+```bash
+pip install -e ".[embeddings]"
+```
+
+Otherwise the server falls back to a lightweight TF-IDF embedder.
+
+## Configuration in Kimi Code
+
+Add to your `mcpServers` config (usually `~/.kimi-code/config.toml` or via the UI):
+
+```toml
+[mcpServers.obsidian-mcp]
+command = "/path/to/servers/obsidian-mcp/.venv/bin/python"
+args = ["-m", "obsidian_mcp.main"]
+```
+
+Or via JSON:
+
+```json
+{
+  "mcpServers": {
+    "obsidian-mcp": {
+      "command": "/path/to/servers/obsidian-mcp/.venv/bin/python",
+      "args": ["-m", "obsidian_mcp.main"]
+    }
+  }
+}
+```
+
+## MCP Tools
+
+- `read_note` — read a note from the vault
+- `search_notes` — search by text, vector, graph, or hybrid
+- `create_note` — create a new note
+- `update_note` — update or append content to an existing note
+- `delete_note` — delete a note
+- `list_notes` — list notes in the vault
+- `get_related_notes` — get related notes via links and graph traversal
+- `sync_from_bundle` — re-index the bundle and local vault
+- `learn_from_text` — detect concepts/decisions in text and create notes automatically
+
+## First Use
+
+1. Make sure Obsidian is installed.
+2. Create / open the vault at `~/.lea` in Obsidian.
+3. Run `sync_from_bundle` to index the bundle as the initial knowledge base.
+4. Use `search_notes` to query knowledge.
+5. Use `learn_from_text` whenever new concepts or decisions appear in conversation.
+
+## Tests
+
+```bash
+pytest tests/ -q
+```

package/servers/obsidian-mcp/pyproject.toml

@@ -0,0 +1,32 @@
+[project]
+name = "obsidian-mcp"
+version = "0.1.0"
+description = "MCP server connecting loop-engineering-agents to a local Obsidian vault"
+requires-python = ">=3.10"
+dependencies = [
+    "mcp>=1.0.0",
+    "pyyaml>=6.0",
+    "scikit-learn>=1.3.0",
+]
+
+[project.optional-dependencies]
+embeddings = [
+    "sentence-transformers>=2.2.0",
+]
+dev = [
+    "pytest>=7.0.0",
+    "pytest-asyncio>=0.21.0",
+]
+
+[project.scripts]
+obsidian-mcp = "obsidian_mcp.main:main"
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/obsidian_mcp"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

package/servers/obsidian-mcp/src/obsidian_mcp/config.py

@@ -0,0 +1,47 @@
+import os
+from dataclasses import dataclass, field
+from pathlib import Path
+
+
+def _default_bundle_path() -> Path:
+    env_path = os.environ.get("CREWLOOP_BUNDLE_PATH")
+    if env_path:
+        return Path(env_path)
+    return Path(__file__).resolve().parents[4]
+
+
+@dataclass
+class PrivacyConfig:
+    enabled: bool = True
+    block_api_keys: bool = True
+    block_private_keys: bool = True
+    block_env_files: bool = True
+    block_emails: bool = True
+    block_credit_cards: bool = True
+    allowed_strings: list[str] = field(default_factory=list)
+
+
+@dataclass(frozen=True)
+class Config:
+    vault_path: Path = field(default_factory=lambda: Path.home() / ".lea")
+    index_dir: Path = field(default_factory=lambda: Path.home() / ".lea" / ".index")
+    bundle_path: Path = field(default_factory=_default_bundle_path)
+    embedding_model: str = "sentence-transformers/all-MiniLM-L6-v2"
+    chunk_size: int = 512
+    chunk_overlap: int = 64
+    vector_limit: int = 20
+    text_limit: int = 20
+    graph_depth: int = 1
+    sensitive_patterns: list[str] = field(default_factory=list)
+    privacy: PrivacyConfig = field(default_factory=PrivacyConfig)
+
+    def __post_init__(self):
+        object.__setattr__(self, "vault_path", Path(self.vault_path).expanduser().resolve())
+        object.__setattr__(self, "index_dir", Path(self.index_dir).expanduser().resolve())
+        object.__setattr__(self, "bundle_path", Path(self.bundle_path).expanduser().resolve())
+        privacy = self.privacy
+        if isinstance(privacy, dict):
+            privacy = PrivacyConfig(**privacy)
+        elif not isinstance(privacy, PrivacyConfig):
+            privacy = PrivacyConfig()
+        object.__setattr__(self, "privacy", privacy)

package/servers/obsidian-mcp/src/obsidian_mcp/indexer/embeddings.py

@@ -0,0 +1,105 @@
+import hashlib
+import logging
+from typing import Protocol
+
+from sklearn.feature_extraction.text import TfidfVectorizer
+
+logger = logging.getLogger(__name__)
+
+
+class Embedder(Protocol):
+    def encode(self, texts: list[str]) -> list[list[float]]: ...
+    def is_available(self) -> bool: ...
+    def uses_stored_embeddings(self) -> bool: ...
+
+
+class SentenceTransformerEmbedder:
+    def __init__(self, model_name: str):
+        self.model_name = model_name
+        self._model = None
+
+    def _load(self):
+        if self._model is None:
+            try:
+                from sentence_transformers import SentenceTransformer
+                self._model = SentenceTransformer(self.model_name)
+            except Exception as exc:
+                logger.warning("sentence-transformers not available: %s", exc)
+                raise
+
+    def is_available(self) -> bool:
+        try:
+            from sentence_transformers import SentenceTransformer
+            return SentenceTransformer is not None
+        except Exception:
+            return False
+
+    def uses_stored_embeddings(self) -> bool:
+        return True
+
+    def encode(self, texts: list[str]) -> list[list[float]]:
+        self._load()
+        embeddings = self._model.encode(texts, convert_to_numpy=True, show_progress_bar=False)
+        return [e.tolist() for e in embeddings]
+
+
+class TfidfEmbedder:
+    def __init__(self):
+        self._vectorizer = TfidfVectorizer()
+        self._fitted = False
+
+    def is_available(self) -> bool:
+        return True
+
+    def uses_stored_embeddings(self) -> bool:
+        return False
+
+    def encode(self, texts: list[str]) -> list[list[float]]:
+        if not self._fitted:
+            matrix = self._vectorizer.fit_transform(texts)
+            self._fitted = True
+        else:
+            matrix = self._vectorizer.transform(texts)
+        return matrix.toarray().tolist()
+
+
+class EmbedderFactory:
+    @staticmethod
+    def create(model_name: str) -> Embedder:
+        st = SentenceTransformerEmbedder(model_name)
+        if st.is_available():
+            return st
+        logger.warning("falling back to TF-IDF embedder")
+        return TfidfEmbedder()
+
+
+def chunk_text(text: str, chunk_size: int, overlap: int) -> list[tuple[str, int, int]]:
+    if not text:
+        return []
+    chunks = []
+    start = 0
+    text_len = len(text)
+    while start < text_len:
+        end = min(start + chunk_size, text_len)
+        if end < text_len:
+            expanded = end
+            while expanded < text_len and text[expanded] not in ("\n", " "):
+                expanded += 1
+            if expanded < text_len:
+                end = expanded
+        chunk_value = text[start:end].strip()
+        if chunk_value:
+            start_line = text.count("\n", 0, start) + 1
+            end_line = text.count("\n", 0, end) + 1
+            chunks.append((chunk_value, start_line, end_line))
+        if end >= text_len:
+            break
+        next_start = end - overlap
+        if next_start <= start:
+            next_start = end
+        start = next_start
+    return chunks
+
+
+def chunk_id(note_path: str, text: str) -> str:
+    return hashlib.sha256(f"{note_path}:{text}".encode()).hexdigest()[:16]

package/servers/obsidian-mcp/src/obsidian_mcp/indexer/indexer.py

@@ -0,0 +1,79 @@
+import hashlib
+import logging
+
+from obsidian_mcp.config import Config
+from obsidian_mcp.indexer.embeddings import EmbedderFactory, chunk_id, chunk_text
+from obsidian_mcp.indexer.store import IndexStore
+from obsidian_mcp.models import Chunk, GraphEdge, Note
+from obsidian_mcp.vault.repository import VaultRepository
+
+logger = logging.getLogger(__name__)
+
+
+class Indexer:
+    def __init__(self, config: Config, vault: VaultRepository, store: IndexStore | None = None):
+        self.config = config
+        self.vault = vault
+        self.store = store or IndexStore(config.index_dir / "index.db")
+        self.embedder = EmbedderFactory.create(config.embedding_model)
+
+    def _hash(self, text: str) -> str:
+        return hashlib.sha256(text.encode()).hexdigest()[:16]
+
+    def index_note(self, note: Note, force: bool = False):
+        meta = self.store.get_note_meta(note.path)
+        mtime = note.mtime.timestamp() if note.mtime else 0.0
+        content_hash = self._hash(note.content)
+        if not force and meta and meta["mtime"] == mtime and meta["hash"] == content_hash:
+            return
+
+        self.store.delete_chunks_for_note(note.path)
+        self.store.delete_edges_for_note(note.path)
+
+        chunks_data = chunk_text(note.content, self.config.chunk_size, self.config.chunk_overlap)
+        chunks = [
+            Chunk(
+                id=chunk_id(note.path, text),
+                note_path=note.path,
+                text=text,
+                start_line=start,
+                end_line=end,
+            )
+            for text, start, end in chunks_data
+        ]
+
+        if chunks and self.embedder.uses_stored_embeddings():
+            try:
+                embeddings = self.embedder.encode([c.text for c in chunks])
+                for chunk, emb in zip(chunks, embeddings):
+                    chunk.embedding = emb
+            except Exception as exc:
+                logger.warning("embedding failed for %s: %s", note.path, exc)
+
+        self.store.upsert_chunks(chunks)
+
+        edges = [
+            GraphEdge(source=note.path, target=target, relation="links", weight=1.0)
+            for target in note.links
+        ]
+        self.store.upsert_edges(edges)
+        self.store.upsert_note_meta(note.path, mtime, content_hash)
+
+    def index_all(self, force: bool = False):
+        notes = self.vault.read_all()
+        for note in notes:
+            self.index_note(note, force=force)
+
+    def compute_backlinks(self):
+        self.store.delete_backlinks()
+        edges = self.store.get_all_edges()
+        targets = {}
+        for edge in edges:
+            if edge.relation == "links":
+                targets.setdefault(edge.target, []).append(edge.source)
+        backlink_edges = [
+            GraphEdge(source=target, target=source, relation="backlink", weight=0.8)
+            for target, sources in targets.items()
+            for source in sources
+        ]
+        self.store.upsert_edges(backlink_edges)

package/servers/obsidian-mcp/src/obsidian_mcp/indexer/store.py

@@ -0,0 +1,141 @@
+import json
+import sqlite3
+from pathlib import Path
+
+from obsidian_mcp.models import Chunk, GraphEdge
+
+
+class IndexStore:
+    def __init__(self, db_path: Path):
+        self.db_path = db_path
+        self.db_path.parent.mkdir(parents=True, exist_ok=True)
+        self._init_db()
+
+    def _connect(self):
+        return sqlite3.connect(self.db_path)
+
+    def _init_db(self):
+        with self._connect() as conn:
+            conn.execute("""
+                CREATE TABLE IF NOT EXISTS chunks (
+                    id TEXT PRIMARY KEY,
+                    note_path TEXT NOT NULL,
+                    text TEXT NOT NULL,
+                    embedding TEXT,
+                    start_line INTEGER,
+                    end_line INTEGER,
+                    mtime REAL
+                )
+            """)
+            conn.execute("""
+                CREATE TABLE IF NOT EXISTS edges (
+                    source TEXT NOT NULL,
+                    target TEXT NOT NULL,
+                    relation TEXT NOT NULL,
+                    weight REAL,
+                    PRIMARY KEY (source, target, relation)
+                )
+            """)
+            conn.execute("""
+                CREATE TABLE IF NOT EXISTS notes_meta (
+                    path TEXT PRIMARY KEY,
+                    mtime REAL,
+                    hash TEXT
+                )
+            """)
+            conn.commit()
+
+    def get_note_meta(self, path: str) -> dict | None:
+        with self._connect() as conn:
+            row = conn.execute(
+                "SELECT mtime, hash FROM notes_meta WHERE path = ?", (path,)
+            ).fetchone()
+        if row is None:
+            return None
+        return {"mtime": row[0], "hash": row[1]}
+
+    def upsert_note_meta(self, path: str, mtime: float, hash: str):
+        with self._connect() as conn:
+            conn.execute(
+                "INSERT OR REPLACE INTO notes_meta (path, mtime, hash) VALUES (?, ?, ?)",
+                (path, mtime, hash),
+            )
+            conn.commit()
+
+    def delete_chunks_for_note(self, note_path: str):
+        with self._connect() as conn:
+            conn.execute("DELETE FROM chunks WHERE note_path = ?", (note_path,))
+            conn.commit()
+
+    def upsert_chunks(self, chunks: list[Chunk]):
+        with self._connect() as conn:
+            for chunk in chunks:
+                conn.execute(
+                    """
+                    INSERT OR REPLACE INTO chunks
+                    (id, note_path, text, embedding, start_line, end_line)
+                    VALUES (?, ?, ?, ?, ?, ?)
+                    """,
+                    (
+                        chunk.id,
+                        chunk.note_path,
+                        chunk.text,
+                        json.dumps(chunk.embedding) if chunk.embedding is not None else None,
+                        chunk.start_line,
+                        chunk.end_line,
+                    ),
+                )
+            conn.commit()
+
+    def get_all_chunks(self) -> list[Chunk]:
+        with self._connect() as conn:
+            rows = conn.execute(
+                "SELECT id, note_path, text, embedding, start_line, end_line FROM chunks"
+            ).fetchall()
+        return [
+            Chunk(
+                id=r[0],
+                note_path=r[1],
+                text=r[2],
+                embedding=json.loads(r[3]) if r[3] else None,
+                start_line=r[4],
+                end_line=r[5],
+            )
+            for r in rows
+        ]
+
+    def delete_edges_for_note(self, note_path: str):
+        with self._connect() as conn:
+            conn.execute("DELETE FROM edges WHERE source = ?", (note_path,))
+            conn.commit()
+
+    def delete_backlinks(self):
+        with self._connect() as conn:
+            conn.execute("DELETE FROM edges WHERE relation = 'backlink'")
+            conn.commit()
+
+    def upsert_edges(self, edges: list[GraphEdge]):
+        with self._connect() as conn:
+            for edge in edges:
+                conn.execute(
+                    """
+                    INSERT OR REPLACE INTO edges (source, target, relation, weight)
+                    VALUES (?, ?, ?, ?)
+                    """,
+                    (edge.source, edge.target, edge.relation, edge.weight),
+                )
+            conn.commit()
+
+    def get_all_edges(self) -> list[GraphEdge]:
+        with self._connect() as conn:
+            rows = conn.execute(
+                "SELECT source, target, relation, weight FROM edges"
+            ).fetchall()
+        return [GraphEdge(source=r[0], target=r[1], relation=r[2], weight=r[3]) for r in rows]
+
+    def clear(self):
+        with self._connect() as conn:
+            conn.execute("DELETE FROM chunks")
+            conn.execute("DELETE FROM edges")
+            conn.execute("DELETE FROM notes_meta")
+            conn.commit()

package/servers/obsidian-mcp/src/obsidian_mcp/indexer/sync.py

@@ -0,0 +1,37 @@
+import logging
+from pathlib import Path
+
+from obsidian_mcp.config import Config
+from obsidian_mcp.indexer.indexer import Indexer
+from obsidian_mcp.models import Note
+from obsidian_mcp.vault.parser import parse_note
+from obsidian_mcp.vault.repository import VaultRepository
+
+logger = logging.getLogger(__name__)
+
+
+class BundleSync:
+    def __init__(self, config: Config, indexer: Indexer, vault: VaultRepository):
+        self.config = config
+        self.indexer = indexer
+        self.vault = vault
+
+    def _bundle_files(self) -> list[Path]:
+        root = self.config.bundle_path
+        files = []
+        for pattern in ["skills/**/*.md", "references/**/*.md", "README.md", "AGENTS.md"]:
+            files.extend(root.glob(pattern))
+        return files
+
+    def sync(self, force: bool = False) -> dict:
+        indexed = 0
+        for path in self._bundle_files():
+            rel = path.relative_to(self.config.bundle_path).as_posix()
+            try:
+                note = parse_note(rel, path)
+                self.indexer.index_note(note, force=force)
+                indexed += 1
+            except Exception as exc:
+                logger.warning("failed to index bundle file %s: %s", rel, exc)
+        self.indexer.compute_backlinks()
+        return {"indexed_bundle_files": indexed}

package/servers/obsidian-mcp/src/obsidian_mcp/learning/detector.py

@@ -0,0 +1,66 @@
+import hashlib
+import re
+
+from obsidian_mcp.config import Config
+
+
+class Learning:
+    def __init__(self, type: str, title: str, body: str, tags: list[str]):
+        self.type = type
+        self.title = title
+        self.body = body
+        self.tags = tags
+        self.id = hashlib.sha256(f"{type}:{title}".encode()).hexdigest()[:12]
+
+
+class LearningDetector:
+    def __init__(self, config: Config | None = None):
+        self.config = config or Config()
+        self._seen = set()
+
+    def _slug(self, title: str) -> str:
+        return re.sub(r"[^a-z0-9]+", "-", title.lower()).strip("-")
+
+    def detect(self, text: str) -> list[Learning]:
+        findings = []
+        concept = self._extract_concept(text)
+        if concept:
+            findings.append(concept)
+        decision = self._extract_decision(text)
+        if decision:
+            findings.append(decision)
+        return [f for f in findings if f.id not in self._seen]
+
+    def mark_seen(self, learnings: list[Learning]):
+        for learning in learnings:
+            self._seen.add(learning.id)
+
+    def _extract_concept(self, text: str) -> Learning | None:
+        match = re.search(
+            r"(?i)(?:novo\s+)?(?:conceito|concept)\s*[:\-]?\s*([A-Z][A-Za-z0-9\s\-_]{2,60})",
+            text,
+        )
+        if match:
+            title = match.group(1).strip()
+            return Learning(
+                type="concept",
+                title=title,
+                body=text.strip(),
+                tags=["concept", "auto-generated"],
+            )
+        return None
+
+    def _extract_decision(self, text: str) -> Learning | None:
+        match = re.search(
+            r"(?i)(?:decidimos|decision|decis[ãa]o)\s+(?:que|to|by)?\s*[:\-]?\s*(.+?)(?:\.|\n)",
+            text,
+        )
+        if match:
+            title = match.group(1).strip()[:80]
+            return Learning(
+                type="decision",
+                title=title,
+                body=text.strip(),
+                tags=["decision", "auto-generated"],
+            )
+        return None