kompend 2026.10.4__py3-none-any.whl

kompend/AGENTS.md

@@ -0,0 +1,123 @@
+# DOX — kompend Python source tree
+
+<!-- on entry: loaded when agent enters src/kompend/ -->
+<!-- parent: ../../AGENTS.md | .agents/POLICIES.md, .agents/CODE.md, .agents/CODE.md, .agents/FILES.md, .agents/CHECKS.md -->
+
+## Purpose
+
+Coding conventions and quality standards for the kompend Python source tree at
+`src/kompend/`. Governs how Python code is written and how human-readable text is
+handled. Code policies (antipatterns, boundaries, error handling) are universal
+and live in `.agents/CODE.md`.
+
+## Ownership
+
+- `src/kompend/` — all kompend Python source code
+- `src/kompend/templates/` — all LLM prompt and instruction templates
+- `src/kompend/packs/` — prompt packs (see `packs/AGENTS.md`)
+
+## Local Contracts
+
+### No text in code
+
+**LLM prompts, system instructions, template strings, configuration data, and
+any other human-readable text must NOT be embedded in Python source files.**
+Text in string constants is invisible to the domain experts who review prompts,
+editable only by touching Python, versioned with code that has a different
+review cycle, and unverifiable by tooling.
+
+| Content | Format | Location |
+| ------- | ------ | -------- |
+| LLM system/user prompts | Jinja2 (`.md.j2`) | `templates/<module>/<role>_system.md.j2` |
+| Page kind definitions | TOML | `<module>/kinds.toml` or config |
+| Pack prompts and roles | TOML | `packs/<name>/pack.toml` |
+| Any other human-readable string | TOML / Jinja2 | the appropriate config dir |
+
+**Acceptable exceptions:** short (≤5 words) status messages to `typer.echo()` or
+a logger; error messages that must name code-level constructs (variables,
+types); single words as enum members, dict keys, or field names; docstrings and
+type annotations.
+
+Template layout, loader search order, frontmatter schema, and variable
+validation — see `core/templates/AGENTS.md`. Pack layout — see
+`packs/AGENTS.md`. Known legacy violations — see `.agents/HISTORY.md`.
+
+### Python antipatterns
+
+Antipattern table, code boundaries (config fields, no absolute paths, no
+`TYPE_CHECKING`, banned suppressions, lazy heavy deps), and the error-handling
+policy — **see `.agents/CODE.md`**. They apply to production code and tests
+alike. Two are enforced by script; run both before committing (see
+`.agents/CHECKS.md`).
+
+### Output format — OKF v0.2
+
+kompend emits **OKF v0.2** bundles by default; OKF is a compliance spec, not an
+output format. Conformance rules (frontmatter, trust fields, provenance fields,
+portable cross-links, reserved filenames, page kinds, footnote attribution,
+permissive consumption) — **see `output/AGENTS.md`** and
+`docs/concepts/open-knowledge-format.md`. Generated output is rumdl-clean: there
+is one markdown writer, no exempt variants.
+
+### LLM agents
+
+Every LLM call routes through the PydanticAI agents in `agents/` — class
+hierarchy, `KompendAgentBase` responsibilities, `KompendModel`, deps types, and the
+rules for adding an agent — **see `agents/AGENTS.md`**. Model construction goes
+through `kompend.llm.models.build_model()`; see `llm/AGENTS.md`.
+
+### ACP agent
+
+The Agent Client Protocol adapter (`kompend agent` on stdio) — protocol flow,
+module map, and modification guidance — **see `acp/AGENTS.md`** and
+`docs/integrations/acp-agent.md`.
+
+## Work Guidance
+
+- Python 3.11+ only. Modern typing (`X | None`, `list[str]`).
+- Every module in `src/kompend/` has an `__init__.py` that defines its public API
+  via re-exports. Import from the package root, not the defining submodule.
+- `src/kompend/core/paths.py` is the single source of truth for on-disk artifact
+  paths — see `.agents/FILES.md`.
+- `src/kompend/core/config/` owns all configuration and `DEFAULTS` — see
+  `core/config/AGENTS.md`.
+- Heavy optional dependencies stay lazy so CLI startup stays fast — pattern in
+  `.agents/CODE.md`.
+
+## Verification
+
+Commands — see `.agents/CHECKS.md`. For this subtree the relevant ones are
+`ruff check`, `ty check src/ tests/`, `pytest tests/unit`, and the two AST gates.
+`mise lint` and `mise typecheck` wrap the first two.
+
+## Child DOX Index
+
+| Path | Scope |
+| ---- | ----- |
+| `acp/` | ACP adapter — TOML-defined conversational agents, tool set, provider store |
+| `agents/` | PydanticAI agents for every LLM call; `KompendAgentBase`, `KompendModel`, deps |
+| `capture/` | Capture/distill/learn pipeline for URLs, files, stdin |
+| `chunking/` | Tier 2 — split parsed documents into context-window-sized `ChunkTree` |
+| `cli/` | CLI shell over the core library — Typer commands, argument declarations |
+| `cli/args/` | All CLI argument/option definitions (`Annotated` aliases) |
+| `cli/cmds/` | All CLI command entry points, organized by domain |
+| `compilation/` | Tier 4 — knowledge graph build, WiCER loop, page kinds |
+| `core/` | Foundation — models, pipeline, config, cache, templates, trust, utils |
+| `core/cache/` | Content-addressed JSON cache with dependency validation |
+| `core/config/` | Configuration resolution engine and `DEFAULTS` |
+| `core/models/` | Canonical data models and serialization contracts |
+| `core/pipeline/` | Tier orchestration, pipeline config, tier state, ingest |
+| `core/reporting/` | On-demand report builders for CLI observability |
+| `core/resolution/` | Tier 3 — entity extraction/dedup, cross-reference checking |
+| `core/store/` | The vault SQLite store (`.kompend/kompend.db`) |
+| `core/templates/` | Jinja2 loading, frontmatter parsing, variable validation |
+| `core/trust/` | Actor identity and trust/privacy concerns |
+| `core/utils/` | Pure helpers, zero domain logic |
+| `lint/` | Structural + LLM lint rules, gap analysis, remediation |
+| `llm/` | Model construction, context-window discovery, formats, usage tracking |
+| `mcp/` | MCP server exposing kompend operations as tools (fastmcp) |
+| `memory/` | Query memory store and compile-time lifecycle |
+| `output/` | Tier 5 — OKF markdown/JSON/Typst writers, indexes, log |
+| `preprocessing/parsers/` | Pluggable ingestion backends and their registry |
+| `rag/` | zvec vector store — fields, index build, filters, drift, pruning |
+| `search/` | Multi-retriever semantic search with adaptive selection |

kompend/__init__.py

@@ -0,0 +1,10 @@
+"""kompend package."""
+
+from __future__ import annotations
+
+import importlib.metadata
+
+try:
+    __version__ = importlib.metadata.version(__name__)
+except importlib.metadata.PackageNotFoundError:
+    __version__ = "0.0.0"  # Fallback for development mode

kompend/__main__.py

@@ -0,0 +1,8 @@
+"""Entry point for My Project CLI."""
+
+from __future__ import annotations
+
+from kompend.cli.app import main
+
+if __name__ == "__main__":
+    main()

kompend/acp/AGENTS.md

@@ -0,0 +1,65 @@
+# DOX — kompend.acp
+
+<!-- on entry: loaded when agent enters src/kompend/acp -->
+<!-- parent: ../../../AGENTS.md | .agents/POLICIES.md, .agents/CODE.md, .agents/CODE.md, .agents/FILES.md, .agents/CHECKS.md -->
+
+## Purpose
+
+ACP (Agent Client Protocol) adapter — serve kompend to editors and AI agents over
+stdio (`kompend agent`) with TOML-defined conversational agents. The client injects
+its own LLM endpoint through `providers/set`, so kompend carries no credentials of
+its own. User-facing guide: `docs/integrations/acp-agent.md`.
+
+## Ownership
+
+- `src/kompend/acp/` — agent definitions, registry, tool set, provider store,
+  harness adapter, shipped agent TOML
+
+## Architecture
+
+| Module | Contents |
+| ------ | -------- |
+| `definitions.py` | `AgentDefinition`, `AgentTask`, `AgentLLMConfig`, `AgentPromptConfig` — the TOML schema |
+| `registry.py` | `AgentRegistry` — discovery (built-in / vault / explicit), `active_for` resolution |
+| `tools.py` | Native PydanticAI `FunctionToolset` wrapping the MCP server functions; `base_dir` injected by closure |
+| `harness_adapter.py` | `build_kompend_agent`, `kompend_session_config` — per-session agent factory for `run_acp_stdio_sync` |
+| `provider_store.py` | Thread-safe store for the `providers/set` endpoint; reconfigures env-var-based model resolution |
+| `builtin/*.toml` | Shipped agents: `generic`, `compile`, `query`, `capture` |
+
+Protocol flow: `initialize` (advertises `providers` capability) → `providers/list`
+→ `providers/set` (endpoint stored, model reconfigured) → `session/new` →
+prompts. `/kompend <command>` prompts dispatch the corresponding CLI command; free
+text goes to the active conversational agent. The protocol itself is handled by
+`pydantic-ai-harness` (`run_acp_stdio_sync`), not by this package.
+
+## Local Contracts
+
+- Import: `from kompend.acp.registry import AgentRegistry`,
+  `from kompend.acp.definitions import AgentDefinition`.
+- Agent definitions live in TOML, never in Python source (see `AGENTS.md`:
+  "no text in code"). Built-ins ship in `builtin/*.toml`; user agents go in
+  `<wiki>/.kompend/agents/*.toml`. The active agent per task is stored in
+  `config.toml` under `[agents]`.
+- `providers/set` is an RFD, not stable ACP. When it is never called, model
+  resolution falls back to env vars → `config.toml` → `DEFAULTS`.
+- Sessions are stateless: each prompt turn is independent, with no accumulated
+  conversation history.
+- Tools are the MCP server's functions — add a tool by adding an MCP tool, not a
+  parallel implementation.
+- `ProviderCredentials` is thread-safe (`threading.Lock`) and restores the
+  previous environment on `disable()`.
+
+## Work Guidance
+
+- New built-in agent: add TOML under `builtin/` with a name, description, system
+  prompt, and tool list; no Python change is required.
+- New provider method (`providers/*`): extend the `ext_method` handler in the
+  harness adapter and `provider_store.py`.
+- Reaching the pipeline from a tool: call the core function or the CLI helper
+  that the MCP tool calls — never shell out.
+
+## Verification
+
+- `uv run pytest tests/unit/test_agent.py tests/unit/test_agent_definitions.py -q`
+  — provider store, credentials, agent install/remove, definitions, registry.
+  The protocol itself lives in `pydantic-ai-harness` and is not covered here.

kompend/acp/__init__.py

@@ -0,0 +1,24 @@
+"""ACP agent — kompend as an ACP-compatible agent for credential-free LLM injection.
+
+Heavy submodules (``harness_adapter`` → pydantic-ai + the MCP server chain) are
+imported lazily via ``lazy_loader``, so importing anything under ``kompend.acp.*``
+— e.g. the lightweight :mod:`kompend.acp.definitions` enum used by the CLI — does
+not pull in pydantic-ai, the MCP server, and the agent/pipeline stack on every
+``kompend`` invocation (notably ``--help``). Pattern per scientific-python
+SPEC-0001; ``EAGER_IMPORT=1`` disables it for debugging.
+"""
+
+from __future__ import annotations
+
+import lazy_loader as lazy
+
+__getattr__, __dir__, __all__ = lazy.attach(
+    __name__,
+    submodules=["definitions", "registry", "provider_store"],
+    submod_attrs={
+        "definitions": ["AgentDefinition", "AgentTask"],
+        "registry": ["AgentRegistry"],
+        "provider_store": ["LlmApiType", "ProviderConfig", "ProviderCredentials"],
+        "harness_adapter": ["build_kompend_agent", "kompend_session_config"],
+    },
+)

kompend/acp/builtin/capture.toml

@@ -0,0 +1,24 @@
+name = "note-taker"
+description = "Capture-focused agent — helps capture notes, decisions, and observations into the knowledge base."
+task = "capture"
+
+[llm]
+temperature = 0.4
+max_tokens = 2048
+
+[prompt]
+system = """\
+You are a kompend note-taking assistant. You help users capture decisions, \
+discoveries, corrections, and references into the knowledge base.
+
+When the user shares a note or observation, use wiki_capture to extract \
+structured knowledge items from it. Confirm what was captured and suggest \
+tags if appropriate.
+"""
+
+[tools]
+enabled = ["wiki_capture", "wiki_status"]
+
+[behavior]
+max_turns = 5
+stream_progress = true

kompend/acp/builtin/compile.toml

@@ -0,0 +1,26 @@
+name = "compiler"
+description = "Compilation-focused agent — helps compile, recompile, and lint the knowledge wiki."
+task = "compile"
+
+[llm]
+temperature = 0.2
+max_tokens = 4096
+
+[prompt]
+system = """\
+You are a kompend compilation assistant. Your primary job is to help the user \
+compile documents into a structured knowledge wiki.
+
+Before compiling, check the wiki status. After compiling, report the result \
+(page count, duration, any issues). Offer to lint if the compilation succeeded.
+
+If the user asks to change compilation settings, use config_get/config_set. \
+Always confirm before triggering a recompile (it may take a long time).
+"""
+
+[tools]
+enabled = ["compile_wiki", "wiki_status", "config_get", "config_set"]
+
+[behavior]
+max_turns = 8
+stream_progress = true

kompend/acp/builtin/generic.toml

@@ -0,0 +1,28 @@
+name = "generic"
+description = "Generic kompend assistant — handles any task by reasoning over available tools. Always available as fallback."
+task = "generic"
+
+[llm]
+temperature = 0.3
+max_tokens = 4096
+
+[prompt]
+system = """\
+You are a kompend knowledge base assistant. You help users compile, query, \
+capture, and organize documents into a structured wiki.
+
+Use the available tools to accomplish the user's request. If the request \
+is ambiguous, ask for clarification before proceeding. Report results \
+clearly and concisely.
+
+Available operations:
+- Compile sources into a wiki (compile_wiki)
+- Query the compiled wiki (query_wiki)
+- Capture notes and learn from content (wiki_capture, wiki_learn)
+- Check wiki status and health (wiki_status)
+- Read and modify configuration (config_get, config_set)
+"""
+
+[behavior]
+max_turns = 10
+stream_progress = true

kompend/acp/builtin/query.toml

@@ -0,0 +1,28 @@
+name = "researcher"
+description = "Query-focused agent — answers questions about the compiled wiki using retrieval-augmented generation."
+task = "query"
+
+[llm]
+temperature = 0.3
+max_tokens = 4096
+
+[prompt]
+system = """\
+You are a kompend research assistant. You answer questions about the compiled \
+knowledge wiki using the query_wiki tool.
+
+When a user asks a question:
+1. Call query_wiki with their question
+2. Synthesize a clear, factual answer from the results
+3. Cite source pages when relevant
+
+If no wiki is compiled, inform the user and suggest running a compilation first. \
+If the question is ambiguous, ask for clarification.
+"""
+
+[tools]
+enabled = ["query_wiki", "wiki_status"]
+
+[behavior]
+max_turns = 6
+stream_progress = true

kompend/acp/definitions.py

@@ -0,0 +1,130 @@
+"""Pydantic models for TOML-defined conversational agents.
+
+Each agent is a single TOML file with a system prompt, tool list, and
+optional model/behavior configuration. Loaded by :class:`AgentRegistry`.
+
+TOML schema::
+
+    name = "wiki-compiler"
+    description = "Compiles documents into a knowledge wiki"
+    task = "compile"  # compile | query | capture | organize | generic
+
+    [llm]
+    model = null  # null = vault default
+    temperature = 0.2
+    max_tokens = 4096
+
+    [prompt]
+    system = "You are a kompend compilation assistant..."
+
+    [tools]
+    enabled = ["compile_wiki", "wiki_status"]  # null/omitted = all tools
+
+    [behavior]
+    max_turns = 10
+    stream_progress = true
+"""
+
+from __future__ import annotations
+
+import tomllib
+from enum import StrEnum
+from pathlib import Path
+from typing import Any
+
+from pydantic import BaseModel, ConfigDict, Field
+
+
+class AgentTask(StrEnum):
+    """User-facing task categories for agent selection.
+
+    Tasks map to the kind of operation the user wants to perform, not to
+    individual LLM calls within the pipeline.
+    """
+
+    COMPILE = "compile"
+    QUERY = "query"
+    CAPTURE = "capture"
+    ORGANIZE = "organize"
+    GENERIC = "generic"
+
+
+class AgentLLMConfig(BaseModel):
+    """Per-agent LLM model and generation parameters."""
+
+    model_config = ConfigDict(extra="forbid")
+
+    model: str | None = Field(default=None, description="Model identifier. None = vault default.")
+    temperature: float = Field(default=0.3, ge=0.0, le=2.0)
+    max_tokens: int = Field(default=4096, ge=1)
+
+
+class AgentPromptConfig(BaseModel):
+    """System prompt for the agent. Taken as-is from TOML."""
+
+    model_config = ConfigDict(extra="forbid")
+
+    system: str = Field(description="System prompt text. Written as-is; default language is English.")
+
+
+class AgentToolsConfig(BaseModel):
+    """Tool access configuration for the agent."""
+
+    model_config = ConfigDict(extra="forbid")
+
+    enabled: list[str] | None = Field(
+        default=None,
+        description="Tool names the agent may call. None = all available tools.",
+    )
+
+
+class AgentBehaviorConfig(BaseModel):
+    """Behavioral limits for the conversational loop."""
+
+    model_config = ConfigDict(extra="forbid")
+
+    max_turns: int = Field(default=10, ge=1, description="Maximum LLM round-trips per prompt.")
+    stream_progress: bool = Field(default=True, description="Forward tool progress as session/update chunks.")
+
+
+class AgentDefinition(BaseModel):
+    """A single TOML-defined conversational agent.
+
+    Loaded from a TOML file via :meth:`from_toml` or constructed directly.
+    Used by :class:`~kompend.acp.conversational.ConversationalAgent` and
+    :class:`~kompend.acp.registry.AgentRegistry`.
+    """
+
+    model_config = ConfigDict(extra="forbid")
+
+    name: str = Field(description="Unique agent name (matches filename without .toml).")
+    description: str = Field(description="Human-readable description shown in `kompend agents list`.")
+    task: AgentTask = Field(description="Task category for selection.")
+    llm: AgentLLMConfig = Field(default_factory=AgentLLMConfig)
+    prompt: AgentPromptConfig
+    tools: AgentToolsConfig = Field(default_factory=AgentToolsConfig)
+    behavior: AgentBehaviorConfig = Field(default_factory=AgentBehaviorConfig)
+
+    @classmethod
+    def from_toml(cls, path: str | Path) -> AgentDefinition:
+        """Load an agent definition from a TOML file.
+
+        Args:
+            path: Path to the TOML file.
+
+        Returns:
+            Parsed and validated :class:`AgentDefinition`.
+
+        Raises:
+            FileNotFoundError: If *path* does not exist.
+            ValidationError: If the TOML content doesn't match the schema.
+        """
+        path = Path(path)
+        with path.open("rb") as f:
+            data: dict[str, Any] = tomllib.load(f)
+        return cls.model_validate(data)
+
+    @classmethod
+    def from_dict(cls, data: dict[str, Any]) -> AgentDefinition:
+        """Construct from a pre-parsed dict (e.g. from config.toml inline agent)."""
+        return cls.model_validate(data)

kompend/acp/harness_adapter.py

@@ -0,0 +1,65 @@
+"""Harness adapter — builds a PydanticAI Agent for ACP serving.
+
+Provides :func:`build_kompend_agent` (a generic agent with all kompend tools) and
+:func:`kompend_session_config` (a per-session factory that roots tools at the
+client's workspace and resolves the active agent definition).
+
+Used by :mod:`kompend.cli.cmds.agent` to serve kompend over ACP via
+``run_acp_stdio_sync``.
+"""
+
+from __future__ import annotations
+
+from pydantic_ai import Agent
+from pydantic_ai_harness.experimental.acp import AcpSession, AcpSessionConfig
+
+from kompend.acp.definitions import AgentTask
+from kompend.acp.registry import AgentRegistry
+from kompend.acp.tools import build_toolset
+from kompend.logging import get_logger
+
+logger = get_logger(__name__)
+
+
+def build_kompend_agent() -> Agent[None, str]:
+    """Build a generic PydanticAI Agent with a placeholder model.
+
+    The actual model is resolved per-session via the config or
+    ``providers/set`` override. The agent uses a minimal system prompt —
+    the per-session ``session_config`` factory overrides instructions
+    based on the active agent definition.
+    """
+    return Agent(
+        "noop",
+        instructions="You are a kompend knowledge base assistant.",
+        name="acp-session-agent",
+    )
+
+
+def kompend_session_config(session: AcpSession) -> AcpSessionConfig[None]:
+    """Per-session factory for ``run_acp_stdio_sync(session_config=...)``.
+
+    Resolves the active agent definition for the session's workspace and
+    returns an ``AcpSessionConfig`` with the appropriate toolset and
+    instructions.
+
+    Args:
+        session: The :class:`AcpSession` from the harness (has ``cwd``,
+            ``mcp_servers``, ``client_capabilities``).
+
+    Returns:
+        An ``AcpSessionConfig`` with deps=None and the resolved toolset.
+    """
+    cwd = session.cwd
+    registry = AgentRegistry(base_dir=cwd)
+    definition = registry.active_for(AgentTask.GENERIC, {})
+
+    toolset = build_toolset(
+        base_dir=cwd,
+        enabled=definition.tools.enabled,
+    )
+
+    return AcpSessionConfig(
+        deps=None,
+        toolsets=[toolset],
+    )