mdengine 0.13.4__tar.gz → 0.14.1__tar.gz

{mdengine-0.13.4 → mdengine-0.14.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mdengine
-Version: 0.13.4
+Version: 0.14.1
 Summary: Convert PDF, Office, images, audio, video, text/JSON/XML, ZIP archives, web URLs, databases, graphs, and OpenAPI specs to Markdown.
 Author: mdengine contributors
 License-Expression: MIT
@@ -107,6 +107,24 @@ Provides-Extra: graph
 Requires-Dist: networkx>=3.2.0; extra == "graph"
 Requires-Dist: neo4j>=5.14.0; extra == "graph"
 Requires-Dist: pyyaml>=6.0.0; extra == "graph"
+Provides-Extra: vector
+Requires-Dist: qdrant-client>=1.7.0; extra == "vector"
+Requires-Dist: numpy>=1.24.0; extra == "vector"
+Provides-Extra: langchain
+Requires-Dist: langchain>=0.1.0; extra == "langchain"
+Requires-Dist: langchain-core>=0.1.0; extra == "langchain"
+Requires-Dist: langgraph>=0.0.10; extra == "langchain"
+Provides-Extra: crewai
+Requires-Dist: crewai>=0.28.0; extra == "crewai"
+Provides-Extra: integrations
+Requires-Dist: qdrant-client>=1.7.0; extra == "integrations"
+Requires-Dist: numpy>=1.24.0; extra == "integrations"
+Requires-Dist: langchain>=0.1.0; extra == "integrations"
+Requires-Dist: langchain-core>=0.1.0; extra == "integrations"
+Requires-Dist: langgraph>=0.0.10; extra == "integrations"
+Requires-Dist: crewai>=0.28.0; extra == "integrations"
+Requires-Dist: networkx>=3.2.0; extra == "integrations"
+Requires-Dist: neo4j>=5.14.0; extra == "integrations"
 Provides-Extra: openapi
 Requires-Dist: pyyaml>=6.0.0; extra == "openapi"
 Requires-Dist: prance>=23.6.0; extra == "openapi"
@@ -243,7 +261,7 @@ Dynamic: license-file
 
 # mdengine
 
-Single Python distribution for converting **PDF**, **Word (.docx)**, **PowerPoint (.pptx)**, **Excel (.xlsx/.xlsm)**, **images** (OCR), **plain text / JSON / XML**, **ZIP archives**, **audio / video** (Whisper transcription → Markdown), **database metadata** (SQL + Mongo), **graphs** (Neo4j / NetworkX → Markdown), **OpenAPI** specs, **Playwright**-captured web pages (including SPAs), **application logs** (plain, JSON, CSV, ZIP bundles → Markdown with optional clustering and semantic grouping), **OpenTelemetry** traces (**OTLP** JSON or protobuf → Markdown trace summaries; optional correlation with log exports via `otel_path`), **OData CSDL metadata** (V1–V4, XML/JSON → Markdown catalogs via **`md-odata`**), **SAP artifacts** (ABAP, CDS, ADT DDIC XML, HANA views, BW, Datasphere, BAPI, IDoc, transport → AI-ready Markdown; OData catalogs via **`md-sap`**, standalone **`md-odata`**, or shared **`md_generator.odata`**), and **source code** (codeflow → architecture Markdown) into **Markdown** (and related assets). Install only the extras you need; everything imports under the **`md_generator`** package.
+Single Python distribution for converting **PDF**, **Word (.docx)**, **PowerPoint (.pptx)**, **Excel (.xlsx/.xlsm)**, **images** (OCR), **plain text / JSON / XML**, **ZIP archives**, **audio / video** (Whisper transcription → Markdown), **database metadata** (SQL + Mongo), **graphs** (Neo4j / NetworkX → Markdown), **OpenAPI** specs, **Playwright**-captured web pages (including SPAs), **application logs** (plain, JSON, CSV, ZIP bundles → Markdown with optional clustering and semantic grouping), **OpenTelemetry** traces (**OTLP** JSON or protobuf → Markdown trace summaries; optional correlation with log exports via `otel_path`), **OData CSDL metadata** (V1–V4, XML/JSON → Markdown catalogs via **`md-odata`**), **SAP artifacts** (ABAP, CDS, ADT DDIC XML, HANA views, BW, Datasphere, BAPI, IDoc, transport, **ABAP execution journeys & call graphs** → AI-ready Markdown; OData catalogs via **`md-sap`**, standalone **`md-odata`**, or shared **`md_generator.odata`**), and **source code** (codeflow → architecture Markdown) into **Markdown** (and related assets). Install only the extras you need; everything imports under the **`md_generator`** package.
 
 - **PyPI name:** `mdengine` (import package: `md_generator`)
 - **Source:** [github.com/mdengine/md-generator](https://github.com/mdengine/md-generator)
@@ -442,7 +460,7 @@ If the shell reports “command not found”, ensure the Python **Scripts** dire
 
 **odata-to-md (`md-odata`):** parses **OData CSDL** metadata (**V1–V4**, XML or JSON) from **`--file`**, **`--folder`**, **`--zip`**, or **`--url`** (`$metadata`) into catalog Markdown under **`odata/`** (services, entity sets, types, actions, functions). Optional **`--graph`** (relationship JSON/Mermaid) and **`--chunk`** (semantic chunks for RAG). Works **standalone** with **`md-odata`**; when OData files sit beside ABAP/CDS in a SAP tree, use **`md-sap`** instead — see [odata-to-md/docs/dual-mode.md](odata-to-md/docs/dual-mode.md). Library: [`md_generator/odata/`](src/md_generator/odata/). Module README: [`odata-to-md/README.md`](odata-to-md/README.md); catalog layout: [odata-to-md/docs/output-layout.md](odata-to-md/docs/output-layout.md); SAP-integrated catalog notes: [sap-to-md/docs/odata-catalog.md](sap-to-md/docs/odata-catalog.md).
 
-**sap-to-md (`md-sap`):** parses **ABAP**, **CDS** (views, **`define table`**, **`define type`** structures), **DDIC** (ADT dictionary XML, abapGit **`TABL`**, DD02/DD03 CSV), **OData** metadata (via shared **`md_generator.odata`** when **`odata_catalog`** is enabled), **BAPI**, **IDoc**, and transport artifacts into chunked Markdown knowledge packs. Also supports **SAP HANA** (calculation, analytic, attribute, SQL, and HDI views), **SAP BW/4HANA** (ADSO, composite providers, DTP, InfoObjects, transformations), **SAP Datasphere** (`.dsview`, `.dataflow`, analytical model JSON exports), and optional **external lineage** parsers (dbt `manifest.json`, Snowflake DDL, Kafka `.avsc`, Informatica mapping XML) when **`parser.include_external: true`** in YAML.
+**sap-to-md (`md-sap`):** parses **ABAP**, **CDS** (views, **`define table`**, **`define type`** structures), **DDIC** (ADT dictionary XML, abapGit **`TABL`**, DD02/DD03 CSV), **OData** metadata (via shared **`md_generator.odata`** when **`odata_catalog`** is enabled), **BAPI**, **IDoc**, and transport artifacts into chunked Markdown knowledge packs. Also supports **ABAP execution journeys & call graphs** (`abap_journey` static caller-callee resolution, traversal limits, statement extractors, fallback journey roots, `.journey-cache`, Mermaid/Graphviz/Markdown renderers), **SAP HANA** (calculation, analytic, attribute, SQL, and HDI views), **SAP BW/4HANA** (ADSO, composite providers, DTP, InfoObjects, transformations), **SAP Datasphere** (`.dsview`, `.dataflow`, analytical model JSON exports), and optional **external lineage** parsers (dbt `manifest.json`, Snowflake DDL, Kafka `.avsc`, Informatica mapping XML) when **`parser.include_external: true`** in YAML.
 
 **DDIC / ADT:** Eclipse ADT exports under `wbobj/dictionary/` (data elements, domains, tables, structures, table types, range types, reference types) are parsed by **`ddic.adt`**. Modern **`TABL/DT`** objects with DDL references accept sidecar DDL source when present. Normalized **`ddic_canonical`** metadata powers v2 generators; legacy entity keys remain for v1 **`entities/`** output. Output layout: **`ddic/data-elements/`**, **`ddic/domains/`**, **`ddic/tables/`**, … and shared **`structures/{name}.md`** for ADT structures and CDS **`define type`**. See [sap-to-md/docs/ddic-adt.md](sap-to-md/docs/ddic-adt.md).
 
@@ -885,6 +903,7 @@ Library code lives under [`src/md_generator/sap/`](src/md_generator/sap/). The p
 | Package area | Role |
 |--------------|------|
 | `parser/` | Plugin registry — ABAP, CDS (view / **`define table`** / **`define type`**), DDIC (ADT XML, abapGit TABL, CSV), OData, BAPI, IDoc, transport, HANA, BW, Datasphere, external lineage |
+| `abap_journey/` | ABAP execution journeys & call graphs — static call resolution, traversal limits, statement extractors, fallback journey roots, incremental caching, Mermaid/Graphviz/Markdown renderers |
 | `normalizer/` + `canonical/` | **`SapObject` → `CanonicalArtifact`**, typed **`ddic_canonical`** metadata, transformation graphs |
 | `analyzer/` | Relationships, governance, validation, authorization, lineage, semantics |
 | `generators/` + `markdown/` | v1 entity docs (12 sections) and v2 typed Markdown (DDIC, HANA, CDS, BW, Datasphere); optional **semantic narrative** blocks |
@@ -1284,6 +1303,7 @@ Static call graphs and per-entry Markdown/Mermaid/JSON for **Python**, **Java**
 More detail: **[`codeflow-to-md/docs/graph-and-outputs.md`](codeflow-to-md/docs/graph-and-outputs.md)** (node/edge model, `graph-schema.json`, `methods/` layout).
 
 - **Formats:** pass **`--formats md,mermaid,json,html`** (comma-separated). **`--output`** is the **output directory**, not the format list.
+- **Journey analysis & export formats:** pass **`--journey-format md,json,mermaid,html,dot,graphml,gexf`** to generate cross-file execution journey representations and export graph visualizations (DOT, GraphML, GEXF) alongside Markdown/Mermaid outputs.
 - **Graph schema export:** optional **`--emit-graph-schema`** (with `json` in formats) writes **`graph-schema.json`** — stable Node/Edge view with derived File/Class nodes and `CONTAINS` edges. **`graph-full.json`** stays unchanged for existing consumers.
 - **Markdown intelligence:** flow and entry Markdown include **Called By** and **Impact** lists (capped with **`--intelligence-list-cap`**, default 80).
 - **IR / CFG (optional):** **`--emit-cfg`** builds a language-agnostic IR and writes **`cfg.json`**, **`cfg.mmd`**, and appends a CFG Mermaid block to **`flow.md`** (when `md` is in formats). Cap with **`--cfg-max-nodes`** (default 500).

{mdengine-0.13.4 → mdengine-0.14.1}/README.md

@@ -1,6 +1,6 @@
 # mdengine
 
-Single Python distribution for converting **PDF**, **Word (.docx)**, **PowerPoint (.pptx)**, **Excel (.xlsx/.xlsm)**, **images** (OCR), **plain text / JSON / XML**, **ZIP archives**, **audio / video** (Whisper transcription → Markdown), **database metadata** (SQL + Mongo), **graphs** (Neo4j / NetworkX → Markdown), **OpenAPI** specs, **Playwright**-captured web pages (including SPAs), **application logs** (plain, JSON, CSV, ZIP bundles → Markdown with optional clustering and semantic grouping), **OpenTelemetry** traces (**OTLP** JSON or protobuf → Markdown trace summaries; optional correlation with log exports via `otel_path`), **OData CSDL metadata** (V1–V4, XML/JSON → Markdown catalogs via **`md-odata`**), **SAP artifacts** (ABAP, CDS, ADT DDIC XML, HANA views, BW, Datasphere, BAPI, IDoc, transport → AI-ready Markdown; OData catalogs via **`md-sap`**, standalone **`md-odata`**, or shared **`md_generator.odata`**), and **source code** (codeflow → architecture Markdown) into **Markdown** (and related assets). Install only the extras you need; everything imports under the **`md_generator`** package.
+Single Python distribution for converting **PDF**, **Word (.docx)**, **PowerPoint (.pptx)**, **Excel (.xlsx/.xlsm)**, **images** (OCR), **plain text / JSON / XML**, **ZIP archives**, **audio / video** (Whisper transcription → Markdown), **database metadata** (SQL + Mongo), **graphs** (Neo4j / NetworkX → Markdown), **OpenAPI** specs, **Playwright**-captured web pages (including SPAs), **application logs** (plain, JSON, CSV, ZIP bundles → Markdown with optional clustering and semantic grouping), **OpenTelemetry** traces (**OTLP** JSON or protobuf → Markdown trace summaries; optional correlation with log exports via `otel_path`), **OData CSDL metadata** (V1–V4, XML/JSON → Markdown catalogs via **`md-odata`**), **SAP artifacts** (ABAP, CDS, ADT DDIC XML, HANA views, BW, Datasphere, BAPI, IDoc, transport, **ABAP execution journeys & call graphs** → AI-ready Markdown; OData catalogs via **`md-sap`**, standalone **`md-odata`**, or shared **`md_generator.odata`**), and **source code** (codeflow → architecture Markdown) into **Markdown** (and related assets). Install only the extras you need; everything imports under the **`md_generator`** package.
 
 - **PyPI name:** `mdengine` (import package: `md_generator`)
 - **Source:** [github.com/mdengine/md-generator](https://github.com/mdengine/md-generator)
@@ -199,7 +199,7 @@ If the shell reports “command not found”, ensure the Python **Scripts** dire
 
 **odata-to-md (`md-odata`):** parses **OData CSDL** metadata (**V1–V4**, XML or JSON) from **`--file`**, **`--folder`**, **`--zip`**, or **`--url`** (`$metadata`) into catalog Markdown under **`odata/`** (services, entity sets, types, actions, functions). Optional **`--graph`** (relationship JSON/Mermaid) and **`--chunk`** (semantic chunks for RAG). Works **standalone** with **`md-odata`**; when OData files sit beside ABAP/CDS in a SAP tree, use **`md-sap`** instead — see [odata-to-md/docs/dual-mode.md](odata-to-md/docs/dual-mode.md). Library: [`md_generator/odata/`](src/md_generator/odata/). Module README: [`odata-to-md/README.md`](odata-to-md/README.md); catalog layout: [odata-to-md/docs/output-layout.md](odata-to-md/docs/output-layout.md); SAP-integrated catalog notes: [sap-to-md/docs/odata-catalog.md](sap-to-md/docs/odata-catalog.md).
 
-**sap-to-md (`md-sap`):** parses **ABAP**, **CDS** (views, **`define table`**, **`define type`** structures), **DDIC** (ADT dictionary XML, abapGit **`TABL`**, DD02/DD03 CSV), **OData** metadata (via shared **`md_generator.odata`** when **`odata_catalog`** is enabled), **BAPI**, **IDoc**, and transport artifacts into chunked Markdown knowledge packs. Also supports **SAP HANA** (calculation, analytic, attribute, SQL, and HDI views), **SAP BW/4HANA** (ADSO, composite providers, DTP, InfoObjects, transformations), **SAP Datasphere** (`.dsview`, `.dataflow`, analytical model JSON exports), and optional **external lineage** parsers (dbt `manifest.json`, Snowflake DDL, Kafka `.avsc`, Informatica mapping XML) when **`parser.include_external: true`** in YAML.
+**sap-to-md (`md-sap`):** parses **ABAP**, **CDS** (views, **`define table`**, **`define type`** structures), **DDIC** (ADT dictionary XML, abapGit **`TABL`**, DD02/DD03 CSV), **OData** metadata (via shared **`md_generator.odata`** when **`odata_catalog`** is enabled), **BAPI**, **IDoc**, and transport artifacts into chunked Markdown knowledge packs. Also supports **ABAP execution journeys & call graphs** (`abap_journey` static caller-callee resolution, traversal limits, statement extractors, fallback journey roots, `.journey-cache`, Mermaid/Graphviz/Markdown renderers), **SAP HANA** (calculation, analytic, attribute, SQL, and HDI views), **SAP BW/4HANA** (ADSO, composite providers, DTP, InfoObjects, transformations), **SAP Datasphere** (`.dsview`, `.dataflow`, analytical model JSON exports), and optional **external lineage** parsers (dbt `manifest.json`, Snowflake DDL, Kafka `.avsc`, Informatica mapping XML) when **`parser.include_external: true`** in YAML.
 
 **DDIC / ADT:** Eclipse ADT exports under `wbobj/dictionary/` (data elements, domains, tables, structures, table types, range types, reference types) are parsed by **`ddic.adt`**. Modern **`TABL/DT`** objects with DDL references accept sidecar DDL source when present. Normalized **`ddic_canonical`** metadata powers v2 generators; legacy entity keys remain for v1 **`entities/`** output. Output layout: **`ddic/data-elements/`**, **`ddic/domains/`**, **`ddic/tables/`**, … and shared **`structures/{name}.md`** for ADT structures and CDS **`define type`**. See [sap-to-md/docs/ddic-adt.md](sap-to-md/docs/ddic-adt.md).
 
@@ -642,6 +642,7 @@ Library code lives under [`src/md_generator/sap/`](src/md_generator/sap/). The p
 | Package area | Role |
 |--------------|------|
 | `parser/` | Plugin registry — ABAP, CDS (view / **`define table`** / **`define type`**), DDIC (ADT XML, abapGit TABL, CSV), OData, BAPI, IDoc, transport, HANA, BW, Datasphere, external lineage |
+| `abap_journey/` | ABAP execution journeys & call graphs — static call resolution, traversal limits, statement extractors, fallback journey roots, incremental caching, Mermaid/Graphviz/Markdown renderers |
 | `normalizer/` + `canonical/` | **`SapObject` → `CanonicalArtifact`**, typed **`ddic_canonical`** metadata, transformation graphs |
 | `analyzer/` | Relationships, governance, validation, authorization, lineage, semantics |
 | `generators/` + `markdown/` | v1 entity docs (12 sections) and v2 typed Markdown (DDIC, HANA, CDS, BW, Datasphere); optional **semantic narrative** blocks |
@@ -1041,6 +1042,7 @@ Static call graphs and per-entry Markdown/Mermaid/JSON for **Python**, **Java**
 More detail: **[`codeflow-to-md/docs/graph-and-outputs.md`](codeflow-to-md/docs/graph-and-outputs.md)** (node/edge model, `graph-schema.json`, `methods/` layout).
 
 - **Formats:** pass **`--formats md,mermaid,json,html`** (comma-separated). **`--output`** is the **output directory**, not the format list.
+- **Journey analysis & export formats:** pass **`--journey-format md,json,mermaid,html,dot,graphml,gexf`** to generate cross-file execution journey representations and export graph visualizations (DOT, GraphML, GEXF) alongside Markdown/Mermaid outputs.
 - **Graph schema export:** optional **`--emit-graph-schema`** (with `json` in formats) writes **`graph-schema.json`** — stable Node/Edge view with derived File/Class nodes and `CONTAINS` edges. **`graph-full.json`** stays unchanged for existing consumers.
 - **Markdown intelligence:** flow and entry Markdown include **Called By** and **Impact** lists (capped with **`--intelligence-list-cap`**, default 80).
 - **IR / CFG (optional):** **`--emit-cfg`** builds a language-agnostic IR and writes **`cfg.json`**, **`cfg.mmd`**, and appends a CFG Mermaid block to **`flow.md`** (when `md` is in formats). Cap with **`--cfg-max-nodes`** (default 500).

{mdengine-0.13.4 → mdengine-0.14.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "mdengine"
-version = "0.13.4"
+version = "0.14.1"
 description = "Convert PDF, Office, images, audio, video, text/JSON/XML, ZIP archives, web URLs, databases, graphs, and OpenAPI specs to Markdown."
 readme = "README.md"
 requires-python = ">=3.10"
@@ -109,6 +109,28 @@ graph = [
   "neo4j>=5.14.0",
   "pyyaml>=6.0.0",
 ]
+vector = [
+  "qdrant-client>=1.7.0",
+  "numpy>=1.24.0",
+]
+langchain = [
+  "langchain>=0.1.0",
+  "langchain-core>=0.1.0",
+  "langgraph>=0.0.10",
+]
+crewai = [
+  "crewai>=0.28.0",
+]
+integrations = [
+  "qdrant-client>=1.7.0",
+  "numpy>=1.24.0",
+  "langchain>=0.1.0",
+  "langchain-core>=0.1.0",
+  "langgraph>=0.0.10",
+  "crewai>=0.28.0",
+  "networkx>=3.2.0",
+  "neo4j>=5.14.0",
+]
 openapi = [
   "pyyaml>=6.0.0",
   "prance>=23.6.0",
@@ -329,8 +351,10 @@ include = ["md_generator*"]
 "md_generator.tools.assistant" = ["data/**/*"]
 
 [tool.pytest.ini_options]
+addopts = "--import-mode=importlib"
 pythonpath = ["src"]
 testpaths = [
+  "tests",
   "tool-assistant/tests",
   "pdf-to-md/tests",
   "word-to-md/tests",

{mdengine-0.13.4 → mdengine-0.14.1}/src/md_generator/__init__.py

@@ -1,5 +1,5 @@
 """md_generator: PDF, Office, image, text, and ZIP to Markdown conversion."""
 
-__version__ = "0.13.4"
+__version__ = "0.14.1"
 
 __all__ = ["__version__"]

mdengine-0.14.1/src/md_generator/adapters/__init__.py

@@ -0,0 +1 @@
+"""Adapters bridging md_generator to external AI ecosystems."""

mdengine-0.14.1/src/md_generator/adapters/crewai/__init__.py

@@ -0,0 +1,20 @@
+"""Public facade for mdengine CrewAI tool adapters."""
+
+from md_generator.adapters.crewai.exceptions import (
+    CrewAIAdapterError,
+    CrewAIDependencyError,
+    CrewAIToolExecutionError,
+    TenantBoundaryError,
+)
+from md_generator.adapters.crewai.schema import MDSearchInput
+from md_generator.adapters.crewai.tools import MDSearchTool, detect_crewai_base_tool
+
+__all__ = [
+    "MDSearchTool",
+    "detect_crewai_base_tool",
+    "MDSearchInput",
+    "CrewAIAdapterError",
+    "CrewAIDependencyError",
+    "CrewAIToolExecutionError",
+    "TenantBoundaryError",
+]

mdengine-0.14.1/src/md_generator/adapters/crewai/exceptions.py

@@ -0,0 +1,29 @@
+"""Standalone exception hierarchy for mdengine CrewAI tool adapters.
+
+Uncoupled from vector store errors and LangGraph errors, while preserving
+direct propagation of the shared TenantBoundaryError for security violations.
+"""
+
+from md_generator.vector.exceptions import TenantBoundaryError
+
+__all__ = [
+    "CrewAIAdapterError",
+    "CrewAIDependencyError",
+    "CrewAIToolExecutionError",
+    "TenantBoundaryError",
+]
+
+
+class CrewAIAdapterError(Exception):
+    """Base exception for all CrewAI adapter operations."""
+    pass
+
+
+class CrewAIDependencyError(ImportError, CrewAIAdapterError):
+    """Raised when CrewAI is installed but cannot be imported due to syntax or dependency errors."""
+    pass
+
+
+class CrewAIToolExecutionError(CrewAIAdapterError):
+    """Raised on invalid document scores, execution failures, or tool operational faults."""
+    pass

mdengine-0.14.1/src/md_generator/adapters/crewai/schema.py

@@ -0,0 +1,37 @@
+"""Strict Pydantic v2 input argument schemas for CrewAI tools."""
+
+from pydantic import BaseModel, ConfigDict, Field, field_validator
+
+__all__ = ["MDSearchInput"]
+
+
+class MDSearchInput(BaseModel):
+    """Strict input argument schema for mdengine CrewAI search tool.
+    
+    Invariants:
+        1. extra="forbid": Unknown fields (including agent-supplied tenant_id) are strictly rejected.
+        2. strict=True: Rejects loose type coercions (e.g., int/float to string).
+        3. frozen=True: Input instance is immutable.
+        4. Option A: Retrieval uses the retriever's configured top_k setting without a conflicting per-call limit.
+    """
+    model_config = ConfigDict(
+        extra="forbid",
+        strict=True,
+        frozen=True,
+    )
+
+    query: str = Field(
+        ...,
+        description="Search query to locate relevant enterprise documentation, code, and architecture specifications.",
+        min_length=1,
+    )
+
+    @field_validator("query")
+    @classmethod
+    def validate_non_whitespace_query(cls, v: str) -> str:
+        if not isinstance(v, str):
+            raise ValueError("Query must be a string.")
+        stripped = v.strip()
+        if not stripped:
+            raise ValueError("Query string cannot be empty or whitespace-only.")
+        return stripped

mdengine-0.14.1/src/md_generator/adapters/crewai/tools.py

@@ -0,0 +1,284 @@
+"""MDSearchTool adapter for CrewAI agents with bound tenant isolation and score filtering."""
+
+import importlib.util
+import math
+from typing import Any, Optional, Type
+from pydantic import BaseModel, ConfigDict, PrivateAttr
+
+from md_generator.adapters.crewai.exceptions import (
+    CrewAIDependencyError,
+    CrewAIToolExecutionError,
+    TenantBoundaryError,
+)
+from md_generator.adapters.crewai.schema import MDSearchInput
+from md_generator.adapters.langchain.context import PromptContextBuilder
+from md_generator.adapters.langchain.retriever import MDVectorStoreRetriever
+
+__all__ = ["MDSearchTool", "detect_crewai_base_tool"]
+
+
+def detect_crewai_base_tool() -> Optional[Type[Any]]:
+    """Detects BaseTool from crewai or crewai_tools without masking broken installations.
+    
+    Supported Version Range:
+        - CrewAI >= 0.11.0 to current: imports from `crewai.tools.BaseTool`
+        - CrewAI tool packages: imports from `crewai_tools.BaseTool`
+        
+    Resolution Logic (P2B-R03):
+        1. Genuine Absence: If neither `crewai` nor `crewai_tools` can be located by
+           importlib.util.find_spec, returns None to activate `_FallbackBaseTool`.
+        2. Installed Package: If either package is located on disk, attempts to import BaseTool.
+           A failure indicates a broken installation (missing internal dependencies, syntax
+           errors, or version incompatibilities) and raises CrewAIDependencyError.
+    """
+    has_crewai = False
+    has_crewai_tools = False
+
+    try:
+        has_crewai = importlib.util.find_spec("crewai") is not None
+    except (ImportError, AttributeError, ValueError) as err:
+        raise CrewAIDependencyError(f"Corrupted crewai package metadata: {err}") from err
+
+    try:
+        has_crewai_tools = importlib.util.find_spec("crewai_tools") is not None
+    except (ImportError, AttributeError, ValueError) as err:
+        raise CrewAIDependencyError(f"Corrupted crewai_tools package metadata: {err}") from err
+
+    # Case 1: Genuinely absent
+    if not has_crewai and not has_crewai_tools:
+        return None
+
+    # Case 2: crewai package located
+    if has_crewai:
+        try:
+            from crewai.tools import BaseTool as DetectedBaseTool
+            return DetectedBaseTool
+        except ImportError as err:
+            if has_crewai_tools:
+                try:
+                    from crewai_tools import BaseTool as DetectedToolsBaseTool
+                    return DetectedToolsBaseTool
+                except ImportError as err_tools:
+                    raise CrewAIDependencyError(
+                        f"CrewAI packages installed but BaseTool could not be imported: "
+                        f"crewai: {err}; crewai_tools: {err_tools}"
+                    ) from err_tools
+            raise CrewAIDependencyError(
+                f"crewai is installed but BaseTool cannot be imported: {err}"
+            ) from err
+
+    # Case 3: only crewai_tools package located
+    if has_crewai_tools:
+        try:
+            from crewai_tools import BaseTool as DetectedToolsBaseTool
+            return DetectedToolsBaseTool
+        except ImportError as err:
+            raise CrewAIDependencyError(
+                f"crewai_tools is installed but BaseTool cannot be imported: {err}"
+            ) from err
+
+    return None
+
+
+CrewAIBaseTool: Optional[Type[Any]] = detect_crewai_base_tool()
+
+
+class _FallbackBaseTool(BaseModel):
+    """Pydantic v2 fallback providing the identical public compatibility surface when CrewAI is absent."""
+    model_config = ConfigDict(arbitrary_types_allowed=True)
+
+    name: str = "enterprise_knowledge_search"
+    description: str = (
+        "Search enterprise markdown and code documentation with token-budgeted prompt context and source citations."
+    )
+    args_schema: Type[BaseModel] = MDSearchInput
+
+    def run(self, *args: Any, **kwargs: Any) -> str:
+        return self._run(*args, **kwargs)
+
+    def _run(self, *args: Any, **kwargs: Any) -> str:
+        raise NotImplementedError
+
+
+_ToolBase = CrewAIBaseTool if CrewAIBaseTool is not None else _FallbackBaseTool
+
+
+class MDSearchTool(_ToolBase):
+    """CrewAI search tool adapter bridging mdengine retrieval to multi-agent ecosystems.
+    
+    Invariants:
+        1. Mandatory Bound Tenant Isolation (C1): Bound to a single immutable tenant_id at construction.
+        2. Input Schema Contract (C2): Pydantic v2 strict input schema with extra='forbid'.
+        3. Document Score Contract (C3): Scores validated in [0.0, 1.0] and filtered by min_score.
+        4. Deterministic Output (C4): Structured PromptContextBuilder output; informative string on 0 results.
+        5. Zero In-Place Mutation: Never alters retriever, vector store, or document states.
+    """
+    name: str = "enterprise_knowledge_search"
+    description: str = (
+        "Search enterprise markdown and code documentation with token-budgeted prompt context and source citations."
+    )
+    args_schema: Type[BaseModel] = MDSearchInput
+
+    _retriever: Any = PrivateAttr()
+    _tenant_id: str = PrivateAttr()
+    _min_score: float = PrivateAttr()
+    _builder: PromptContextBuilder = PrivateAttr()
+    _is_frozen: bool = PrivateAttr(default=False)
+
+    def __init__(
+        self,
+        retriever: MDVectorStoreRetriever,
+        tenant_id: str,
+        min_score: float = 0.0,
+        builder: Optional[PromptContextBuilder] = None,
+        **kwargs: Any,
+    ) -> None:
+        # 1. Mandatory construction-time tenant validation (P2B-V04)
+        if tenant_id is None or not isinstance(tenant_id, str) or not tenant_id.strip():
+            raise TenantBoundaryError("MDSearchTool requires a mandatory non-empty string 'tenant_id'.")
+        clean_tenant = tenant_id.strip()
+
+        if not hasattr(retriever, "tenant_id"):
+            raise TenantBoundaryError("retriever must provide a valid 'tenant_id' attribute.")
+        retriever_tenant = getattr(retriever, "tenant_id")
+        if not retriever_tenant or clean_tenant != retriever_tenant.strip():
+            raise TenantBoundaryError(
+                f"Tool tenant_id '{clean_tenant}' conflicts with retriever tenant_id '{retriever_tenant}'."
+            )
+
+        # 2. Strict threshold validation (P2B-V05)
+        if isinstance(min_score, bool):
+            raise ValueError("min_score cannot be a boolean.")
+        if not isinstance(min_score, (int, float)):
+            raise ValueError(f"min_score must be a numeric float or int, got {type(min_score).__name__}.")
+        min_score_val = float(min_score)
+        if math.isnan(min_score_val) or math.isinf(min_score_val):
+            raise ValueError("min_score cannot be NaN or Infinite.")
+        if not (0.0 <= min_score_val <= 1.0):
+            raise ValueError(f"min_score must be between 0.0 and 1.0, got {min_score_val}.")
+
+        # 3. Builder initialization
+        if builder is None:
+            builder = PromptContextBuilder()
+
+        super().__init__(**kwargs)
+        self._retriever = retriever
+        self._tenant_id = clean_tenant
+        self._min_score = min_score_val
+        self._builder = builder
+        self._is_frozen = True
+
+    @property
+    def tenant_id(self) -> str:
+        """Immutable trusted tenant identifier bound at construction."""
+        return self._tenant_id
+
+    @property
+    def min_score(self) -> float:
+        """Minimum relevance threshold for document inclusion."""
+        return self._min_score
+
+    @property
+    def retriever(self) -> Any:
+        """Bound vector store retriever."""
+        return self._retriever
+
+    @property
+    def builder(self) -> PromptContextBuilder:
+        """Bound context builder."""
+        return self._builder
+
+    def __setattr__(self, name: str, value: Any) -> None:
+        if getattr(self, "_is_frozen", False) and name in ("tenant_id", "_tenant_id"):
+            raise AttributeError("tenant_id is immutable and cannot be reassigned.")
+        super().__setattr__(name, value)
+
+    def _run(self, *args: Any, **kwargs: Any) -> str:
+        """Executes tenant-isolated retrieval, document validation, score filtering, and context synthesis.
+        
+        Invocation Signatures (P2B-R01):
+            - Keyword: run(query="...") or _run(query="...")
+            - Positional: run("...") or _run("...")
+            
+        Schema Enforcement:
+            All inputs (positional or keyword) are validated through MDSearchInput:
+            - Missing arguments -> pydantic.ValidationError
+            - Non-string or whitespace queries -> pydantic.ValidationError
+            - Extra fields (e.g. tenant_id) -> pydantic.ValidationError
+            - Multiple or ambiguous positional arguments -> pydantic.ValidationError
+        """
+        # 1. Parse arguments and normalize into kwargs for MDSearchInput validation
+        call_kwargs = dict(kwargs)
+        if args:
+            if len(args) == 1:
+                if "query" in call_kwargs:
+                    call_kwargs["__duplicate_positional_query__"] = args[0]
+                else:
+                    call_kwargs["query"] = args[0]
+            else:
+                for idx, arg in enumerate(args):
+                    call_kwargs[f"__positional_{idx}__"] = arg
+
+        validated = MDSearchInput.model_validate(call_kwargs)
+        query_str = validated.query
+
+        # 2. Invoke retriever
+        try:
+            raw_docs = self._retriever.invoke(query_str) or []
+        except TenantBoundaryError:
+            raise
+        except Exception as err:
+            raise CrewAIToolExecutionError(f"Retriever invocation failed for query '{query_str}': {err}") from err
+
+        # 3. Deep document tenant ownership validation (P2B-V04)
+        for i, doc in enumerate(raw_docs):
+            meta = doc.metadata if hasattr(doc, "metadata") else (doc.get("metadata", {}) if isinstance(doc, dict) else {})
+            doc_tenant = meta.get("tenant_id")
+            if doc_tenant is None or not isinstance(doc_tenant, str) or not doc_tenant.strip():
+                raise TenantBoundaryError(
+                    f"Retrieved document at index {i} missing mandatory non-empty 'tenant_id' in metadata."
+                )
+            if doc_tenant.strip() != self._tenant_id:
+                raise TenantBoundaryError(
+                    f"Retrieved document at index {i} belongs to tenant '{doc_tenant.strip()}', "
+                    f"conflicting with bound tool tenant '{self._tenant_id}'."
+                )
+
+        # 4. Document score validation & threshold filtering (P2B-V05, P2B-R04)
+        qualifying_docs = []
+        for i, doc in enumerate(raw_docs):
+            meta = doc.metadata if hasattr(doc, "metadata") else (doc.get("metadata", {}) if isinstance(doc, dict) else {})
+            if "score" not in meta:
+                raise CrewAIToolExecutionError(f"Retrieved document at index {i} missing mandatory 'score' in metadata.")
+            raw_score = meta["score"]
+            if raw_score is None:
+                raise CrewAIToolExecutionError(f"Retrieved document at index {i} has None 'score'.")
+            if isinstance(raw_score, bool):
+                raise CrewAIToolExecutionError(f"Retrieved document at index {i} has boolean 'score'.")
+            if not isinstance(raw_score, (int, float)):
+                raise CrewAIToolExecutionError(
+                    f"Retrieved document at index {i} has non-numeric score of type '{type(raw_score).__name__}'."
+                )
+            if math.isnan(raw_score) or math.isinf(raw_score):
+                raise CrewAIToolExecutionError(f"Retrieved document at index {i} has non-finite score: {raw_score}.")
+            if not (0.0 <= raw_score <= 1.0):
+                raise CrewAIToolExecutionError(f"Retrieved document at index {i} score {raw_score} is outside [0.0, 1.0].")
+
+            if float(raw_score) >= self._min_score:
+                qualifying_docs.append(doc)
+
+        # 5. Empty results contract (P2B-V06, P2B-V07)
+        if not qualifying_docs:
+            return f"No relevant documents found for query: '{query_str}'."
+
+        # 6. Context synthesis via PromptContextBuilder
+        try:
+            return self._builder.build_context(qualifying_docs)
+        except TenantBoundaryError:
+            raise
+        except Exception as err:
+            raise CrewAIToolExecutionError(f"Context synthesis failed: {err}") from err
+
+    def run(self, *args: Any, **kwargs: Any) -> str:
+        """Public execution entry point conforming to CrewAI tool invocation protocol."""
+        return self._run(*args, **kwargs)

mdengine-0.14.1/src/md_generator/adapters/langchain/__init__.py

@@ -0,0 +1,33 @@
+"""LangChain adapters for md_generator."""
+
+from md_generator.adapters.langchain.exceptions import (
+    LangChainAdapterError,
+    LangChainDependencyError,
+)
+from md_generator.adapters.langchain.loader import (
+    MDDocumentLoader,
+    extract_semantic_key,
+    resolve_tenant_id,
+)
+from md_generator.adapters.langchain.retriever import (
+    MDVectorStoreRetriever,
+)
+from md_generator.adapters.langchain.context import (
+    PromptContextBuilder,
+    canonical_document_fingerprint,
+    extract_and_validate_score,
+    estimate_token_count,
+)
+
+__all__ = [
+    "LangChainAdapterError",
+    "LangChainDependencyError",
+    "MDDocumentLoader",
+    "extract_semantic_key",
+    "resolve_tenant_id",
+    "MDVectorStoreRetriever",
+    "PromptContextBuilder",
+    "canonical_document_fingerprint",
+    "extract_and_validate_score",
+    "estimate_token_count",
+]