deepdoc 2.3.2__tar.gz → 2.3.4__tar.gz

{deepdoc-2.3.2 → deepdoc-2.3.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: deepdoc
-Version: 2.3.2
+Version: 2.3.4
 Summary: Auto-generate beautiful docs from any codebase
 Author: Pranav Kumar
 License: MIT
@@ -82,7 +82,7 @@ DeepDoc scans your repo, builds a bucket-based documentation plan, generates ric
 - **Bucket-based documentation architecture** — system, feature, endpoint, integration, and database buckets instead of one-file-per-page noise.
 - **Multi-step AI planner** — classifies the repo, proposes buckets, then assigns files, symbols, artifacts, and dependencies into a final reader-first plan.
 - **Giant-file handling** — large files are decomposed into feature-aligned clusters so a single controller can feed multiple doc pages.
-- **MDX compile gate** — every page is compile-checked before being written; broken MDX is auto-fixed via LLM retry or degraded to safe Markdown rather than shipped.
+- **MDX-safe generation** — generated pages are repaired and validated in Python before being written; deploy-time quality gates block failed, invalid, or stub pages before the Fumadocs build.
 - **Evidence-first chatbot answers** — final code proof is hydrated from archived source snippets with exact file paths and line ranges, not just retrieval guesses.
 - **Incremental updates** — `deepdoc update` regenerates only stale or structurally affected docs against the last synced commit.
 - **OpenAPI-aware API docs** — auto-detects OpenAPI/Swagger specs and stages interactive `/api/*` pages in the generated site.
@@ -129,7 +129,7 @@ pip install click litellm gitpython rich pyyaml jinja2
 pip install -e . --no-deps
 ```
 
-You will also need **Node 18+** on `PATH`. DeepDoc uses Node at generate time to compile-check each MDX page before writing it, and at `serve`/`deploy` time for the Fumadocs site. The small set of MDX validation dependencies installs into `deepdoc/generator/mdx_validator/node_modules/` automatically on first `deepdoc generate`.
+You will also need **Node 18+** on `PATH` when you run `deepdoc serve` or `deepdoc deploy`. DeepDoc generation itself uses Python-side repair and validation; Node is only needed for the generated Fumadocs site.
 
 ### Verify installation
 
@@ -346,7 +346,7 @@ deepdoc generate --exclude "tests/**"
 
 1. **Phase 1: Scan** — Walk the repo, parse supported languages, detect endpoints, config/setup artifacts, runtime surfaces, integration signals, and OpenAPI specs.
 2. **Phase 2: Plan** — Run the multi-step bucket planner. It classifies the repo, proposes bucket candidates, and assigns files/symbols/artifacts to the final doc structure.
-3. **Phase 3: Generate** — Generate bucket pages in batches with parallel workers. High-level buckets are AI-planned; scanned endpoints enrich grouped API-reference pages instead of creating one page per route. Each page passes through an MDX compile-check (with bounded LLM-driven retry on failure) before being written to disk.
+3. **Phase 3: Generate** — Generate bucket pages in batches with parallel workers. High-level buckets are AI-planned; scanned endpoints enrich grouped API-reference pages instead of creating one page per route. Each page passes through Python-side MDX repair, grounding validation, and bounded quality retries before being written to disk.
 4. **Phase 4: API Ref** — Stage OpenAPI assets for the generated Fumadocs `/api/*` pages when a spec exists.
 5. **Phase 5: Build** — Write the generated `site/` Fumadocs scaffold, page tree, search route, and static assets from the generated plan.
 
@@ -1562,7 +1562,7 @@ deepdoc generate --force           # Full regen with new model
 ## Requirements
 
 - Python 3.10+
-- **Node 18+** on `PATH` — used at generate time for MDX compile validation, and at `serve`/`deploy` time for the Fumadocs site
+- **Node 18+** on `PATH` — used by `deepdoc serve` and `deepdoc deploy` for the generated Fumadocs site
 - Git (for `deepdoc update` and `deepdoc deploy`)
 - An LLM API key (or Ollama running locally)
 

{deepdoc-2.3.2 → deepdoc-2.3.4}/README.md

@@ -43,7 +43,7 @@ DeepDoc scans your repo, builds a bucket-based documentation plan, generates ric
 - **Bucket-based documentation architecture** — system, feature, endpoint, integration, and database buckets instead of one-file-per-page noise.
 - **Multi-step AI planner** — classifies the repo, proposes buckets, then assigns files, symbols, artifacts, and dependencies into a final reader-first plan.
 - **Giant-file handling** — large files are decomposed into feature-aligned clusters so a single controller can feed multiple doc pages.
-- **MDX compile gate** — every page is compile-checked before being written; broken MDX is auto-fixed via LLM retry or degraded to safe Markdown rather than shipped.
+- **MDX-safe generation** — generated pages are repaired and validated in Python before being written; deploy-time quality gates block failed, invalid, or stub pages before the Fumadocs build.
 - **Evidence-first chatbot answers** — final code proof is hydrated from archived source snippets with exact file paths and line ranges, not just retrieval guesses.
 - **Incremental updates** — `deepdoc update` regenerates only stale or structurally affected docs against the last synced commit.
 - **OpenAPI-aware API docs** — auto-detects OpenAPI/Swagger specs and stages interactive `/api/*` pages in the generated site.
@@ -90,7 +90,7 @@ pip install click litellm gitpython rich pyyaml jinja2
 pip install -e . --no-deps
 ```
 
-You will also need **Node 18+** on `PATH`. DeepDoc uses Node at generate time to compile-check each MDX page before writing it, and at `serve`/`deploy` time for the Fumadocs site. The small set of MDX validation dependencies installs into `deepdoc/generator/mdx_validator/node_modules/` automatically on first `deepdoc generate`.
+You will also need **Node 18+** on `PATH` when you run `deepdoc serve` or `deepdoc deploy`. DeepDoc generation itself uses Python-side repair and validation; Node is only needed for the generated Fumadocs site.
 
 ### Verify installation
 
@@ -307,7 +307,7 @@ deepdoc generate --exclude "tests/**"
 
 1. **Phase 1: Scan** — Walk the repo, parse supported languages, detect endpoints, config/setup artifacts, runtime surfaces, integration signals, and OpenAPI specs.
 2. **Phase 2: Plan** — Run the multi-step bucket planner. It classifies the repo, proposes bucket candidates, and assigns files/symbols/artifacts to the final doc structure.
-3. **Phase 3: Generate** — Generate bucket pages in batches with parallel workers. High-level buckets are AI-planned; scanned endpoints enrich grouped API-reference pages instead of creating one page per route. Each page passes through an MDX compile-check (with bounded LLM-driven retry on failure) before being written to disk.
+3. **Phase 3: Generate** — Generate bucket pages in batches with parallel workers. High-level buckets are AI-planned; scanned endpoints enrich grouped API-reference pages instead of creating one page per route. Each page passes through Python-side MDX repair, grounding validation, and bounded quality retries before being written to disk.
 4. **Phase 4: API Ref** — Stage OpenAPI assets for the generated Fumadocs `/api/*` pages when a spec exists.
 5. **Phase 5: Build** — Write the generated `site/` Fumadocs scaffold, page tree, search route, and static assets from the generated plan.
 
@@ -1523,7 +1523,7 @@ deepdoc generate --force           # Full regen with new model
 ## Requirements
 
 - Python 3.10+
-- **Node 18+** on `PATH` — used at generate time for MDX compile validation, and at `serve`/`deploy` time for the Fumadocs site
+- **Node 18+** on `PATH` — used by `deepdoc serve` and `deepdoc deploy` for the generated Fumadocs site
 - Git (for `deepdoc update` and `deepdoc deploy`)
 - An LLM API key (or Ollama running locally)
 

{deepdoc-2.3.2 → deepdoc-2.3.4}/deepdoc/__init__.py

@@ -1,3 +1,3 @@
 """DeepDoc — Auto-generate beautiful docs from any codebase."""
 
-__version__ = "2.3.2"
+__version__ = "2.3.4"

{deepdoc-2.3.2 → deepdoc-2.3.4}/deepdoc/generator/generation.py

@@ -611,6 +611,10 @@ class BucketGenerationEngine:
             content = repair_unbalanced_code_fences(content)
             content = normalize_explanatory_lines_outside_fences(content)
             content = repair_dangling_plain_fences(content)
+            content = normalize_fumadocs_directives(content)
+            content = fix_frontmatter_description(content)
+            content = fix_bare_mermaid_fences(content)
+            content = fix_leaf_card_directives(content)
             content = escape_mdx_angle_hazards(content)
             content = repair_internal_doc_links(
                 content,
@@ -650,6 +654,10 @@ class BucketGenerationEngine:
                     content = repair_unbalanced_code_fences(content)
                     content = normalize_explanatory_lines_outside_fences(content)
                     content = repair_dangling_plain_fences(content)
+                    content = normalize_fumadocs_directives(content)
+                    content = fix_frontmatter_description(content)
+                    content = fix_bare_mermaid_fences(content)
+                    content = fix_leaf_card_directives(content)
                     content = escape_mdx_angle_hazards(content)
                     content = repair_internal_doc_links(
                         content,
@@ -692,6 +700,10 @@ class BucketGenerationEngine:
                     content = repair_unbalanced_code_fences(content)
                     content = normalize_explanatory_lines_outside_fences(content)
                     content = repair_dangling_plain_fences(content)
+                    content = normalize_fumadocs_directives(content)
+                    content = fix_frontmatter_description(content)
+                    content = fix_bare_mermaid_fences(content)
+                    content = fix_leaf_card_directives(content)
                     content = escape_mdx_angle_hazards(content)
                     content = repair_internal_doc_links(
                         content,

{deepdoc-2.3.2 → deepdoc-2.3.4}/deepdoc/generator/post_processors.py

@@ -207,9 +207,13 @@ def _fix_mermaid_diagram(diagram: str) -> str:
                 dupes.append(nid)
             seen.add(nid)
         if dupes:
+            # Insert comment AFTER the diagram type declaration line, not before it
+            # (Mermaid requires diagram type as the very first line)
+            first_nl = result.index("\n") if "\n" in result else len(result)
             result = (
-                f"%% Note: possible duplicate node IDs: {', '.join(set(dupes))}\n"
-                + result
+                result[: first_nl + 1]
+                + f"%% Note: possible duplicate node IDs: {', '.join(set(dupes))}\n"
+                + result[first_nl + 1 :]
             )
 
     return result
@@ -800,6 +804,132 @@ def inject_source_files_disclosure(content: str, evidence_files: list[str]) -> s
     return patched if count else content
 
 
+def normalize_fumadocs_directives(content: str) -> str:
+    """Normalize fumadocs callout directive names to the supported set.
+
+    Valid fumadocs directive types: note, tip, info, warning, danger, caution.
+    The LLM frequently uses shorthands like :::warn, :::error, :::success.
+    """
+    _alias: dict[str, str] = {
+        "warn": "warning",
+        "caution": "warning",
+        "alert": "warning",
+        "error": "danger",
+        "success": "tip",
+        "check": "tip",
+        "hint": "tip",
+        "important": "note",
+        "notice": "note",
+    }
+
+    def replace_directive(m: re.Match) -> str:
+        name = m.group(1).strip().lower()
+        canonical = _alias.get(name, name)
+        return f":::{canonical}"
+
+    # Only replace opening directives (:::name), not closing :::
+    return re.sub(r":::([a-z]+)", replace_directive, content)
+
+
+def fix_frontmatter_description(content: str) -> str:
+    """Strip directive artefacts (trailing ::, :::) from frontmatter description field.
+
+    The LLM sometimes writes description values that end with :: or ::: which
+    the remark-directive plugin then partially parses, corrupting the nav sidebar.
+    """
+    def clean_description(m: re.Match) -> str:
+        prefix = m.group(1)   # "description: "
+        value = m.group(2)    # the value only
+        value = re.sub(r'\s*:{2,3}\s*$', '', value.rstrip())
+        return f'{prefix}{value}'
+
+    return re.sub(
+        r'^(description:\s*)(["\']?.*?["\']?)\s*$',
+        clean_description,
+        content,
+        flags=re.MULTILINE,
+    )
+
+
+def fix_bare_mermaid_fences(content: str) -> str:
+    """Repair mermaid diagrams where the LLM omitted the opening code fence.
+
+    The LLM sometimes writes just the word 'mermaid' on its own line followed
+    by diagram content and a closing ``` — instead of the correct ```mermaid
+    opening fence.  This produces a bare 'mermaid' paragraph in MDX and leaves
+    the diagram body (including {node} labels) unescaped, causing acorn parse
+    errors.
+
+    Pattern matched:
+        <text ending in : or .\n>
+        mermaid\n
+        <diagram type line, e.g. sequenceDiagram>\n
+        ...diagram body...
+        ```
+    """
+    return re.sub(
+        r'(?m)^mermaid\n(?=(sequenceDiagram|flowchart|graph|classDiagram|stateDiagram|erDiagram|gantt|pie|gitGraph|mindmap|timeline|journey|quadrantChart|xychart|block|packet|kanban|architecture))',
+        '```mermaid\n',
+        content,
+    )
+
+
+def fix_leaf_card_directives(content: str) -> str:
+    """Convert LLM-invented ::card{...}\\nCONTENT\\n:: to :::card{...}\\nCONTENT\\n:::.
+
+    The fumadocs remark-directive plugin expects 'card' to be a *container*
+    directive (:::card ... :::) so it can carry child content.  The LLM
+    frequently uses the *leaf* directive form (::card) with a standalone ::
+    close marker — which remark-directive does not recognise, rendering both
+    the content and the :: as raw text.
+    """
+    def fix_cards_block(m: re.Match) -> str:
+        inner = m.group(1)
+        # ::card{...}\nCONTENT\n::  →  :::card{...}\nCONTENT\n:::
+        inner = re.sub(
+            r'^::card(\{[^}]*\})\n(.*?)\n^::$',
+            r':::card\1\n\2\n:::',
+            inner,
+            flags=re.MULTILINE | re.DOTALL,
+        )
+        return f':::cards\n{inner}\n:::'
+
+    return re.sub(
+        r':::cards\n(.*?)\n:::',
+        fix_cards_block,
+        content,
+        flags=re.DOTALL,
+    )
+
+
+def unwrap_markdown_trapped_in_code_fences(content: str) -> str:
+    """Detect code fences that contain markdown content and unwrap them.
+
+    The LLM sometimes forgets to close a code fence, leaving section headers
+    (## ...), callout directives (:::), and horizontal rules (---) trapped
+    inside a code block. These render as literal text instead of MDX.
+    """
+    # Pattern: a fenced block that contains markdown indicators
+    MD_INDICATORS = re.compile(
+        r"^(?:#{1,6}\s|:::|\-\-\-\s*$|^\*\*\*\s*$)",
+        re.MULTILINE,
+    )
+
+    def maybe_unwrap(m: re.Match) -> str:
+        lang = m.group(1)  # may be empty for plain ```
+        body = m.group(2)
+        # If body contains markdown indicators, unwrap it
+        if MD_INDICATORS.search(body):
+            return body
+        return m.group(0)  # leave intact
+
+    return re.sub(
+        r"```([A-Za-z0-9_+-]*)\n([\s\S]*?)\n```",
+        maybe_unwrap,
+        content,
+    )
+
+
 def extract_glossary_terms(glossary_content: str) -> list[tuple[str, str]]:
     """Parse `### TermName` (h3) headings from glossary MDX → [(term, anchor-slug)].
 

{deepdoc-2.3.2 → deepdoc-2.3.4}/deepdoc/llm/client.py

@@ -19,8 +19,10 @@ class LLMClient:
         # max_tokens=None means don't cap — let the model use its full output capacity
         self.max_tokens = llm_cfg.get("max_tokens", None)
         self.temperature = llm_cfg.get("temperature", 0.2)
-        self.base_url = llm_cfg.get("base_url")
-        self.api_version = llm_cfg.get("api_version")
+        self.base_url = str(llm_cfg.get("base_url") or "") or None
+        # YAML parses bare dates like 2024-12-01 as datetime.date — coerce to str
+        _api_version = llm_cfg.get("api_version")
+        self.api_version = str(_api_version).strip() if _api_version is not None else None
         self.usage: dict[str, int] = {
             "calls": 0,
             "prompt_chars": 0,
@@ -65,8 +67,9 @@ class LLMClient:
 
         is_azure = provider.lower() == "azure" or model.lower().startswith("azure/")
         if is_azure:
-            base_url = (llm_cfg.get("base_url") or "").strip()
-            api_version = (llm_cfg.get("api_version") or "").strip()
+            base_url = str(llm_cfg.get("base_url") or "").strip()
+            # YAML parses bare dates like 2025-07-01 as datetime.date — coerce to str
+            api_version = str(llm_cfg.get("api_version") or "").strip()
             missing = []
             if not base_url:
                 missing.append("llm.base_url  (your Azure OpenAI endpoint URL)")

{deepdoc-2.3.2 → deepdoc-2.3.4}/deepdoc/site/builder/scaffold_files.py

@@ -1670,21 +1670,8 @@ def _mermaid_component_tsx() -> str:
           }
           render() {
             if (this.state.error) {
-              return (
-                <pre
-                  style={{
-                    padding: '0.75rem 1rem',
-                    background: 'var(--color-fd-muted)',
-                    borderRadius: '6px',
-                    fontSize: '0.75rem',
-                    color: 'var(--color-fd-muted-foreground)',
-                    whiteSpace: 'pre-wrap',
-                    wordBreak: 'break-word',
-                  }}
-                >
-                  {'[diagram parse error] ' + this.state.error}
-                </pre>
-              );
+              // Silently hide diagrams that fail to parse — noise-free fallback
+              return null;
             }
             return this.props.children;
           }

{deepdoc-2.3.2 → deepdoc-2.3.4}/deepdoc.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: deepdoc
-Version: 2.3.2
+Version: 2.3.4
 Summary: Auto-generate beautiful docs from any codebase
 Author: Pranav Kumar
 License: MIT
@@ -82,7 +82,7 @@ DeepDoc scans your repo, builds a bucket-based documentation plan, generates ric
 - **Bucket-based documentation architecture** — system, feature, endpoint, integration, and database buckets instead of one-file-per-page noise.
 - **Multi-step AI planner** — classifies the repo, proposes buckets, then assigns files, symbols, artifacts, and dependencies into a final reader-first plan.
 - **Giant-file handling** — large files are decomposed into feature-aligned clusters so a single controller can feed multiple doc pages.
-- **MDX compile gate** — every page is compile-checked before being written; broken MDX is auto-fixed via LLM retry or degraded to safe Markdown rather than shipped.
+- **MDX-safe generation** — generated pages are repaired and validated in Python before being written; deploy-time quality gates block failed, invalid, or stub pages before the Fumadocs build.
 - **Evidence-first chatbot answers** — final code proof is hydrated from archived source snippets with exact file paths and line ranges, not just retrieval guesses.
 - **Incremental updates** — `deepdoc update` regenerates only stale or structurally affected docs against the last synced commit.
 - **OpenAPI-aware API docs** — auto-detects OpenAPI/Swagger specs and stages interactive `/api/*` pages in the generated site.
@@ -129,7 +129,7 @@ pip install click litellm gitpython rich pyyaml jinja2
 pip install -e . --no-deps
 ```
 
-You will also need **Node 18+** on `PATH`. DeepDoc uses Node at generate time to compile-check each MDX page before writing it, and at `serve`/`deploy` time for the Fumadocs site. The small set of MDX validation dependencies installs into `deepdoc/generator/mdx_validator/node_modules/` automatically on first `deepdoc generate`.
+You will also need **Node 18+** on `PATH` when you run `deepdoc serve` or `deepdoc deploy`. DeepDoc generation itself uses Python-side repair and validation; Node is only needed for the generated Fumadocs site.
 
 ### Verify installation
 
@@ -346,7 +346,7 @@ deepdoc generate --exclude "tests/**"
 
 1. **Phase 1: Scan** — Walk the repo, parse supported languages, detect endpoints, config/setup artifacts, runtime surfaces, integration signals, and OpenAPI specs.
 2. **Phase 2: Plan** — Run the multi-step bucket planner. It classifies the repo, proposes bucket candidates, and assigns files/symbols/artifacts to the final doc structure.
-3. **Phase 3: Generate** — Generate bucket pages in batches with parallel workers. High-level buckets are AI-planned; scanned endpoints enrich grouped API-reference pages instead of creating one page per route. Each page passes through an MDX compile-check (with bounded LLM-driven retry on failure) before being written to disk.
+3. **Phase 3: Generate** — Generate bucket pages in batches with parallel workers. High-level buckets are AI-planned; scanned endpoints enrich grouped API-reference pages instead of creating one page per route. Each page passes through Python-side MDX repair, grounding validation, and bounded quality retries before being written to disk.
 4. **Phase 4: API Ref** — Stage OpenAPI assets for the generated Fumadocs `/api/*` pages when a spec exists.
 5. **Phase 5: Build** — Write the generated `site/` Fumadocs scaffold, page tree, search route, and static assets from the generated plan.
 
@@ -1562,7 +1562,7 @@ deepdoc generate --force           # Full regen with new model
 ## Requirements
 
 - Python 3.10+
-- **Node 18+** on `PATH` — used at generate time for MDX compile validation, and at `serve`/`deploy` time for the Fumadocs site
+- **Node 18+** on `PATH` — used by `deepdoc serve` and `deepdoc deploy` for the generated Fumadocs site
 - Git (for `deepdoc update` and `deepdoc deploy`)
 - An LLM API key (or Ollama running locally)
 

{deepdoc-2.3.2 → deepdoc-2.3.4}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "deepdoc"
-version = "2.3.2"
+version = "2.3.4"
 description = "Auto-generate beautiful docs from any codebase"
 readme = "README.md"
 authors = [
@@ -57,9 +57,6 @@ Issues = "https://github.com/tss-pranavkumar/deepdoc/issues"
 where = ["."]
 include = ["deepdoc*"]
 
-[tool.setuptools.package-data]
-"deepdoc.generator.mdx_validator" = ["validate.mjs", "package.json"]
-
 [tool.pytest.ini_options]
 testpaths = ["tests"]
 python_files = ["test_*.py"]