deepdoc 0.4.2__tar.gz → 0.5.0__tar.gz

{deepdoc-0.4.2 → deepdoc-0.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: deepdoc
-Version: 0.4.2
+Version: 0.5.0
 Summary: Auto-generate beautiful docs from any codebase
 Author: Pranav Kumar
 License: MIT
@@ -88,7 +88,15 @@ DeepDoc scans your repo, builds a bucket-based documentation plan, generates ric
 - **OpenAPI-aware API docs** — auto-detects OpenAPI/Swagger specs and renders an interactive Swagger UI page in the generated site.
 - **Local-first Next.js + Fumadocs site** — generates a `site/` shell (Fumadocs UI, Mermaid, syntax highlighting, built-in search); `deepdoc deploy` runs `next build` and exports a static site to any host. Requires Node.js ≥18.
 
-Works with Anthropic, OpenAI, Azure OpenAI, Google Gemini, Ollama, and any other LiteLLM-compatible provider. Parses Python, JavaScript/TypeScript, Go, PHP, and Vue.
+Works with Anthropic, OpenAI, Azure OpenAI, Google Gemini, Ollama, and any other LiteLLM-compatible provider.
+
+> **Use DeepDoc for its supported stacks.** DeepDoc is built and tested for
+> Python backends using **Django, Django REST Framework, or Falcon**;
+> JavaScript/TypeScript backends using **Express, Fastify, or NestJS**;
+> **Laravel** PHP applications; and conventional **Go** services. It also
+> extracts symbols from Vue SFCs. It is deliberately **not** a generic
+> framework-agnostic documentation generator: do not rely on it for FastAPI,
+> Flask, Nuxt, or unsupported frameworks just because their files can be parsed.
 
 ---
 
@@ -396,7 +404,7 @@ Generation writes quality artifacts under `.deepdoc/`:
 - `.deepdoc/generation_quality.json` records invalid/degraded pages, coverage metrics, local setup warnings, and consistency summary data.
 - `.deepdoc/consistency_warnings.json` records warning-only cross-page identifier consistency findings.
 
-Generated Markdown pages include provenance frontmatter such as `deepdoc_generated_commit`, `deepdoc_generated_at`, `deepdoc_generated_version`, `deepdoc_status`, `deepdoc_evidence_files`, and `deepdoc_prereqs` (prerequisite page slugs). The Fumadocs UI renders these pages with built-in navigation, search, and table of contents.
+Generated Markdown pages include provenance frontmatter such as `deepdoc_generated_commit`, `deepdoc_generated_at`, `deepdoc_generated_version`, `deepdoc_status`, `deepdoc_evidence_files`, `deepdoc_prereqs`, and evidence-budget usage/trimming fields. The Fumadocs UI renders these pages with built-in navigation, search, and table of contents.
 
 **Options:**
 
@@ -416,6 +424,20 @@ deepdoc status
 
 This is useful after `generate` or `update` when you want a quick health check without opening the site.
 
+### `deepdoc performance`
+
+Show the latest local pipeline timings, LLM token usage, retries, and I/O counters.
+
+```bash
+deepdoc performance
+```
+
+DeepDoc stores one sanitized JSON record per completed `generate` or `update`
+under `.deepdoc/performance/runs.jsonl`. Records contain measurements only—never
+source code, prompts, generated content, API keys, or endpoint URLs. The history
+rotates when it reaches 10 MB, and the command compares the latest duration with
+the previous run when available.
+
 ### `deepdoc serve`
 
 Preview the generated docs locally with live reload using the Next.js + Fumadocs site in `site/`.
@@ -627,8 +649,16 @@ llm:
   model: claude-3-5-sonnet-20241022
   api_key_env: ANTHROPIC_API_KEY
   base_url: null                    # Set for Ollama/custom endpoints
-  max_tokens: null                  # null = no cap (recommended); set a number to limit output
+  max_tokens: null                  # null = provider default; set a number to request an explicit cap
+  base_model: null                  # Set only for a custom/Azure deployment alias
+  context_window_tokens: null       # null = resolve locally from LiteLLM model metadata
+  output_reserve_tokens: null       # null = derive from the resolved model output capability
   temperature: 0.2
+  rate_limits:
+    max_concurrency: 6
+    requests_per_minute: 60
+    tokens_per_minute: 250000
+    adaptive_backoff: true
 
 languages:
   - python
@@ -690,8 +720,15 @@ site:
 | `llm.model` | `claude-3-5-sonnet-20241022` | Model name (use provider prefix for non-Anthropic, e.g. `azure/gpt-4.1`) |
 | `llm.api_key_env` | `ANTHROPIC_API_KEY` | Environment variable that holds the API key |
 | `llm.base_url` | `null` | Custom endpoint URL (required for Ollama, optional for Azure) |
-| `llm.max_tokens` | `null` | Max output tokens per LLM call. `null` = no cap (recommended). Set explicitly if your provider requires it (e.g. some Azure deployments). Typical values: `4096` for shorter pages, `8192`–`16384` for detailed docs |
+| `llm.base_model` | `null` | Canonical LiteLLM model name for an otherwise unknown Azure/custom deployment alias |
+| `llm.max_tokens` | `null` | Requested provider output cap. `null` omits the parameter and uses the provider default; configured values are clamped to `output_reserve_tokens` |
 | `llm.temperature` | `0.2` | LLM sampling temperature |
+| `llm.context_window_tokens` | `null` | Model context window resolved locally from LiteLLM; set an integer for an unknown/private deployment alias |
+| `llm.output_reserve_tokens` | `null` | Output reservation derived from the resolved model; set an integer to override or cap explicit `max_tokens` |
+| `llm.rate_limits.max_concurrency` | `6` | Maximum simultaneous documentation-model requests |
+| `llm.rate_limits.requests_per_minute` | `60` | Rolling request limit for the documentation model |
+| `llm.rate_limits.tokens_per_minute` | `250000` | Rolling estimated input-token limit for the documentation model |
+| `llm.rate_limits.adaptive_backoff` | `true` | Share provider `429`/`Retry-After` cooldown across workers |
 | **Generation** | | |
 | `generation_mode` | `feature_buckets` | Documentation generation mode |
 | `max_pages` | `0` | Max pages to generate. `0` = no cap |
@@ -701,8 +738,9 @@ site:
 | `include_endpoint_pages` | `true` | Generate endpoint documentation pages |
 | `include_integration_pages` | `true` | Generate integration documentation pages |
 | **Parallelism** | | |
-| `max_parallel_workers` | `6` | Concurrent LLM calls. Increase for Azure PTU or high-TPM deployments |
-| `batch_size` | `10` | Pages per batch before rate-limit pause |
+| `max_parallel_workers` | `6` | Generation executor size; actual hosted request concurrency is also bounded by `llm.rate_limits.max_concurrency` |
+| `batch_size` | `10` | Submission-throttle cadence retained for compatibility; generation uses one rolling executor, not batch barriers |
+| `scan.max_workers` | `8` | Bounded local workers for source reads, parsing, and per-file endpoint detection (`1` forces serial scanning) |
 | **File filters** | | |
 | `languages` | `[python, javascript, typescript, go, php, vue]` | Languages to parse |
 | `include` | `[]` | Glob patterns to include (empty = everything) |
@@ -748,7 +786,7 @@ All chatbot endpoints now share one response contract:
 - `evidence[]` is the canonical source of right-pane code/config snippets. Each item has an ID like `E1`, `file_path`, `start_line`, `end_line`, and `snippet`.
 - `references[]` contains generated docs or repo-authored docs. These are read-next links, not implementation proof.
 - Legacy fields such as `code_citations`, `doc_links`, `code_workspace_citations`, and `file_inventory` are derived for compatibility.
-- SQLite FTS, FAISS vectors, symbol chunks, and relationship chunks are candidate retrieval artifacts only. A candidate becomes evidence only after it is hydrated from the source archive/catalog.
+- SQLite FTS, FAISS vectors, symbol chunks, and relationship chunks are candidate retrieval artifacts only. A candidate becomes evidence only after it is hydrated from the content-addressed SQLite source archive/catalog. Existing gzip source archives migrate automatically on the next update.
 - Generated/internal paths such as `.deepdoc*`, `docs/`, `site/`, and `chatbot_backend/` are excluded from source evidence.
 - The answer validator rejects invented source paths, `line unknown`, unknown evidence IDs, and docs used as implementation proof. If a retry still fails, the backend returns a conservative answer with diagnostics.
 
@@ -966,7 +1004,7 @@ The defaults work for almost every project. Expand below only when you need to t
 |-----|---------|-------------|
 | **General** | | |
 | `chatbot.enabled` | `false` | Enable chatbot indexing and backend (set automatically by `deepdoc init --with-chatbot`) |
-| `chatbot.index_dir` | `.deepdoc/chatbot` | Directory for source archive/catalog, SQLite lexical index, vector indexes, relationship artifacts, and chunk data |
+| `chatbot.index_dir` | `.deepdoc/chatbot` | Directory for the transactional SQLite source archive/catalog, SQLite lexical index, vector indexes, relationship artifacts, and chunk data |
 | **Indexing** | | |
 | `chatbot.indexing.include_repo_docs` | `true` | Index selected repo-authored docs such as README/design notes in a separate corpus |
 | `chatbot.indexing.include_tests` | `false` | Allow test/example/fixture docs into the repo-doc corpus |
@@ -1244,7 +1282,26 @@ deepdoc clean --yes    # Skip confirmation prompt
 
 ## Supported Languages & Frameworks
 
-**Parsing (tree-sitter AST + regex fallback):**
+> ## Supported Stacks Only
+>
+> **Use DeepDoc when your repository is primarily one of these targets:**
+>
+> - **Python:** Django, Django REST Framework, Falcon
+> - **JavaScript / TypeScript:** Express, Fastify, NestJS
+> - **PHP:** Laravel
+> - **Go:** conventional HTTP services and supported route helpers
+> - **Vue:** component and symbol extraction, not a standalone backend route target
+>
+> DeepDoc can parse a number of source formats, but parsing is not the same as
+> full framework support. Endpoint resolution, runtime discovery, call-graph
+> enrichment, and generated API documentation are only guaranteed for the
+> supported stacks above. **Do not adopt DeepDoc for FastAPI, Flask, Nuxt, or an
+> unlisted framework unless you are prepared to extend scanner coverage first.**
+
+### Supported Source Parsing
+
+These file types receive AST/symbol extraction. This alone does not imply
+framework-level route or runtime coverage.
 
 | Language | Extensions | Extracts |
 |----------|-----------|----------|
@@ -1255,7 +1312,7 @@ deepdoc clean --yes    # Skip confirmation prompt
 | PHP | `.php` | Functions, classes, methods, namespaces |
 | Vue | `.vue` | SFC script symbols, props/emits/slots, router/store usage |
 
-**High-confidence framework support (fixture-backed):**
+### Supported Framework Targets
 
 | Framework | Language | Proven patterns |
 |-----------|----------|-----------------|
@@ -1263,14 +1320,28 @@ deepdoc clean --yes    # Skip confirmation prompt
 | Django / DRF | Python | `path()`, `re_path()`, `@api_view`, `as_view()`, DRF routers, `@action` |
 | Express | JS/TS | Mounted routers via `app.use()`, nested prefixes, chained `route()` calls |
 | Fastify | JS/TS | Plugin `register(..., { prefix })`, shorthand methods, `route({ ... })`, schema hints |
+| NestJS | JS/TS | Decorator controllers and method routes |
 | Falcon | Python | `app.add_route()`, responder classes, imported resources, app middleware |
-| Vue | Vue SFC | Component detection, `defineProps`, `defineEmits`, `defineModel`, `defineSlots`, router/store signals |
+| Go services | Go | Conventional HTTP handlers and common route helpers |
 
-**Supported but not headline-high-confidence yet:**
+### Parser Coverage, Not A Supported Backend Target
 
 | Framework | Language | Current coverage |
 |-----------|----------|------------------|
-| Gin / Echo / Fiber | Go | Common route helpers (`GET`, `POST`, `HandleFunc`) |
+| Vue SFC | Vue | Component detection, `defineProps`, `defineEmits`, `defineModel`, `defineSlots`, router/store signals |
+| Gin / Echo / Fiber | Go | Common route helpers (`GET`, `POST`, `HandleFunc`); validate against your codebase before adoption |
+
+### Explicitly Unsupported Scan Targets
+
+| Framework | Why this matters |
+|-----------|------------------|
+| FastAPI | No supported route-resolution or scanner contract |
+| Flask | No supported route-resolution or scanner contract |
+| Nuxt | No supported route-resolution or scanner contract |
+
+If your framework is not listed under **Supported Framework Targets**, treat it
+as unsupported. Extend `deepdoc/scanner/` and route fixtures before using
+DeepDoc as the source of truth for that stack.
 
 **Runtime/background surface extraction:**
 

{deepdoc-0.4.2 → deepdoc-0.5.0}/README.md

@@ -49,7 +49,15 @@ DeepDoc scans your repo, builds a bucket-based documentation plan, generates ric
 - **OpenAPI-aware API docs** — auto-detects OpenAPI/Swagger specs and renders an interactive Swagger UI page in the generated site.
 - **Local-first Next.js + Fumadocs site** — generates a `site/` shell (Fumadocs UI, Mermaid, syntax highlighting, built-in search); `deepdoc deploy` runs `next build` and exports a static site to any host. Requires Node.js ≥18.
 
-Works with Anthropic, OpenAI, Azure OpenAI, Google Gemini, Ollama, and any other LiteLLM-compatible provider. Parses Python, JavaScript/TypeScript, Go, PHP, and Vue.
+Works with Anthropic, OpenAI, Azure OpenAI, Google Gemini, Ollama, and any other LiteLLM-compatible provider.
+
+> **Use DeepDoc for its supported stacks.** DeepDoc is built and tested for
+> Python backends using **Django, Django REST Framework, or Falcon**;
+> JavaScript/TypeScript backends using **Express, Fastify, or NestJS**;
+> **Laravel** PHP applications; and conventional **Go** services. It also
+> extracts symbols from Vue SFCs. It is deliberately **not** a generic
+> framework-agnostic documentation generator: do not rely on it for FastAPI,
+> Flask, Nuxt, or unsupported frameworks just because their files can be parsed.
 
 ---
 
@@ -357,7 +365,7 @@ Generation writes quality artifacts under `.deepdoc/`:
 - `.deepdoc/generation_quality.json` records invalid/degraded pages, coverage metrics, local setup warnings, and consistency summary data.
 - `.deepdoc/consistency_warnings.json` records warning-only cross-page identifier consistency findings.
 
-Generated Markdown pages include provenance frontmatter such as `deepdoc_generated_commit`, `deepdoc_generated_at`, `deepdoc_generated_version`, `deepdoc_status`, `deepdoc_evidence_files`, and `deepdoc_prereqs` (prerequisite page slugs). The Fumadocs UI renders these pages with built-in navigation, search, and table of contents.
+Generated Markdown pages include provenance frontmatter such as `deepdoc_generated_commit`, `deepdoc_generated_at`, `deepdoc_generated_version`, `deepdoc_status`, `deepdoc_evidence_files`, `deepdoc_prereqs`, and evidence-budget usage/trimming fields. The Fumadocs UI renders these pages with built-in navigation, search, and table of contents.
 
 **Options:**
 
@@ -377,6 +385,20 @@ deepdoc status
 
 This is useful after `generate` or `update` when you want a quick health check without opening the site.
 
+### `deepdoc performance`
+
+Show the latest local pipeline timings, LLM token usage, retries, and I/O counters.
+
+```bash
+deepdoc performance
+```
+
+DeepDoc stores one sanitized JSON record per completed `generate` or `update`
+under `.deepdoc/performance/runs.jsonl`. Records contain measurements only—never
+source code, prompts, generated content, API keys, or endpoint URLs. The history
+rotates when it reaches 10 MB, and the command compares the latest duration with
+the previous run when available.
+
 ### `deepdoc serve`
 
 Preview the generated docs locally with live reload using the Next.js + Fumadocs site in `site/`.
@@ -588,8 +610,16 @@ llm:
   model: claude-3-5-sonnet-20241022
   api_key_env: ANTHROPIC_API_KEY
   base_url: null                    # Set for Ollama/custom endpoints
-  max_tokens: null                  # null = no cap (recommended); set a number to limit output
+  max_tokens: null                  # null = provider default; set a number to request an explicit cap
+  base_model: null                  # Set only for a custom/Azure deployment alias
+  context_window_tokens: null       # null = resolve locally from LiteLLM model metadata
+  output_reserve_tokens: null       # null = derive from the resolved model output capability
   temperature: 0.2
+  rate_limits:
+    max_concurrency: 6
+    requests_per_minute: 60
+    tokens_per_minute: 250000
+    adaptive_backoff: true
 
 languages:
   - python
@@ -651,8 +681,15 @@ site:
 | `llm.model` | `claude-3-5-sonnet-20241022` | Model name (use provider prefix for non-Anthropic, e.g. `azure/gpt-4.1`) |
 | `llm.api_key_env` | `ANTHROPIC_API_KEY` | Environment variable that holds the API key |
 | `llm.base_url` | `null` | Custom endpoint URL (required for Ollama, optional for Azure) |
-| `llm.max_tokens` | `null` | Max output tokens per LLM call. `null` = no cap (recommended). Set explicitly if your provider requires it (e.g. some Azure deployments). Typical values: `4096` for shorter pages, `8192`–`16384` for detailed docs |
+| `llm.base_model` | `null` | Canonical LiteLLM model name for an otherwise unknown Azure/custom deployment alias |
+| `llm.max_tokens` | `null` | Requested provider output cap. `null` omits the parameter and uses the provider default; configured values are clamped to `output_reserve_tokens` |
 | `llm.temperature` | `0.2` | LLM sampling temperature |
+| `llm.context_window_tokens` | `null` | Model context window resolved locally from LiteLLM; set an integer for an unknown/private deployment alias |
+| `llm.output_reserve_tokens` | `null` | Output reservation derived from the resolved model; set an integer to override or cap explicit `max_tokens` |
+| `llm.rate_limits.max_concurrency` | `6` | Maximum simultaneous documentation-model requests |
+| `llm.rate_limits.requests_per_minute` | `60` | Rolling request limit for the documentation model |
+| `llm.rate_limits.tokens_per_minute` | `250000` | Rolling estimated input-token limit for the documentation model |
+| `llm.rate_limits.adaptive_backoff` | `true` | Share provider `429`/`Retry-After` cooldown across workers |
 | **Generation** | | |
 | `generation_mode` | `feature_buckets` | Documentation generation mode |
 | `max_pages` | `0` | Max pages to generate. `0` = no cap |
@@ -662,8 +699,9 @@ site:
 | `include_endpoint_pages` | `true` | Generate endpoint documentation pages |
 | `include_integration_pages` | `true` | Generate integration documentation pages |
 | **Parallelism** | | |
-| `max_parallel_workers` | `6` | Concurrent LLM calls. Increase for Azure PTU or high-TPM deployments |
-| `batch_size` | `10` | Pages per batch before rate-limit pause |
+| `max_parallel_workers` | `6` | Generation executor size; actual hosted request concurrency is also bounded by `llm.rate_limits.max_concurrency` |
+| `batch_size` | `10` | Submission-throttle cadence retained for compatibility; generation uses one rolling executor, not batch barriers |
+| `scan.max_workers` | `8` | Bounded local workers for source reads, parsing, and per-file endpoint detection (`1` forces serial scanning) |
 | **File filters** | | |
 | `languages` | `[python, javascript, typescript, go, php, vue]` | Languages to parse |
 | `include` | `[]` | Glob patterns to include (empty = everything) |
@@ -709,7 +747,7 @@ All chatbot endpoints now share one response contract:
 - `evidence[]` is the canonical source of right-pane code/config snippets. Each item has an ID like `E1`, `file_path`, `start_line`, `end_line`, and `snippet`.
 - `references[]` contains generated docs or repo-authored docs. These are read-next links, not implementation proof.
 - Legacy fields such as `code_citations`, `doc_links`, `code_workspace_citations`, and `file_inventory` are derived for compatibility.
-- SQLite FTS, FAISS vectors, symbol chunks, and relationship chunks are candidate retrieval artifacts only. A candidate becomes evidence only after it is hydrated from the source archive/catalog.
+- SQLite FTS, FAISS vectors, symbol chunks, and relationship chunks are candidate retrieval artifacts only. A candidate becomes evidence only after it is hydrated from the content-addressed SQLite source archive/catalog. Existing gzip source archives migrate automatically on the next update.
 - Generated/internal paths such as `.deepdoc*`, `docs/`, `site/`, and `chatbot_backend/` are excluded from source evidence.
 - The answer validator rejects invented source paths, `line unknown`, unknown evidence IDs, and docs used as implementation proof. If a retry still fails, the backend returns a conservative answer with diagnostics.
 
@@ -927,7 +965,7 @@ The defaults work for almost every project. Expand below only when you need to t
 |-----|---------|-------------|
 | **General** | | |
 | `chatbot.enabled` | `false` | Enable chatbot indexing and backend (set automatically by `deepdoc init --with-chatbot`) |
-| `chatbot.index_dir` | `.deepdoc/chatbot` | Directory for source archive/catalog, SQLite lexical index, vector indexes, relationship artifacts, and chunk data |
+| `chatbot.index_dir` | `.deepdoc/chatbot` | Directory for the transactional SQLite source archive/catalog, SQLite lexical index, vector indexes, relationship artifacts, and chunk data |
 | **Indexing** | | |
 | `chatbot.indexing.include_repo_docs` | `true` | Index selected repo-authored docs such as README/design notes in a separate corpus |
 | `chatbot.indexing.include_tests` | `false` | Allow test/example/fixture docs into the repo-doc corpus |
@@ -1205,7 +1243,26 @@ deepdoc clean --yes    # Skip confirmation prompt
 
 ## Supported Languages & Frameworks
 
-**Parsing (tree-sitter AST + regex fallback):**
+> ## Supported Stacks Only
+>
+> **Use DeepDoc when your repository is primarily one of these targets:**
+>
+> - **Python:** Django, Django REST Framework, Falcon
+> - **JavaScript / TypeScript:** Express, Fastify, NestJS
+> - **PHP:** Laravel
+> - **Go:** conventional HTTP services and supported route helpers
+> - **Vue:** component and symbol extraction, not a standalone backend route target
+>
+> DeepDoc can parse a number of source formats, but parsing is not the same as
+> full framework support. Endpoint resolution, runtime discovery, call-graph
+> enrichment, and generated API documentation are only guaranteed for the
+> supported stacks above. **Do not adopt DeepDoc for FastAPI, Flask, Nuxt, or an
+> unlisted framework unless you are prepared to extend scanner coverage first.**
+
+### Supported Source Parsing
+
+These file types receive AST/symbol extraction. This alone does not imply
+framework-level route or runtime coverage.
 
 | Language | Extensions | Extracts |
 |----------|-----------|----------|
@@ -1216,7 +1273,7 @@ deepdoc clean --yes    # Skip confirmation prompt
 | PHP | `.php` | Functions, classes, methods, namespaces |
 | Vue | `.vue` | SFC script symbols, props/emits/slots, router/store usage |
 
-**High-confidence framework support (fixture-backed):**
+### Supported Framework Targets
 
 | Framework | Language | Proven patterns |
 |-----------|----------|-----------------|
@@ -1224,14 +1281,28 @@ deepdoc clean --yes    # Skip confirmation prompt
 | Django / DRF | Python | `path()`, `re_path()`, `@api_view`, `as_view()`, DRF routers, `@action` |
 | Express | JS/TS | Mounted routers via `app.use()`, nested prefixes, chained `route()` calls |
 | Fastify | JS/TS | Plugin `register(..., { prefix })`, shorthand methods, `route({ ... })`, schema hints |
+| NestJS | JS/TS | Decorator controllers and method routes |
 | Falcon | Python | `app.add_route()`, responder classes, imported resources, app middleware |
-| Vue | Vue SFC | Component detection, `defineProps`, `defineEmits`, `defineModel`, `defineSlots`, router/store signals |
+| Go services | Go | Conventional HTTP handlers and common route helpers |
 
-**Supported but not headline-high-confidence yet:**
+### Parser Coverage, Not A Supported Backend Target
 
 | Framework | Language | Current coverage |
 |-----------|----------|------------------|
-| Gin / Echo / Fiber | Go | Common route helpers (`GET`, `POST`, `HandleFunc`) |
+| Vue SFC | Vue | Component detection, `defineProps`, `defineEmits`, `defineModel`, `defineSlots`, router/store signals |
+| Gin / Echo / Fiber | Go | Common route helpers (`GET`, `POST`, `HandleFunc`); validate against your codebase before adoption |
+
+### Explicitly Unsupported Scan Targets
+
+| Framework | Why this matters |
+|-----------|------------------|
+| FastAPI | No supported route-resolution or scanner contract |
+| Flask | No supported route-resolution or scanner contract |
+| Nuxt | No supported route-resolution or scanner contract |
+
+If your framework is not listed under **Supported Framework Targets**, treat it
+as unsupported. Extend `deepdoc/scanner/` and route fixtures before using
+DeepDoc as the source of truth for that stack.
 
 **Runtime/background surface extraction:**
 

{deepdoc-0.4.2 → deepdoc-0.5.0}/deepdoc/__init__.py

@@ -1,3 +1,3 @@
 """DeepDoc — Auto-generate beautiful docs from any codebase."""
 
-__version__ = "3.3.1"
+__version__ = "0.5.0"

{deepdoc-0.4.2 → deepdoc-0.5.0}/deepdoc/chatbot/answer_mixin.py

@@ -57,13 +57,17 @@ class AnswerMixin:
         current = answer
         retries = 0
         while retries < max_retries and self._answer_looks_incomplete(current):
-            continuation_prompt = (
+            instruction = (
                 "The previous answer appears incomplete and ended abruptly. "
                 "Continue from the exact point where it stopped. "
                 "Do not repeat earlier sections. Complete any unfinished bullets, "
-                "headings, or sentences, and end with `## Summary`.\n\n"
-                "Previous answer tail:\n"
-                f"{current[-context_chars:]}"
+                "headings, or sentences, and end with `## Summary`."
+            )
+            fit_continuation = getattr(self.chat_client, "fit_continuation_prompt", None)
+            continuation_prompt = (
+                fit_continuation(system, instruction, current, context_chars)
+                if callable(fit_continuation)
+                else instruction + "\n\nPrevious answer tail:\n" + current[-context_chars:]
             )
             continuation = self.chat_client.complete(system, continuation_prompt)
             if not continuation or not continuation.strip():
@@ -182,11 +186,11 @@ class AnswerMixin:
             if content:
                 history_lines.append(f"{role.title()}: {content}")
 
-        sections = [f"Question: {question}"]
+        required_sections = [f"Question: {question}"]
         if history_lines:
-            sections.append("Conversation:\n" + "\n".join(history_lines))
+            required_sections.append("Conversation:\n" + "\n".join(history_lines))
 
-        used = sum(len(s) for s in sections)
+        used = sum(len(section) for section in required_sections)
         evidence, _ = self._evidence_from_workspace_rows(
             self._code_workspace_citations(
                 {
@@ -220,7 +224,7 @@ class AnswerMixin:
                 "do not cite docs as source proof:\n"
                 + "\n\n".join(evidence_blocks)
             )
-            sections.append(evidence_text)
+            required_sections.append(evidence_text)
             used += len(evidence_text)
 
         sections_by_mode = {
@@ -249,6 +253,7 @@ class AnswerMixin:
                 ("Docs context", doc_hits),
             ],
         }
+        optional_sections: list[tuple[str, str]] = []
         for label, hits in sections_by_mode.get(
             profile.get("query_mode", "general"),
             [
@@ -268,9 +273,37 @@ class AnswerMixin:
                 parts.append(chunk_text)
                 used += len(chunk_text) + 2  # account for join separator
             if len(parts) > 1:
-                sections.append("\n\n".join(parts))
+                optional_sections.append((label, "\n\n".join(parts)))
+
+        required_text = "\n\n".join(required_sections)
+        fit_prompt = getattr(self.chat_client, "fit_prompt_sections", None)
+        if not callable(fit_prompt):
+            return "\n\n".join(
+                [required_text, *(text for _, text in optional_sections)]
+            )
+
+        def _render(sections: dict[str, str]) -> str:
+            return "\n\n".join(
+                [
+                    sections["required"],
+                    *(
+                        sections.get(f"optional_{index}", "")
+                        for index, _ in enumerate(optional_sections)
+                    ),
+                ]
+            ).strip()
 
-        return "\n\n".join(sections)
+        fitted = fit_prompt(
+            system=self._system_prompt(),
+            render_prompt=_render,
+            required_sections={"required": required_text},
+            optional_sections=[
+                (f"optional_{index}", [text])
+                for index, (_, text) in enumerate(optional_sections)
+            ],
+            step_name="chatbot answer",
+        )
+        return fitted.prompt
 
     def _system_prompt(self) -> str:
         return (
@@ -334,6 +367,7 @@ class AnswerMixin:
                 original_prompt,
                 response,
                 errors,
+                system_prompt=system_prompt,
             )
             corrected = self.chat_client.complete(system_prompt, correction_prompt)
             if corrected and corrected.strip():
@@ -911,6 +945,8 @@ class AnswerMixin:
         original_prompt: str,
         response: dict[str, Any],
         errors: list[str],
+        *,
+        system_prompt: str = "",
     ) -> str:
         evidence_blocks = []
         for item in response.get("evidence", []) or []:
@@ -927,7 +963,7 @@ class AnswerMixin:
             f"- {item.get('kind')}: {item.get('path')} ({item.get('title') or item.get('url') or ''})"
             for item in response.get("references", []) or []
         ]
-        return (
+        required_prompt = (
             "Your previous answer failed evidence validation.\n\n"
             f"Question: {question}\n\n"
             "Validation errors:\n"
@@ -942,10 +978,31 @@ class AnswerMixin:
             + ("\n\n".join(evidence_blocks) if evidence_blocks else "(none)")
             + "\n\nReference-only docs:\n"
             + ("\n".join(references) if references else "(none)")
-            + "\n\nOriginal retrieval prompt:\n"
-            + original_prompt
-            + "\n\nReturn only the corrected answer."
         )
+        fit_prompt = getattr(self.chat_client, "fit_prompt_sections", None)
+        if not callable(fit_prompt):
+            return (
+                required_prompt
+                + "\n\nOriginal retrieval prompt:\n"
+                + original_prompt
+                + "\n\nReturn only the corrected answer."
+            )
+
+        def _render(sections: dict[str, str]) -> str:
+            return (
+                sections["required"]
+                + "\n\nOriginal retrieval prompt:\n"
+                + sections.get("original", "")
+                + "\n\nReturn only the corrected answer."
+            )
+
+        return fit_prompt(
+            system=system_prompt or self._system_prompt(),
+            render_prompt=_render,
+            required_sections={"required": required_prompt},
+            optional_sections=[("original", [original_prompt])],
+            step_name="chatbot evidence correction",
+        ).prompt
 
     def _conservative_grounded_answer(
         self,

{deepdoc-0.4.2 → deepdoc-0.5.0}/deepdoc/chatbot/chunker.py

@@ -68,8 +68,6 @@ IGNORED_PARTS = {
     "coverage",
 }
 
-# Safe limit for text-embedding-3-small (8,192 token limit ≈ 6,000 chars for code)
-MAX_CHUNK_CHARS = 6000
 EMPTY_LINK_INFO = LinkInfo()
 
 
@@ -474,11 +472,6 @@ def _format_code_chunk_text(
     parts.append("")
     parts.append(snippet.strip())
     result = "\n".join(parts).strip()
-    if len(result) > MAX_CHUNK_CHARS:
-        # Truncate the snippet portion to stay within embedding model limits
-        header = result[: result.index("\n\n") + 2] if "\n\n" in result else ""
-        budget = MAX_CHUNK_CHARS - len(header) - 20  # room for truncation marker
-        result = header + snippet.strip()[:budget] + "\n... [truncated]"
     return result
 
 
@@ -651,11 +644,7 @@ def build_artifact_chunks(
         for start_line, end_line, section_name in windows:
             snippet = "\n".join(lines[start_line - 1 : end_line])
             header = f"Artifact: {rel_path}\nType: {artifact_type}\nSection: {section_name}\nLines: {start_line}-{end_line}\n\n"
-            body = snippet.strip()
-            if len(header) + len(body) > MAX_CHUNK_CHARS:
-                budget = MAX_CHUNK_CHARS - len(header) - 20
-                body = body[:budget] + "\n... [truncated]"
-            text = (header + body).strip()
+            text = (header + snippet.strip()).strip()
             chunk_hash = _hash_text(text)
             chunks.append(
                 ChunkRecord(
@@ -879,9 +868,6 @@ def build_relationship_chunks(
                 f"{Path(rel_path).name} defines {len(parsed.symbols)} symbols:\n"
                 + "\n".join(symbol_lines)
             )
-            # Respect embedding limits
-            if len(text) > MAX_CHUNK_CHARS:
-                text = text[: MAX_CHUNK_CHARS - 20] + "\n... [truncated]"
             chunk_hash = _hash_text(text)
             chunks.append(
                 ChunkRecord(