deepcode-hku 1.0.3__tar.gz → 1.0.5__tar.gz

{deepcode_hku-1.0.3/deepcode_hku.egg-info → deepcode_hku-1.0.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: deepcode-hku
-Version: 1.0.3
+Version: 1.0.5
 Summary: AI Research Engine - Transform research papers into working code automatically
 Home-page: https://github.com/HKUDS/DeepCode
 Author: DeepCodeTeam
@@ -60,6 +60,10 @@ Dynamic: summary
 </tr>
 </table>
 
+<div align="center">
+<a href="https://trendshift.io/repositories/14665" target="_blank"><img src="https://trendshift.io/api/badge/repositories/14665" alt="HKUDS%2FDeepCode | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
+</div>
+
 <!-- <img src="https://readme-typing-svg.herokuapp.com?font=Russo+One&size=28&duration=2000&pause=800&color=06B6D4&background=00000000&center=true&vCenter=true&width=800&height=50&lines=%E2%9A%A1+OPEN+AGENTIC+CODING+%E2%9A%A1" alt="DeepCode Tech Subtitle" style="margin-top: 5px; filter: drop-shadow(0 0 12px #06B6D4) drop-shadow(0 0 24px rgba(6,182,212,0.4));"/> -->
 
 # <img src="https://github.com/Zongwei9888/Experiment_Images/raw/43c585dca3d21b8e4b6390d835cdd34dc4b4b23d/DeepCode_images/title_logo.svg" alt="DeepCode Logo" width="32" height="32" style="vertical-align: middle; margin-right: 8px;"/> DeepCode: Open Agentic Coding
@@ -135,6 +139,32 @@ Dynamic: summary
 </tr>
 </table>
 
+---
+
+<div align="center">
+
+### 🎬 **Introduction Video**
+
+<div style="margin: 20px 0;">
+  <a href="https://youtu.be/PRgmP8pOI08" target="_blank">
+    <img src="https://img.youtube.com/vi/PRgmP8pOI08/maxresdefault.jpg"
+         alt="DeepCode Introduction Video"
+         width="75%"
+         style="border-radius: 12px; box-shadow: 0 8px 25px rgba(0,0,0,0.15); transition: transform 0.3s ease;"/>
+  </a>
+</div>
+
+*🎯 **Watch our complete introduction** - See how DeepCode transforms research papers and natural language into production-ready code*
+
+<p>
+  <a href="https://youtu.be/PRgmP8pOI08" target="_blank">
+    <img src="https://img.shields.io/badge/▶️_Watch_Video-FF0000?style=for-the-badge&logo=youtube&logoColor=white" alt="Watch Video"/>
+  </a>
+</p>
+
+</div>
+
+---
 
 
 
@@ -316,21 +346,35 @@ Advanced retrieval-augmented generation combining semantic vector embeddings wit
 
 #### 🛠️ **Implementation Tools Matrix**
 
-| 🛠️ Tool | 🔧 Function | 💡 Purpose |
-|----------|-------------|------------|
-| **📄 read_code_mem** | Read code summary from memory | Efficient code context retrieval |
-| **✍️ write_file** | Write file contents | Code file generation |
-| **🐍 execute_python** | Execute Python code | Code testing and validation |
-| **⚡ execute_bash** | Execute bash commands | Environment setup and package management |
-| **🔗 search_code_reference** | Search GitHub repositories | Automated CodeRAG and repository discovery |
-| **📁 get_file_structure** | Get directory structure | Project organization |
-| **⚙️ set_workspace** | Set working directory | Environment configuration |
-| **📊 get_operation_history** | Track operations | Process monitoring |
-| **🔍 brave** | Web search via Brave API | Real-time information retrieval |
-| **📂 filesystem** | File system operations | File and directory management |
-| **🌐 fetch** | Fetch content from URLs | Web content extraction |
-| **📥 github-downloader** | Download GitHub repositories | Repository content acquisition |
-| **📋 file-downloader** | Download files and PDFs | Document processing |
+**🔧 Powered by MCP (Model Context Protocol)**
+
+DeepCode leverages the **Model Context Protocol (MCP)** standard to seamlessly integrate with various tools and services. This standardized approach ensures reliable communication between AI agents and external systems, enabling powerful automation capabilities.
+
+##### 📡 **MCP Servers & Tools**
+
+| 🛠️ **MCP Server** | 🔧 **Primary Function** | 💡 **Purpose & Capabilities** |
+|-------------------|-------------------------|-------------------------------|
+| **🔍 brave** | Web Search Engine | Real-time information retrieval via Brave Search API |
+| **🌐 bocha-mcp** | Alternative Search | Secondary search option with independent API access |
+| **📂 filesystem** | File System Operations | Local file and directory management, read/write operations |
+| **🌐 fetch** | Web Content Retrieval | Fetch and extract content from URLs and web resources |
+| **📥 github-downloader** | Repository Management | Clone and download GitHub repositories for analysis |
+| **📋 file-downloader** | Document Processing | Download and convert files (PDF, DOCX, etc.) to Markdown |
+| **⚡ command-executor** | System Commands | Execute bash/shell commands for environment management |
+| **🧬 code-implementation** | Code Generation Hub | Comprehensive code reproduction with execution and testing |
+| **📚 code-reference-indexer** | Smart Code Search | Intelligent indexing and search of code repositories |
+| **📄 document-segmentation** | Smart Document Analysis | Intelligent document segmentation for large papers and technical documents |
+
+##### 🔧 **Legacy Tool Functions** *(for reference)*
+
+| 🛠️ **Function** | 🎯 **Usage Context** |
+|-----------------|---------------------|
+| **📄 read_code_mem** | Efficient code context retrieval from memory |
+| **✍️ write_file** | Direct file content generation and modification |
+| **🐍 execute_python** | Python code testing and validation |
+| **📁 get_file_structure** | Project structure analysis and organization |
+| **⚙️ set_workspace** | Dynamic workspace and environment configuration |
+| **📊 get_operation_history** | Process monitoring and operation tracking |
 
 
 ---
@@ -466,8 +510,15 @@ curl -O https://raw.githubusercontent.com/HKUDS/DeepCode/main/mcp_agent.secrets.
 # - openai: api_key, base_url (for OpenAI/custom endpoints)
 # - anthropic: api_key (for Claude models)
 
-# 🔑 Configure BRAVE_API_KEY for web search (optional)
-# Edit mcp_agent.config.yaml and update BRAVE_API_KEY under brave.env section
+# 🔑 Configure search API keys for web search (optional)
+# Edit mcp_agent.config.yaml to set your API keys:
+# - For Brave Search: Set BRAVE_API_KEY: "your_key_here" in brave.env section (line ~28)
+# - For Bocha-MCP: Set BOCHA_API_KEY: "your_key_here" in bocha-mcp.env section (line ~74)
+
+# 📄 Configure document segmentation (optional)
+# Edit mcp_agent.config.yaml to control document processing:
+# - enabled: true/false (whether to use intelligent document segmentation)
+# - size_threshold_chars: 50000 (document size threshold to trigger segmentation)
 ```
 
 #### 🔧 **Development Installation (From Source)**
@@ -495,8 +546,15 @@ uv pip install -r requirements.txt
 # - openai: api_key, base_url (for OpenAI/custom endpoints)
 # - anthropic: api_key (for Claude models)
 
-# 🔑 Configure BRAVE_API_KEY for web search (optional)
-# Edit mcp_agent.config.yaml and update BRAVE_API_KEY under brave.env section
+# 🔑 Configure search API keys for web search (optional)
+# Edit mcp_agent.config.yaml to set your API keys:
+# - For Brave Search: Set BRAVE_API_KEY: "your_key_here" in brave.env section (line ~28)
+# - For Bocha-MCP: Set BOCHA_API_KEY: "your_key_here" in bocha-mcp.env section (line ~74)
+
+# 📄 Configure document segmentation (optional)
+# Edit mcp_agent.config.yaml to control document processing:
+# - enabled: true/false (whether to use intelligent document segmentation)
+# - size_threshold_chars: 50000 (document size threshold to trigger segmentation)
 ```
 
 ##### 🐍 **Using Traditional pip**
@@ -514,8 +572,15 @@ pip install -r requirements.txt
 # - openai: api_key, base_url (for OpenAI/custom endpoints)
 # - anthropic: api_key (for Claude models)
 
-# 🔑 Configure BRAVE_API_KEY for web search (optional)
-# Edit mcp_agent.config.yaml and update BRAVE_API_KEY under brave.env section
+# 🔑 Configure search API keys for web search (optional)
+# Edit mcp_agent.config.yaml to set your API keys:
+# - For Brave Search: Set BRAVE_API_KEY: "your_key_here" in brave.env section (line ~28)
+# - For Bocha-MCP: Set BOCHA_API_KEY: "your_key_here" in bocha-mcp.env section (line ~74)
+
+# 📄 Configure document segmentation (optional)
+# Edit mcp_agent.config.yaml to control document processing:
+# - enabled: true/false (whether to use intelligent document segmentation)
+# - size_threshold_chars: 50000 (document size threshold to trigger segmentation)
 ```
 
 </details>
@@ -548,6 +613,47 @@ mcp:
 
 > **Note**: Replace the path with your actual global node_modules path from step 2.
 
+#### 🔍 **Search Server Configuration (Optional)**
+
+DeepCode supports multiple search servers for web search functionality. You can configure your preferred option in `mcp_agent.config.yaml`:
+
+```yaml
+# Default search server configuration
+# Options: "brave" or "bocha-mcp"
+default_search_server: "brave"
+```
+
+**Available Options:**
+- **🔍 Brave Search** (`"brave"`):
+  - Default option with high-quality search results
+  - Requires BRAVE_API_KEY configuration
+  - Recommended for most users
+
+- **🌐 Bocha-MCP** (`"bocha-mcp"`):
+  - Alternative search server option
+  - Requires BOCHA_API_KEY configuration
+  - Uses local Python server implementation
+
+**API Key Configuration in mcp_agent.config.yaml:**
+```yaml
+# For Brave Search (default) - around line 28
+brave:
+  command: "npx"
+  args: ["-y", "@modelcontextprotocol/server-brave-search"]
+  env:
+    BRAVE_API_KEY: "your_brave_api_key_here"
+
+# For Bocha-MCP (alternative) - around line 74
+bocha-mcp:
+  command: "python"
+  args: ["tools/bocha_search_server.py"]
+  env:
+    PYTHONPATH: "."
+    BOCHA_API_KEY: "your_bocha_api_key_here"
+```
+
+> **💡 Tip**: Both search servers require API key configuration. Choose the one that best fits your API access and requirements.
+
 ### ⚡ **Step 2: Launch Application**
 
 #### 🚀 **Using Installed Package (Recommended)**
@@ -657,6 +763,14 @@ python cli/main_cli.py
 
 
 
+### 🆕 **Recent Updates**
+
+#### 📄 **Smart Document Segmentation (v1.2.0)**
+- **Intelligent Processing**: Automatically handles large research papers and technical documents that exceed LLM token limits
+- **Configurable Control**: Toggle segmentation via configuration with size-based thresholds
+- **Semantic Analysis**: Advanced content understanding with algorithm, concept, and formula preservation
+- **Backward Compatibility**: Seamlessly falls back to traditional processing for smaller documents
+
 ### 🚀 **Coming Soon**
 
 We're continuously enhancing DeepCode with exciting new features:

{deepcode_hku-1.0.3 → deepcode_hku-1.0.5}/README.md

@@ -16,6 +16,10 @@
 </tr>
 </table>
 
+<div align="center">
+<a href="https://trendshift.io/repositories/14665" target="_blank"><img src="https://trendshift.io/api/badge/repositories/14665" alt="HKUDS%2FDeepCode | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
+</div>
+
 <!-- <img src="https://readme-typing-svg.herokuapp.com?font=Russo+One&size=28&duration=2000&pause=800&color=06B6D4&background=00000000&center=true&vCenter=true&width=800&height=50&lines=%E2%9A%A1+OPEN+AGENTIC+CODING+%E2%9A%A1" alt="DeepCode Tech Subtitle" style="margin-top: 5px; filter: drop-shadow(0 0 12px #06B6D4) drop-shadow(0 0 24px rgba(6,182,212,0.4));"/> -->
 
 # <img src="https://github.com/Zongwei9888/Experiment_Images/raw/43c585dca3d21b8e4b6390d835cdd34dc4b4b23d/DeepCode_images/title_logo.svg" alt="DeepCode Logo" width="32" height="32" style="vertical-align: middle; margin-right: 8px;"/> DeepCode: Open Agentic Coding
@@ -91,6 +95,32 @@
 </tr>
 </table>
 
+---
+
+<div align="center">
+
+### 🎬 **Introduction Video**
+
+<div style="margin: 20px 0;">
+  <a href="https://youtu.be/PRgmP8pOI08" target="_blank">
+    <img src="https://img.youtube.com/vi/PRgmP8pOI08/maxresdefault.jpg"
+         alt="DeepCode Introduction Video"
+         width="75%"
+         style="border-radius: 12px; box-shadow: 0 8px 25px rgba(0,0,0,0.15); transition: transform 0.3s ease;"/>
+  </a>
+</div>
+
+*🎯 **Watch our complete introduction** - See how DeepCode transforms research papers and natural language into production-ready code*
+
+<p>
+  <a href="https://youtu.be/PRgmP8pOI08" target="_blank">
+    <img src="https://img.shields.io/badge/▶️_Watch_Video-FF0000?style=for-the-badge&logo=youtube&logoColor=white" alt="Watch Video"/>
+  </a>
+</p>
+
+</div>
+
+---
 
 
 
@@ -272,21 +302,35 @@ Advanced retrieval-augmented generation combining semantic vector embeddings wit
 
 #### 🛠️ **Implementation Tools Matrix**
 
-| 🛠️ Tool | 🔧 Function | 💡 Purpose |
-|----------|-------------|------------|
-| **📄 read_code_mem** | Read code summary from memory | Efficient code context retrieval |
-| **✍️ write_file** | Write file contents | Code file generation |
-| **🐍 execute_python** | Execute Python code | Code testing and validation |
-| **⚡ execute_bash** | Execute bash commands | Environment setup and package management |
-| **🔗 search_code_reference** | Search GitHub repositories | Automated CodeRAG and repository discovery |
-| **📁 get_file_structure** | Get directory structure | Project organization |
-| **⚙️ set_workspace** | Set working directory | Environment configuration |
-| **📊 get_operation_history** | Track operations | Process monitoring |
-| **🔍 brave** | Web search via Brave API | Real-time information retrieval |
-| **📂 filesystem** | File system operations | File and directory management |
-| **🌐 fetch** | Fetch content from URLs | Web content extraction |
-| **📥 github-downloader** | Download GitHub repositories | Repository content acquisition |
-| **📋 file-downloader** | Download files and PDFs | Document processing |
+**🔧 Powered by MCP (Model Context Protocol)**
+
+DeepCode leverages the **Model Context Protocol (MCP)** standard to seamlessly integrate with various tools and services. This standardized approach ensures reliable communication between AI agents and external systems, enabling powerful automation capabilities.
+
+##### 📡 **MCP Servers & Tools**
+
+| 🛠️ **MCP Server** | 🔧 **Primary Function** | 💡 **Purpose & Capabilities** |
+|-------------------|-------------------------|-------------------------------|
+| **🔍 brave** | Web Search Engine | Real-time information retrieval via Brave Search API |
+| **🌐 bocha-mcp** | Alternative Search | Secondary search option with independent API access |
+| **📂 filesystem** | File System Operations | Local file and directory management, read/write operations |
+| **🌐 fetch** | Web Content Retrieval | Fetch and extract content from URLs and web resources |
+| **📥 github-downloader** | Repository Management | Clone and download GitHub repositories for analysis |
+| **📋 file-downloader** | Document Processing | Download and convert files (PDF, DOCX, etc.) to Markdown |
+| **⚡ command-executor** | System Commands | Execute bash/shell commands for environment management |
+| **🧬 code-implementation** | Code Generation Hub | Comprehensive code reproduction with execution and testing |
+| **📚 code-reference-indexer** | Smart Code Search | Intelligent indexing and search of code repositories |
+| **📄 document-segmentation** | Smart Document Analysis | Intelligent document segmentation for large papers and technical documents |
+
+##### 🔧 **Legacy Tool Functions** *(for reference)*
+
+| 🛠️ **Function** | 🎯 **Usage Context** |
+|-----------------|---------------------|
+| **📄 read_code_mem** | Efficient code context retrieval from memory |
+| **✍️ write_file** | Direct file content generation and modification |
+| **🐍 execute_python** | Python code testing and validation |
+| **📁 get_file_structure** | Project structure analysis and organization |
+| **⚙️ set_workspace** | Dynamic workspace and environment configuration |
+| **📊 get_operation_history** | Process monitoring and operation tracking |
 
 
 ---
@@ -422,8 +466,15 @@ curl -O https://raw.githubusercontent.com/HKUDS/DeepCode/main/mcp_agent.secrets.
 # - openai: api_key, base_url (for OpenAI/custom endpoints)
 # - anthropic: api_key (for Claude models)
 
-# 🔑 Configure BRAVE_API_KEY for web search (optional)
-# Edit mcp_agent.config.yaml and update BRAVE_API_KEY under brave.env section
+# 🔑 Configure search API keys for web search (optional)
+# Edit mcp_agent.config.yaml to set your API keys:
+# - For Brave Search: Set BRAVE_API_KEY: "your_key_here" in brave.env section (line ~28)
+# - For Bocha-MCP: Set BOCHA_API_KEY: "your_key_here" in bocha-mcp.env section (line ~74)
+
+# 📄 Configure document segmentation (optional)
+# Edit mcp_agent.config.yaml to control document processing:
+# - enabled: true/false (whether to use intelligent document segmentation)
+# - size_threshold_chars: 50000 (document size threshold to trigger segmentation)
 ```
 
 #### 🔧 **Development Installation (From Source)**
@@ -451,8 +502,15 @@ uv pip install -r requirements.txt
 # - openai: api_key, base_url (for OpenAI/custom endpoints)
 # - anthropic: api_key (for Claude models)
 
-# 🔑 Configure BRAVE_API_KEY for web search (optional)
-# Edit mcp_agent.config.yaml and update BRAVE_API_KEY under brave.env section
+# 🔑 Configure search API keys for web search (optional)
+# Edit mcp_agent.config.yaml to set your API keys:
+# - For Brave Search: Set BRAVE_API_KEY: "your_key_here" in brave.env section (line ~28)
+# - For Bocha-MCP: Set BOCHA_API_KEY: "your_key_here" in bocha-mcp.env section (line ~74)
+
+# 📄 Configure document segmentation (optional)
+# Edit mcp_agent.config.yaml to control document processing:
+# - enabled: true/false (whether to use intelligent document segmentation)
+# - size_threshold_chars: 50000 (document size threshold to trigger segmentation)
 ```
 
 ##### 🐍 **Using Traditional pip**
@@ -470,8 +528,15 @@ pip install -r requirements.txt
 # - openai: api_key, base_url (for OpenAI/custom endpoints)
 # - anthropic: api_key (for Claude models)
 
-# 🔑 Configure BRAVE_API_KEY for web search (optional)
-# Edit mcp_agent.config.yaml and update BRAVE_API_KEY under brave.env section
+# 🔑 Configure search API keys for web search (optional)
+# Edit mcp_agent.config.yaml to set your API keys:
+# - For Brave Search: Set BRAVE_API_KEY: "your_key_here" in brave.env section (line ~28)
+# - For Bocha-MCP: Set BOCHA_API_KEY: "your_key_here" in bocha-mcp.env section (line ~74)
+
+# 📄 Configure document segmentation (optional)
+# Edit mcp_agent.config.yaml to control document processing:
+# - enabled: true/false (whether to use intelligent document segmentation)
+# - size_threshold_chars: 50000 (document size threshold to trigger segmentation)
 ```
 
 </details>
@@ -504,6 +569,47 @@ mcp:
 
 > **Note**: Replace the path with your actual global node_modules path from step 2.
 
+#### 🔍 **Search Server Configuration (Optional)**
+
+DeepCode supports multiple search servers for web search functionality. You can configure your preferred option in `mcp_agent.config.yaml`:
+
+```yaml
+# Default search server configuration
+# Options: "brave" or "bocha-mcp"
+default_search_server: "brave"
+```
+
+**Available Options:**
+- **🔍 Brave Search** (`"brave"`):
+  - Default option with high-quality search results
+  - Requires BRAVE_API_KEY configuration
+  - Recommended for most users
+
+- **🌐 Bocha-MCP** (`"bocha-mcp"`):
+  - Alternative search server option
+  - Requires BOCHA_API_KEY configuration
+  - Uses local Python server implementation
+
+**API Key Configuration in mcp_agent.config.yaml:**
+```yaml
+# For Brave Search (default) - around line 28
+brave:
+  command: "npx"
+  args: ["-y", "@modelcontextprotocol/server-brave-search"]
+  env:
+    BRAVE_API_KEY: "your_brave_api_key_here"
+
+# For Bocha-MCP (alternative) - around line 74
+bocha-mcp:
+  command: "python"
+  args: ["tools/bocha_search_server.py"]
+  env:
+    PYTHONPATH: "."
+    BOCHA_API_KEY: "your_bocha_api_key_here"
+```
+
+> **💡 Tip**: Both search servers require API key configuration. Choose the one that best fits your API access and requirements.
+
 ### ⚡ **Step 2: Launch Application**
 
 #### 🚀 **Using Installed Package (Recommended)**
@@ -613,6 +719,14 @@ python cli/main_cli.py
 
 
 
+### 🆕 **Recent Updates**
+
+#### 📄 **Smart Document Segmentation (v1.2.0)**
+- **Intelligent Processing**: Automatically handles large research papers and technical documents that exceed LLM token limits
+- **Configurable Control**: Toggle segmentation via configuration with size-based thresholds
+- **Semantic Analysis**: Advanced content understanding with algorithm, concept, and formula preservation
+- **Backward Compatibility**: Seamlessly falls back to traditional processing for smaller documents
+
 ### 🚀 **Coming Soon**
 
 We're continuously enhancing DeepCode with exciting new features:

{deepcode_hku-1.0.3 → deepcode_hku-1.0.5}/__init__.py

@@ -5,7 +5,7 @@ DeepCode - AI Research Engine
 ⚡ Transform research papers into working code automatically
 """
 
-__version__ = "1.0.3"
+__version__ = "1.0.5"
 __author__ = "DeepCode Team"
 __url__ = "https://github.com/HKUDS/DeepCode"
 

{deepcode_hku-1.0.3 → deepcode_hku-1.0.5}/cli/cli_app.py

@@ -37,6 +37,7 @@ class CLIApp:
         self.app = None  # Will be initialized by workflow adapter
         self.logger = None
         self.context = None
+        # Document segmentation will be managed by CLI interface
 
     async def initialize_mcp_app(self):
         """初始化MCP应用 - 使用工作流适配器"""
@@ -50,6 +51,8 @@ class CLIApp:
     async def process_input(self, input_source: str, input_type: str):
         """处理输入源（URL或文件）- 使用升级版智能体编排引擎"""
         try:
+            # Document segmentation configuration is managed by CLI interface
+
             self.cli.print_separator()
             self.cli.print_status(
                 "🚀 Starting intelligent agent orchestration...", "processing"
@@ -237,6 +240,7 @@ class CLIApp:
                     self.cli.show_history()
 
                 elif choice in ["c", "config", "configure"]:
+                    # Show configuration menu - all settings managed by CLI interface
                     self.cli.show_configuration_menu()
 
                 else:

{deepcode_hku-1.0.3 → deepcode_hku-1.0.5}/cli/cli_interface.py

@@ -41,6 +41,66 @@ class CLIInterface:
         self.processing_history = []
         self.enable_indexing = True  # Default configuration
 
+        # Load segmentation config from the same source as UI
+        self._load_segmentation_config()
+
+        # Initialize tkinter availability
+        self._init_tkinter()
+
+    def _load_segmentation_config(self):
+        """Load segmentation configuration from mcp_agent.config.yaml"""
+        try:
+            from utils.llm_utils import get_document_segmentation_config
+
+            seg_config = get_document_segmentation_config()
+            self.segmentation_enabled = seg_config.get("enabled", True)
+            self.segmentation_threshold = seg_config.get("size_threshold_chars", 50000)
+        except Exception as e:
+            print(f"⚠️ Warning: Failed to load segmentation config: {e}")
+            # Fall back to defaults
+            self.segmentation_enabled = True
+            self.segmentation_threshold = 50000
+
+    def _save_segmentation_config(self):
+        """Save segmentation configuration to mcp_agent.config.yaml"""
+        import yaml
+        import os
+
+        # Get the project root directory (where mcp_agent.config.yaml is located)
+        current_file = os.path.abspath(__file__)
+        cli_dir = os.path.dirname(current_file)  # cli directory
+        project_root = os.path.dirname(cli_dir)  # project root
+        config_path = os.path.join(project_root, "mcp_agent.config.yaml")
+
+        try:
+            # Read current config
+            with open(config_path, "r", encoding="utf-8") as f:
+                config = yaml.safe_load(f)
+
+            # Update document segmentation settings
+            if "document_segmentation" not in config:
+                config["document_segmentation"] = {}
+
+            config["document_segmentation"]["enabled"] = self.segmentation_enabled
+            config["document_segmentation"]["size_threshold_chars"] = (
+                self.segmentation_threshold
+            )
+
+            # Write updated config
+            with open(config_path, "w", encoding="utf-8") as f:
+                yaml.dump(config, f, default_flow_style=False, allow_unicode=True)
+
+            print(
+                f"{Colors.OKGREEN}✅ Document segmentation configuration updated{Colors.ENDC}"
+            )
+
+        except Exception as e:
+            print(
+                f"{Colors.WARNING}⚠️ Failed to update segmentation config: {str(e)}{Colors.ENDC}"
+            )
+
+    def _init_tkinter(self):
+        """Initialize tkinter availability check"""
         # Check tkinter availability for file dialogs
         self.tkinter_available = True
         try:
@@ -125,6 +185,9 @@ class CLIInterface:
         # Display current configuration
         pipeline_mode = "🧠 COMPREHENSIVE" if self.enable_indexing else "⚡ OPTIMIZED"
         index_status = "✅ Enabled" if self.enable_indexing else "🔶 Disabled"
+        segmentation_mode = (
+            "📄 SMART" if self.segmentation_enabled else "📋 TRADITIONAL"
+        )
 
         menu = f"""
 {Colors.BOLD}{Colors.CYAN}╔═══════════════════════════════════════════════════════════════════════════════╗
@@ -135,6 +198,7 @@ class CLIInterface:
 ║                                                                               ║
 ║  {Colors.BOLD}🤖 Current Pipeline Mode: {pipeline_mode}{Colors.CYAN}                          ║
 ║  {Colors.BOLD}🗂️  Codebase Indexing: {index_status}{Colors.CYAN}                                    ║
+║  {Colors.BOLD}📄 Document Processing: {segmentation_mode}{Colors.CYAN}                               ║
 ║                                                                               ║
 ║  {Colors.YELLOW}📝 URL Processing:{Colors.CYAN}                                                         ║
 ║  {Colors.YELLOW}   ▶ Enter research paper URL (arXiv, IEEE, ACM, etc.)                    {Colors.CYAN}║
@@ -693,6 +757,11 @@ class CLIInterface:
     def show_configuration_menu(self):
         """Show configuration options menu"""
         self.clear_screen()
+
+        # Get segmentation config status
+        segmentation_enabled = getattr(self, "segmentation_enabled", True)
+        segmentation_threshold = getattr(self, "segmentation_threshold", 50000)
+
         print(f"""
 {Colors.BOLD}{Colors.CYAN}╔═══════════════════════════════════════════════════════════════════════════════╗
 ║                           CONFIGURATION MENU                                  ║
@@ -716,9 +785,23 @@ class CLIInterface:
 ║         ✗ Repository Acquisition (Skipped)                                   ║
 ║         ✗ Codebase Intelligence Orchestration (Skipped)                     ║
 ║                                                                               ║
-║  {Colors.YELLOW}Current Setting:{Colors.CYAN} {'🧠 Comprehensive Mode' if self.enable_indexing else '⚡ Optimized Mode'}                              ║
+║  {Colors.OKCYAN}[2] Document Processing:{Colors.CYAN}                                                   ║
+║      {Colors.BOLD}📄 Smart Segmentation{Colors.CYAN} - Intelligent document analysis (Default)      ║
+║         ✓ Semantic boundary detection                                        ║
+║         ✓ Algorithm integrity preservation                                   ║
+║         ✓ Formula chain recognition                                          ║
+║         ✓ Adaptive character limits                                          ║
+║                                                                               ║
+║      {Colors.BOLD}📋 Traditional Processing{Colors.CYAN} - Full document reading                       ║
+║         ✓ Complete document analysis                                         ║
+║         ✗ Smart segmentation (Disabled)                                      ║
 ║                                                                               ║
-║  {Colors.OKGREEN}[T] Toggle Pipeline Mode    {Colors.CYAN}│  {Colors.FAIL}[B] Back to Main Menu{Colors.CYAN}            ║
+║  {Colors.YELLOW}Current Settings:{Colors.CYAN}                                                         ║
+║    Pipeline: {'🧠 Comprehensive Mode' if self.enable_indexing else '⚡ Optimized Mode'}                                          ║
+║    Document: {'📄 Smart Segmentation' if segmentation_enabled else '📋 Traditional Processing'}                                ║
+║    Threshold: {segmentation_threshold} characters                                    ║
+║                                                                               ║
+║  {Colors.OKGREEN}[T] Toggle Pipeline    {Colors.BLUE}[S] Toggle Segmentation    {Colors.FAIL}[B] Back{Colors.CYAN}     ║
 ╚═══════════════════════════════════════════════════════════════════════════════╝{Colors.ENDC}
 """)
 
@@ -737,8 +820,27 @@ class CLIInterface:
                 self.show_configuration_menu()
                 return
 
+            elif choice in ["s", "segmentation"]:
+                current_state = getattr(self, "segmentation_enabled", True)
+                self.segmentation_enabled = not current_state
+                # Save the configuration to file
+                self._save_segmentation_config()
+                seg_mode = (
+                    "📄 Smart Segmentation"
+                    if self.segmentation_enabled
+                    else "📋 Traditional Processing"
+                )
+                self.print_status(
+                    f"Document processing switched to: {seg_mode}", "success"
+                )
+                time.sleep(1)
+                self.show_configuration_menu()
+                return
+
             elif choice in ["b", "back"]:
                 return
 
             else:
-                self.print_status("Invalid choice. Please enter 'T' or 'B'.", "warning")
+                self.print_status(
+                    "Invalid choice. Please enter 'T', 'S', or 'B'.", "warning"
+                )