deepcode-hku 1.0.4__tar.gz → 1.0.6__tar.gz

{deepcode_hku-1.0.4/deepcode_hku.egg-info → deepcode_hku-1.0.6}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: deepcode-hku
-Version: 1.0.4
+Version: 1.0.6
 Summary: AI Research Engine - Transform research papers into working code automatically
 Home-page: https://github.com/HKUDS/DeepCode
 Author: DeepCodeTeam
@@ -27,6 +27,7 @@ Requires-Dist: docling
 Requires-Dist: mcp-agent
 Requires-Dist: mcp-server-git
 Requires-Dist: nest_asyncio
+Requires-Dist: openai
 Requires-Dist: pathlib2
 Requires-Dist: PyPDF2>=2.0.0
 Requires-Dist: reportlab>=3.5.0
@@ -60,6 +61,10 @@ Dynamic: summary
 </tr>
 </table>
 
+<div align="center">
+<a href="https://trendshift.io/repositories/14665" target="_blank"><img src="https://trendshift.io/api/badge/repositories/14665" alt="HKUDS%2FDeepCode | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
+</div>
+
 <!-- <img src="https://readme-typing-svg.herokuapp.com?font=Russo+One&size=28&duration=2000&pause=800&color=06B6D4&background=00000000&center=true&vCenter=true&width=800&height=50&lines=%E2%9A%A1+OPEN+AGENTIC+CODING+%E2%9A%A1" alt="DeepCode Tech Subtitle" style="margin-top: 5px; filter: drop-shadow(0 0 12px #06B6D4) drop-shadow(0 0 24px rgba(6,182,212,0.4));"/> -->
 
 # <img src="https://github.com/Zongwei9888/Experiment_Images/raw/43c585dca3d21b8e4b6390d835cdd34dc4b4b23d/DeepCode_images/title_logo.svg" alt="DeepCode Logo" width="32" height="32" style="vertical-align: middle; margin-right: 8px;"/> DeepCode: Open Agentic Coding
@@ -92,6 +97,15 @@ Dynamic: summary
   </a>
 </div>
 
+<div align="center" style="margin-top: 10px;">
+  <a href="README.md">
+    <img src="https://img.shields.io/badge/English-00d4ff?style=for-the-badge&logo=readme&logoColor=white&labelColor=1a1a2e" alt="English">
+  </a>
+  <a href="README_ZH.md">
+    <img src="https://img.shields.io/badge/中文-00d4ff?style=for-the-badge&logo=readme&logoColor=white&labelColor=1a1a2e" alt="中文">
+  </a>
+</div>
+
 ### 🖥️ **Interface Showcase**
 
 <table align="center" width="100%" style="border: none; border-collapse: collapse; margin: 30px 0;">
@@ -173,14 +187,30 @@ Dynamic: summary
 
 ## 📑 Table of Contents
 
+- [📰 News](#-news)
 - [🚀 Key Features](#-key-features)
 - [🏗️ Architecture](#️-architecture)
+- [📊 Experimental Results](#-experimental-results)
 - [🚀 Quick Start](#-quick-start)
 - [💡 Examples](#-examples)
   - [🎬 Live Demonstrations](#-live-demonstrations)
 - [⭐ Star History](#-star-history)
 - [📄 License](#-license)
 
+
+---
+
+## 📰 News
+
+🎉 **[2025-10] 🎉 [2025-10-28] DeepCode Achieves SOTA on PaperBench!**
+
+DeepCode sets new benchmarks on OpenAI's PaperBench Code-Dev across all categories:
+
+- 🏆 **Surpasses Human Experts**: **75.9%** (DeepCode) vs Top Machine Learning PhDs 72.4% (+3.5%).
+- 🥇 **Outperforms SOTA Commercial Code Agents**: **84.8%** (DeepCode) vs Leading Commercial Code Agents (+26.1%) (Cursor, Claude Code, and Codex).
+- 🔬 **Advances Scientific Coding**: **73.5%** (DeepCode) vs PaperCoder 51.1% (+22.4%).
+- 🚀 **Beats LLM Agents**: **73.5%** (DeepCode) vs best LLM frameworks 43.3% (+30.2%).
+
 ---
 
 ## 🚀 Key Features
@@ -257,7 +287,58 @@ Dynamic: summary
 
 <br/>
 
-### 🎯 **Autonomous Multi-Agent Workflow**
+---
+
+## 📊 Experimental Results
+
+<div align="center">
+    <img src='./assets/result_main02.jpg' /><br>
+</div>
+<br/>
+
+We evaluate **DeepCode** on the [*PaperBench*](https://openai.com/index/paperbench/) benchmark (released by OpenAI), a rigorous testbed requiring AI agents to independently reproduce 20 ICML 2024 papers from scratch. The benchmark comprises 8,316 gradable components assessed using SimpleJudge with hierarchical weighting.
+
+Our experiments compare DeepCode against four baseline categories: **(1) Human Experts**, **(2) State-of-the-Art Commercial Code Agents**, **(3) Scientific Code Agents**, and **(4) LLM-Based Agents**.
+
+### ① 🧠 Human Expert Performance (Top Machine Learning PhD)
+
+**DeepCode: 75.9% vs. Top Machine Learning PhD: 72.4% (+3.5%)**
+
+DeepCode achieves **75.9%** on the 3-paper human evaluation subset, **surpassing the best-of-3 human expert baseline (72.4%) by +3.5 percentage points**. This demonstrates that our framework not only matches but exceeds expert-level code reproduction capabilities, representing a significant milestone in autonomous scientific software engineering.
+
+### ② 💼 State-of-the-Art Commercial Code Agents
+
+**DeepCode: 84.8% vs. Best Commercial Agent: 58.7% (+26.1%)**
+
+On the 5-paper subset, DeepCode substantially outperforms leading commercial coding tools:
+- Cursor: 58.4%
+- Claude Code: 58.7%
+- Codex: 40.0%
+- **DeepCode: 84.8%**
+
+This represents a **+26.1% improvement** over the leading commercial code agent. All commercial agents utilize Claude Sonnet 4.5 or GPT-5 Codex-high, highlighting that **DeepCode's superior architecture**—rather than base model capability—drives this performance gap.
+
+### ③ 🔬 Scientific Code Agents
+
+**DeepCode: 73.5% vs. PaperCoder: 51.1% (+22.4%)**
+
+Compared to PaperCoder (**51.1%**), the state-of-the-art scientific code reproduction framework, DeepCode achieves **73.5%**, demonstrating a **+22.4% relative improvement**. This substantial margin validates our multi-module architecture combining planning, hierarchical task decomposition, code generation, and iterative debugging over simpler pipeline-based approaches.
+
+### ④ 🤖 LLM-Based Agents
+
+**DeepCode: 73.5% vs. Best LLM Agent: 43.3% (+30.2%)**
+
+DeepCode significantly outperforms all tested LLM agents:
+- Claude 3.5 Sonnet + IterativeAgent: 27.5%
+- o1 + IterativeAgent (36 hours): 42.4%
+- o1 BasicAgent: 43.3%
+- **DeepCode: 73.5%**
+
+The **+30.2% improvement** over the best-performing LLM agent demonstrates that sophisticated agent scaffolding, rather than extended inference time or larger models, is critical for complex code reproduction tasks.
+
+---
+
+### 🎯 **Autonomous Self-Orchestrating Multi-Agent Architecture**
 
 **The Challenges**:
 
@@ -485,6 +566,7 @@ Implementation Generation • Testing • Documentation
 
 ---
 
+
 ## 🚀 Quick Start
 
 

{deepcode_hku-1.0.4 → deepcode_hku-1.0.6}/README.md

@@ -16,6 +16,10 @@
 </tr>
 </table>
 
+<div align="center">
+<a href="https://trendshift.io/repositories/14665" target="_blank"><img src="https://trendshift.io/api/badge/repositories/14665" alt="HKUDS%2FDeepCode | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
+</div>
+
 <!-- <img src="https://readme-typing-svg.herokuapp.com?font=Russo+One&size=28&duration=2000&pause=800&color=06B6D4&background=00000000&center=true&vCenter=true&width=800&height=50&lines=%E2%9A%A1+OPEN+AGENTIC+CODING+%E2%9A%A1" alt="DeepCode Tech Subtitle" style="margin-top: 5px; filter: drop-shadow(0 0 12px #06B6D4) drop-shadow(0 0 24px rgba(6,182,212,0.4));"/> -->
 
 # <img src="https://github.com/Zongwei9888/Experiment_Images/raw/43c585dca3d21b8e4b6390d835cdd34dc4b4b23d/DeepCode_images/title_logo.svg" alt="DeepCode Logo" width="32" height="32" style="vertical-align: middle; margin-right: 8px;"/> DeepCode: Open Agentic Coding
@@ -48,6 +52,15 @@
   </a>
 </div>
 
+<div align="center" style="margin-top: 10px;">
+  <a href="README.md">
+    <img src="https://img.shields.io/badge/English-00d4ff?style=for-the-badge&logo=readme&logoColor=white&labelColor=1a1a2e" alt="English">
+  </a>
+  <a href="README_ZH.md">
+    <img src="https://img.shields.io/badge/中文-00d4ff?style=for-the-badge&logo=readme&logoColor=white&labelColor=1a1a2e" alt="中文">
+  </a>
+</div>
+
 ### 🖥️ **Interface Showcase**
 
 <table align="center" width="100%" style="border: none; border-collapse: collapse; margin: 30px 0;">
@@ -129,14 +142,30 @@
 
 ## 📑 Table of Contents
 
+- [📰 News](#-news)
 - [🚀 Key Features](#-key-features)
 - [🏗️ Architecture](#️-architecture)
+- [📊 Experimental Results](#-experimental-results)
 - [🚀 Quick Start](#-quick-start)
 - [💡 Examples](#-examples)
   - [🎬 Live Demonstrations](#-live-demonstrations)
 - [⭐ Star History](#-star-history)
 - [📄 License](#-license)
 
+
+---
+
+## 📰 News
+
+🎉 **[2025-10] 🎉 [2025-10-28] DeepCode Achieves SOTA on PaperBench!**
+
+DeepCode sets new benchmarks on OpenAI's PaperBench Code-Dev across all categories:
+
+- 🏆 **Surpasses Human Experts**: **75.9%** (DeepCode) vs Top Machine Learning PhDs 72.4% (+3.5%).
+- 🥇 **Outperforms SOTA Commercial Code Agents**: **84.8%** (DeepCode) vs Leading Commercial Code Agents (+26.1%) (Cursor, Claude Code, and Codex).
+- 🔬 **Advances Scientific Coding**: **73.5%** (DeepCode) vs PaperCoder 51.1% (+22.4%).
+- 🚀 **Beats LLM Agents**: **73.5%** (DeepCode) vs best LLM frameworks 43.3% (+30.2%).
+
 ---
 
 ## 🚀 Key Features
@@ -213,7 +242,58 @@
 
 <br/>
 
-### 🎯 **Autonomous Multi-Agent Workflow**
+---
+
+## 📊 Experimental Results
+
+<div align="center">
+    <img src='./assets/result_main02.jpg' /><br>
+</div>
+<br/>
+
+We evaluate **DeepCode** on the [*PaperBench*](https://openai.com/index/paperbench/) benchmark (released by OpenAI), a rigorous testbed requiring AI agents to independently reproduce 20 ICML 2024 papers from scratch. The benchmark comprises 8,316 gradable components assessed using SimpleJudge with hierarchical weighting.
+
+Our experiments compare DeepCode against four baseline categories: **(1) Human Experts**, **(2) State-of-the-Art Commercial Code Agents**, **(3) Scientific Code Agents**, and **(4) LLM-Based Agents**.
+
+### ① 🧠 Human Expert Performance (Top Machine Learning PhD)
+
+**DeepCode: 75.9% vs. Top Machine Learning PhD: 72.4% (+3.5%)**
+
+DeepCode achieves **75.9%** on the 3-paper human evaluation subset, **surpassing the best-of-3 human expert baseline (72.4%) by +3.5 percentage points**. This demonstrates that our framework not only matches but exceeds expert-level code reproduction capabilities, representing a significant milestone in autonomous scientific software engineering.
+
+### ② 💼 State-of-the-Art Commercial Code Agents
+
+**DeepCode: 84.8% vs. Best Commercial Agent: 58.7% (+26.1%)**
+
+On the 5-paper subset, DeepCode substantially outperforms leading commercial coding tools:
+- Cursor: 58.4%
+- Claude Code: 58.7%
+- Codex: 40.0%
+- **DeepCode: 84.8%**
+
+This represents a **+26.1% improvement** over the leading commercial code agent. All commercial agents utilize Claude Sonnet 4.5 or GPT-5 Codex-high, highlighting that **DeepCode's superior architecture**—rather than base model capability—drives this performance gap.
+
+### ③ 🔬 Scientific Code Agents
+
+**DeepCode: 73.5% vs. PaperCoder: 51.1% (+22.4%)**
+
+Compared to PaperCoder (**51.1%**), the state-of-the-art scientific code reproduction framework, DeepCode achieves **73.5%**, demonstrating a **+22.4% relative improvement**. This substantial margin validates our multi-module architecture combining planning, hierarchical task decomposition, code generation, and iterative debugging over simpler pipeline-based approaches.
+
+### ④ 🤖 LLM-Based Agents
+
+**DeepCode: 73.5% vs. Best LLM Agent: 43.3% (+30.2%)**
+
+DeepCode significantly outperforms all tested LLM agents:
+- Claude 3.5 Sonnet + IterativeAgent: 27.5%
+- o1 + IterativeAgent (36 hours): 42.4%
+- o1 BasicAgent: 43.3%
+- **DeepCode: 73.5%**
+
+The **+30.2% improvement** over the best-performing LLM agent demonstrates that sophisticated agent scaffolding, rather than extended inference time or larger models, is critical for complex code reproduction tasks.
+
+---
+
+### 🎯 **Autonomous Self-Orchestrating Multi-Agent Architecture**
 
 **The Challenges**:
 
@@ -441,6 +521,7 @@ Implementation Generation • Testing • Documentation
 
 ---
 
+
 ## 🚀 Quick Start
 
 

{deepcode_hku-1.0.4 → deepcode_hku-1.0.6}/__init__.py

@@ -5,7 +5,7 @@ DeepCode - AI Research Engine
 ⚡ Transform research papers into working code automatically
 """
 
-__version__ = "1.0.4"
+__version__ = "1.0.6"
 __author__ = "DeepCode Team"
 __url__ = "https://github.com/HKUDS/DeepCode"
 

{deepcode_hku-1.0.4 → deepcode_hku-1.0.6}/cli/cli_app.py

@@ -37,8 +37,7 @@ class CLIApp:
         self.app = None  # Will be initialized by workflow adapter
         self.logger = None
         self.context = None
-        # Document segmentation configuration
-        self.segmentation_config = {"enabled": True, "size_threshold_chars": 50000}
+        # Document segmentation will be managed by CLI interface
 
     async def initialize_mcp_app(self):
         """初始化MCP应用 - 使用工作流适配器"""
@@ -49,50 +48,10 @@ class CLIApp:
         """清理MCP应用 - 使用工作流适配器"""
         await self.workflow_adapter.cleanup_mcp_app()
 
-    def update_segmentation_config(self):
-        """Update document segmentation configuration in mcp_agent.config.yaml"""
-        import yaml
-        import os
-
-        config_path = os.path.join(
-            os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
-            "mcp_agent.config.yaml",
-        )
-
-        try:
-            # Read current config
-            with open(config_path, "r", encoding="utf-8") as f:
-                config = yaml.safe_load(f)
-
-            # Update document segmentation settings
-            if "document_segmentation" not in config:
-                config["document_segmentation"] = {}
-
-            config["document_segmentation"]["enabled"] = self.segmentation_config[
-                "enabled"
-            ]
-            config["document_segmentation"]["size_threshold_chars"] = (
-                self.segmentation_config["size_threshold_chars"]
-            )
-
-            # Write updated config
-            with open(config_path, "w", encoding="utf-8") as f:
-                yaml.dump(config, f, default_flow_style=False, allow_unicode=True)
-
-            self.cli.print_status(
-                "📄 Document segmentation configuration updated", "success"
-            )
-
-        except Exception as e:
-            self.cli.print_status(
-                f"⚠️ Failed to update segmentation config: {str(e)}", "warning"
-            )
-
     async def process_input(self, input_source: str, input_type: str):
         """处理输入源（URL或文件）- 使用升级版智能体编排引擎"""
         try:
-            # Update segmentation configuration before processing
-            self.update_segmentation_config()
+            # Document segmentation configuration is managed by CLI interface
 
             self.cli.print_separator()
             self.cli.print_status(
@@ -281,20 +240,9 @@ class CLIApp:
                     self.cli.show_history()
 
                 elif choice in ["c", "config", "configure"]:
-                    # Sync current segmentation config from CLI interface
-                    self.segmentation_config["enabled"] = self.cli.segmentation_enabled
-                    self.segmentation_config["size_threshold_chars"] = (
-                        self.cli.segmentation_threshold
-                    )
-
+                    # Show configuration menu - all settings managed by CLI interface
                     self.cli.show_configuration_menu()
 
-                    # Sync back from CLI interface after configuration changes
-                    self.segmentation_config["enabled"] = self.cli.segmentation_enabled
-                    self.segmentation_config["size_threshold_chars"] = (
-                        self.cli.segmentation_threshold
-                    )
-
                 else:
                     self.cli.print_status(
                         "Invalid choice. Please select U, F, T, C, H, or Q.", "warning"

{deepcode_hku-1.0.4 → deepcode_hku-1.0.6}/cli/cli_interface.py

@@ -39,10 +39,70 @@ class CLIInterface:
         self.uploaded_file = None
         self.is_running = True
         self.processing_history = []
-        self.enable_indexing = True  # Default configuration
-        self.segmentation_enabled = True  # Default to smart segmentation
-        self.segmentation_threshold = 50000  # Default threshold
+        self.enable_indexing = (
+            False  # Default configuration (matching UI: fast mode by default)
+        )
+
+        # Load segmentation config from the same source as UI
+        self._load_segmentation_config()
+
+        # Initialize tkinter availability
+        self._init_tkinter()
+
+    def _load_segmentation_config(self):
+        """Load segmentation configuration from mcp_agent.config.yaml"""
+        try:
+            from utils.llm_utils import get_document_segmentation_config
+
+            seg_config = get_document_segmentation_config()
+            self.segmentation_enabled = seg_config.get("enabled", True)
+            self.segmentation_threshold = seg_config.get("size_threshold_chars", 50000)
+        except Exception as e:
+            print(f"⚠️ Warning: Failed to load segmentation config: {e}")
+            # Fall back to defaults
+            self.segmentation_enabled = True
+            self.segmentation_threshold = 50000
+
+    def _save_segmentation_config(self):
+        """Save segmentation configuration to mcp_agent.config.yaml"""
+        import yaml
+        import os
+
+        # Get the project root directory (where mcp_agent.config.yaml is located)
+        current_file = os.path.abspath(__file__)
+        cli_dir = os.path.dirname(current_file)  # cli directory
+        project_root = os.path.dirname(cli_dir)  # project root
+        config_path = os.path.join(project_root, "mcp_agent.config.yaml")
+
+        try:
+            # Read current config
+            with open(config_path, "r", encoding="utf-8") as f:
+                config = yaml.safe_load(f)
+
+            # Update document segmentation settings
+            if "document_segmentation" not in config:
+                config["document_segmentation"] = {}
+
+            config["document_segmentation"]["enabled"] = self.segmentation_enabled
+            config["document_segmentation"]["size_threshold_chars"] = (
+                self.segmentation_threshold
+            )
+
+            # Write updated config
+            with open(config_path, "w", encoding="utf-8") as f:
+                yaml.dump(config, f, default_flow_style=False, allow_unicode=True)
+
+            print(
+                f"{Colors.OKGREEN}✅ Document segmentation configuration updated{Colors.ENDC}"
+            )
+
+        except Exception as e:
+            print(
+                f"{Colors.WARNING}⚠️ Failed to update segmentation config: {str(e)}{Colors.ENDC}"
+            )
 
+    def _init_tkinter(self):
+        """Initialize tkinter availability check"""
         # Check tkinter availability for file dialogs
         self.tkinter_available = True
         try:
@@ -765,6 +825,8 @@ class CLIInterface:
             elif choice in ["s", "segmentation"]:
                 current_state = getattr(self, "segmentation_enabled", True)
                 self.segmentation_enabled = not current_state
+                # Save the configuration to file
+                self._save_segmentation_config()
                 seg_mode = (
                     "📄 Smart Segmentation"
                     if self.segmentation_enabled

{deepcode_hku-1.0.4 → deepcode_hku-1.0.6}/cli/main_cli.py

@@ -214,15 +214,17 @@ async def main():
         # 创建CLI应用
         app = CLIApp()
 
-        # 设置配置
+        # 设置配置 - 默认禁用索引功能以加快处理速度
         if args.optimized:
             app.cli.enable_indexing = False
             print(
                 f"\n{Colors.YELLOW}⚡ Optimized mode enabled - indexing disabled{Colors.ENDC}"
             )
         else:
+            # 默认也禁用索引功能
+            app.cli.enable_indexing = False
             print(
-                f"\n{Colors.GREEN}🧠 Comprehensive mode enabled - full intelligence analysis{Colors.ENDC}"
+                f"\n{Colors.YELLOW}⚡ Fast mode enabled - indexing disabled by default{Colors.ENDC}"
             )
 
         # Configure document segmentation settings
@@ -230,18 +232,16 @@ async def main():
             print(
                 f"\n{Colors.MAGENTA}📄 Document segmentation disabled - using traditional processing{Colors.ENDC}"
             )
-            app.segmentation_config = {
-                "enabled": False,
-                "size_threshold_chars": args.segmentation_threshold,
-            }
+            app.cli.segmentation_enabled = False
+            app.cli.segmentation_threshold = args.segmentation_threshold
+            app.cli._save_segmentation_config()
         else:
             print(
                 f"\n{Colors.BLUE}📄 Smart document segmentation enabled (threshold: {args.segmentation_threshold} chars){Colors.ENDC}"
             )
-            app.segmentation_config = {
-                "enabled": True,
-                "size_threshold_chars": args.segmentation_threshold,
-            }
+            app.cli.segmentation_enabled = True
+            app.cli.segmentation_threshold = args.segmentation_threshold
+            app.cli._save_segmentation_config()
 
         # 检查是否为直接处理模式
         if args.file or args.url or args.chat:
@@ -250,7 +250,9 @@ async def main():
                 if not os.path.exists(args.file):
                     print(f"{Colors.FAIL}❌ File not found: {args.file}{Colors.ENDC}")
                     sys.exit(1)
-                success = await run_direct_processing(app, args.file, "file")
+                # 使用 file:// 前缀保持与交互模式一致，确保文件被复制而非移动
+                file_url = f"file://{os.path.abspath(args.file)}"
+                success = await run_direct_processing(app, file_url, "file")
             elif args.url:
                 success = await run_direct_processing(app, args.url, "url")
             elif args.chat: