cnhkmcp 1.8.10__py3-none-any.whl → 2.0__py3-none-any.whl

cnhkmcp/untracked/APP/MODULAR_STRUCTURE.md

@@ -4,21 +4,30 @@ This document describes the modular structure implemented for the BRAIN Expressi
 
 ## Overview
 
-The application has been refactored to use Flask blueprints for better code organization and maintainability. The paper analysis functionality has been separated into its own module.
+The application has been refactored to use Flask blueprints for better code organization and maintainability. The application is divided into several functional modules including paper analysis, feature engineering, idea generation, and inspiration.
 
 ## Project Structure
 
 ```
 BRAINProject/
-├── app.py                          # Main Flask application
+├── 运行打开我.py                   # Main Flask application entry point
 ├── blueprints/                     # Blueprint modules
 │   ├── __init__.py                # Package initialization
+│   ├── feature_engineering.py     # Feature engineering blueprint
+│   ├── idea_house.py              # Idea house blueprint (Coze integration)
+│   ├── inspiration_house.py       # Inspiration house blueprint
 │   └── paper_analysis.py          # Paper analysis blueprint
 ├── templates/
 │   ├── index.html                 # Main page template
+│   ├── feature_engineering.html   # Feature engineering page template
+│   ├── idea_house.html            # Idea house page template
+│   ├── inspiration_house.html     # Inspiration house page template
 │   └── paper_analysis.html        # Paper analysis page template
 ├── static/
 │   ├── script.js                  # Main application JavaScript
+│   ├── feature_engineering.js     # Feature engineering JavaScript
+│   ├── idea_house.js              # Idea house JavaScript
+│   ├── inspiration_house.js       # Inspiration house JavaScript
 │   ├── paper_analysis.js          # Paper analysis JavaScript
 │   ├── brain.js                   # BRAIN API functions
 │   ├── decoder.js                 # Template decoder functions
@@ -29,34 +38,29 @@ BRAINProject/
 ## Blueprint Structure
 
 ### Paper Analysis Blueprint (`blueprints/paper_analysis.py`)
-
-The paper analysis functionality has been moved to a separate blueprint that includes:
-
-#### Routes:
-- `GET /paper-analysis/` - Paper analysis page
-- `POST /paper-analysis/api/test-deepseek` - Test Deepseek API connection
-- `POST /paper-analysis/api/analyze-paper` - Analyze uploaded papers
-
-#### Functions:
-- File processing functions for various formats (PDF, DOCX, RTF, LaTeX, Markdown, plain text)
-- Deepseek API integration functions
-- Text extraction and processing utilities
-
-### Main Application (`app.py`)
-
-The main application now focuses on:
-
-#### Routes:
-- BRAIN API authentication and session management
-- BRAIN operators and data fields API
-- Expression testing functionality
-- Main application routes
-
-#### Features:
-- Auto-dependency installation
-- BRAIN API integration
-- Session management for multiple users
-- Template decoding functionality
+Handles paper analysis functionality.
+- Routes: `/paper-analysis/`
+- Features: File processing (PDF, DOCX, etc.), Deepseek API integration.
+
+### Feature Engineering Blueprint (`blueprints/feature_engineering.py`)
+Handles feature engineering tasks.
+- Routes: `/feature-engineering/`
+- Features: Deepseek/Kimi API integration for feature generation.
+
+### Idea House Blueprint (`blueprints/idea_house.py`)
+Handles idea generation using Coze API.
+- Routes: `/idea-house/`
+- Features: Coze API integration for processing data fields.
+
+### Inspiration House Blueprint (`blueprints/inspiration_house.py`)
+Handles inspiration generation.
+- Routes: `/inspiration-house/`
+- Features: Deepseek/Kimi API integration.
+
+### Main Application (`运行打开我.py`)
+The main application entry point.
+- Routes: Main application routes, BRAIN API authentication.
+- Features: Auto-dependency installation, Blueprint registration.
 
 ## Benefits of This Structure
 
@@ -66,17 +70,6 @@ The main application now focuses on:
 4. **Separation of Concerns**: Each blueprint handles a specific domain
 5. **Testability**: Individual modules can be tested independently
 
-## URL Structure
-
-### Before Refactoring:
-- `/paper-analysis` - Paper analysis page
-- `/api/test-deepseek` - Test Deepseek API
-- `/api/analyze-paper` - Analyze paper
-
-### After Refactoring:
-- `/paper-analysis/` - Paper analysis page (blueprint)
-- `/paper-analysis/api/test-deepseek` - Test Deepseek API (blueprint)
-- `/paper-analysis/api/analyze-paper` - Analyze paper (blueprint)
 
 ## Dependencies
 
@@ -86,13 +79,9 @@ The main application now focuses on:
 - requests
 - pandas (for BRAIN API integration)
 
-### Paper Analysis Dependencies:
-- All main dependencies plus:
-- PyPDF2 or pdfplumber (for PDF processing)
-- python-docx (for Word documents)
-- docx2txt (for legacy DOC files)
-- striprtf (for RTF files)
-- PyMuPDF (alternative PDF library)
+### Module Specific Dependencies:
+- **Paper Analysis**: PyPDF2, pdfplumber, python-docx, docx2txt, striprtf, PyMuPDF
+- **Idea House**: cozepy (for Coze API)
 
 ## Adding New Blueprints
 
@@ -109,7 +98,7 @@ To add a new blueprint:
    def index():
        return render_template('new_feature.html')
    ```
-3. Import and register the blueprint in `app.py`:
+3. Import and register the blueprint in the main application file (`运行打开我.py`):
    ```python
    from blueprints.new_feature import new_blueprint
    app.register_blueprint(new_blueprint)

cnhkmcp/untracked/APP/Tranformer/Transformer.py

@@ -5,6 +5,58 @@ import asyncio
 import openai
 import re
 from typing import Optional, Union # Added this import
+try:
+    from .validator_hooks import is_valid_template_expr, has_empty_datafield_candidates
+except Exception:
+    # Fallback for direct script execution
+    try:
+        from validator_hooks import is_valid_template_expr, has_empty_datafield_candidates
+    except Exception:
+        is_valid_template_expr = None
+        has_empty_datafield_candidates = None
+
+# --- Validation wrappers to integrate into the pipeline ---
+def _filter_valid_templates(
+    proposed_templates: dict,
+    operators_meta,
+    brain_session,
+    settings: dict,
+    parse_alpha_code_func,
+):
+    """Return dict of only templates that pass validation.
+
+    Safe no-op if validation helpers are unavailable.
+    """
+    if not is_valid_template_expr or not parse_alpha_code_func:
+        return proposed_templates
+    filtered = {}
+    for template_expr, template_expl in proposed_templates.items():
+        try:
+            if is_valid_template_expr(
+                template_expr,
+                operators_meta,
+                brain_session,
+                settings,
+                parse_alpha_code_func,
+            ):
+                filtered[template_expr] = template_expl
+        except Exception:
+            # Be conservative: drop on exceptions
+            continue
+    return filtered
+
+
+def _should_skip_due_to_empty_candidates(populated_info: dict) -> bool:
+    """True if any data_field placeholder has zero candidates.
+
+    Safe no-op fallback when helper is missing.
+    """
+    if not has_empty_datafield_candidates:
+        return False
+    try:
+        return has_empty_datafield_candidates(populated_info)
+    except Exception:
+        return False
 import logging
 import pandas as pd
 import os
@@ -36,7 +88,8 @@ DATA_CATEGORIES = None
 
 template_summary = """# BRAIN论坛Alpha模板精华总结
 
-本文档旨在系统性地整理和总结优秀Alpha模板。所有内容均源自社区顾问的无私分享或公开文件，并经过了初步的筛选和整合。每个模板都附有其核心思想、变量说明、适用场景及原帖链接，方便您理解、应用和进一步探索。
+本文档旨在系统性地整理和总结优秀Alpha模板,它是一种可复用的标准化框架性表达式，它承载着特定的经济逻辑，并预留出若干 “配置项”（包括数据字段、算子、分组方式、衰减规则、中性化方案等），用于生成多个候选阿尔法因子。其典型流程为：数据清洗（数据回填、缩尾处理）→ 跨时间或跨标的维度进行转换 / 对比 → 排序 / 中性化处理 →（可选步骤）衰减调整 / 换手率优化。这种模板模式能够推动系统化的因子挖掘、复用与多元化配置，同时确保每一个因子都具备清晰可追溯的经济逻辑支撑。
+以下每个模板都附有其核心思想、变量说明、适用场景及原帖链接，方便您理解、应用和进一步探索。
 使用时请思考如何将下列模板与有的Alpha表达式结合，创造出新的模板来捕捉和发现市场规律，找到”好“公司和”坏“公司
 **使用前请注意：**
 *   **过拟合风险**：部分模板可能存在过拟合风险，请谨慎使用，并结合IS-Ladder测试、多市场回测等方法进行验证。
@@ -387,6 +440,61 @@ template_summary = """# BRAIN论坛Alpha模板精华总结
 *   **优化方向**:
     *   **事件驱动**: 在财报日前后缩短 `ts_mean` 的窗口，提高灵敏度。
 
+---
+
+## 新增模板（CAPM與估值、分析師期限、期權、搜尋優化）
+
+### 1. CAPM殘差模板（市場/行業中性收益）
+*   **表達式**: `ts_regression(returns, group_mean(returns, log(ts_mean(cap,21)), sector), 252, rettype=0)`。
+*   **核心思想**: 回歸剔除市場/行業暴露，保留超額收益殘差作為Alpha。
+*   **適用場景**: 通用起手式，回歸殘差可作後續動量或價值信號的底板。
+*   **優化**: 改`rettype=2`獲取beta斜率，用於風險排序或低/高beta組合；可加入`winsorize`、`ts_backfill`預處理。
+
+### 2. CAPM廣義殘差（任意特徵）
+*   **表達式**: `data = winsorize(ts_backfill(<data>,63), std=4); gpm = group_mean(data, log(ts_mean(cap,21)), sector); resid = ts_regression(data, gpm, 252, rettype=0)`。
+*   **核心思想**: 將任意特徵去除組均值成分，提取行業相對的特異性部分。
+*   **適用場景**: 基本面、情緒、替代數據的組內殘差提純。
+*   **優化**: 先`group_zscore`再回歸；對`resid`再做`ts_zscore`或`ts_mean`平滑。
+
+### 3. CAPM Beta排序模板
+*   **表達式**: `target_data = winsorize(ts_backfill(<target>,63), std=4); market_data = winsorize(ts_backfill(<market>,63), std=4); beta = ts_regression(target_data, group_mean(market_data, log(ts_mean(cap,21)), sector), 252, rettype=2)`。
+*   **核心思想**: 提取行業內相對beta，作為風險/防禦排序；低beta偏防禦，高beta偏進攻。
+*   **優化**: 行業或國家分組；可按beta分桶做長低/短高，或反向用於高波段套利。
+
+### 4. 實際-預估差異模板（Analyst Surprise）
+*   **表達式**: `group_zscore(subtract(group_zscore(<act>, industry), group_zscore(<est>, industry)), industry)`。
+*   **核心思想**: 行業內標準化後的實際值與預估值差，捕捉超預期或低於預期的驚喜。
+*   **適用場景**: analyst7/analyst14/earnings估值類字段。
+*   **優化**: 對差分再做`ts_zscore`；門檻交易只在|z|>1.5時開倉。
+
+### 5. 分析師期限結構模板（近遠期預估斜率）
+*   **表達式**: `group_zscore(subtract(group_zscore(anl14_mean_eps_<p1>, industry), group_zscore(anl14_mean_eps_<p2>, industry)), industry)`，`<p1>/<p2>`為fp1/fp2/fy1/fy2等。
+*   **核心思想**: 比較短期與長期預估的行業內斜率，捕捉預期加速或鈍化。
+*   **適用場景**: analyst14/15 期別字段；適用成長/拐點挖掘。
+*   **優化**: 擴展到多期間差分或`ts_delta`跟蹤斜率變化；對斜率做`rank`或`winsorize`。
+
+### 6. 期權Greeks淨值模板
+*   **表達式**: `group_operator(<put_greek> - <call_greek>, <group>)`，Greek可選Delta/Gamma/Vega/Theta。
+*   **核心思想**: 同組內看多vs看空的期權敏感度差，反映隱含情緒或凸性差異。
+*   **適用場景**: Option數據集；行業或市值分組下的情緒/波動信號。
+*   **優化**: 多Greek加權組合；對淨值再`ts_mean`平滑；事件期(財報)可降權或過濾。
+
+### 7. IV Skew動量擴展
+*   **表達式**: `ts_delta(implied_volatility_call_<w>, <p>) - ts_delta(implied_volatility_put_<w>, <p>)`。
+*   **核心思想**: Call與Put隱含波動變化差捕捉情緒轉折；可做多情緒改善、做空情緒惡化。
+*   **優化**: 加`trade_when(abs(skew)>thr)`門檻；財報前後縮窗；行業中性。
+
+### 8. 殘差動量精簡版
+*   **表達式**: `res = regression_neut(returns, <common_factor_matrix>); ts_mean(res, <window>)`。
+*   **核心思想**: 先剝離市場/風格暴露，再對特異收益做動量；較原版多重回歸更輕量。
+*   **優化**: 使用`ts_decay_linear`增加近期權重；行業內`group_rank`提升截面穩定度。
+
+### 9. 分紅/現金流組間殘差（簡版）
+*   **表達式**: `alpha = ts_zscore(ts_backfill(<cf_or_div_field>,90)); g = group_mean(alpha, <group>, <weight_opt>); resid = alpha - g; group_zscore(resid, <group>)`。
+*   **核心思想**: 先回填平滑，再對組均值做殘差，捕捉組內相對高/低分紅或現金流質量。
+*   **適用場景**: fnd8/fnd6/topdiv等分紅現金流字段；行業/國家分組。
+*   **優化**: 權重可用log(cap)或vol逆；對resid再做`ts_mean`平滑。
+
 """
 
 class SingleSession(requests.Session):
@@ -1607,6 +1715,23 @@ async def generate_new_alphas(alpha_description, brain_session, template_summary
     print("\n--- Proposed Alpha Templates (JSON) (建议的Alpha模板,多样性会受到模型和模板总结文档的影响) ---")
     print(json.dumps(proposed_templates, indent=4))
 
+    # --- Validation: Drop templates with suspicious literal identifiers ---
+    try:
+        operators_meta = get_brain_operators().get('operators', [])
+        proposed_templates = _filter_valid_templates(
+            proposed_templates,
+            operators_meta,
+            brain_session,
+            details.get('settings', {}),
+            parse_alpha_code,
+        )
+    except Exception as e:
+        print(f"⚠ 模板校验步骤出现异常，跳过校验: {e}")
+
+    if not proposed_templates:
+        print("❌ 所有模板在校验后被丢弃，无法继续。")
+        sys.exit(1)
+
     # --- Step 5: Process all proposed templates and gather candidates ---
     # --- Step 6: Prepare for Output ---
     # Ensure the output directory exists next to this script
@@ -1625,6 +1750,11 @@ async def generate_new_alphas(alpha_description, brain_session, template_summary
         print(f"\n--- Populating template (正在填充模板): '{template_expr}' ---")
         try:
             populated_info = await populate_template(brain_session, details, template_expr, template_expl, operator_summary, llm_client, top_n_datafield=top_n_datafield, user_region=user_region, user_universe=user_universe, user_delay=user_delay, user_category=user_category, user_data_type=user_data_type)
+
+            # Skip templates where any data_field placeholder has zero candidates
+            if _should_skip_due_to_empty_candidates(populated_info):
+                print("⚠ 该模板存在数据字段候选为空的占位符，跳过此模板。")
+                continue
             
             final_output[template_expr] = {
                 "template_explanation": template_expl,