@roll-agent/octopus-agent 0.0.1

package/README.md

@@ -0,0 +1,109 @@
+# Octopus Sponge NL2SQL Agent
+
+本仓库实现 Octopus 对接 Sponge MCP Server 的第一版 MVP。
+
+## 能力
+
+- Schema Cache（表结构缓存）：本地保存 Sponge 返回的 schema。
+- MCP Client（MCP 客户端）：调用 `get_schema`、`get_allowed_context`、`validate_sql`、`execute_sql`。
+- MCP Sampling（采样）：由 roll-core 代调 LLM 生成和修正 SQL。
+- SQL Guard（SQL 保护规则）：只允许单条 `SELECT`，必须带 `LIMIT`。
+- Audit Log（审计日志）：记录 trace、session、SQL hash、执行摘要。
+
+## 本地运行
+
+```bash
+python -m octopus_sponge_skill "查询我能看到的品牌列表" --user-id 456
+```
+
+运行参数由 Agent 内置；用户本地只需要配置 Sponge MCP Server 地址和 Token。
+
+## roll-core 接入
+
+本仓库支持纯 Python stdio MCP 子 Agent 接入 roll-core。
+
+### npm 包安装
+
+发布到 npm 后，可通过 `roll agent install` 安装：
+
+```bash
+roll agent install @roll-agent/octopus-agent
+roll run octopus-agent diagnostic_status --json
+```
+
+该 npm package（npm 包）直接包含 Python 源码；`roll agent install` 不会再从 PyPI 下载 `octopus-sponge-skill`。
+安装机器上的 `python3` 需要是 Python 3.11+。
+
+### 本地源码注册
+
+```bash
+roll agent add .
+roll run octopus-agent diagnostic_status --json
+roll run octopus-agent refresh_schema --json \
+  --input-json '{"traceId":"trace_schema_10001","refreshReason":"manual"}'
+roll run octopus-agent query_sponge --json \
+  --question "查询我能看到的品牌列表" \
+  --user-id 456
+```
+
+`refresh_schema` 必填 `traceId`。Agent 会自动读取本地缓存里的 `schemaVersion` 并传给 Sponge。
+
+指定输出格式：
+
+```bash
+roll run octopus-agent query_sponge --json \
+  --input-json '{"question":"查询我能看到的品牌列表","userId":456,"resultFormat":"markdown"}'
+```
+
+支持 `text`、`yaml`、`md`、`markdown`、`html`。不传时会从问题里自动识别。
+
+调试 SQL：
+
+```bash
+roll run octopus-agent query_sponge --json \
+  --input-json '{"question":"查询我能看到的品牌列表","userId":456,"includeSql":true}'
+```
+
+`includeSql=true` 时会返回 `generatedSql`、`normalizedSql`、`repairedSql` 和 `validation`。
+
+`package.json` 中声明了 roll-core 启动方式：
+
+```json
+{
+  "rollAgent": {
+    "runtime": {"ownership": "on-demand", "transport": "stdio"},
+    "start": {
+      "command": "python3",
+      "args": ["-m", "octopus_sponge_skill.roll_server"]
+    }
+  }
+}
+```
+
+环境变量可放到 `roll.config.yaml`：
+
+```yaml
+agents:
+  env:
+    octopus-agent:
+      SPONGE_MCP_BASE_URL: "https://sponge-mcp.duliday.com"
+      SPONGE_MCP_ACCESS_TOKEN: "xxxx"
+```
+
+`SPONGE_MCP_BASE_URL` 只填域名根地址，不要带 `/mcp/admin/health`；Agent 会调用 `/mcp/tools/...`。
+
+LLM 配置读取 roll-core 现有 `llm` 配置；Agent 不读取模型 API Key。
+`query_sponge` 会通过 MCP Sampling 请求 roll-core 代调模型。
+
+## 本地持久化
+
+- `data/sponge-schema-cache.json`：schema 缓存。
+- `logs/octopus-sponge-audit.log`：审计日志。
+
+不新建数据库，不保存真实项目 ID、品牌 ID、数据库账号密码。
+
+## 自测
+
+```bash
+PYTHONPATH=src python -m unittest discover -s tests
+```

package/SKILL.md

@@ -0,0 +1,47 @@
+---
+name: octopus-agent
+description: Octopus Sponge NL2SQL Agent，把自然语言问题转换为只读 SQL，并通过 Sponge MCP Server 校验和执行。
+metadata:
+  roll-env-file: references/env.yaml
+---
+
+# Octopus Sponge Agent
+
+## 职责
+
+本 Agent 只负责 Sponge 数据查询：
+
+- 读取和刷新 Schema Cache（表结构缓存）。
+- 基于用户问题和 schema context，通过 MCP Sampling（采样）请求 roll-core 代调 LLM 生成只读 SQL。
+- 调用 Sponge MCP Server 做权限上下文、SQL 校验和 SQL 执行。
+- 记录 Audit Log（审计日志）。
+
+## Tools
+
+| Tool | 中文解释 | 用途 |
+|-|-|-|
+| `diagnostic_status` | 诊断状态 | 检查配置、缓存和日志路径 |
+| `refresh_schema` | 刷新表结构 | 人工刷新本地 schema 缓存 |
+| `query_sponge` | 查询 Sponge 数据 | 自然语言查询入口 |
+
+`refresh_schema` 必填 `traceId`；Agent 会自动读取本地缓存里的 `schemaVersion` 并传给 Sponge。
+
+`query_sponge` 支持 `resultFormat`：`text`、`yaml`、`md`、`markdown`、`html`。不传时按用户问题自动识别。
+调试时可传 `includeSql=true` 返回 `generatedSql`、`normalizedSql`、`repairedSql` 和 `validation`；默认不返回 SQL。
+
+## 上层调用规则
+
+1. 查询前如需排查环境，先调用 `diagnostic_status`。
+2. schema 过期或缺失时，调用 `refresh_schema`。
+3. 用户提出业务数据查询时，调用 `query_sponge`。
+4. 不要求上层提供真实项目 ID 或品牌 ID；权限由 Sponge MCP Server 处理。
+5. 不允许绕过本 Agent 直接连业务数据库。
+6. 运行参数由 Agent 内置；本地只通过环境变量配置 Sponge 地址和 Token。
+7. LLM 由 roll-core 通过 MCP Sampling 代调，本 Agent 不读取模型 API Key。
+
+## 持久化
+
+- `data/sponge-schema-cache.json`：Schema Cache（表结构缓存）。
+- `logs/octopus-sponge-audit.log`：Audit Log（审计日志）。
+
+不新建数据库，不保存真实项目 ID、品牌 ID、数据库账号密码。

package/octopus_sponge_skill/__init__.py

@@ -0,0 +1,11 @@
+from __future__ import annotations
+
+from pathlib import Path
+
+_src_package = Path(__file__).resolve().parent.parent / "src" / "octopus_sponge_skill"
+if _src_package.exists():
+    __path__.append(str(_src_package))
+
+from ._version import __version__
+
+__all__ = ["__version__"]

package/package.json

@@ -0,0 +1,37 @@
+{
+  "name": "@roll-agent/octopus-agent",
+  "version": "0.0.1",
+  "description": "Octopus Sponge NL2SQL Agent",
+  "license": "UNLICENSED",
+  "keywords": [
+    "roll-agent",
+    "mcp",
+    "nl2sql"
+  ],
+  "publishConfig": {
+    "access": "public"
+  },
+  "files": [
+    "octopus_sponge_skill",
+    "src",
+    "references",
+    "SKILL.md",
+    "pyproject.toml",
+    "!**/__pycache__/**",
+    "!**/*.pyc",
+    "!**/*.egg-info/**"
+  ],
+  "rollAgent": {
+    "runtime": {
+      "ownership": "on-demand",
+      "transport": "stdio"
+    },
+    "start": {
+      "command": "python3",
+      "args": [
+        "-m",
+        "octopus_sponge_skill.roll_server"
+      ]
+    }
+  }
+}

package/pyproject.toml

@@ -0,0 +1,17 @@
+[project]
+name = "octopus-skill"
+version = "0.0.1"
+description = "Octopus Sponge NL2SQL Agent"
+readme = "README.md"
+requires-python = ">=3.11"
+dependencies = []
+
+[project.scripts]
+octopus-sponge = "octopus_sponge_skill.main:main"
+
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[tool.setuptools.packages.find]
+where = ["src"]

package/references/env.yaml

@@ -0,0 +1,8 @@
+env:
+  SPONGE_MCP_BASE_URL:
+    required: true
+    purpose: Sponge MCP Server 地址
+    example: https://sponge-mcp.duliday.com
+  SPONGE_MCP_ACCESS_TOKEN:
+    required: true
+    purpose: Sponge MCP Server 固定访问 Token

package/src/octopus_sponge_skill/__init__.py

@@ -0,0 +1,5 @@
+"""Octopus Sponge NL2SQL Agent."""
+
+from ._version import __version__
+
+__all__ = ["__version__"]

package/src/octopus_sponge_skill/__main__.py

@@ -0,0 +1,4 @@
+from .main import main
+
+main()
+

package/src/octopus_sponge_skill/_version.py

@@ -0,0 +1,3 @@
+from __future__ import annotations
+
+__version__ = "0.0.1"

package/src/octopus_sponge_skill/agent.py

@@ -0,0 +1,288 @@
+from __future__ import annotations
+
+import re
+from dataclasses import dataclass, field
+from typing import Any
+
+from .audit_logger import AuditLogger, sql_hash
+from .config import AppConfig
+from .context import UserContext, new_session_id, new_trace_id
+from .llm_sql_generator import Sampler, SamplingSqlGenerator
+from .mcp_client import SpongeMcpClient
+from .result_renderer import render_result
+from .schema_cache import SchemaCache
+from .schema_context_retriever import retrieve_schema_context
+from .sql_generator import RuleBasedSqlGenerator, enforce_sql_rules
+
+
+@dataclass(frozen=True)
+class QueryResult:
+    answer: str
+    generated_sql: str
+    normalized_sql: str
+    repaired_sql: str | None
+    validation: dict[str, Any]
+
+
+@dataclass
+class SpongeAgent:
+    config: AppConfig
+    mcp_client: SpongeMcpClient
+    schema_cache: SchemaCache
+    audit_logger: AuditLogger
+    _last_validation_error: dict[str, Any] = field(default_factory=dict, init=False)
+    _last_validation_result: dict[str, Any] = field(default_factory=dict, init=False)
+
+    @classmethod
+    def from_config(cls, config: AppConfig) -> "SpongeAgent":
+        return cls(
+            config=config,
+            mcp_client=SpongeMcpClient(
+                base_url=config.sponge_mcp.base_url,
+                access_token=config.sponge_mcp.access_token,
+                timeout_ms=config.sponge_mcp.timeout_ms,
+            ),
+            schema_cache=SchemaCache(config.schema.cache_path),
+            audit_logger=AuditLogger(config.audit.log_path, enabled=config.audit.enabled),
+        )
+
+    def answer(
+        self,
+        *,
+        question: str,
+        user_id: int,
+        session_id: str | None = None,
+        device_id: str | None = None,
+        sampler: Sampler | None = None,
+        result_format: str | None = None,
+    ) -> str:
+        return self.query(
+            question=question,
+            user_id=user_id,
+            session_id=session_id,
+            device_id=device_id,
+            sampler=sampler,
+            result_format=result_format,
+        ).answer
+
+    def query(
+        self,
+        *,
+        question: str,
+        user_id: int,
+        session_id: str | None = None,
+        device_id: str | None = None,
+        sampler: Sampler | None = None,
+        result_format: str | None = None,
+    ) -> QueryResult:
+        context = UserContext(
+            user_id=user_id,
+            session_id=session_id or new_session_id(),
+            device_id=device_id,
+        )
+        sql_question = _strip_result_format_instruction(question)
+        schema = self._ensure_schema()
+        allowed_context = self._get_allowed_context(context, sql_question)
+        scope_hints = allowed_context.get("scopeHints", {})
+        if sampler is not None:
+            schema_context = retrieve_schema_context(schema, sql_question)
+            generator = SamplingSqlGenerator(
+                sampler=sampler,
+                default_limit=self.config.sql.default_limit,
+                max_limit=self.config.sql.max_limit,
+            )
+            generated = generator.generate(
+                question=sql_question,
+                schema_context=schema_context,
+                scope_hints=scope_hints,
+            )
+        else:
+            schema_context = None
+            generator = RuleBasedSqlGenerator(
+                default_limit=self.config.sql.default_limit,
+                max_limit=self.config.sql.max_limit,
+            )
+            generated = generator.generate(
+                question=sql_question,
+                schema=schema,
+                scope_hints=scope_hints,
+            )
+        enforce_sql_rules(generated.sql, max_limit=self.config.sql.max_limit)
+        normalized_sql = self._validate_sql(context, question, generated.sql)
+        repaired_sql: str | None = None
+        if sampler is not None and normalized_sql is None and schema_context is not None:
+            repaired = generator.repair(
+                question=sql_question,
+                original_sql=generated.sql,
+                validation_error=self._last_validation_error,
+                schema_context=schema_context,
+                scope_hints=scope_hints,
+            )
+            repaired_sql = repaired.sql
+            enforce_sql_rules(repaired.sql, max_limit=self.config.sql.max_limit)
+            normalized_sql = self._validate_sql(context, question, repaired.sql, raise_on_error=True)
+        if normalized_sql is None:
+            raise RuntimeError("SQL validation failed")
+        result = self._execute_sql(context, question, normalized_sql)
+        return QueryResult(
+            answer=render_result(question, result, result_format),
+            generated_sql=generated.sql,
+            normalized_sql=normalized_sql,
+            repaired_sql=repaired_sql,
+            validation=self._last_validation_result,
+        )
+
+    def _ensure_schema(self) -> dict[str, Any]:
+        cached_schema = self.schema_cache.load()
+        cached_version = str(cached_schema.get("schemaVersion")) if cached_schema and cached_schema.get("schemaVersion") else None
+        trace_id = new_trace_id("trace_schema")
+        try:
+            schema_response = self.mcp_client.get_schema(
+                trace_id=trace_id,
+                schema_version=cached_version,
+                refresh_reason="startup",
+            )
+        except Exception:
+            if cached_schema is not None:
+                self.audit_logger.log(
+                    {
+                        "traceId": trace_id,
+                        "tool": "get_schema",
+                        "schemaVersion": cached_version,
+                        "fallback": "cached_schema",
+                    }
+                )
+                return cached_schema
+            raise
+
+        if schema_response.get("changed") is False and cached_schema is not None:
+            self.audit_logger.log(
+                {
+                    "traceId": trace_id,
+                    "tool": "get_schema",
+                    "schemaVersion": cached_version,
+                    "changed": False,
+                }
+            )
+            return cached_schema
+
+        if schema_response.get("changed") is False and cached_schema is None:
+            schema_response = self.mcp_client.get_schema(
+                trace_id=new_trace_id("trace_schema"),
+                schema_version=None,
+                refresh_reason="startup",
+            )
+
+        if "tables" in schema_response:
+            self.schema_cache.save(schema_response)
+            self.audit_logger.log(
+                {
+                    "traceId": trace_id,
+                    "tool": "get_schema",
+                    "schemaVersion": schema_response.get("schemaVersion"),
+                    "changed": schema_response.get("changed"),
+                }
+            )
+            return schema_response
+
+        if cached_schema is not None:
+            return cached_schema
+        raise RuntimeError("Sponge get_schema did not return tables and no local schema cache exists")
+
+    def _get_allowed_context(self, context: UserContext, question: str) -> dict[str, Any]:
+        trace_id = new_trace_id()
+        result = self.mcp_client.get_allowed_context(
+            user_id=context.user_id,
+            trace_id=trace_id,
+            session_id=context.session_id,
+            device_id=context.device_id,
+            question=question,
+        )
+        self.audit_logger.log(
+            {
+                "traceId": trace_id,
+                "sessionId": context.session_id,
+                "userId": context.user_id,
+                "question": question,
+                "tool": "get_allowed_context",
+            }
+        )
+        return result
+
+    def _validate_sql(
+        self,
+        context: UserContext,
+        question: str,
+        sql: str,
+        *,
+        raise_on_error: bool = False,
+    ) -> str | None:
+        trace_id = new_trace_id()
+        result = self.mcp_client.validate_sql(
+            user_id=context.user_id,
+            trace_id=trace_id,
+            session_id=context.session_id,
+            sql=sql,
+        )
+        self._last_validation_result = result
+        self.audit_logger.log(
+            {
+                "traceId": trace_id,
+                "sessionId": context.session_id,
+                "userId": context.user_id,
+                "question": question,
+                "tool": "validate_sql",
+                "sqlHash": sql_hash(sql),
+                "validateResult": result,
+                "errorCode": result.get("error", {}).get("code") if isinstance(result.get("error"), dict) else None,
+            }
+        )
+        if not result.get("valid"):
+            error = result.get("error", {})
+            self._last_validation_error = error if isinstance(error, dict) else {"code": "UNKNOWN"}
+            code = error.get("code") if isinstance(error, dict) else "UNKNOWN"
+            if raise_on_error:
+                raise RuntimeError(f"SQL validation failed: {code}")
+            return None
+        return str(result.get("normalizedSql") or sql)
+
+    def _execute_sql(self, context: UserContext, question: str, sql: str) -> dict[str, Any]:
+        trace_id = new_trace_id()
+        result = self.mcp_client.execute_sql(
+            user_id=context.user_id,
+            trace_id=trace_id,
+            session_id=context.session_id,
+            sql=sql,
+            timeout_ms=self.config.sponge_mcp.timeout_ms,
+        )
+        self.audit_logger.log(
+            {
+                "traceId": trace_id,
+                "sessionId": context.session_id,
+                "userId": context.user_id,
+                "question": question,
+                "tool": "execute_sql",
+                "sqlHash": sql_hash(sql),
+                "executeResult": {
+                    "success": result.get("success"),
+                    "rowCount": result.get("rowCount"),
+                    "elapsedMs": result.get("elapsedMs"),
+                },
+                "rowCount": result.get("rowCount"),
+                "elapsedMs": result.get("elapsedMs"),
+            }
+        )
+        return result
+
+
+def _strip_result_format_instruction(question: str) -> str:
+    cleaned = question
+    patterns = [
+        r"[，,。；;]?\s*输出\s*(?:yaml|html|markdown|md)\s*(?:格式)?",
+        r"[，,。；;]?\s*用\s*表格\s*展示",
+        r"[，,。；;]?\s*以\s*(?:yaml|html|markdown|md)\s*(?:格式)?\s*输出",
+        r"[，,。；;]?\s*return\s+(?:yaml|html|markdown|md)",
+    ]
+    for pattern in patterns:
+        cleaned = re.sub(pattern, "", cleaned, flags=re.IGNORECASE)
+    return cleaned.strip() or question

package/src/octopus_sponge_skill/audit_logger.py

@@ -0,0 +1,29 @@
+from __future__ import annotations
+
+import hashlib
+import json
+from dataclasses import dataclass
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any
+
+
+@dataclass
+class AuditLogger:
+    path: Path
+    enabled: bool = True
+
+    def log(self, event: dict[str, Any]) -> None:
+        if not self.enabled:
+            return
+        self.path.parent.mkdir(parents=True, exist_ok=True)
+        record = {
+            "occurredAt": datetime.now(timezone.utc).isoformat(),
+            **event,
+        }
+        with self.path.open("a", encoding="utf-8") as file:
+            file.write(json.dumps(record, ensure_ascii=False, sort_keys=True) + "\n")
+
+
+def sql_hash(sql: str) -> str:
+    return hashlib.sha256(sql.encode("utf-8")).hexdigest()

package/src/octopus_sponge_skill/config.py

@@ -0,0 +1,73 @@
+from __future__ import annotations
+
+import os
+from dataclasses import dataclass
+from pathlib import Path
+
+
+PACKAGE_ROOT = Path.cwd()
+DEFAULT_SPONGE_MCP_BASE_URL = "https://sponge-mcp.duliday.com"
+DEFAULT_SPONGE_MCP_ACCESS_TOKEN = ""
+
+
+@dataclass(frozen=True)
+class SpongeMcpConfig:
+    base_url: str
+    access_token: str
+    timeout_ms: int
+
+
+@dataclass(frozen=True)
+class SchemaConfig:
+    cache_path: Path
+    refresh_on_start: bool
+    refresh_interval_minutes: int
+
+
+@dataclass(frozen=True)
+class SqlConfig:
+    max_repair_attempts: int
+    default_limit: int
+    max_limit: int
+
+
+@dataclass(frozen=True)
+class AuditConfig:
+    enabled: bool
+    log_path: Path
+
+
+@dataclass(frozen=True)
+class AppConfig:
+    sponge_mcp: SpongeMcpConfig
+    schema: SchemaConfig
+    sql: SqlConfig
+    audit: AuditConfig
+
+
+def load_config() -> AppConfig:
+    return AppConfig(
+        sponge_mcp=SpongeMcpConfig(
+            base_url=os.getenv("SPONGE_MCP_BASE_URL", DEFAULT_SPONGE_MCP_BASE_URL).rstrip("/"),
+            access_token=os.getenv("SPONGE_MCP_ACCESS_TOKEN", DEFAULT_SPONGE_MCP_ACCESS_TOKEN),
+            timeout_ms=30000,
+        ),
+        schema=SchemaConfig(
+            cache_path=PACKAGE_ROOT / "data/sponge-schema-cache.json",
+            refresh_on_start=True,
+            refresh_interval_minutes=1440,
+        ),
+        sql=SqlConfig(
+            max_repair_attempts=2,
+            default_limit=100,
+            max_limit=500,
+        ),
+        audit=AuditConfig(
+            enabled=True,
+            log_path=PACKAGE_ROOT / "logs/octopus-sponge-audit.log",
+        ),
+    )
+
+
+def load_config_from_env() -> AppConfig:
+    return load_config()

package/src/octopus_sponge_skill/context.py

@@ -0,0 +1,20 @@
+from __future__ import annotations
+
+from dataclasses import dataclass
+from uuid import uuid4
+
+
+@dataclass(frozen=True)
+class UserContext:
+    user_id: int
+    session_id: str
+    device_id: str | None = None
+
+
+def new_trace_id(prefix: str = "trace") -> str:
+    return f"{prefix}_{uuid4().hex}"
+
+
+def new_session_id() -> str:
+    return f"session_{uuid4().hex}"
+