mcp-query-table 0.2.6__tar.gz → 0.3.2__tar.gz

{mcp_query_table-0.2.6 → mcp_query_table-0.3.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mcp_query_table
-Version: 0.2.6
+Version: 0.3.2
 Summary: query table from website, support MCP
 Author-email: wukan <wu-kan@163.com>
 License: MIT License
@@ -39,13 +39,20 @@ Dynamic: license-file
 
 # mcp_query_table
 
-基于`playwright`实现的财经网页表格爬虫，支持`Model Context Protocol (MCP) `。目前可查询来源为
+1. 基于`playwright`实现的财经网页表格爬虫，支持`Model Context Protocol (MCP) `。目前可查询来源为
 
-- [同花顺i问财](http://iwencai.com/)
-- [通达信问小达](https://wenda.tdx.com.cn/)
-- [东方财富条件选股](https://xuangu.eastmoney.com/)
+    - [同花顺i问财](http://iwencai.com/)
+    - [通达信问小达](https://wenda.tdx.com.cn/)
+    - [东方财富条件选股](https://xuangu.eastmoney.com/)
 
-实盘时，如果某网站宕机或改版，可以立即切换到其他网站。(注意：不同网站的表格结构不同，需要提前做适配)
+   实盘时，如果某网站宕机或改版，可以立即切换到其他网站。(注意：不同网站的表格结构不同，需要提前做适配)
+
+2. 基于`playwright`实现的大语言模型调用爬虫。目前可用来源为
+    - [纳米搜索](https://www.n.cn/)
+    - [腾讯元宝](https://yuanbao.tencent.com/)
+    - [百度AI搜索](https://chat.baidu.com/)
+
+   `RooCode`提供了`Human Reply`功能。但发现`纳米搜索`网页版复制时格式破坏，所以研发了此功能
 
 ## 安装
 
@@ -63,7 +70,7 @@ from mcp_query_table import *
 
 
 async def main() -> None:
-    async with BrowserManager(port=9222, browser_path=None, debug=True) as bm:
+    async with BrowserManager(cdp_endpoint="http://127.0.0.1:9222", executable_path=None, debug=True) as bm:
         # 问财需要保证浏览器宽度>768，防止界面变成适应手机
         page = await bm.get_page()
         df = await query(page, '收益最好的200只ETF', query_type=QueryType.ETF, max_page=1, site=Site.THS)
@@ -75,8 +82,14 @@ async def main() -> None:
         # TODO 东财翻页要提前登录
         df = await query(page, '今日涨幅前5的概念板块;', query_type=QueryType.Board, max_page=3, site=Site.EastMoney)
         print(df)
-        bm.release_page(page)
+
+        output = await chat(page, "1+2等于多少？", provider=Provider.YuanBao)
+        print(output)
+        output = await chat(page, "3+4等于多少？", provider=Provider.YuanBao, create=True)
+        print(output)
+
         print('done')
+        bm.release_page(page)
         await page.wait_for_timeout(2000)
 
 
@@ -119,7 +132,7 @@ if __name__ == '__main__':
 
 确保可以在控制台中执行`python -m mcp_query_table -h`。如果不能，可能要先`pip install mcp_query_table`
 
-在`Cline`中可以配置如下。其中`command`是`python`的绝对路径，`browser_path`是`Chrome`的绝对路径。
+在`Cline`中可以配置如下。其中`command`是`python`的绝对路径，`executable_path`是`Chrome`的绝对路径。
 
 ### STDIO方式
 
@@ -133,7 +146,9 @@ if __name__ == '__main__':
         "mcp_query_table",
         "--format",
         "markdown",
-        "--browser_path",
+        "--cdp_endpoint",
+        "http://127.0.0.1:9222",
+        "--executable_path",
         "C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe"
       ]
     }
@@ -146,11 +161,20 @@ if __name__ == '__main__':
 先在控制台中执行如下命令，启动`MCP`服务
 
 ```commandline
-python -m mcp_query_table --format markdown --browser_path "C:\Program Files\Google\Chrome\Application\chrome.exe" --transport sse --mcp_port 8000
+python -m mcp_query_table --format markdown --transport sse --port 8000
 ```
 
 然后就可以连接到`MCP`服务了
-http://localhost:8000/sse
+
+```json
+{
+  "mcpServers": {
+    "mcp_query_table": {
+      "url": "http://127.0.0.1:8000/sse"
+    }
+  }
+}
+```
 
 ## 使用`MCP Inspector`进行调试
 
@@ -158,7 +182,8 @@ http://localhost:8000/sse
 npx @modelcontextprotocol/inspector python -m mcp_query_table --format markdown
 ```
 
-打开浏览器并翻页是一个比较耗时的操作，会导致`MCP Inspector`页面超时，可以`http://localhost:5173/?timeout=60000` 表示超时时间为60秒
+打开浏览器并翻页是一个比较耗时的操作，会导致`MCP Inspector`页面超时，可以`http://localhost:5173/?timeout=600000`
+表示超时时间为600秒
 
 第一次尝试编写`MCP`项目，可能会有各种问题，欢迎大家交流。
 

{mcp_query_table-0.2.6 → mcp_query_table-0.3.2}/README.md

@@ -1,12 +1,19 @@
 # mcp_query_table
 
-基于`playwright`实现的财经网页表格爬虫，支持`Model Context Protocol (MCP) `。目前可查询来源为
+1. 基于`playwright`实现的财经网页表格爬虫，支持`Model Context Protocol (MCP) `。目前可查询来源为
 
-- [同花顺i问财](http://iwencai.com/)
-- [通达信问小达](https://wenda.tdx.com.cn/)
-- [东方财富条件选股](https://xuangu.eastmoney.com/)
+    - [同花顺i问财](http://iwencai.com/)
+    - [通达信问小达](https://wenda.tdx.com.cn/)
+    - [东方财富条件选股](https://xuangu.eastmoney.com/)
 
-实盘时，如果某网站宕机或改版，可以立即切换到其他网站。(注意：不同网站的表格结构不同，需要提前做适配)
+   实盘时，如果某网站宕机或改版，可以立即切换到其他网站。(注意：不同网站的表格结构不同，需要提前做适配)
+
+2. 基于`playwright`实现的大语言模型调用爬虫。目前可用来源为
+    - [纳米搜索](https://www.n.cn/)
+    - [腾讯元宝](https://yuanbao.tencent.com/)
+    - [百度AI搜索](https://chat.baidu.com/)
+
+   `RooCode`提供了`Human Reply`功能。但发现`纳米搜索`网页版复制时格式破坏，所以研发了此功能
 
 ## 安装
 
@@ -24,7 +31,7 @@ from mcp_query_table import *
 
 
 async def main() -> None:
-    async with BrowserManager(port=9222, browser_path=None, debug=True) as bm:
+    async with BrowserManager(cdp_endpoint="http://127.0.0.1:9222", executable_path=None, debug=True) as bm:
         # 问财需要保证浏览器宽度>768，防止界面变成适应手机
         page = await bm.get_page()
         df = await query(page, '收益最好的200只ETF', query_type=QueryType.ETF, max_page=1, site=Site.THS)
@@ -36,8 +43,14 @@ async def main() -> None:
         # TODO 东财翻页要提前登录
         df = await query(page, '今日涨幅前5的概念板块;', query_type=QueryType.Board, max_page=3, site=Site.EastMoney)
         print(df)
-        bm.release_page(page)
+
+        output = await chat(page, "1+2等于多少？", provider=Provider.YuanBao)
+        print(output)
+        output = await chat(page, "3+4等于多少？", provider=Provider.YuanBao, create=True)
+        print(output)
+
         print('done')
+        bm.release_page(page)
         await page.wait_for_timeout(2000)
 
 
@@ -80,7 +93,7 @@ if __name__ == '__main__':
 
 确保可以在控制台中执行`python -m mcp_query_table -h`。如果不能，可能要先`pip install mcp_query_table`
 
-在`Cline`中可以配置如下。其中`command`是`python`的绝对路径，`browser_path`是`Chrome`的绝对路径。
+在`Cline`中可以配置如下。其中`command`是`python`的绝对路径，`executable_path`是`Chrome`的绝对路径。
 
 ### STDIO方式
 
@@ -94,7 +107,9 @@ if __name__ == '__main__':
         "mcp_query_table",
         "--format",
         "markdown",
-        "--browser_path",
+        "--cdp_endpoint",
+        "http://127.0.0.1:9222",
+        "--executable_path",
         "C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe"
       ]
     }
@@ -107,11 +122,20 @@ if __name__ == '__main__':
 先在控制台中执行如下命令，启动`MCP`服务
 
 ```commandline
-python -m mcp_query_table --format markdown --browser_path "C:\Program Files\Google\Chrome\Application\chrome.exe" --transport sse --mcp_port 8000
+python -m mcp_query_table --format markdown --transport sse --port 8000
 ```
 
 然后就可以连接到`MCP`服务了
-http://localhost:8000/sse
+
+```json
+{
+  "mcpServers": {
+    "mcp_query_table": {
+      "url": "http://127.0.0.1:8000/sse"
+    }
+  }
+}
+```
 
 ## 使用`MCP Inspector`进行调试
 
@@ -119,7 +143,8 @@ http://localhost:8000/sse
 npx @modelcontextprotocol/inspector python -m mcp_query_table --format markdown
 ```
 
-打开浏览器并翻页是一个比较耗时的操作，会导致`MCP Inspector`页面超时，可以`http://localhost:5173/?timeout=60000` 表示超时时间为60秒
+打开浏览器并翻页是一个比较耗时的操作，会导致`MCP Inspector`页面超时，可以`http://localhost:5173/?timeout=600000`
+表示超时时间为600秒
 
 第一次尝试编写`MCP`项目，可能会有各种问题，欢迎大家交流。
 

mcp_query_table-0.3.2/mcp_query_table/__init__.py

@@ -0,0 +1,6 @@
+from ._version import __version__
+
+from .enums import QueryType, Site, Provider
+from .tool import BrowserManager, query, chat
+
+TIMEOUT = 1000 * 60 * 2  # 2分钟，在抓取EventStream数据时等待数据返回，防止外层30秒超时

{mcp_query_table-0.2.6 → mcp_query_table-0.3.2}/mcp_query_table/__main__.py

@@ -10,19 +10,20 @@ def main():
 
     parser.add_argument("--format", type=str, help="输出格式",
                         default='markdown', choices=['markdown', 'csv', 'json'])
-    parser.add_argument("--cdp_port", type=int, help="浏览器远程调试端口",
-                        default=9222)
-    parser.add_argument("--browser_path", type=str, help="浏览器类型",
+    parser.add_argument("--cdp_endpoint", type=str, help="浏览器CDP调试地址",
+                        default="http://127.0.0.1:9222")
+    parser.add_argument("--executable_path", type=str, help="浏览器类型",
                         default=r'C:\Program Files\Google\Chrome\Application\chrome.exe')
 
     parser.add_argument("--transport", type=str, help="传输类型",
                         default='stdio', choices=['stdio', 'sse'])
-    parser.add_argument("--mcp_host", type=str, help="MCP服务端地址",
+    parser.add_argument("--host", type=str, help="MCP服务端绑定地址",
                         default='0.0.0.0')
-    parser.add_argument("--mcp_port", type=int, help="MCP服务端端口",
+    parser.add_argument("--port", type=int, help="MCP服务端绑定端口",
                         default='8000')
     args = parser.parse_args()
-    serve(args.format, args.cdp_port, args.browser_path, args.transport, args.mcp_host, args.mcp_port)
+    serve(args.format, args.cdp_endpoint, args.executable_path,
+          args.transport, args.host, args.port)
 
 
 if __name__ == "__main__":

mcp_query_table-0.3.2/mcp_query_table/_version.py

@@ -0,0 +1 @@
+__version__ = "0.3.2"

{mcp_query_table-0.2.6 → mcp_query_table-0.3.2}/mcp_query_table/enums.py

@@ -19,3 +19,10 @@ class Site(Enum):
     EastMoney = '东方财富'  # 东方财富 条件选股
     TDX = '通达信'  # 通达信 问小达
     THS = '同花顺'  # 同花顺 i问财
+
+
+class Provider(Enum):
+    """提供商"""
+    Nami = '纳米搜索'  # 360 纳米搜索
+    YuanBao = '腾讯元宝'  # 腾讯元宝
+    BaiDu = '百度AI搜索'  # 百度AI搜索

mcp_query_table-0.3.2/mcp_query_table/providers/baidu.py

@@ -0,0 +1,82 @@
+"""
+百度AI搜索
+
+限制了输入长度为5000，很多时候会被截断，导致MCP无法正常工作
+"""
+import json
+
+from playwright.async_api import Page
+
+import mcp_query_table
+from mcp_query_table.tool import GlobalVars
+
+_PAGE0_ = "https://chat.baidu.com/search"
+_PAGE1_ = "https://chat.baidu.com/aichat/api/conversation"
+
+G = GlobalVars()
+
+
+def read_event_stream(text):
+    text1 = []
+    text2 = []
+    for event in text.split('\n\n'):
+        if '"component":"thinkingSteps"' in event:
+            if '"reasoningContent":' not in event:
+                continue
+            lines = event.split('\n')
+            for line in lines:
+                if line.startswith('data:'):
+                    t = line[5:]
+                    t = json.loads(t)['data']['message']['content']['generator']['data']['reasoningContent']
+                    text1.append(t)
+        if '"component":"markdown-yiyan"' in event:
+            lines = event.split('\n')
+            for line in lines:
+                if line.startswith('data:'):
+                    t = line[5:]
+                    t = json.loads(t)['data']['message']['content']['generator']['data']['value']
+                    text2.append(t)
+
+    text2 = ''.join(text2)
+    if len(text1) == 0:
+        return text2
+    else:
+        text1 = ''.join(text1)
+        return f"<thinking>{text1}</thinking>\n\n{text2}"
+
+
+async def on_response(response):
+    if response.url.startswith(_PAGE1_):
+        # print("on_response", response.url)
+        text = await response.text()
+        G.set_text(read_event_stream(text))
+
+
+async def on_route(route):
+    # 避免出现 Protocol error (Network.getResponseBody): No data found for resource with given identifier
+    # print("on_route", route.request.url)
+    if route.request.url == _PAGE1_:
+        # TODO 为何只要转发一下就没事了？
+        response = await route.fetch(timeout=mcp_query_table.TIMEOUT)
+        await route.fulfill(response=response)
+    else:
+        await route.continue_()
+
+
+async def chat(page: Page,
+               prompt: str,
+               create: bool,
+               ) -> str:
+    if not page.url.startswith(_PAGE0_):
+        create = True
+
+    if create:
+        await page.goto(_PAGE0_)
+
+    await page.route(_PAGE1_, on_route)
+    async with page.expect_response(_PAGE1_, timeout=mcp_query_table.TIMEOUT) as response_info:
+        await page.locator("#chat-input-box").fill(prompt)
+        await page.locator("#chat-input-box").press("Enter")
+    await on_response(await response_info.value)
+
+    return G.get_text()

mcp_query_table-0.3.2/mcp_query_table/providers/n.py

@@ -0,0 +1,77 @@
+"""
+360 纳米搜索
+"""
+import json
+
+from playwright.async_api import Page
+
+import mcp_query_table
+from mcp_query_table.tool import GlobalVars
+
+_PAGE0_ = "https://www.n.cn"
+_PAGE1_ = "https://www.n.cn/search"
+_PAGE2_ = "https://www.n.cn/api/common/chat/v2"
+
+G = GlobalVars()
+
+
+def read_event_stream(text):
+    text1 = []
+    text2 = []
+    for event in text.split('\n\n'):
+        if "event: 102" in event:
+            if 'data: {"type":"reasoning_text"' in event:
+                lines = event.split('\n')
+                for line in lines:
+                    if line.startswith('data: '):
+                        t = line[6:]
+                        t = json.loads(t)['message']
+                        text1.append(t)
+        if "event: 200" in event:
+            lines = event.split('\n')
+            for line in lines:
+                if line.startswith('data: '):
+                    t = line[6:]
+                    if t == '':
+                        text2.append('\n')
+                    elif t == ' ':
+                        text2.append('\n')
+                    else:
+                        text2.append(t)
+
+    text2 = ''.join(text2)
+    if len(text1) == 0:
+        return text2
+    else:
+        text1 = ''.join(text1)
+        return f"<thinking>{text1}</thinking>\n\n{text2}"
+
+
+async def on_response(response):
+    if response == _PAGE2_:
+        # print("on_response", response.url)
+        text = await response.text()
+        G.set_text(read_event_stream(text))
+
+
+async def chat(page: Page,
+               prompt: str,
+               create: bool,
+               ) -> str:
+    if not create:
+        if not page.url.startswith(_PAGE1_):
+            create = True
+
+    if create:
+        await page.goto(_PAGE0_)
+        name = "输入任何问题"
+    else:
+        name = "提出后续问题，Enter发送，Shift+Enter 换行"
+
+    async with page.expect_response(_PAGE2_, timeout=mcp_query_table.TIMEOUT) as response_info:
+        textbox = page.get_by_role("textbox", name=name)
+        await textbox.fill(prompt)
+        await textbox.press("Enter")
+    await on_response(await response_info.value)
+
+    return G.get_text()

mcp_query_table-0.3.2/mcp_query_table/providers/yuanbao.py

@@ -0,0 +1,82 @@
+"""
+腾讯元宝
+"""
+import json
+
+from playwright.async_api import Page
+
+import mcp_query_table
+from mcp_query_table.tool import GlobalVars
+
+_PAGE0_ = "https://yuanbao.tencent.com/"
+_PAGE1_ = "https://yuanbao.tencent.com/api/chat"
+
+G = GlobalVars()
+
+
+def read_event_stream(text):
+    text1 = []
+    text2 = []
+    for event in text.split('\n\n'):
+        if 'data: {"type":"think"' in event:
+            lines = event.split('\n')
+            for line in lines:
+                if line.startswith('data: '):
+                    t = line[6:]
+                    t = json.loads(t)['content']
+                    text1.append(t)
+        if 'data: {"type":"text"' in event:
+            lines = event.split('\n')
+            for line in lines:
+                if line.startswith('data: '):
+                    t = line[6:]
+                    t = json.loads(t).get('msg', "")
+                    text2.append(t)
+
+    text2 = ''.join(text2)
+    if len(text1) == 0:
+        return text2
+    else:
+        text1 = ''.join(text1)
+        return f"<thinking>{text1}</thinking>\n\n{text2}"
+
+
+async def on_response(response):
+    if response.url.startswith(_PAGE1_):
+        # print("on_response", response.url)
+        text = await response.text()
+        G.set_text(read_event_stream(text))
+
+
+async def on_route(route):
+    # print("on_route", route.request.url)
+    if route.request.url.startswith(_PAGE1_):
+        # TODO 这里会导致数据全部加载，逻辑变了，所以界面可能混乱
+        response = await route.fetch(timeout=mcp_query_table.TIMEOUT)
+        await route.fulfill(
+            # 强行加utf-8，否则编码搞不定
+            content_type="text/event-stream; charset=utf-8",
+            response=response,
+        )
+    else:
+        await route.continue_()
+
+
+async def chat(page: Page,
+               prompt: str,
+               create: bool,
+               ) -> str:
+    if not page.url.startswith(_PAGE0_):
+        create = True
+
+    if create:
+        await page.goto(_PAGE0_)
+
+    await page.route(f"{_PAGE1_}/*", on_route)
+    async with page.expect_response(f"{_PAGE1_}/*", timeout=mcp_query_table.TIMEOUT) as response_info:
+        textbox = page.locator(".ql-editor")
+        await textbox.fill(prompt)
+        await textbox.press("Enter")
+    await on_response(await response_info.value)
+
+    return G.get_text()

{mcp_query_table-0.2.6 → mcp_query_table-0.3.2}/mcp_query_table/server.py

@@ -4,18 +4,22 @@ from loguru import logger
 from mcp.server.fastmcp import FastMCP
 from pydantic import Field
 
-from mcp_query_table import QueryType, Site, query as query_table_query
+from mcp_query_table import QueryType, Site, query as qt_query, chat as qt_chat
+from mcp_query_table.enums import Provider
 from mcp_query_table.tool import BrowserManager
 
 
 class QueryServer:
-    def __init__(self, format: str = 'markdown', port: int = 9222, browser_path: Optional[str] = None) -> None:
+    def __init__(self,
+                 format: str = 'markdown',
+                 cdp_endpoint: Optional[str] = 'http://127.0.0.1:9222',
+                 executable_path: Optional[str] = None) -> None:
         self.format: str = format
-        self.browser = BrowserManager(port=port, browser_path=browser_path, debug=False)
+        self.browser = BrowserManager(cdp_endpoint=cdp_endpoint, executable_path=executable_path, debug=False)
 
     async def query(self, query_input: str, query_type: QueryType, max_page: int, site: Site):
         page = await self.browser.get_page()
-        df = await query_table_query(page, query_input, query_type, max_page, site)
+        df = await qt_query(page, query_input, query_type, max_page, site)
         self.browser.release_page(page)
 
         if self.format == 'csv':
@@ -25,6 +29,12 @@ class QueryServer:
         if self.format == 'json':
             return df.to_json(force_ascii=False, indent=2)
 
+    async def chat(self, prompt: str, create: bool, provider: Provider):
+        page = await self.browser.get_page()
+        txt = await qt_chat(page, prompt, create, provider)
+        self.browser.release_page(page)
+        return txt
+
 
 # !!!log_level这一句非常重要，否则Cline/MCP Server/Tools工作不正常
 mcp = FastMCP("query_table_mcp", log_level="ERROR")
@@ -43,14 +53,25 @@ async def query(
     return await qsv.query(query_input, query_type, max_page, site)
 
 
-def serve(format, cdp_port, browser_path, transport, mcp_host, mcp_port):
+@mcp.tool(description="大语言模型对话")
+async def chat(
+        prompt: Annotated[str, Field(description="提示词。如：`9.9大还是9.11大？`")],
+        create: Annotated[bool, Field(default=False, description="是否创建新对话")],
+        provider: Annotated[
+            Provider, Field(default=Provider.Nami, description="提供商。支持`纳米搜索`、`腾讯元宝`、`百度AI搜索`")]
+) -> str:
+    return await qsv.chat(prompt, create, provider)
+
+
+def serve(format, cdp_endpoint, executable_path, transport, host, port):
     qsv.format = format
-    qsv.port = cdp_port
-    qsv.browser_path = browser_path
-    logger.info("serve:{},{},{},{}", qsv.format, qsv.port, qsv.browser_path, transport)
+    qsv.cdp_endpoint = cdp_endpoint
+    qsv.executable_path = executable_path
+    logger.info(f"{format=},{transport=}")
+    logger.info(f"{cdp_endpoint=},{executable_path=}")
     if transport == 'sse':
-        logger.info("mcp:{},{}:{}", transport, mcp_host, mcp_port)
+        logger.info(f"{host=},{port=}", transport, host, port)
 
-    mcp.settings.host = mcp_host
-    mcp.settings.port = mcp_port
+    mcp.settings.host = host
+    mcp.settings.port = port
     mcp.run(transport=transport)

mcp_query_table-0.3.2/mcp_query_table/tool.py

@@ -0,0 +1,263 @@
+import subprocess
+import sys
+import time
+from pathlib import Path
+from typing import Optional
+
+import pandas as pd
+from loguru import logger
+from playwright.async_api import async_playwright, Playwright, Page
+
+from mcp_query_table.enums import QueryType, Site, Provider
+
+
+def create_detached_process(command):
+    # 设置通用参数
+    kwargs = {}
+
+    if sys.platform == 'win32':
+        kwargs.update({
+            # 在PyCharm中运行还是会出现新建进程被关闭
+            'creationflags': subprocess.DETACHED_PROCESS | subprocess.CREATE_NEW_PROCESS_GROUP
+        })
+    else:
+        # Unix-like 系统（Linux, macOS）特定设置
+        kwargs.update({
+            'start_new_session': True  # 创建新的会话
+        })
+    return subprocess.Popen(command, **kwargs)
+
+
+def is_local_url(url: str) -> bool:
+    """判断url是否是本地地址"""
+    for local in ('localhost', '127.0.0.1'):
+        if local in url.lower():
+            return True
+    return False
+
+
+def get_executable_path(executable_path) -> Optional[str]:
+    """获取浏览器可执行文件路径"""
+    browsers = {
+        "default": executable_path,
+        "chrome.exe": r"C:\Program Files\Google\Chrome\Application\chrome.exe",
+        "msedge.exe": r"C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe",
+    }
+    for k, v in browsers.items():
+        if v is None:
+            continue
+        if Path(v).exists():
+            return v
+    return None
+
+
+class BrowserManager:
+    async def __aenter__(self):
+        return self
+
+    async def __aexit__(self, exc_type, exc_val, exc_tb):
+        await self.cleanup()
+
+    def __init__(self,
+                 cdp_endpoint: Optional[str] = None,
+                 executable_path: Optional[str] = None,
+                 debug: bool = False):
+        """
+
+        Parameters
+        ----------
+        cdp_endpoint:str
+            浏览器CDP地址
+        executable_path:str
+            浏览器可执行文件路径。推荐使用chrome，因为Microsoft Edge必须在任务管理器中完全退出才能启动调试端口
+        debug:bool
+            是否显示开发者工具
+
+        """
+        self.cdp_endpoint = cdp_endpoint or 'http://127.0.0.1:9222'
+        self.executable_path = executable_path
+        self.debug = debug
+
+        self.playwright: Optional[Playwright] = None
+        self.browser = None
+        self.context = None
+        # 空闲page池
+        self.pages = []
+
+    async def cleanup(self):
+        if self.browser:
+            await self.browser.close()
+        if self.playwright:
+            await self.playwright.stop()
+
+    async def _connect_to_local(self) -> None:
+        """连接本地浏览器"""
+        port = self.cdp_endpoint.split(':')[-1]
+        executable_path = get_executable_path(self.executable_path)
+        command = [executable_path, f'--remote-debugging-port={port}', '--start-maximized']
+        if self.debug:
+            command.append('--auto-open-devtools-for-tabs')
+
+        for i in range(2):
+            try:
+                self.browser = await self.playwright.chromium.connect_over_cdp(self.cdp_endpoint,
+                                                                               timeout=10000, slow_mo=1000)
+                break
+            except:
+                if i == 0:
+                    logger.info(f"start browser:{command}")
+                    create_detached_process(command)
+                    time.sleep(3)
+                    continue
+                if i == 1:
+                    raise ConnectionError(
+                        f"已提前打开了浏览器，但未开启远程调试端口？请关闭浏览器全部进程后重试 `taskkill /f /im {Path(executable_path).name}`")
+
+    async def _connect_to_remote(self) -> None:
+        """连接远程浏览器"""
+        try:
+            self.browser = await self.playwright.chromium.connect_over_cdp(self.cdp_endpoint,
+                                                                           timeout=10000, slow_mo=1000)
+        except:
+            raise ConnectionError(f"连接远程浏览器失败，请检查CDP地址和端口是否正确。{self.cdp_endpoint}")
+
+    async def _launch(self) -> None:
+        """启动浏览器，并连接CDP协议
+
+        References
+        ----------
+        https://blog.csdn.net/qq_30576521/article/details/142370538
+
+        """
+        self.playwright = await async_playwright().start()
+
+        if is_local_url(self.cdp_endpoint):
+            await self._connect_to_local()
+        else:
+            await self._connect_to_remote()
+
+        self.context = self.browser.contexts[0]
+        # 复用打开的page
+        for page in self.context.pages:
+            # 防止开发者工具被使用
+            if page.url.startswith("devtools://"):
+                continue
+            self.pages.append(page)
+
+    async def _try_launch(self) -> None:
+        if self.browser is None:
+            await self._launch()
+        if not self.browser.is_connected():
+            await self._launch()
+
+    async def get_page(self) -> Page:
+        """获取可用Page。无空闲标签时会打开新标签"""
+        await self._try_launch()
+
+        # 反复取第一个tab
+        while len(self.pages) > 0:
+            page = self.pages.pop()
+            if page.is_closed():
+                continue
+            return page
+
+        # 不够，新建一个
+        return await self.context.new_page()
+
+    def release_page(self, page) -> None:
+        """用完的Page释放到池中。如果用完不放回，get_page会一直打开新标签"""
+        if page.is_closed():
+            return
+        # 放回
+        self.pages.append(page)
+
+
+class GlobalVars:
+    """全局变量"""
+
+    def __init__(self):
+        self.text = ""
+
+    def set_text(self, text):
+        self.text = text
+
+    def get_text(self):
+        return self.text
+
+
+async def query(
+        page: Page,
+        query_input: str = "收盘价>100元",
+        query_type: QueryType = QueryType.CNStock,
+        max_page: int = 5,
+        site: Site = Site.THS) -> pd.DataFrame:
+    """查询表格
+
+    Parameters
+    ----------
+    page : playwright.sync_api.Page
+        页面
+    query_input : str, optional
+        查询条件, by default "收盘价>100元"
+    query_type : QueryType, optional
+        查询类型, by default QueryType.astock
+    max_page : int, optional
+        最大页数, by default 5
+    site : Site, optional
+        站点, by default Site.iwencai
+
+    Returns
+    -------
+    pd.DataFrame
+        查询结果
+
+    """
+
+    if site == Site.EastMoney:
+        from mcp_query_table.sites.eastmoney import query
+        return await query(page, query_input, query_type, max_page)
+    if site == Site.THS:
+        from mcp_query_table.sites.iwencai import query
+        return await query(page, query_input, query_type, max_page)
+    if site == Site.TDX:
+        from mcp_query_table.sites.tdx import query
+        return await query(page, query_input, query_type, max_page)
+
+    raise ValueError(f"未支持的站点:{site}")
+
+
+async def chat(
+        page: Page,
+        prompt: str = "9.9大还是9.11大？",
+        create: bool = False,
+        provider: Provider = Provider.Nami) -> str:
+    """大语言对话
+
+    Parameters
+    ----------
+    page : playwright.sync_api.Page
+        页面
+    prompt : str, optional
+        对话内容, by default "9.9大还是9.11大？"
+    create : bool, optional
+        是否创建新对话, by default False
+    provider : Provider, optional
+        提供商, by default Provider.N
+
+    Returns
+    -------
+    str
+        对话结果
+
+    """
+    if provider == Provider.Nami:
+        from mcp_query_table.providers.n import chat
+        return await chat(page, prompt, create)
+    if provider == Provider.YuanBao:
+        from mcp_query_table.providers.yuanbao import chat
+        return await chat(page, prompt, create)
+    if provider == Provider.BaiDu:
+        from mcp_query_table.providers.baidu import chat
+        return await chat(page, prompt, create)
+
+    raise ValueError(f"未支持的提供商:{provider}")

{mcp_query_table-0.2.6 → mcp_query_table-0.3.2}/mcp_query_table.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mcp_query_table
-Version: 0.2.6
+Version: 0.3.2
 Summary: query table from website, support MCP
 Author-email: wukan <wu-kan@163.com>
 License: MIT License
@@ -39,13 +39,20 @@ Dynamic: license-file
 
 # mcp_query_table
 
-基于`playwright`实现的财经网页表格爬虫，支持`Model Context Protocol (MCP) `。目前可查询来源为
+1. 基于`playwright`实现的财经网页表格爬虫，支持`Model Context Protocol (MCP) `。目前可查询来源为
 
-- [同花顺i问财](http://iwencai.com/)
-- [通达信问小达](https://wenda.tdx.com.cn/)
-- [东方财富条件选股](https://xuangu.eastmoney.com/)
+    - [同花顺i问财](http://iwencai.com/)
+    - [通达信问小达](https://wenda.tdx.com.cn/)
+    - [东方财富条件选股](https://xuangu.eastmoney.com/)
 
-实盘时，如果某网站宕机或改版，可以立即切换到其他网站。(注意：不同网站的表格结构不同，需要提前做适配)
+   实盘时，如果某网站宕机或改版，可以立即切换到其他网站。(注意：不同网站的表格结构不同，需要提前做适配)
+
+2. 基于`playwright`实现的大语言模型调用爬虫。目前可用来源为
+    - [纳米搜索](https://www.n.cn/)
+    - [腾讯元宝](https://yuanbao.tencent.com/)
+    - [百度AI搜索](https://chat.baidu.com/)
+
+   `RooCode`提供了`Human Reply`功能。但发现`纳米搜索`网页版复制时格式破坏，所以研发了此功能
 
 ## 安装
 
@@ -63,7 +70,7 @@ from mcp_query_table import *
 
 
 async def main() -> None:
-    async with BrowserManager(port=9222, browser_path=None, debug=True) as bm:
+    async with BrowserManager(cdp_endpoint="http://127.0.0.1:9222", executable_path=None, debug=True) as bm:
         # 问财需要保证浏览器宽度>768，防止界面变成适应手机
         page = await bm.get_page()
         df = await query(page, '收益最好的200只ETF', query_type=QueryType.ETF, max_page=1, site=Site.THS)
@@ -75,8 +82,14 @@ async def main() -> None:
         # TODO 东财翻页要提前登录
         df = await query(page, '今日涨幅前5的概念板块;', query_type=QueryType.Board, max_page=3, site=Site.EastMoney)
         print(df)
-        bm.release_page(page)
+
+        output = await chat(page, "1+2等于多少？", provider=Provider.YuanBao)
+        print(output)
+        output = await chat(page, "3+4等于多少？", provider=Provider.YuanBao, create=True)
+        print(output)
+
         print('done')
+        bm.release_page(page)
         await page.wait_for_timeout(2000)
 
 
@@ -119,7 +132,7 @@ if __name__ == '__main__':
 
 确保可以在控制台中执行`python -m mcp_query_table -h`。如果不能，可能要先`pip install mcp_query_table`
 
-在`Cline`中可以配置如下。其中`command`是`python`的绝对路径，`browser_path`是`Chrome`的绝对路径。
+在`Cline`中可以配置如下。其中`command`是`python`的绝对路径，`executable_path`是`Chrome`的绝对路径。
 
 ### STDIO方式
 
@@ -133,7 +146,9 @@ if __name__ == '__main__':
         "mcp_query_table",
         "--format",
         "markdown",
-        "--browser_path",
+        "--cdp_endpoint",
+        "http://127.0.0.1:9222",
+        "--executable_path",
         "C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe"
       ]
     }
@@ -146,11 +161,20 @@ if __name__ == '__main__':
 先在控制台中执行如下命令，启动`MCP`服务
 
 ```commandline
-python -m mcp_query_table --format markdown --browser_path "C:\Program Files\Google\Chrome\Application\chrome.exe" --transport sse --mcp_port 8000
+python -m mcp_query_table --format markdown --transport sse --port 8000
 ```
 
 然后就可以连接到`MCP`服务了
-http://localhost:8000/sse
+
+```json
+{
+  "mcpServers": {
+    "mcp_query_table": {
+      "url": "http://127.0.0.1:8000/sse"
+    }
+  }
+}
+```
 
 ## 使用`MCP Inspector`进行调试
 
@@ -158,7 +182,8 @@ http://localhost:8000/sse
 npx @modelcontextprotocol/inspector python -m mcp_query_table --format markdown
 ```
 
-打开浏览器并翻页是一个比较耗时的操作，会导致`MCP Inspector`页面超时，可以`http://localhost:5173/?timeout=60000` 表示超时时间为60秒
+打开浏览器并翻页是一个比较耗时的操作，会导致`MCP Inspector`页面超时，可以`http://localhost:5173/?timeout=600000`
+表示超时时间为600秒
 
 第一次尝试编写`MCP`项目，可能会有各种问题，欢迎大家交流。
 

{mcp_query_table-0.2.6 → mcp_query_table-0.3.2}/mcp_query_table.egg-info/SOURCES.txt

@@ -12,6 +12,10 @@ mcp_query_table.egg-info/SOURCES.txt
 mcp_query_table.egg-info/dependency_links.txt
 mcp_query_table.egg-info/requires.txt
 mcp_query_table.egg-info/top_level.txt
+mcp_query_table/providers/__init__.py
+mcp_query_table/providers/baidu.py
+mcp_query_table/providers/n.py
+mcp_query_table/providers/yuanbao.py
 mcp_query_table/sites/__init__.py
 mcp_query_table/sites/eastmoney.py
 mcp_query_table/sites/iwencai.py

{mcp_query_table-0.2.6 → mcp_query_table-0.3.2}/pyproject.toml

@@ -29,6 +29,7 @@ dynamic = ["version"]
 packages = [
     "mcp_query_table",
     "mcp_query_table.sites",
+    "mcp_query_table.providers",
 ]
 
 [tool.setuptools.dynamic]

mcp_query_table-0.2.6/mcp_query_table/__init__.py

@@ -1,4 +0,0 @@
-from ._version import __version__
-
-from .enums import QueryType, Site
-from .tool import BrowserManager, query

mcp_query_table-0.2.6/mcp_query_table/_version.py

@@ -1 +0,0 @@
-__version__ = "0.2.6"

mcp_query_table-0.2.6/mcp_query_table/tool.py

@@ -1,165 +0,0 @@
-import subprocess
-import time
-from pathlib import Path
-from typing import Optional
-
-import pandas as pd
-from loguru import logger
-from playwright.async_api import async_playwright, Playwright, Page
-
-from mcp_query_table.enums import QueryType, Site
-
-
-class BrowserManager:
-    async def __aenter__(self):
-        return self
-
-    async def __aexit__(self, exc_type, exc_val, exc_tb):
-        await self.cleanup()
-
-    def __init__(self, port: int = 9222, browser_path: Optional[str] = None, debug: bool = False):
-        """
-
-        Parameters
-        ----------
-        port:int
-            浏览器调试端口
-        browser_path
-            浏览器可执行路径。推荐使用chrome，因为Microsoft Edge必须在任务管理器中完全退出才能启动调试端口
-        debug:bool
-            是否显示开发者工具
-
-        """
-        if browser_path is None:
-            browser_path = r'C:\Program Files\Google\Chrome\Application\chrome.exe'
-            if not Path(browser_path).exists():
-                # Microsoft Edge必须在任务管理器中完全退出才能启动调试端口
-                browser_path = r'C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe'
-                if not Path(browser_path).exists():
-                    raise ValueError("未找到浏览器可执行文件")
-
-        self.port = port
-        self.browser_path = browser_path
-        self.debug = debug
-
-        self.playwright: Optional[Playwright] = None
-        self.browser = None
-        self.context = None
-        # 空闲page池
-        self.pages = []
-
-    async def cleanup(self):
-        if self.browser:
-            await self.browser.close()
-        if self.playwright:
-            await self.playwright.stop()
-
-    async def _launch(self) -> None:
-        """启动浏览器，并连接CDP协议
-
-        References
-        ----------
-        https://blog.csdn.net/qq_30576521/article/details/142370538
-
-        """
-        self.playwright = await async_playwright().start()
-
-        try:
-            # 尝试连接已打开的浏览器
-            self.browser = await self.playwright.chromium.connect_over_cdp(f"http://127.0.0.1:{self.port}",
-                                                                           slow_mo=1000,
-                                                                           timeout=5000)
-        except:
-
-            # 执行完成后不会关闭浏览器
-            if self.debug:
-                command = f'"{self.browser_path}" --remote-debugging-port={self.port} --start-maximized --auto-open-devtools-for-tabs'
-            else:
-                command = f'"{self.browser_path}" --remote-debugging-port={self.port} --start-maximized'
-            logger.info(f"start browser:{command}")
-            subprocess.Popen(command, shell=True)
-            time.sleep(3)
-
-            try:
-                self.browser = await self.playwright.chromium.connect_over_cdp(f"http://127.0.0.1:{self.port}",
-                                                                               slow_mo=1000,
-                                                                               timeout=5000)
-            except:
-                logger.warning("是否提前打开了浏览器，但未开启远程调试端口？请关闭浏览器全部进程后重试")
-                raise
-
-        self.context = self.browser.contexts[0]
-        # 复用打开的page
-        for page in self.context.pages:
-            # 防止开发者工具被使用
-            if page.url.startswith("devtools://"):
-                continue
-            self.pages.append(page)
-
-    async def _try_launch(self) -> None:
-        if self.browser is None:
-            await self._launch()
-        if not self.browser.is_connected():
-            await self._launch()
-
-    async def get_page(self) -> Page:
-        """获取可用Page。无空闲标签时会打开新标签"""
-        await self._try_launch()
-
-        # 反复取第一个tab
-        while len(self.pages) > 0:
-            page = self.pages.pop()
-            if page.is_closed():
-                continue
-            return page
-
-        # 不够，新建一个
-        return await self.context.new_page()
-
-    def release_page(self, page) -> None:
-        """用完的Page释放到池中。如果用完不放回，get_page会一直打开新标签"""
-        if page.is_closed():
-            return
-        # 放回
-        self.pages.append(page)
-
-
-async def query(
-        page: Page,
-        query_input: str = "收盘价>100元",
-        query_type: QueryType = QueryType.CNStock,
-        max_page: int = 5,
-        site: Site = Site.THS) -> pd.DataFrame:
-    """查询表格
-
-    Parameters
-    ----------
-    page : playwright.sync_api.Page
-        页面
-    query_input : str, optional
-        查询条件, by default "收盘价>100元"
-    query_type : QueryType, optional
-        查询类型, by default QueryType.astock
-    max_page : int, optional
-        最大页数, by default 5
-    site : Site, optional
-        站点, by default Site.iwencai
-
-    Returns
-    -------
-    pd.DataFrame
-        查询结果
-
-    """
-
-    if site == Site.EastMoney:
-        from mcp_query_table.sites.eastmoney import query
-        return await query(page, query_input, query_type, max_page)
-    if site == Site.THS:
-        from mcp_query_table.sites.iwencai import query
-        return await query(page, query_input, query_type, max_page)
-    if site == Site.TDX:
-        from mcp_query_table.sites.tdx import query
-        return await query(page, query_input, query_type, max_page)
-
-    raise ValueError(f"未支持的站点:{site}")