codebee 0.1.13 → 0.1.15
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +15 -0
- package/README.md +42 -8
- package/app/core/automation.py +7 -0
- package/app/core/catalog.py +9 -1
- package/app/core/env_scrub.py +5 -3
- package/app/core/flows.py +3 -0
- package/app/core/goal_service.py +11 -2
- package/app/core/jobs.py +38 -11
- package/app/core/market_remote.py +6 -3
- package/app/core/modelhub.py +45 -18
- package/app/core/notify.py +96 -0
- package/app/core/paihang.py +106 -0
- package/app/core/pipeline.py +26 -15
- package/app/core/publish/browser.py +63 -3
- package/app/core/publish/fanqie.py +16 -0
- package/app/core/publish/flow.py +25 -1
- package/app/core/publish/flows-fanqie-calibrated.json +37 -0
- package/app/core/publish/flows-qimao-calibrated.json +233 -0
- package/app/core/publish/manager.py +73 -13
- package/app/core/publish/qimao.py +13 -0
- package/app/core/settings.py +3 -1
- package/app/core/settings_schema.py +15 -2
- package/app/core/share_page.py +111 -0
- package/app/core/step_runner.py +23 -1
- package/app/core/telemetry.py +3 -2
- package/app/core/tlsctx.py +64 -0
- package/app/core/zentao.py +1302 -0
- package/app/main.py +89 -4
- package/app/ui/app.js +212 -1
- package/app/ui/i18n.js +54 -0
- package/app/ui/index.html +50 -0
- package/app/ui/style.css +27 -0
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,21 @@ CodeBee 的用户可感知变更记录。发布新版时:最上面加一节,
|
|
|
4
4
|
`<!-- relnotes:start -->…<!-- relnotes:end -->` 段(那段会被 `npm view` 的
|
|
5
5
|
README 元数据带回,供老版本在「发现新版本」时展示新版更新内容)。
|
|
6
6
|
|
|
7
|
+
## v0.1.15(2026-09-19)
|
|
8
|
+
|
|
9
|
+
- 禅道集成:定时扫描激活 Bug → 自动建修复任务 → 跑完自动合并 + resolve(fixed) + 报告回写 + 群通知(REST v1,SSRF 守卫;设置页「禅道」子页)
|
|
10
|
+
- 扫榜双源聚合:七猫+番茄 Web 版排行榜合并分析,跨平台重合题材单独标记(信号更强)
|
|
11
|
+
- 可分享报告页:运行报告导出为自包含单文件 HTML,发给任何人浏览器直接打开
|
|
12
|
+
- 出站 HTTPS 加固:统一证书校验上下文,修复 macOS python.org 发行版缺 CA 导致的全部外联失败(校验只增不减)
|
|
13
|
+
- 引擎调参卡:设置页可直接调压缩阈值/尾部保留/goal 轮数等引擎参数(schema 驱动,revision CAS 防并发覆盖)
|
|
14
|
+
- 发布浏览器 profile 迁至 ~/.codebee(登录态自动迁移),不再占用仓库目录与拖慢扫描
|
|
15
|
+
- 连载发布真机校准续:番茄编辑器结构校准、发章导航超时放宽、真实鼠标事件提交(合成点击被平台检测)
|
|
16
|
+
|
|
17
|
+
## v0.1.14(2026-09-19)
|
|
18
|
+
|
|
19
|
+
- 新任务类型「扫榜选材」:抓取七猫排行榜公开数据,AI 提炼热门题材 Top3/高频人设套路/差异化切入建议,直出选题洞察报告
|
|
20
|
+
- 运行结果群推送:设置群机器人 webhook(钉钉/飞书/企微自动适配),任务跑完自动汇报状态与评分
|
|
21
|
+
|
|
7
22
|
## v0.1.13(2026-09-19)
|
|
8
23
|
|
|
9
24
|
- 封面图生成:连载任务建书面板新增「生成封面」——调编排者供应商图像 API(cogview 候选,竖版优先)产出 cover.png 到运行目录,SSRF 边界校验 + curl 子进程落盘(Python 不经手图像字节)
|
package/README.md
CHANGED
|
@@ -21,14 +21,14 @@ Kimi Code、MiMo Code、Grok Build、Pi、DeepSeek Harness……),提供
|
|
|
21
21
|
不会把你的任务内容交给任何第三方。
|
|
22
22
|
|
|
23
23
|
<!-- relnotes:start -->
|
|
24
|
-
### 最新版更新内容(v0.1.
|
|
25
|
-
|
|
26
|
-
-
|
|
27
|
-
-
|
|
28
|
-
-
|
|
29
|
-
-
|
|
30
|
-
-
|
|
31
|
-
-
|
|
24
|
+
### 最新版更新内容(v0.1.15)
|
|
25
|
+
|
|
26
|
+
- **禅道集成**:定时扫描激活 Bug → 自动建修复任务 → 跑完自动合并 + resolve + 报告回写 + 群通知
|
|
27
|
+
- **扫榜选材(双源)**:七猫+番茄排行榜抓取聚合,AI 提炼热门题材/人设套路/差异化切入,跨平台重合题材单独标记
|
|
28
|
+
- **可分享报告页**:运行报告一键导出自包含单文件 HTML
|
|
29
|
+
- **引擎调参卡**:设置页直接调压缩阈值/goal 轮数等引擎参数(防并发覆盖)
|
|
30
|
+
- 出站 HTTPS 证书校验加固(修复部分 Mac 上的全外联失败);发布浏览器 profile 迁出仓库目录
|
|
31
|
+
- 群推送(钉钉/飞书/企微自动适配)与「扫榜选材」任务类型见 v0.1.14
|
|
32
32
|
<!-- relnotes:end -->
|
|
33
33
|
|
|
34
34
|
---
|
|
@@ -136,6 +136,40 @@ codebee
|
|
|
136
136
|
数据存放在用户目录(Windows `%APPDATA%\CodeBee`,macOS/Linux `~/.codebee`),
|
|
137
137
|
升级/重装不影响;老版本 Tutti 目录(`%APPDATA%\Tutti`)会被自动沿用,无需迁移。
|
|
138
138
|
|
|
139
|
+
### macOS:npm 全局安装报 EACCES(permission denied)
|
|
140
|
+
|
|
141
|
+
官方 pkg 安装的 Node,全局目录 `/usr/local/lib/node_modules` 归 root,直接
|
|
142
|
+
`npm install -g codebee` 会报
|
|
143
|
+
`EACCES: permission denied, mkdir '/usr/local/lib/node_modules/codebee'`。
|
|
144
|
+
**不要用 sudo 装**:装完目录归 root,应用内「一键升级」(以普通用户跑
|
|
145
|
+
`npm install -g codebee@latest`)之后每次都会撞同样的权限错误。
|
|
146
|
+
正确做法是把 npm 全局目录改到用户目录下,一次配好、安装与自动升级都畅通:
|
|
147
|
+
|
|
148
|
+
```bash
|
|
149
|
+
mkdir -p ~/.npm-global
|
|
150
|
+
npm config set prefix "~/.npm-global"
|
|
151
|
+
echo 'export PATH="$HOME/.npm-global/bin:$PATH"' >> ~/.zshrc
|
|
152
|
+
source ~/.zshrc
|
|
153
|
+
npm install -g codebee
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
默认 shell 是 bash 的话,把第 3 行的 `~/.zshrc` 换成 `~/.bash_profile`。
|
|
157
|
+
用 Homebrew 装的 Node(prefix 在 `/opt/homebrew`)没有这个问题,可直接安装。
|
|
158
|
+
验证 `which codebee` 指向 `~/.npm-global/bin/codebee` 即配置成功。
|
|
159
|
+
|
|
160
|
+
### macOS:HTTPS 请求报 SSL 证书错误(CERTIFICATE_VERIFY_FAILED)
|
|
161
|
+
|
|
162
|
+
python.org 安装的 Python 不读 macOS 系统钥匙串,本机又没有可用的 CA 束时,
|
|
163
|
+
「获取模型列表 / 适配测试 / 插件市场」等所有 HTTPS 检测会报
|
|
164
|
+
`certificate verify failed: unable to get local issuer certificate`。
|
|
165
|
+
CodeBee 已内置兜底:出站请求自动加载 certifi 与 macOS 系统证书束
|
|
166
|
+
(`/etc/ssl/cert.pem`),多数机器无需任何操作。若仍报错,补齐证书后重试:
|
|
167
|
+
|
|
168
|
+
```bash
|
|
169
|
+
python3 -m pip install --upgrade certifi # CodeBee 自动采用,装完即生效
|
|
170
|
+
# 或:Finder → 应用程序 → Python 3.x → 双击 Install Certificates.command
|
|
171
|
+
```
|
|
172
|
+
|
|
139
173
|
**方式二:源码运行(开发者)**
|
|
140
174
|
|
|
141
175
|
```bat
|
package/app/core/automation.py
CHANGED
|
@@ -387,6 +387,13 @@ def _tick():
|
|
|
387
387
|
log.info("automation: 定时发布触发 %d 个任务", n)
|
|
388
388
|
except Exception:
|
|
389
389
|
log.debug("automation: 定时发布检查跳过", exc_info=True)
|
|
390
|
+
# 禅道 Bug 扫描联动:fire_due 内部自节流(未启用/没到点零开销),
|
|
391
|
+
# 对账回写也在同一入口,这里只当调度宿主(同定时发布模式)。
|
|
392
|
+
try:
|
|
393
|
+
from . import zentao as _zentao
|
|
394
|
+
_zentao.fire_due()
|
|
395
|
+
except Exception:
|
|
396
|
+
log.debug("automation: 禅道扫描跳过", exc_info=True)
|
|
390
397
|
|
|
391
398
|
|
|
392
399
|
def _loop():
|
package/app/core/catalog.py
CHANGED
|
@@ -248,7 +248,7 @@ def npm_pkg_name(cmd):
|
|
|
248
248
|
|
|
249
249
|
|
|
250
250
|
def derive_uninstall(cmd):
|
|
251
|
-
"""从安装命令推导卸载命令(npm / winget / pip
|
|
251
|
+
"""从安装命令推导卸载命令(npm / winget / pip / uv / brew 本机渠道)。
|
|
252
252
|
|
|
253
253
|
卸载命令不单独维护一份,避免与安装命令不同步;认不出渠道返回 None,
|
|
254
254
|
此时条目可显式配置 uninstall 字段覆盖。
|
|
@@ -275,6 +275,14 @@ def derive_uninstall(cmd):
|
|
|
275
275
|
pkgs = [t for t in m.group(1).split() if not t.startswith("-")]
|
|
276
276
|
if pkgs:
|
|
277
277
|
return "uv tool uninstall %s" % pkgs[-1]
|
|
278
|
+
m = re.search(r"brew\s+install\b(.*)$", c)
|
|
279
|
+
if m:
|
|
280
|
+
# brew uninstall 按名字自动识别 formula/cask,--cask 这类 flag 丢了也
|
|
281
|
+
# 能卸对(Mac 渠道,AI 修复白名单已放行 brew install,编辑安装命令
|
|
282
|
+
# 也会填这种形态)
|
|
283
|
+
pkgs = [t for t in m.group(1).split() if not t.startswith("-")]
|
|
284
|
+
if pkgs:
|
|
285
|
+
return "brew uninstall %s" % pkgs[0]
|
|
278
286
|
return None
|
|
279
287
|
|
|
280
288
|
|
package/app/core/env_scrub.py
CHANGED
|
@@ -16,9 +16,10 @@ _DENY_RE = re.compile(r"(KEY|SECRET|TOKEN|PASSWORD|PASSWD|CREDENTIAL)", re.IGNOR
|
|
|
16
16
|
|
|
17
17
|
# 白名单:精确匹配(不分大小写),用于覆盖黑名单。
|
|
18
18
|
_ALLOW_EXACT = frozenset({
|
|
19
|
-
#
|
|
20
|
-
|
|
21
|
-
"
|
|
19
|
+
# 系统/环境(TEMP/TMP/PATHEXT 等是 Windows 名,TMPDIR 是 macOS 每用户
|
|
20
|
+
# 临时目录——三边各留各的,多出的名字在别的平台上不存在、无副作用)
|
|
21
|
+
"PATH", "PATHEXT", "SYSTEMROOT", "TEMP", "TMP", "TMPDIR",
|
|
22
|
+
"USERPROFILE", "USERDNSDOMAIN", "HOMEPATH", "HOMEDRIVE",
|
|
22
23
|
"LANG", "LC_ALL", "LC_CTYPE",
|
|
23
24
|
"HOME", "SHELL", "TERM", "PYTHONIOENCODING",
|
|
24
25
|
"PYTHONPATH", "PYTHONUNBUFFERED",
|
|
@@ -38,6 +39,7 @@ _ALLOW_PREFIX = (
|
|
|
38
39
|
"TUTTI_", "ALLINONE_", "DSH_",
|
|
39
40
|
"LANG", "LC_", # locale 系列
|
|
40
41
|
"PYTHON", # PYTHONPATH / PYTHONIOENCODING 等
|
|
42
|
+
"XDG_", # XDG_CONFIG_HOME 等(macOS/Linux CLI 认它找配置/缓存落点)
|
|
41
43
|
"CHROME", "EDGE", "PLAYWRIGHT", # 本机测试工具
|
|
42
44
|
)
|
|
43
45
|
|
package/app/core/flows.py
CHANGED
|
@@ -74,6 +74,9 @@ BUILTIN_FLOWS = [
|
|
|
74
74
|
"threshold": 7.0, "rounds": 2,
|
|
75
75
|
"goal_hint": "翻译什么(源文本位置 / 目标语言 / 要求)",
|
|
76
76
|
"note": "起草 → 多维评审 → 修订循环 → 发布门禁"},
|
|
77
|
+
{"id": "rank_scan", "name": "扫榜选材", "icon": "i-chart", "engine": "direct", "builtin": True,
|
|
78
|
+
"goal_hint": "想写哪个方向(一句话,可留空默认分析总榜热门题材)",
|
|
79
|
+
"note": "抓取七猫排行榜公开数据 → AI 提炼热门题材/人设/差异化切入点(快档直出报告)"},
|
|
77
80
|
{"id": "research", "name": "调研报告", "icon": "i-file-search", "engine": "review", "builtin": True,
|
|
78
81
|
"manuscript": "report.md",
|
|
79
82
|
"rubric": ["全面性", "深度", "论据可靠", "可读性", "结论质量"],
|
package/app/core/goal_service.py
CHANGED
|
@@ -86,9 +86,18 @@ def get(goal_id: str):
|
|
|
86
86
|
return dict(g) if g else None
|
|
87
87
|
|
|
88
88
|
|
|
89
|
-
def create(title: str, description: str = "", *, rounds_max: int =
|
|
89
|
+
def create(title: str, description: str = "", *, rounds_max: int | None = None,
|
|
90
90
|
metadata: dict | None = None) -> dict:
|
|
91
|
-
"""创建新 goal。已有未完结 goal 时拒绝(单一当前目标,仿 dsh)。
|
|
91
|
+
"""创建新 goal。已有未完结 goal 时拒绝(单一当前目标,仿 dsh)。
|
|
92
|
+
|
|
93
|
+
rounds_max 缺省读 settings_v2 orchestrator.max_goal_rounds(读不到回落 5)。"""
|
|
94
|
+
if rounds_max is None:
|
|
95
|
+
try:
|
|
96
|
+
from .settings_schema import get as ss_get, register_default_namespaces
|
|
97
|
+
register_default_namespaces()
|
|
98
|
+
rounds_max = int(ss_get("orchestrator", "max_goal_rounds") or 5)
|
|
99
|
+
except Exception:
|
|
100
|
+
rounds_max = 5
|
|
92
101
|
with _LOCK:
|
|
93
102
|
cur = current()
|
|
94
103
|
if cur is not None:
|
package/app/core/jobs.py
CHANGED
|
@@ -5,11 +5,13 @@
|
|
|
5
5
|
隔离,store 层有全局锁。目标并发数可在设置页调整;调小后多余线程在取到
|
|
6
6
|
新任务前自行退出,调大即时补齐。安装/升级失败时自动触发 AI 诊断修复:
|
|
7
7
|
由真实智能体读取失败日志与本机环境给出修正命令;仅当命令命中白名单前缀
|
|
8
|
-
(npm/winget/pip
|
|
8
|
+
(npm/winget/brew/pip 安装类,按平台取对应渠道)才自动执行,否则把建议命令
|
|
9
|
+
记录在运行记录里等人工确认。
|
|
9
10
|
"""
|
|
10
11
|
from __future__ import annotations
|
|
11
12
|
|
|
12
13
|
import queue
|
|
14
|
+
import sys
|
|
13
15
|
import threading
|
|
14
16
|
import traceback
|
|
15
17
|
|
|
@@ -27,10 +29,11 @@ WATCHDOG_INTERVAL_S = 60 # 队列看门狗巡检周期
|
|
|
27
29
|
WATCHDOG_STALE_S = 120 # queued 超过该秒数视为掉队(正常入队到被拿起 ≤5s)
|
|
28
30
|
_watchdog_started = False
|
|
29
31
|
|
|
30
|
-
AI_REPAIR_PROMPT = """你是环境工程师。在
|
|
32
|
+
AI_REPAIR_PROMPT = """你是环境工程师。在 __OS__ 上执行下面的安装命令失败了,请诊断原因并给出修正命令。
|
|
31
33
|
只输出一个 ```json 代码块,不要输出其他内容。JSON 结构:
|
|
32
34
|
{"diagnosis": "失败原因(一句话)", "command": "修正后的完整安装命令", "safe": true/false}
|
|
33
|
-
硬性约束:command 只能是本机包管理器的安装命令,前缀必须是 __ALLOW__
|
|
35
|
+
硬性约束:command 只能是本机包管理器的安装命令,前缀必须是 __ALLOW__ 之一,且不要带 sudo
|
|
36
|
+
(命令非交互执行,sudo 会挂死;brew/npm 本身也拒绝 sudo)。
|
|
34
37
|
给不出符合约束的安全命令时,safe 设为 false 且 command 留空。
|
|
35
38
|
|
|
36
39
|
## 失败的命令
|
|
@@ -43,14 +46,29 @@ __LOG__
|
|
|
43
46
|
__ENV__"""
|
|
44
47
|
|
|
45
48
|
# AI 修复命令白名单:只放行包管理器的安装类命令(提示词里的前缀清单由它生成,
|
|
46
|
-
#
|
|
47
|
-
|
|
48
|
-
|
|
49
|
+
# 两处永远不会漂移)。按平台分组:winget/py 启动器是 Windows 独有,brew/python3
|
|
50
|
+
# 是 macOS/Linux 渠道——Mac 上不放行 brew 的话,AI 给出正确的修正命令也会被拒。
|
|
51
|
+
_AI_REPAIR_ALLOW_WIN = ("npm install ", "winget install", "py -3.13 -m pip install",
|
|
52
|
+
"uv tool install")
|
|
53
|
+
_AI_REPAIR_ALLOW_UNIX = ("npm install ", "brew install", "python3 -m pip install",
|
|
54
|
+
"uv tool install")
|
|
55
|
+
AI_REPAIR_ALLOW = _AI_REPAIR_ALLOW_WIN # 兼容旧引用:Windows 本机即此表
|
|
49
56
|
|
|
50
57
|
|
|
51
|
-
def
|
|
58
|
+
def _repair_os_label(platform=None):
|
|
59
|
+
return {"win32": "Windows", "darwin": "macOS"}.get(platform or sys.platform, "Linux")
|
|
60
|
+
|
|
61
|
+
|
|
62
|
+
def _repair_allow(platform=None):
|
|
63
|
+
if (platform or sys.platform) == "win32":
|
|
64
|
+
return _AI_REPAIR_ALLOW_WIN
|
|
65
|
+
return _AI_REPAIR_ALLOW_UNIX
|
|
66
|
+
|
|
67
|
+
|
|
68
|
+
def _repair_command_allowed(cmd, platform=None):
|
|
52
69
|
cmd = (cmd or "").strip()
|
|
53
|
-
return cmd.startswith(
|
|
70
|
+
return (cmd.startswith(_repair_allow(platform)) and "|" not in cmd
|
|
71
|
+
and "&" not in cmd and ">" not in cmd)
|
|
54
72
|
|
|
55
73
|
|
|
56
74
|
def configure(max_workers):
|
|
@@ -449,6 +467,11 @@ def _worker():
|
|
|
449
467
|
_maybe_auto_resume(run_id) # 连载失败自动续跑(继承已完成章)
|
|
450
468
|
except Exception:
|
|
451
469
|
pass
|
|
470
|
+
try:
|
|
471
|
+
from . import notify
|
|
472
|
+
notify.push_run_async(run_id) # 结果推群(借鉴 agency-orchestrator --notify)
|
|
473
|
+
except Exception:
|
|
474
|
+
pass
|
|
452
475
|
_QUEUE.task_done()
|
|
453
476
|
finally:
|
|
454
477
|
with _pool_lock:
|
|
@@ -587,17 +610,21 @@ def _ai_repair(run_id, entry, ev, failed_cmd, orig_log):
|
|
|
587
610
|
except Exception:
|
|
588
611
|
log_tail = "(日志不可读)"
|
|
589
612
|
import shutil
|
|
613
|
+
is_win = sys.platform == "win32"
|
|
590
614
|
env_lines = [
|
|
591
|
-
"OS:
|
|
615
|
+
"OS: %s" % _repair_os_label(),
|
|
592
616
|
"node: %s" % (shutil.which("node") or "缺失"),
|
|
593
617
|
"npm: %s" % (shutil.which("npm") or "缺失"),
|
|
594
618
|
"pnpm: %s" % (shutil.which("pnpm") or "缺失"),
|
|
595
|
-
"python: %s" % (shutil.which("python") or "缺失"),
|
|
619
|
+
"python: %s" % (shutil.which("python3" if not is_win else "python") or "缺失"),
|
|
596
620
|
]
|
|
621
|
+
if not is_win:
|
|
622
|
+
env_lines.append("brew: %s" % (shutil.which("brew") or "缺失"))
|
|
597
623
|
prompt = (AI_REPAIR_PROMPT.replace("__CMD__", failed_cmd or "(未知)")
|
|
598
624
|
.replace("__LOG__", log_tail)
|
|
599
625
|
.replace("__ENV__", "\n".join(env_lines))
|
|
600
|
-
.replace("
|
|
626
|
+
.replace("__OS__", _repair_os_label())
|
|
627
|
+
.replace("__ALLOW__", " / ".join(p.strip() for p in _repair_allow())))
|
|
601
628
|
step, log_abs = store.add_step(run_id, "ai-repair", agent["id"], agent.get("label"),
|
|
602
629
|
note="自动诊断修复")
|
|
603
630
|
res = runner.run_agent(agent, prompt, readonly=True, timeout=300,
|
|
@@ -36,7 +36,7 @@ import zipfile
|
|
|
36
36
|
from pathlib import Path
|
|
37
37
|
from urllib.parse import urlparse
|
|
38
38
|
|
|
39
|
-
from . import market
|
|
39
|
+
from . import market, tlsctx
|
|
40
40
|
|
|
41
41
|
_LOCK = market._LOCK # 安装/记账与 market 共用一把锁,避免交叉写 market.json
|
|
42
42
|
|
|
@@ -157,10 +157,13 @@ def _fetch(url, cap=_CAP_MANIFEST):
|
|
|
157
157
|
return b"".join(chunks)
|
|
158
158
|
|
|
159
159
|
try:
|
|
160
|
-
return _read(lambda: urllib.request.urlopen(
|
|
160
|
+
return _read(lambda: urllib.request.urlopen(
|
|
161
|
+
req, timeout=30, context=tlsctx.context())) # 默认 opener:含系统代理
|
|
161
162
|
except (urllib.error.HTTPError, urllib.error.URLError):
|
|
162
163
|
# 直连重试:ProxyHandler({}) 显式清空代理
|
|
163
|
-
opener = urllib.request.build_opener(
|
|
164
|
+
opener = urllib.request.build_opener(
|
|
165
|
+
urllib.request.ProxyHandler({}),
|
|
166
|
+
urllib.request.HTTPSHandler(context=tlsctx.context()))
|
|
164
167
|
return _read(lambda: opener.open(req, timeout=30))
|
|
165
168
|
|
|
166
169
|
|
package/app/core/modelhub.py
CHANGED
|
@@ -41,7 +41,7 @@ import threading
|
|
|
41
41
|
import time
|
|
42
42
|
import urllib.request
|
|
43
43
|
|
|
44
|
-
from . import paths
|
|
44
|
+
from . import paths, tlsctx
|
|
45
45
|
|
|
46
46
|
|
|
47
47
|
class _NoRedirect(urllib.request.HTTPRedirectHandler):
|
|
@@ -50,6 +50,13 @@ class _NoRedirect(urllib.request.HTTPRedirectHandler):
|
|
|
50
50
|
def redirect_request(self, req, fp, code, msg, headers, newurl):
|
|
51
51
|
return None
|
|
52
52
|
|
|
53
|
+
|
|
54
|
+
def _opener():
|
|
55
|
+
"""出站 opener:禁重定向 + tlsctx 兜底 CA(macOS python.org 版 Python
|
|
56
|
+
缺系统证书,默认上下文验证必挂——见 tlsctx 模块 docstring)。"""
|
|
57
|
+
return urllib.request.build_opener(
|
|
58
|
+
_NoRedirect, urllib.request.HTTPSHandler(context=tlsctx.context()))
|
|
59
|
+
|
|
53
60
|
_LOCK = threading.RLock()
|
|
54
61
|
_FILE = paths.DATA_DIR / "models.json"
|
|
55
62
|
|
|
@@ -206,11 +213,11 @@ def _fetch_models_http(base_url, api_key, protocol, allow_private=False):
|
|
|
206
213
|
else:
|
|
207
214
|
urls = std
|
|
208
215
|
last_err = ""
|
|
209
|
-
opener =
|
|
216
|
+
opener = _opener()
|
|
210
217
|
for url in urls:
|
|
211
|
-
|
|
212
|
-
if
|
|
213
|
-
last_err =
|
|
218
|
+
host, herr = _validate_host(url, allow_private)
|
|
219
|
+
if host is None:
|
|
220
|
+
last_err = herr
|
|
214
221
|
continue
|
|
215
222
|
# auto 不知道是哪条 wire,鉴权头也按两种都试(google 那种单独补上)
|
|
216
223
|
hdrs = _auth_header_variants(api_key, protocol)
|
|
@@ -237,8 +244,16 @@ def _fetch_models_http(base_url, api_key, protocol, allow_private=False):
|
|
|
237
244
|
if names:
|
|
238
245
|
return names, ""
|
|
239
246
|
last_err = url + " 返回 200 但未解析到模型"
|
|
247
|
+
except urllib.error.HTTPError as e:
|
|
248
|
+
if e.code == 404:
|
|
249
|
+
# zcode-plan / open.bigmodel.cn 等 anthropic 形网关没有
|
|
250
|
+
# GET /models——404 与密钥无关,模型以导入/手填为准
|
|
251
|
+
last_err = (url + " 无模型列表接口(HTTP 404)——"
|
|
252
|
+
"模型以导入/手动添加为准,不影响对话调用")
|
|
253
|
+
else:
|
|
254
|
+
last_err = tlsctx.humanize("%s → HTTP %s" % (url, e.code))
|
|
240
255
|
except Exception as e:
|
|
241
|
-
last_err = "%s → %r" % (url, e)
|
|
256
|
+
last_err = tlsctx.humanize("%s → %r" % (url, e))
|
|
242
257
|
return None, last_err
|
|
243
258
|
|
|
244
259
|
|
|
@@ -1512,6 +1527,7 @@ def _src_zcode():
|
|
|
1512
1527
|
default_model = str(d.get("model") or "")
|
|
1513
1528
|
tail = default_model.split("/", 1)[1] if "/" in default_model else default_model
|
|
1514
1529
|
bad = []
|
|
1530
|
+
nokey = []
|
|
1515
1531
|
for pid, p in (d.get("provider") or {}).items():
|
|
1516
1532
|
if not isinstance(p, dict):
|
|
1517
1533
|
continue
|
|
@@ -1523,13 +1539,23 @@ def _src_zcode():
|
|
|
1523
1539
|
"zcode", "zcode:%s" % pid,
|
|
1524
1540
|
model=(tail if tail in names else ""), models=names)
|
|
1525
1541
|
if pr:
|
|
1542
|
+
if not pr.get("api_key"):
|
|
1543
|
+
nokey.append(str(p.get("name") or pid))
|
|
1526
1544
|
out["providers"].append(pr)
|
|
1527
1545
|
else:
|
|
1528
1546
|
bad.append(str(p.get("name") or pid))
|
|
1547
|
+
notes = []
|
|
1529
1548
|
if bad:
|
|
1530
|
-
|
|
1531
|
-
len(bad), "、".join(bad[:4]))
|
|
1532
|
-
|
|
1549
|
+
notes.append("跳过 {0} 个(缺合法 baseURL):{1}".format(
|
|
1550
|
+
len(bad), "、".join(bad[:4])))
|
|
1551
|
+
if nokey:
|
|
1552
|
+
# ZCode 的 builtin Plan 条目(zcode.z.ai/api/v1/zcode-plan 等)密钥不入
|
|
1553
|
+
# config.json(实测 credentials.json 里的 OAuth token 也不能直接当 API
|
|
1554
|
+
# key 用)——静默导入空密钥条目只会让用户在拉列表/适配测试时一头雾水
|
|
1555
|
+
notes.append("{0} 个未带出密钥——请在「模型接入」页手填:{1}".format(
|
|
1556
|
+
len(nokey), "、".join(nokey[:4])))
|
|
1557
|
+
if notes:
|
|
1558
|
+
out["note"] = ";".join(notes)
|
|
1533
1559
|
return out
|
|
1534
1560
|
|
|
1535
1561
|
|
|
@@ -2368,18 +2394,18 @@ def _post_json_http(url, headers, body, allow_private, timeout=20):
|
|
|
2368
2394
|
p = urllib.parse.urlsplit(url)
|
|
2369
2395
|
if p.scheme not in ("http", "https"):
|
|
2370
2396
|
return 0, None, "协议必须是 http/https"
|
|
2371
|
-
|
|
2372
|
-
if
|
|
2373
|
-
return 0, None,
|
|
2397
|
+
host, herr = _validate_host(url, allow_private)
|
|
2398
|
+
if host is None:
|
|
2399
|
+
return 0, None, herr
|
|
2374
2400
|
try:
|
|
2375
2401
|
req = urllib.request.Request(url, method="POST",
|
|
2376
2402
|
headers=dict(headers, **{"Content-Type": "application/json"}),
|
|
2377
2403
|
data=json.dumps(body).encode("utf-8"))
|
|
2378
|
-
with
|
|
2404
|
+
with _opener().open(req, timeout=timeout) as resp:
|
|
2379
2405
|
raw = resp.read(1024 * 1024)
|
|
2380
2406
|
return resp.status, json.loads(raw.decode("utf-8", "replace")), ""
|
|
2381
2407
|
except Exception as e:
|
|
2382
|
-
return 0, None, repr(e)[:300]
|
|
2408
|
+
return 0, None, tlsctx.humanize(repr(e)[:300])
|
|
2383
2409
|
|
|
2384
2410
|
|
|
2385
2411
|
def _sse_parse(proto, obj):
|
|
@@ -2440,15 +2466,16 @@ def _post_sse_http(url, headers, body, allow_private, timeout, proto, on_delta):
|
|
|
2440
2466
|
p = urllib.parse.urlsplit(url)
|
|
2441
2467
|
if p.scheme not in ("http", "https"):
|
|
2442
2468
|
return 0, "", None, "协议必须是 http/https"
|
|
2443
|
-
|
|
2444
|
-
|
|
2469
|
+
host, herr = _validate_host(url, allow_private)
|
|
2470
|
+
if host is None:
|
|
2471
|
+
return 0, "", None, herr
|
|
2445
2472
|
parts, usage = [], {}
|
|
2446
2473
|
resp_status = 0
|
|
2447
2474
|
try:
|
|
2448
2475
|
req = urllib.request.Request(url, method="POST",
|
|
2449
2476
|
headers=dict(headers, **{"Content-Type": "application/json"}),
|
|
2450
2477
|
data=json.dumps(body).encode("utf-8"))
|
|
2451
|
-
with
|
|
2478
|
+
with _opener().open(req, timeout=timeout) as resp:
|
|
2452
2479
|
resp_status = resp.status
|
|
2453
2480
|
if not 200 <= resp.status < 300:
|
|
2454
2481
|
raw = resp.read(65536).decode("utf-8", "replace")
|
|
@@ -2479,7 +2506,7 @@ def _post_sse_http(url, headers, body, allow_private, timeout, proto, on_delta):
|
|
|
2479
2506
|
if sum(len(s) for s in parts) > 2 * 1024 * 1024: # 防失控
|
|
2480
2507
|
break
|
|
2481
2508
|
except Exception as e:
|
|
2482
|
-
return 0, "", None, repr(e)[:300]
|
|
2509
|
+
return 0, "", None, tlsctx.humanize(repr(e)[:300])
|
|
2483
2510
|
text = "".join(parts)
|
|
2484
2511
|
if not usage.get("total"):
|
|
2485
2512
|
usage["total"] = usage.get("input", 0) + usage.get("output", 0) + usage.get("cached", 0)
|
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
# -*- coding: utf-8 -*-
|
|
2
|
+
"""运行结果群推送(借鉴 agency-orchestrator 的 --notify):任务跑完把结果摘要
|
|
3
|
+
推到钉钉/飞书/企业微信群机器人,配合定时自动化就是「AI 团队每天定点交活」。
|
|
4
|
+
|
|
5
|
+
webhook 一个地址全包——按域名自动适配三种机器人格式(同 agency-orchestrator
|
|
6
|
+
思路);也接受任意 https 地址(按钉钉 text 形状发,自建 n8n 等自选)。
|
|
7
|
+
推送在后台线程、失败只记日志——通知永远不影响任务本身。
|
|
8
|
+
"""
|
|
9
|
+
from __future__ import annotations
|
|
10
|
+
|
|
11
|
+
import logging
|
|
12
|
+
|
|
13
|
+
log = logging.getLogger(__name__)
|
|
14
|
+
|
|
15
|
+
|
|
16
|
+
def _payload_for(webhook, text):
|
|
17
|
+
"""按 webhook 域名适配群机器人消息体。返回 None = 无法识别的地址。"""
|
|
18
|
+
from urllib.parse import urlparse
|
|
19
|
+
host = (urlparse(webhook).hostname or "").lower()
|
|
20
|
+
if "dingtalk" in host:
|
|
21
|
+
return {"msgtype": "text", "text": {"content": text}}
|
|
22
|
+
if "feishu" in host or "larksuite" in host:
|
|
23
|
+
return {"msg_type": "text", "content": {"text": text}}
|
|
24
|
+
if "weixin" in host or "wechat" in host or "work.weixin" in host:
|
|
25
|
+
return {"msgtype": "text", "text": {"content": text}}
|
|
26
|
+
if host:
|
|
27
|
+
return {"msgtype": "text", "text": {"content": text}} # 未知域名按钉钉形状
|
|
28
|
+
return None
|
|
29
|
+
|
|
30
|
+
|
|
31
|
+
def _webhook():
|
|
32
|
+
try:
|
|
33
|
+
from . import settings
|
|
34
|
+
return str(settings.load().get("notify_webhook") or "").strip()
|
|
35
|
+
except Exception:
|
|
36
|
+
return ""
|
|
37
|
+
|
|
38
|
+
|
|
39
|
+
def _post(hook, text):
|
|
40
|
+
"""curl 子进程 POST(Python 不经手响应体);网络失败只记日志。"""
|
|
41
|
+
from . import runner
|
|
42
|
+
import json as _json
|
|
43
|
+
body = _json.dumps(_payload_for(hook, text), ensure_ascii=False)
|
|
44
|
+
r = runner.run_process(
|
|
45
|
+
argv=["curl", "-sS", "--max-time", "20",
|
|
46
|
+
"-H", "Content-Type: application/json",
|
|
47
|
+
"-d", body, hook],
|
|
48
|
+
timeout=30)
|
|
49
|
+
return bool(r["ok"])
|
|
50
|
+
|
|
51
|
+
|
|
52
|
+
def push_text(text):
|
|
53
|
+
"""推一条文本到群。配置了 webhook 才推;失败返回 False 不抛错。"""
|
|
54
|
+
hook = _webhook()
|
|
55
|
+
if not hook:
|
|
56
|
+
return False
|
|
57
|
+
if not hook.startswith("https://"):
|
|
58
|
+
log.warning("notify webhook 必须是 https")
|
|
59
|
+
return False
|
|
60
|
+
try:
|
|
61
|
+
return _post(hook, text)
|
|
62
|
+
except Exception as e:
|
|
63
|
+
log.warning("notify push failed: %s", e)
|
|
64
|
+
return False
|
|
65
|
+
|
|
66
|
+
|
|
67
|
+
def push_run_async(run_id):
|
|
68
|
+
"""任务收尾后异步推送结果摘要(jobs 层调用;绝不阻塞/影响任务)。"""
|
|
69
|
+
threading.Thread(target=push_run, daemon=True,
|
|
70
|
+
name="notify-%s" % run_id, args=(run_id,)).start()
|
|
71
|
+
|
|
72
|
+
|
|
73
|
+
def push_run(run_id):
|
|
74
|
+
"""组装 run 结果摘要并推送(webhook 未配置时静默跳过)。"""
|
|
75
|
+
hook = _webhook()
|
|
76
|
+
if not hook:
|
|
77
|
+
return False
|
|
78
|
+
from . import store
|
|
79
|
+
run = store.get_run(run_id) or {}
|
|
80
|
+
if not run:
|
|
81
|
+
return False
|
|
82
|
+
status = str(run.get("status") or "")
|
|
83
|
+
mark = {"done": "✅", "failed": "❌", "cancelled": "⚪"}.get(status, "🔔")
|
|
84
|
+
lines = ["%s CodeBee 任务%s" % (mark, {"done": "完成", "failed": "失败",
|
|
85
|
+
"cancelled": "已取消"}.get(status, status))]
|
|
86
|
+
lines.append("任务:%s" % (run.get("title") or run_id))
|
|
87
|
+
if run.get("error"):
|
|
88
|
+
lines.append("错误:%s" % str(run["error"])[:200])
|
|
89
|
+
v = run.get("verdict") or {}
|
|
90
|
+
if v.get("overall") is not None:
|
|
91
|
+
lines.append("综合评分 %.1f(%s)" % (
|
|
92
|
+
float(v["overall"]), "达标" if v.get("publishable") else "未达标"))
|
|
93
|
+
return push_text("\n".join(lines))
|
|
94
|
+
|
|
95
|
+
|
|
96
|
+
import threading # noqa: E402 (push_run_async 依赖;置底避免顶部循环导入)
|
|
@@ -0,0 +1,106 @@
|
|
|
1
|
+
# -*- coding: utf-8 -*-
|
|
2
|
+
"""扫榜选材:抓取公开排行榜页,产出选题分析素材。
|
|
3
|
+
|
|
4
|
+
数据源(双源聚合,任一失败不影响另一源):
|
|
5
|
+
- 七猫:www.qimao.com/paihang/(公开可抓,2026-09-19 实测 200/106KB)
|
|
6
|
+
- 番茄 Web 版:fanqienovel.com/rank(公开可抓,题材分类+书名在
|
|
7
|
+
中文 a 标签里;App 端接口有 SecuritySign/X-Argus 签名——那才是难路,
|
|
8
|
+
Web 版榜单页不需要签名,别走弯路)
|
|
9
|
+
|
|
10
|
+
下载走 curl 子进程(与 covergen 同款),解析用宽松正则——页面结构变了
|
|
11
|
+
宁可返回空(调用方回落普通直连提示词),不做脆弱的强解析。
|
|
12
|
+
"""
|
|
13
|
+
from __future__ import annotations
|
|
14
|
+
|
|
15
|
+
import logging
|
|
16
|
+
import re
|
|
17
|
+
|
|
18
|
+
log = logging.getLogger(__name__)
|
|
19
|
+
|
|
20
|
+
QIMAO_RANK_URL = "https://www.qimao.com/paihang/"
|
|
21
|
+
FANQIE_RANK_URL = "https://fanqienovel.com/rank"
|
|
22
|
+
_UA = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
|
|
23
|
+
"(KHTML, like Gecko) Chrome/126 Safari/537.36")
|
|
24
|
+
|
|
25
|
+
# 页面 UI 噪音词(非书目/分类)
|
|
26
|
+
_NOISE = {"加入书架", "立即阅读", "开始阅读", "免费阅读", "全部", "分类", "排行",
|
|
27
|
+
"排行榜", "完本", "连载中", "阅读", "书城", "男生", "女生",
|
|
28
|
+
"帮助中心", "作家助手", "客服", "登录", "注册"}
|
|
29
|
+
|
|
30
|
+
_CJK_RE = re.compile(r">([\u4e00-\u9fa5]{2,12})<")
|
|
31
|
+
|
|
32
|
+
|
|
33
|
+
def _curl_text(url):
|
|
34
|
+
"""curl 子进程抓页面文本;失败返回空串。"""
|
|
35
|
+
from . import runner
|
|
36
|
+
try:
|
|
37
|
+
r = runner.run_process(
|
|
38
|
+
argv=["curl", "-sS", "--max-time", "20", "-A", _UA, url],
|
|
39
|
+
timeout=30)
|
|
40
|
+
except Exception:
|
|
41
|
+
return ""
|
|
42
|
+
if not r.get("ok"):
|
|
43
|
+
return ""
|
|
44
|
+
return r.get("stdout") or ""
|
|
45
|
+
|
|
46
|
+
|
|
47
|
+
def _extract_cjk(html, limit):
|
|
48
|
+
"""从 HTML 提取中文元素材(去 UI 噪音、保序去重)。"""
|
|
49
|
+
items, seen = [], set()
|
|
50
|
+
for t in _CJK_RE.findall(html or ""):
|
|
51
|
+
if t in _NOISE or t in seen:
|
|
52
|
+
continue
|
|
53
|
+
seen.add(t)
|
|
54
|
+
items.append(t)
|
|
55
|
+
if len(items) >= limit:
|
|
56
|
+
break
|
|
57
|
+
return items
|
|
58
|
+
|
|
59
|
+
|
|
60
|
+
def fetch_qimao_rank(limit=30):
|
|
61
|
+
"""七猫排行榜:返回元素列表,抓取失败返回 []。"""
|
|
62
|
+
return _extract_cjk(_curl_text(QIMAO_RANK_URL), limit)
|
|
63
|
+
|
|
64
|
+
|
|
65
|
+
def fetch_fanqie_rank(limit=30):
|
|
66
|
+
"""番茄 Web 版榜单:返回元素列表,抓取失败返回 []。"""
|
|
67
|
+
return _extract_cjk(_curl_text(FANQIE_RANK_URL), limit)
|
|
68
|
+
|
|
69
|
+
|
|
70
|
+
def fetch_rank_items(limit=40):
|
|
71
|
+
"""双源聚合:七猫+番茄各抓一份(去重合并,七猫在前)。
|
|
72
|
+
|
|
73
|
+
两源都失败才返回 []——单源挂了另一源仍可用,扫榜永不因单点挡任务。"""
|
|
74
|
+
qm = fetch_qimao_rank(limit)
|
|
75
|
+
fq = fetch_fanqie_rank(limit)
|
|
76
|
+
merged, seen = [], set(qm)
|
|
77
|
+
for t in fq:
|
|
78
|
+
if t not in seen:
|
|
79
|
+
seen.add(t)
|
|
80
|
+
merged.append(t)
|
|
81
|
+
return qm + merged
|
|
82
|
+
|
|
83
|
+
|
|
84
|
+
def rank_scan_prompt(goal):
|
|
85
|
+
"""组装扫榜选材分析提示词。双源都空返回 None(调用方回落直连提示词)。"""
|
|
86
|
+
qm = fetch_qimao_rank(30)
|
|
87
|
+
fq = fetch_fanqie_rank(30)
|
|
88
|
+
qm = [t for t in qm if t not in fq] # 双源重合的只留七猫份
|
|
89
|
+
if not qm and not fq:
|
|
90
|
+
return None
|
|
91
|
+
blocks = []
|
|
92
|
+
if qm:
|
|
93
|
+
blocks.append("### 七猫排行榜素材\n" + "、".join(qm))
|
|
94
|
+
if fq:
|
|
95
|
+
blocks.append("### 番茄排行榜素材\n" + "、".join(fq))
|
|
96
|
+
material = "\n\n".join(blocks)
|
|
97
|
+
return (
|
|
98
|
+
"你是网文选题分析师。以下是刚刚抓取的两个平台排行榜页面的书目与分类素材:\n\n"
|
|
99
|
+
"## 榜单元素材\n%s\n\n"
|
|
100
|
+
"## 用户想写的方向\n%s\n\n"
|
|
101
|
+
"## 你的产出(Markdown 报告)\n"
|
|
102
|
+
"1. **热门题材 Top3**:各自的共同特征与上榜代表书目(跨平台重合的题材单独点出——双平台都热说明是真风口)\n"
|
|
103
|
+
"2. **高频人设/套路总结**:3-5 条,点名反复出现的元素\n"
|
|
104
|
+
"3. **差异化切入建议**:2-3 个,结合用户方向给出「题材+人设」组合与一句话理由\n"
|
|
105
|
+
"直接输出分析报告,不要复述素材清单,不要输出与报告无关的内容。"
|
|
106
|
+
% (material, goal or "(用户未指定方向,按大盘热门分析)"))
|