sdd-flow-kit 1.3.0 → 1.3.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md
CHANGED
|
@@ -35,6 +35,7 @@ npx sdd-flow-kit doctor --agent cursor
|
|
|
35
35
|
```bash
|
|
36
36
|
cd <你的项目目录> # cd 到哪,装到哪
|
|
37
37
|
pnpm add -D sdd-flow-kit # ① postinstall 自动轻量写入 skill
|
|
38
|
+
pnpm add -D sdd-flow-kit@latest
|
|
38
39
|
npx sdd-flow-kit install --project <类型> -y # ② 全量 OPSX(按需)
|
|
39
40
|
```
|
|
40
41
|
|
|
@@ -55,136 +56,9 @@ ls docs/*-doc-skill/SKILL.md
|
|
|
55
56
|
ls .claude/skills/sdd-flow-kit/SKILL.md
|
|
56
57
|
```
|
|
57
58
|
|
|
58
|
-
|
|
59
|
+
如果找不到,则执行下面的命令
|
|
59
60
|
|
|
60
|
-
|
|
61
|
-
- 自动解析项目根:`INIT_CWD`(pnpm/npm)→ 从 `node_modules` 向上找消费方 `package.json` → `process.cwd()`(手动执行时)
|
|
62
|
-
- 全量 postinstall:`SDD_FLOW_KIT_POSTINSTALL_FULL=1 pnpm add -D sdd-flow-kit`
|
|
63
|
-
- 跳过 postinstall:`SDD_FLOW_KIT_SKIP_POSTINSTALL=1 pnpm add -D sdd-flow-kit`
|
|
64
|
-
|
|
65
|
-
> 环境变量覆盖项目类型:`SDD_FLOW_KIT_PROJECT=ADI pnpm add -D sdd-flow-kit`
|
|
66
|
-
|
|
67
|
-
#### 安装失败排查
|
|
68
|
-
|
|
69
|
-
当没有生成 `.cursor/skills/sdd-flow-kit/SKILL.md` 时:
|
|
70
|
-
|
|
71
|
-
1. 检查包是否已安装:`pnpm list sdd-flow-kit --depth 0`
|
|
72
|
-
2. **pnpm v10** 若出现 `Ignored build scripts: sdd-flow-kit`:
|
|
73
|
-
- `pnpm approve-builds` 选中 `sdd-flow-kit`,或
|
|
74
|
-
- 在 `package.json` 加 `"pnpm": { "onlyBuiltDependencies": ["sdd-flow-kit"] }` 后 `pnpm install`
|
|
75
|
-
3. 若出现 `ERR_PNPM_UNEXPECTED_STORE`,对齐 store 后重装:`pnpm install`
|
|
76
|
-
4. 手动触发轻量安装(任选其一,**须在项目根目录**):
|
|
77
|
-
- `node ./node_modules/sdd-flow-kit/dist/postinstall.js`(v1.2.4+ 无需 `INIT_CWD`)
|
|
78
|
-
- `pnpm rebuild sdd-flow-kit`
|
|
79
|
-
- `npx sdd-flow-kit install --project OMS -y`(按项目改类型,全量)
|
|
80
|
-
5. 验证:`ls .cursor/skills/sdd-flow-kit/SKILL.md docs/*-doc-skill/SKILL.md`
|
|
81
|
-
|
|
82
|
-
首次跑任意 CLI(`doctor` / `run` / `gate` 等)也会自动 bootstrap 补装缺失的 SDD skill 或 OPSX。
|
|
83
|
-
|
|
84
|
-
##### `pnpm add` 后依赖全被删掉(`Packages: +1 -932`)
|
|
85
|
-
|
|
86
|
-
**原因**:`pnpm` 会按当前 `package.json` **同步** `node_modules`。若 manifest 里只剩 `sdd-flow-kit`(或文件被误保存成几行空壳),就会卸载其它所有包——**不是** `pnpm add` 单独删包,而是 manifest 已不完整。
|
|
87
|
-
|
|
88
|
-
**恢复**(在业务仓如 `adInsight-web/frontend`):
|
|
89
|
-
|
|
90
|
-
1. 用 Git 恢复完整 `package.json`:`git checkout -- package.json`
|
|
91
|
-
2. 重装:`pnpm install`
|
|
92
|
-
3. 再装 kit(建议 **≥ 1.1.2**,且必须在有完整 `package.json` 的目录执行):
|
|
93
|
-
- `cd frontend && pnpm add -D sdd-flow-kit@1.1.2`
|
|
94
|
-
- 或本地联调:`pnpm add -D file:../../sdd-flow-kit`
|
|
95
|
-
|
|
96
|
-
**v1.1.2+ 防护**:`repairOpsxDeliveryScripts` 仅合并 `scripts`,写入前检查依赖条数与文件体积。
|
|
97
|
-
|
|
98
|
-
**v1.2.4+ postinstall**:默认 light;自动解析消费方项目根(不依赖 `INIT_CWD`);首次安装后会写入 `pnpm.onlyBuiltDependencies` 以便后续 `pnpm install` 执行脚本。
|
|
99
|
-
|
|
100
|
-
#### 使用说明
|
|
101
|
-
|
|
102
|
-
##### 安装根:cd 到哪,装到哪(v1.2+)
|
|
103
|
-
|
|
104
|
-
默认 **`installRoot = 你执行命令时的当前目录`**(`process.cwd()`),也可用 `--project-root` / `SDD_FLOW_KIT_INSTALL_ROOT` 指定。
|
|
105
|
-
|
|
106
|
-
| 你 cd 到 | 会安装到 |
|
|
107
|
-
|----------|----------|
|
|
108
|
-
| `adInsight-web/frontend` | `frontend/package.json`、`.cursor/skills`、`.opsx`、`pnpm run opsx:delivery-pipeline` |
|
|
109
|
-
| `adInsight-web`(仓库根) | 仓库根(需根目录也有 `package.json`,否则会失败) |
|
|
110
|
-
|
|
111
|
-
```bash
|
|
112
|
-
cd /path/to/adInsight-web/frontend
|
|
113
|
-
npx sdd-flow-kit install --project ADI -y
|
|
114
|
-
npx sdd-flow-kit run --product ADI --version 2.3.2 -y
|
|
115
|
-
```
|
|
116
|
-
|
|
117
|
-
**PRD / gate 的 `--project-root`**:与安装根相同;若 `openspec/` 在上级目录,工具会**向上查找**已有 `openspec/PRD`(不把安装写到上级)。
|
|
118
|
-
|
|
119
|
-
旧行为(在仓库根执行时自动落到 `frontend/`):设置 `SDD_FLOW_KIT_LEGACY_MONOREPO=1`。
|
|
120
|
-
|
|
121
|
-
##### 脚本门禁(v1.1+,**不依赖自然语言**)
|
|
122
|
-
|
|
123
|
-
`run` 会在 `openspec/PRD/<runId>/NEXT.md` 写入**必须按顺序执行的命令**。AI 与人都以 **exit code** 为准:失败则不得进入下一阶段、不得改 `src/`。
|
|
124
|
-
|
|
125
|
-
```bash
|
|
126
|
-
# 门禁(失败 exit 1)
|
|
127
|
-
npx sdd-flow-kit gate --project-root . --run-id <runId> --expect prd-fetched
|
|
128
|
-
npx sdd-flow-kit gate --project-root . --run-id <runId> --expect docs-closed # 含 05 已定义 + 无 src 改动
|
|
129
|
-
npx sdd-flow-kit gate --project-root . --run-id <runId> --expect propose-ready --change <name>
|
|
130
|
-
npx sdd-flow-kit gate --project-root . --run-id <runId> --expect ac-ready --change <name>
|
|
131
|
-
npx sdd-flow-kit gate --project-root . --run-id <runId> --expect impl-allowed --change <name>
|
|
132
|
-
npx sdd-flow-kit gate --project-root . --run-id <runId> --expect ac-signed
|
|
133
|
-
npx sdd-flow-kit gate --project-root . --run-id <runId> --expect shipped --change <name>
|
|
134
|
-
|
|
135
|
-
# 阶段推进(更新 .session-state.json + NEXT.md)
|
|
136
|
-
npx sdd-flow-kit phase advance --project-root . --run-id <runId> --to after-prd
|
|
137
|
-
npx sdd-flow-kit phase advance --project-root . --run-id <runId> --to docs-done
|
|
138
|
-
npx sdd-flow-kit phase advance --project-root . --run-id <runId> --to propose-done --change <name>
|
|
139
|
-
npx sdd-flow-kit phase advance --project-root . --run-id <runId> --to impl --change <name>
|
|
140
|
-
npx sdd-flow-kit phase advance --project-root . --run-id <runId> --to shipped --change <name>
|
|
141
|
-
|
|
142
|
-
# 提案 / 交付
|
|
143
|
-
npx sdd-flow-kit propose --project-root . --run-id <runId> --change <name>
|
|
144
|
-
npx sdd-flow-kit deliver --project-root . --run-id <runId> --change <name>
|
|
145
|
-
```
|
|
146
|
-
|
|
147
|
-
`docs-closed` 会在 git 仓库中扫描 `src/`、`frontend/src/` 等路径;文档阶段若已有实现层改动,**gate 失败**。同时要求 `05-验收清单.md` 已标注「定义状态:已定义」且含 P0 项。
|
|
148
|
-
|
|
149
|
-
##### v1.3.0 — 验收清单(AC Matrix)流程
|
|
150
|
-
|
|
151
|
-
| 文档/门禁 | 说明 |
|
|
152
|
-
|-----------|------|
|
|
153
|
-
| `05-验收清单.md` | 从 `source/PRD.md` 逐条抽 AC;**唯一验收 SSOT** |
|
|
154
|
-
| `gate docs-closed` | 含 05 已定义 + 01/02/04 + 03 闭合 |
|
|
155
|
-
| `gate ac-ready` | propose 后:每条 P0 在 `tasks.md`/`spec.md` 含 `[AC-XX]` |
|
|
156
|
-
| `gate ac-signed` | deliver 前:05 中 P0 已验收 + 「签收状态:已签收」 |
|
|
157
|
-
| `LAYERED-TESTING.md` | 安装到 `.cursor/skills/sdd-flow-kit/`(L1–L4 测试策略) |
|
|
158
|
-
|
|
159
|
-
推荐文档顺序:**03 闭合 → 05 → 04(引用 AC)→ 01/02**。
|
|
160
|
-
|
|
161
|
-
原 `05-openspec-提案草稿` / `06-opsx` 已顺延为 **`06-openspec-提案草稿`** / **`07-opsx-自动串联指引`**。
|
|
162
|
-
|
|
163
|
-
##### 需求澄清:用户无需背提示词
|
|
164
|
-
|
|
165
|
-
`install` 会写入 `.cursor/rules/sdd-flow-kit-gates.mdc` 与触发 Skill(已改为**脚本优先**)。
|
|
166
|
-
用户粘贴确认点后:先更新 **05→04→01/02/03**,再执行 `gate docs-closed` → `phase advance --to docs-done`;propose 后 `gate ac-ready`;deliver 前 `gate ac-signed`。**禁止**跳过 gate 直接改代码。
|
|
167
|
-
|
|
168
|
-
##### `/opsx-apply` 与 TDD + Playwright 检查机制
|
|
169
|
-
|
|
170
|
-
| 环节 | 实现 | 说明 |
|
|
171
|
-
|------|------|------|
|
|
172
|
-
| 安装时 | `postinstall`(light) | 只写 sdd skill/rule;**不**自动跑 opsx-workflow(避免 `pnpm i` 卡住) |
|
|
173
|
-
| 完整安装 | `npx sdd-flow-kit install -y` | 缺什么跑什么 → `ensureOpsxWorkflow` → `npx @lixin5257xxx/opsx-workflow setup` |
|
|
174
|
-
| Apply 前 | `npx sdd-flow-kit ensure-opsx -y` | **阶段 D 第 0 步**;doctor 加 `--fix` 等价修复 |
|
|
175
|
-
| Apply 时 | Cursor 读 skill 链 + `pnpm run opsx:delivery-pipeline` | 由 `AGENTS.md` / `06-agent-skill` 约束 AI,非 Node 自动执行 |
|
|
176
|
-
|
|
177
|
-
```bash
|
|
178
|
-
npx sdd-flow-kit ensure-opsx --agent cursor -y # /opsx-apply 前
|
|
179
|
-
npx sdd-flow-kit doctor --agent cursor --fix # 检查并尝试修复
|
|
180
|
-
```
|
|
181
|
-
|
|
182
|
-
常见遗漏:只 `pnpm add sdd-flow-kit` 未跑 `ensure-opsx`;或 AI 跳过 skill 直接改代码。
|
|
183
|
-
|
|
184
|
-
1. `install`(等价 `setup`):提示选择开发环境(cursor/claude-code/codex/openclaw,默认 claude-code)与开发分支(无则自动创建),并检测/安装 openspec+superpower 基础环境,同时写入项目级 Skill 与 Cursor 门禁规则
|
|
185
|
-
2. 安装完成后,本工具主要负责“流程环境与模板”落地;业务需求由你在目标 AI 工具中自然语言驱动
|
|
186
|
-
3. `start`/`run`/`invoke` 属于进阶能力(脚本化或一键编排时使用)
|
|
187
|
-
4. `doctor`:检查 git/npx/agent CLI/openspec/tool 目录,输出可机读诊断结果
|
|
61
|
+
node ./node_modules/sdd-flow-kit/dist/postinstall.js
|
|
188
62
|
|
|
189
63
|
#### 当前自动化边界
|
|
190
64
|
|
|
@@ -204,25 +78,11 @@ npx sdd-flow-kit doctor --agent cursor --fix # 检查并尝试修复
|
|
|
204
78
|
- 非 scoped 或后续版本:`npm publish`
|
|
205
79
|
6. 发布后验证:在任意项目执行 `npm i -D sdd-flow-kit`,再执行 `npx sdd-flow-kit install`
|
|
206
80
|
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
1. 修改版本号:`npm version patch`(或 `minor` / `major`)
|
|
210
|
-
2. 重新执行构建与发布步骤
|
|
211
|
-
3. 在 `README.md` 或变更记录中补充本次升级说明
|
|
212
|
-
|
|
213
|
-
#### 参与贡献
|
|
214
|
-
|
|
215
|
-
1. Fork 本仓库
|
|
216
|
-
2. 新建 Feat_xxx 分支
|
|
217
|
-
3. 提交代码
|
|
218
|
-
4. 新建 Pull Request
|
|
219
|
-
|
|
81
|
+
#### 变更记录
|
|
220
82
|
|
|
221
|
-
|
|
83
|
+
**1.3.3** — Confluence PRD 拉取修复(ADI 长标题页)
|
|
222
84
|
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
5. Gitee 官方提供的使用手册 [https://gitee.com/help](https://gitee.com/help)
|
|
228
|
-
6. Gitee 封面人物是一档用来展示 Gitee 会员风采的栏目 [https://gitee.com/gitee-stars/](https://gitee.com/gitee-stars/)
|
|
85
|
+
- `confluence-doc.py`:匹配时统一 `-`/`_`;ADI 增加 `ADI_V*结算` 关键词
|
|
86
|
+
- ADI 需求页无「PRD」面包屑时按 `ADI_V*` / 结算开票回款标题识别
|
|
87
|
+
- 支持 `CONFLUENCE_PAGE_ID` 直链(如 V2.3.4 → `109609740`)
|
|
88
|
+
- `dosearchsite` 搜索改用 `domcontentloaded`,降低网络抖动失败率
|
package/package.json
CHANGED
|
@@ -12,11 +12,31 @@ python3 confluence-doc.py {{VERSION_EXAMPLE}}
|
|
|
12
12
|
|
|
13
13
|
版本号可传:`2.3.2`、`V2.3.2`、`ADI_V2.3.2`、`ADI-V2.3.2`(脚本会归一化)。
|
|
14
14
|
|
|
15
|
-
2.
|
|
15
|
+
2. 脚本会依次搜索(含用户原样输入):`ADI_V*` → `ADI-V*` → `ADI_*` → `ADI-*` → `ADI_v*` → `ADI-v*` → `ADI V*` → `ADI_V*结算` → `V*` 等。
|
|
16
16
|
|
|
17
|
-
3.
|
|
17
|
+
3. **禁止**在 Confluence 脚本未跑完或未确认失败前,用本地 Word/原型替代 `source/PRD.md`。
|
|
18
|
+
|
|
19
|
+
4. 成功后把生成的 `docs/*.md` **主文档**复制/回填到本 run 目录:
|
|
18
20
|
- `source/PRD.md`
|
|
19
21
|
|
|
22
|
+
## 直链兜底(已知 pageId 时推荐)
|
|
23
|
+
|
|
24
|
+
ADI V2.3.4 示例(`yyzcpt` 空间 pageId `109609740`):
|
|
25
|
+
|
|
26
|
+
```bash
|
|
27
|
+
cd {{DOC_SKILL_REL}}
|
|
28
|
+
export DOC_PRODUCT_PREFIX="{{PRODUCT_PREFIX}}"
|
|
29
|
+
export CONFLUENCE_PAGE_ID=109609740
|
|
30
|
+
python3 confluence-doc.py 2.3.4
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
或:
|
|
34
|
+
|
|
35
|
+
```bash
|
|
36
|
+
export CONFLUENCE_PAGE_URL="https://confluence.huan.tv/spaces/yyzcpt/pages/109609740/ADI_V2.3.4结算、开票、回款数据对接欢聚"
|
|
37
|
+
python3 confluence-doc.py 2.3.4
|
|
38
|
+
```
|
|
39
|
+
|
|
20
40
|
## 路径说明
|
|
21
41
|
|
|
22
42
|
- 推荐脚本目录:`{{DOC_SKILL_REL}}`(已向上解析,可能是 `docs/adi-doc-skill` 或 `../docs/adi-doc-skill`)
|
|
@@ -24,6 +44,10 @@ python3 confluence-doc.py {{VERSION_EXAMPLE}}
|
|
|
24
44
|
|
|
25
45
|
## 失败排查
|
|
26
46
|
|
|
27
|
-
- 输出 `❌ 未找到`:到 Confluence
|
|
47
|
+
- 输出 `❌ 未找到`:到 Confluence 确认页面标题(ADI 常为 `ADI_V2.3.4结算、开票…`,**下划线**而非 `ADI-V`)
|
|
48
|
+
- 页面存在但搜不到:设置 `CONFLUENCE_PAGE_ID=<pageId>` 或 `CONFLUENCE_PAGE_URL=<完整URL>` 直链下载
|
|
49
|
+
- Playwright 失败且设置 `CONFLUENCE_USE_REST=1` 时才回退 Token;登录最多重试 3 次,并用 `expect_navigation` 避免竞态
|
|
50
|
+
- 登录偶发失败:用 `CONFLUENCE_HEADED=1 CONFLUENCE_SLOW_MO=500` 有头观察;失败截图 `confluence-login-fail-*.png`
|
|
28
51
|
- `playwright` 未安装:`pip install playwright && playwright install chromium`
|
|
52
|
+
- Token 仅作应急回退:`CONFLUENCE_USE_REST=1`(Token 会过期,不作为默认路径)
|
|
29
53
|
- 仍失败:在仓库根执行 `npx sdd-flow-kit install --project {{PRODUCT}} -y` 后重试
|
|
@@ -1,11 +1,14 @@
|
|
|
1
1
|
#!/usr/bin/env python3
|
|
2
2
|
"""Confluence 需求文档获取脚本(Playwright - 通用版)"""
|
|
3
3
|
|
|
4
|
+
import json
|
|
4
5
|
import sys
|
|
5
6
|
import os
|
|
6
7
|
import re
|
|
7
8
|
from pathlib import Path
|
|
8
|
-
from urllib.parse import quote, urljoin, urlparse, unquote
|
|
9
|
+
from urllib.parse import quote, urlencode, urljoin, urlparse, unquote
|
|
10
|
+
from urllib.request import Request, urlopen
|
|
11
|
+
from urllib.error import HTTPError, URLError
|
|
9
12
|
|
|
10
13
|
try:
|
|
11
14
|
from playwright.sync_api import sync_playwright
|
|
@@ -15,6 +18,8 @@ except ImportError:
|
|
|
15
18
|
|
|
16
19
|
DEFAULT_CONFLUENCE_USERNAME = "lixinxin"
|
|
17
20
|
DEFAULT_CONFLUENCE_PASSWORD = "Xin@147258"
|
|
21
|
+
# Confluence REST API Bearer Token(优先于 Playwright 登录,确保 Step1 可稳定拉取 PRD)
|
|
22
|
+
DEFAULT_CONFLUENCE_API_TOKEN = "NTc3NDMzNjQxNjE3OiDitymjYiVP9cQ4PfTzkgOeb+DA"
|
|
18
23
|
|
|
19
24
|
|
|
20
25
|
def env(name: str, default: str | None = None) -> str:
|
|
@@ -61,24 +66,64 @@ def version_slug(prefix: str, version: str) -> str:
|
|
|
61
66
|
return prefix.lower() + "-v" + version.replace(".", "")
|
|
62
67
|
|
|
63
68
|
|
|
64
|
-
def
|
|
65
|
-
"""
|
|
69
|
+
def dedupe_keywords(items: list[str]) -> list[str]:
|
|
70
|
+
"""去重并保持顺序。"""
|
|
71
|
+
seen: set[str] = set()
|
|
72
|
+
out: list[str] = []
|
|
73
|
+
for item in items:
|
|
74
|
+
key = item.strip()
|
|
75
|
+
if not key or key in seen:
|
|
76
|
+
continue
|
|
77
|
+
seen.add(key)
|
|
78
|
+
out.append(key)
|
|
79
|
+
return out
|
|
80
|
+
|
|
81
|
+
|
|
82
|
+
def normalize_match_token(text: str) -> str:
|
|
83
|
+
"""匹配时统一 -/_ 与大小写,避免 ADI-V 与 ADI_V 标题不一致导致漏命中。"""
|
|
84
|
+
return text.lower().replace("-", "_").replace(".", "")
|
|
85
|
+
|
|
86
|
+
|
|
87
|
+
def search_keywords(prefix: str, version: str, raw_input: str | None = None) -> list[str]:
|
|
88
|
+
"""固定顺序(与用户 Confluence 全局搜索习惯一致)。
|
|
89
|
+
|
|
90
|
+
ADI 等产品的典型降级链:
|
|
91
|
+
用户原样输入 → ADI_V2.3.4 → ADI-V2.3.4 → ADI_2.3.4 → ADI-2.3.4
|
|
92
|
+
→ ADI_v2.3.4 → ADI-v2.3.4 → ADI V2.3.4 → ADI_V2.3.4结算 → V2.3.4
|
|
93
|
+
"""
|
|
66
94
|
p = prefix.strip()
|
|
67
95
|
# BreakX(破圈引擎)PRD 在 xhs 空间,页面标题多为 V1.0.2 xxx,而非 BreakX_V1.0.2
|
|
68
96
|
if p.upper() in ("BREAKX", "BX"):
|
|
69
|
-
return
|
|
97
|
+
return dedupe_keywords(
|
|
98
|
+
[
|
|
99
|
+
*( [raw_input.strip()] if raw_input and raw_input.strip() else [] ),
|
|
100
|
+
f"V{version}",
|
|
101
|
+
f"BreakX V{version}",
|
|
102
|
+
f"BreakX_V{version}",
|
|
103
|
+
f"BreakX-V{version}",
|
|
104
|
+
f"xhs V{version}",
|
|
105
|
+
]
|
|
106
|
+
)
|
|
107
|
+
adi_long_title_kw: list[str] = []
|
|
108
|
+
if p.upper() == "ADI":
|
|
109
|
+
adi_long_title_kw = [
|
|
110
|
+
f"ADI_V{version}结算",
|
|
111
|
+
f"ADI_V{version}结算、开票、回款",
|
|
112
|
+
]
|
|
113
|
+
return dedupe_keywords(
|
|
114
|
+
[
|
|
115
|
+
*( [raw_input.strip()] if raw_input and raw_input.strip() else [] ),
|
|
116
|
+
f"{prefix}_V{version}",
|
|
117
|
+
f"{prefix}-V{version}",
|
|
118
|
+
f"{prefix}_{version}",
|
|
119
|
+
f"{prefix}-{version}",
|
|
120
|
+
f"{prefix}_v{version}",
|
|
121
|
+
f"{prefix}-v{version}",
|
|
122
|
+
f"{prefix} V{version}",
|
|
123
|
+
*adi_long_title_kw,
|
|
70
124
|
f"V{version}",
|
|
71
|
-
f"BreakX V{version}",
|
|
72
|
-
f"BreakX_V{version}",
|
|
73
|
-
f"BreakX-V{version}",
|
|
74
|
-
f"xhs V{version}",
|
|
75
125
|
]
|
|
76
|
-
|
|
77
|
-
f"{prefix}_V{version}",
|
|
78
|
-
f"{prefix}-V{version}",
|
|
79
|
-
f"{prefix}_{version}",
|
|
80
|
-
f"{prefix}-{version}",
|
|
81
|
-
]
|
|
126
|
+
)
|
|
82
127
|
|
|
83
128
|
|
|
84
129
|
def sanitize_filename(name: str) -> str:
|
|
@@ -126,36 +171,40 @@ def absolutize_href(base_url: str, href: str) -> str:
|
|
|
126
171
|
|
|
127
172
|
|
|
128
173
|
def href_path_matches_keyword(href: str, keyword: str) -> bool:
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
174
|
+
path_norm = normalize_match_token(unquote(urlparse(href).path))
|
|
175
|
+
kw_norm = normalize_match_token(keyword)
|
|
176
|
+
return kw_norm in path_norm
|
|
177
|
+
|
|
178
|
+
|
|
179
|
+
def text_matches_keyword(text: str, keyword: str) -> bool:
|
|
180
|
+
"""标题/链接文案匹配:忽略 -/_ 差异。"""
|
|
181
|
+
return normalize_match_token(keyword) in normalize_match_token(text)
|
|
134
182
|
|
|
135
183
|
|
|
136
184
|
def score_link_match(text: str, href: str, prefix: str, version: str, keyword: str) -> int:
|
|
137
185
|
"""对搜索结果链接打分;>=2 视为命中。"""
|
|
138
186
|
t = (text or "").lower()
|
|
139
187
|
path = unquote(urlparse(href).path).lower()
|
|
140
|
-
|
|
188
|
+
kw_norm = normalize_match_token(keyword)
|
|
141
189
|
p = prefix.lower()
|
|
142
190
|
v_compact = version.replace(".", "")
|
|
143
|
-
|
|
144
|
-
|
|
191
|
+
t_norm = normalize_match_token(text or "")
|
|
192
|
+
t_compact = re.sub(r"\s+", "", t_norm)
|
|
193
|
+
path_norm = normalize_match_token(path)
|
|
145
194
|
|
|
146
195
|
score = 0
|
|
147
|
-
if
|
|
196
|
+
if text_matches_keyword(text, keyword):
|
|
148
197
|
score += 4
|
|
149
|
-
if
|
|
198
|
+
if kw_norm in t_compact:
|
|
150
199
|
score += 3
|
|
151
200
|
if p in t and version in t:
|
|
152
201
|
score += 4
|
|
153
|
-
if p in
|
|
202
|
+
if p in t_norm and v_compact in t_compact:
|
|
154
203
|
score += 3
|
|
155
204
|
if href_path_matches_keyword(href, keyword):
|
|
156
|
-
score +=
|
|
205
|
+
score += 4
|
|
157
206
|
slug = f"{p}v{v_compact}"
|
|
158
|
-
if slug in
|
|
207
|
+
if slug in path_norm or f"{p}-v{v_compact}" in path or f"{p}_v{v_compact}" in path:
|
|
159
208
|
score += 4
|
|
160
209
|
# BreakX:标题以 V1.0.2 开头时加分
|
|
161
210
|
if p in ("breakx", "bx") and (t.startswith(f"v{version}") or f"v{version}" in t[:12]):
|
|
@@ -224,14 +273,44 @@ def read_breadcrumb_text(page) -> str:
|
|
|
224
273
|
return " > ".join(parts)
|
|
225
274
|
|
|
226
275
|
|
|
276
|
+
def is_adi_requirement_page(title: str, crumbs: str, page_url: str) -> bool:
|
|
277
|
+
"""ADI 需求页常无 PRD 字样,标题形如 ADI_V2.3.4结算、开票、回款数据对接欢聚。"""
|
|
278
|
+
title_upper = (title or "").upper()
|
|
279
|
+
crumbs_upper = (crumbs or "").upper()
|
|
280
|
+
if re.search(r"ADI[_-]?V\d+\.\d+\.\d+", title_upper):
|
|
281
|
+
print(f" ✓ ADI 版本标题命中: {title[:80]}")
|
|
282
|
+
return True
|
|
283
|
+
adi_markers = ("结算", "开票", "回款", "对接欢聚", "数据对接")
|
|
284
|
+
if any(m in title for m in adi_markers) and "ADI" in title_upper:
|
|
285
|
+
print(f" ✓ ADI 财务对接标题命中: {title[:80]}")
|
|
286
|
+
return True
|
|
287
|
+
if "ADI" in crumbs_upper and re.search(r"V\d+\.\d+\.\d+", title_upper + crumbs_upper):
|
|
288
|
+
print(f" ✓ ADI 面包屑+版本命中: {crumbs[:80]} / {title[:60]}")
|
|
289
|
+
return True
|
|
290
|
+
if "/spaces/yyzcpt/" in page_url.lower() and re.search(r"ADI[_-]?V", title_upper):
|
|
291
|
+
print(f" ✓ yyzcpt 空间 ADI 页面: {title[:80]}")
|
|
292
|
+
return True
|
|
293
|
+
return False
|
|
294
|
+
|
|
295
|
+
|
|
227
296
|
def is_prd_page(page, page_url: str) -> bool:
|
|
228
297
|
"""检查页面面包屑/父级是否属于 PRD(需求文档)。"""
|
|
229
|
-
page.goto(page_url, wait_until="
|
|
298
|
+
page.goto(page_url, wait_until="domcontentloaded", timeout=60000)
|
|
230
299
|
page.wait_for_timeout(1200)
|
|
231
300
|
|
|
232
301
|
crumbs = read_breadcrumb_text(page)
|
|
233
302
|
crumbs_upper = crumbs.upper()
|
|
234
303
|
prefix = env("DOC_PRODUCT_PREFIX", "").upper()
|
|
304
|
+
|
|
305
|
+
try:
|
|
306
|
+
title = (page.locator("#title-text").inner_text() or "").strip()
|
|
307
|
+
except Exception:
|
|
308
|
+
title = ""
|
|
309
|
+
|
|
310
|
+
# ADI:标题常无「PRD/需求」字样,单独识别
|
|
311
|
+
if prefix == "ADI" and is_adi_requirement_page(title, crumbs, page_url):
|
|
312
|
+
return True
|
|
313
|
+
|
|
235
314
|
# BreakX:xhs 空间 PRD 挂在「产品设计」下,不一定有 PRD 字样
|
|
236
315
|
if prefix in ("BREAKX", "BX") and "/spaces/xhs/" in page_url.lower():
|
|
237
316
|
if "产品设计" in crumbs or "BREAKX" in crumbs_upper or "破圈" in crumbs:
|
|
@@ -241,10 +320,6 @@ def is_prd_page(page, page_url: str) -> bool:
|
|
|
241
320
|
print(f" ✓ PRD 父级命中: {crumbs[:120]}")
|
|
242
321
|
return True
|
|
243
322
|
|
|
244
|
-
try:
|
|
245
|
-
title = (page.locator("#title-text").inner_text() or "").strip()
|
|
246
|
-
except Exception:
|
|
247
|
-
title = ""
|
|
248
323
|
title_upper = title.upper()
|
|
249
324
|
if re.search(r"\bPRD\b", title_upper) or "需求文档" in title:
|
|
250
325
|
print(f" ✓ 标题含 PRD/需求: {title[:80]}")
|
|
@@ -254,10 +329,308 @@ def is_prd_page(page, page_url: str) -> bool:
|
|
|
254
329
|
return False
|
|
255
330
|
|
|
256
331
|
|
|
332
|
+
def resolve_direct_page_url(base_url: str) -> str | None:
|
|
333
|
+
"""支持 CONFLUENCE_PAGE_URL 或 CONFLUENCE_PAGE_ID 直链,跳过搜索。"""
|
|
334
|
+
page_url = env("CONFLUENCE_PAGE_URL")
|
|
335
|
+
if page_url:
|
|
336
|
+
return page_url
|
|
337
|
+
page_id = resolve_direct_page_id()
|
|
338
|
+
if page_id:
|
|
339
|
+
return f"{base_url.rstrip('/')}/pages/viewpage.action?pageId={page_id}"
|
|
340
|
+
return None
|
|
341
|
+
|
|
342
|
+
|
|
343
|
+
def resolve_direct_page_id() -> str | None:
|
|
344
|
+
"""从环境变量或 URL 中解析 pageId。"""
|
|
345
|
+
page_id = env("CONFLUENCE_PAGE_ID")
|
|
346
|
+
if page_id:
|
|
347
|
+
return page_id
|
|
348
|
+
page_url = env("CONFLUENCE_PAGE_URL")
|
|
349
|
+
if not page_url:
|
|
350
|
+
return None
|
|
351
|
+
m = re.search(r"/pages/(\d+)", page_url)
|
|
352
|
+
if m:
|
|
353
|
+
return m.group(1)
|
|
354
|
+
m = re.search(r"pageId=(\d+)", page_url, flags=re.IGNORECASE)
|
|
355
|
+
if m:
|
|
356
|
+
return m.group(1)
|
|
357
|
+
return None
|
|
358
|
+
|
|
359
|
+
|
|
360
|
+
def get_api_token() -> str:
|
|
361
|
+
"""读取 Confluence REST Bearer Token。"""
|
|
362
|
+
return (
|
|
363
|
+
env("CONFLUENCE_API_TOKEN")
|
|
364
|
+
or env("CONFLUENCE_TOKEN")
|
|
365
|
+
or env("CONFLUENCE_BEARER_TOKEN")
|
|
366
|
+
or DEFAULT_CONFLUENCE_API_TOKEN
|
|
367
|
+
)
|
|
368
|
+
|
|
369
|
+
|
|
370
|
+
def rest_request(base_url: str, token: str, path: str, params: dict | None = None) -> dict:
|
|
371
|
+
"""调用 Confluence REST API,失败时抛出可读异常。"""
|
|
372
|
+
query = f"?{urlencode(params)}" if params else ""
|
|
373
|
+
url = f"{base_url.rstrip('/')}{path}{query}"
|
|
374
|
+
req = Request(
|
|
375
|
+
url,
|
|
376
|
+
headers={
|
|
377
|
+
"Authorization": f"Bearer {token}",
|
|
378
|
+
"Accept": "application/json",
|
|
379
|
+
},
|
|
380
|
+
)
|
|
381
|
+
try:
|
|
382
|
+
with urlopen(req, timeout=60) as resp:
|
|
383
|
+
return json.loads(resp.read().decode("utf-8"))
|
|
384
|
+
except HTTPError as e:
|
|
385
|
+
body = e.read().decode("utf-8", errors="replace")[:300]
|
|
386
|
+
raise RuntimeError(f"REST API HTTP {e.code}: {body}") from e
|
|
387
|
+
except URLError as e:
|
|
388
|
+
raise RuntimeError(f"REST API 网络错误: {e}") from e
|
|
389
|
+
|
|
390
|
+
|
|
391
|
+
def rest_fetch_page(base_url: str, token: str, page_id: str) -> dict:
|
|
392
|
+
"""按 pageId 拉取页面标题与 storage HTML。"""
|
|
393
|
+
data = rest_request(
|
|
394
|
+
base_url,
|
|
395
|
+
token,
|
|
396
|
+
f"/rest/api/content/{page_id}",
|
|
397
|
+
{"expand": "body.storage,space"},
|
|
398
|
+
)
|
|
399
|
+
html = data.get("body", {}).get("storage", {}).get("value", "")
|
|
400
|
+
if not html or len(html) < 20:
|
|
401
|
+
raise RuntimeError(f"pageId={page_id} 正文为空或过短")
|
|
402
|
+
return {
|
|
403
|
+
"id": str(data.get("id", page_id)),
|
|
404
|
+
"title": data.get("title", "").strip() or f"page-{page_id}",
|
|
405
|
+
"html": html,
|
|
406
|
+
"space": (data.get("space") or {}).get("key", ""),
|
|
407
|
+
"url": f"{base_url.rstrip('/')}/pages/viewpage.action?pageId={page_id}",
|
|
408
|
+
}
|
|
409
|
+
|
|
410
|
+
|
|
411
|
+
def rest_search_candidates(
|
|
412
|
+
base_url: str, token: str, prefix: str, version: str, keyword: str
|
|
413
|
+
) -> list[tuple[int, str, str]]:
|
|
414
|
+
"""CQL 搜索页面,返回 [(score, page_id, title), ...]。"""
|
|
415
|
+
cql = f'type=page and title ~ "{keyword.replace(chr(34), "")}"'
|
|
416
|
+
space = env("CONFLUENCE_SPACE")
|
|
417
|
+
if space:
|
|
418
|
+
cql = f'space="{space}" and {cql}'
|
|
419
|
+
data = rest_request(
|
|
420
|
+
base_url,
|
|
421
|
+
token,
|
|
422
|
+
"/rest/api/content/search",
|
|
423
|
+
{"cql": cql, "limit": "15"},
|
|
424
|
+
)
|
|
425
|
+
scored: list[tuple[int, str, str]] = []
|
|
426
|
+
for item in data.get("results", []):
|
|
427
|
+
title = (item.get("title") or "").strip()
|
|
428
|
+
page_id = str(item.get("id", ""))
|
|
429
|
+
if not page_id:
|
|
430
|
+
continue
|
|
431
|
+
href = f"/spaces/{(item.get('space') or {}).get('key', '')}/pages/{page_id}/"
|
|
432
|
+
score = score_link_match(title, href, prefix, version, keyword)
|
|
433
|
+
if score >= 2:
|
|
434
|
+
scored.append((score, page_id, title))
|
|
435
|
+
scored.sort(key=lambda x: x[0], reverse=True)
|
|
436
|
+
return scored
|
|
437
|
+
|
|
438
|
+
|
|
439
|
+
def rest_resolve_target(
|
|
440
|
+
base_url: str, token: str, prefix: str, version: str, keywords: list[str]
|
|
441
|
+
) -> tuple[str, str, str, str] | None:
|
|
442
|
+
"""REST 搜索命中页面,返回 (page_id, title, keyword, url)。"""
|
|
443
|
+
direct_id = resolve_direct_page_id()
|
|
444
|
+
if direct_id:
|
|
445
|
+
page = rest_fetch_page(base_url, token, direct_id)
|
|
446
|
+
kw = keywords[0] if keywords else f"{prefix}_V{version}"
|
|
447
|
+
return page["id"], page["title"], kw, page["url"]
|
|
448
|
+
|
|
449
|
+
fallback: tuple[str, str, str, str] | None = None
|
|
450
|
+
for kw in keywords:
|
|
451
|
+
print(f"\n🔍 [REST] 搜索: {kw}")
|
|
452
|
+
candidates = rest_search_candidates(base_url, token, prefix, version, kw)
|
|
453
|
+
if not candidates:
|
|
454
|
+
print(f" 未找到 {kw}")
|
|
455
|
+
continue
|
|
456
|
+
for score, page_id, title in candidates:
|
|
457
|
+
print(f" 候选 (score={score}): {title[:80]}")
|
|
458
|
+
page = rest_fetch_page(base_url, token, page_id)
|
|
459
|
+
if prefix.upper() == "ADI" and is_adi_title_match(title, version):
|
|
460
|
+
return page_id, page["title"], kw, page["url"]
|
|
461
|
+
if score >= 4:
|
|
462
|
+
return page_id, page["title"], kw, page["url"]
|
|
463
|
+
if fallback is None:
|
|
464
|
+
fallback = (page_id, page["title"], kw, page["url"])
|
|
465
|
+
return fallback
|
|
466
|
+
|
|
467
|
+
|
|
468
|
+
def is_adi_title_match(title: str, version: str) -> bool:
|
|
469
|
+
"""ADI 长标题页识别(面包屑可无 PRD 字样)。"""
|
|
470
|
+
title_upper = (title or "").upper()
|
|
471
|
+
v_compact = version.replace(".", "")
|
|
472
|
+
if re.search(rf"ADI[_-]?V{re.escape(version)}", title_upper):
|
|
473
|
+
return True
|
|
474
|
+
if re.search(rf"ADI[_-]?V{v_compact}", normalize_match_token(title)):
|
|
475
|
+
return True
|
|
476
|
+
markers = ("结算", "开票", "回款", "对接欢聚", "数据对接", "蒲公英", "报价单")
|
|
477
|
+
return "ADI" in title_upper and any(m in title for m in markers)
|
|
478
|
+
|
|
479
|
+
|
|
480
|
+
def clean_confluence_storage_html(html: str) -> str:
|
|
481
|
+
"""清理 Confluence storage 宏标签,便于转 Markdown。"""
|
|
482
|
+
html = re.sub(
|
|
483
|
+
r"<ac:structured-macro[^>]*>.*?</ac:structured-macro>",
|
|
484
|
+
"",
|
|
485
|
+
html,
|
|
486
|
+
flags=re.DOTALL,
|
|
487
|
+
)
|
|
488
|
+
html = re.sub(r"<ac:parameter[^>]*>.*?</ac:parameter>", "", html, flags=re.DOTALL)
|
|
489
|
+
html = re.sub(
|
|
490
|
+
r"<ac:plain-text-body><!\[CDATA\[(.*?)\]\]></ac:plain-text-body>",
|
|
491
|
+
r"\1",
|
|
492
|
+
html,
|
|
493
|
+
flags=re.DOTALL,
|
|
494
|
+
)
|
|
495
|
+
html = re.sub(r"<ac:link[^>]*><ri:page[^>]*></ac:link>", "", html)
|
|
496
|
+
html = re.sub(r"<ac:image[^>]*>.*?</ac:image>", "", html, flags=re.DOTALL)
|
|
497
|
+
return html
|
|
498
|
+
|
|
499
|
+
|
|
500
|
+
def download_images_http(
|
|
501
|
+
base_url: str,
|
|
502
|
+
token: str,
|
|
503
|
+
content_html: str,
|
|
504
|
+
version_dir: Path,
|
|
505
|
+
slug: str,
|
|
506
|
+
request_get=None,
|
|
507
|
+
) -> str:
|
|
508
|
+
"""下载正文图片并改写相对路径;request_get 可注入 Playwright page.request.get。"""
|
|
509
|
+
img_urls = re.findall(r'<img[^>]*src="([^"]+)"', content_html)
|
|
510
|
+
for i, img_url in enumerate(img_urls):
|
|
511
|
+
original_url = img_url
|
|
512
|
+
if img_url.startswith("//"):
|
|
513
|
+
img_url = "https:" + img_url
|
|
514
|
+
elif img_url.startswith("/"):
|
|
515
|
+
img_url = base_url.rstrip("/") + img_url
|
|
516
|
+
elif not img_url.startswith("http"):
|
|
517
|
+
img_url = urljoin(base_url, img_url)
|
|
518
|
+
|
|
519
|
+
if not img_url.startswith("http") or img_url.startswith("data:"):
|
|
520
|
+
continue
|
|
521
|
+
parsed = urlparse(img_url)
|
|
522
|
+
ext = os.path.splitext(parsed.path)[1] or ".png"
|
|
523
|
+
filename = f"image_{i + 1}{ext}"
|
|
524
|
+
filepath = version_dir / filename
|
|
525
|
+
try:
|
|
526
|
+
if request_get:
|
|
527
|
+
resp = request_get(img_url)
|
|
528
|
+
body = resp.body() if hasattr(resp, "body") else resp.read()
|
|
529
|
+
ok = resp.ok if hasattr(resp, "ok") else True
|
|
530
|
+
else:
|
|
531
|
+
req = Request(
|
|
532
|
+
img_url,
|
|
533
|
+
headers={"Authorization": f"Bearer {token}"},
|
|
534
|
+
)
|
|
535
|
+
with urlopen(req, timeout=30) as resp:
|
|
536
|
+
body = resp.read()
|
|
537
|
+
ok = True
|
|
538
|
+
if ok and body:
|
|
539
|
+
filepath.write_bytes(body)
|
|
540
|
+
content_html = content_html.replace(
|
|
541
|
+
f'src="{original_url}"', f'src="{slug}/{filename}"'
|
|
542
|
+
)
|
|
543
|
+
except Exception:
|
|
544
|
+
pass
|
|
545
|
+
return content_html
|
|
546
|
+
|
|
547
|
+
|
|
548
|
+
def write_prd_markdown(
|
|
549
|
+
docs_root: Path,
|
|
550
|
+
title: str,
|
|
551
|
+
matched_keyword: str,
|
|
552
|
+
target_url: str,
|
|
553
|
+
content_html: str,
|
|
554
|
+
prefix: str,
|
|
555
|
+
base_url: str,
|
|
556
|
+
token: str,
|
|
557
|
+
version_dir: Path,
|
|
558
|
+
slug: str,
|
|
559
|
+
request_get=None,
|
|
560
|
+
) -> Path:
|
|
561
|
+
"""将 HTML 转 Markdown 并写入 docs 目录。"""
|
|
562
|
+
html = clean_confluence_storage_html(content_html)
|
|
563
|
+
html = download_images_http(base_url, token, html, version_dir, slug, request_get)
|
|
564
|
+
content_md = parse_html_to_markdown(html)
|
|
565
|
+
md_content = f"""# {title}
|
|
566
|
+
|
|
567
|
+
> 版本: {matched_keyword}
|
|
568
|
+
> 来源: {target_url}
|
|
569
|
+
|
|
570
|
+
---
|
|
571
|
+
|
|
572
|
+
{content_md}
|
|
573
|
+
"""
|
|
574
|
+
md_filename = sanitize_filename(title) + ".md"
|
|
575
|
+
if not re.search(rf"{prefix}[-_]?V", md_filename, flags=re.IGNORECASE):
|
|
576
|
+
md_filename = f"{matched_keyword}-{md_filename}"
|
|
577
|
+
md_path = docs_root / md_filename
|
|
578
|
+
md_path.write_text(md_content, encoding="utf-8")
|
|
579
|
+
return md_path
|
|
580
|
+
|
|
581
|
+
|
|
582
|
+
def try_rest_download(
|
|
583
|
+
base_url: str,
|
|
584
|
+
prefix: str,
|
|
585
|
+
version: str,
|
|
586
|
+
keywords: list[str],
|
|
587
|
+
docs_root: Path,
|
|
588
|
+
version_dir: Path,
|
|
589
|
+
slug: str,
|
|
590
|
+
) -> bool:
|
|
591
|
+
"""REST API 主路径:Bearer Token 拉取 PRD,不依赖 Playwright 登录。"""
|
|
592
|
+
token = get_api_token()
|
|
593
|
+
if not token:
|
|
594
|
+
print(" ⚠ 未配置 CONFLUENCE_API_TOKEN,跳过 REST")
|
|
595
|
+
return False
|
|
596
|
+
try:
|
|
597
|
+
resolved = rest_resolve_target(base_url, token, prefix, version, keywords)
|
|
598
|
+
if not resolved:
|
|
599
|
+
print(" REST 未命中任何页面")
|
|
600
|
+
return False
|
|
601
|
+
page_id, title, matched_keyword, target_url = resolved
|
|
602
|
+
print(f"✅ [REST] 命中: {matched_keyword} | {title[:80]}")
|
|
603
|
+
page = rest_fetch_page(base_url, token, page_id)
|
|
604
|
+
md_path = write_prd_markdown(
|
|
605
|
+
docs_root,
|
|
606
|
+
page["title"],
|
|
607
|
+
matched_keyword,
|
|
608
|
+
target_url,
|
|
609
|
+
page["html"],
|
|
610
|
+
prefix,
|
|
611
|
+
base_url,
|
|
612
|
+
token,
|
|
613
|
+
version_dir,
|
|
614
|
+
slug,
|
|
615
|
+
)
|
|
616
|
+
if md_path.stat().st_size < 80:
|
|
617
|
+
raise RuntimeError("生成的 Markdown 过短")
|
|
618
|
+
print("\n✅ 获取完成(REST API)!")
|
|
619
|
+
print(f"📄 主文档: {md_path}")
|
|
620
|
+
print(f"🖼️ 图片目录: {version_dir}")
|
|
621
|
+
return True
|
|
622
|
+
except Exception as e:
|
|
623
|
+
print(f" REST 拉取失败: {e}")
|
|
624
|
+
return False
|
|
625
|
+
|
|
626
|
+
|
|
257
627
|
def run_search_dosearch(page, base_url: str, prefix: str, version: str, keyword: str) -> list[tuple[str, str, int]]:
|
|
258
628
|
url = f"{base_url.rstrip('/')}/dosearchsite.action?queryString={quote(keyword)}"
|
|
259
629
|
print(f" → 全站搜索: {url}")
|
|
260
|
-
|
|
630
|
+
try:
|
|
631
|
+
page.goto(url, wait_until="domcontentloaded", timeout=60000)
|
|
632
|
+
except Exception:
|
|
633
|
+
page.goto(url, wait_until="load", timeout=60000)
|
|
261
634
|
page.wait_for_timeout(2500)
|
|
262
635
|
if is_login_page(page):
|
|
263
636
|
return []
|
|
@@ -324,47 +697,96 @@ def resolve_prd_target(
|
|
|
324
697
|
|
|
325
698
|
|
|
326
699
|
def is_login_page(page) -> bool:
|
|
700
|
+
"""判断是否仍在登录页(禁止用宽泛的 'login' in url,避免误判)。"""
|
|
327
701
|
url = page.url.lower()
|
|
328
|
-
if "login.action" in url
|
|
702
|
+
if "login.action" in url:
|
|
329
703
|
return True
|
|
330
|
-
|
|
331
|
-
|
|
704
|
+
user_field = page.locator("#username-field, input[name='os_username']")
|
|
705
|
+
pass_field = page.locator("#password-field, input[name='os_password']")
|
|
706
|
+
if user_field.count() > 0 and pass_field.count() > 0:
|
|
707
|
+
try:
|
|
708
|
+
if user_field.first.is_visible() and pass_field.first.is_visible():
|
|
709
|
+
return True
|
|
710
|
+
except Exception:
|
|
332
711
|
return True
|
|
333
712
|
return False
|
|
334
713
|
|
|
335
714
|
|
|
336
|
-
def
|
|
337
|
-
|
|
338
|
-
|
|
339
|
-
|
|
340
|
-
|
|
341
|
-
|
|
715
|
+
def is_session_authenticated(page, base_url: str) -> bool:
|
|
716
|
+
"""通过 Confluence 页面 meta 与会话页探测是否已登录(比 URL 判断更可靠)。"""
|
|
717
|
+
try:
|
|
718
|
+
remote_user = page.locator('meta[name="ajs-remote-user"]').get_attribute("content") or ""
|
|
719
|
+
if remote_user.strip():
|
|
720
|
+
return True
|
|
721
|
+
except Exception:
|
|
722
|
+
pass
|
|
723
|
+
return not is_login_page(page)
|
|
342
724
|
|
|
725
|
+
|
|
726
|
+
def ensure_logged_in(page, base_url: str, username: str, password: str, max_retries: int = 3) -> None:
|
|
727
|
+
"""登录 Confluence;用 expect_navigation 包裹点击,并以 remote-user 校验会话。"""
|
|
728
|
+
login_url = f"{base_url.rstrip('/')}/login.action"
|
|
343
729
|
if not username or not password:
|
|
344
730
|
raise RuntimeError("需要登录但未提供 CONFLUENCE_USERNAME / CONFLUENCE_PASSWORD")
|
|
345
731
|
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
355
|
-
|
|
732
|
+
last_err = ""
|
|
733
|
+
for attempt in range(1, max_retries + 1):
|
|
734
|
+
page.goto(login_url, wait_until="domcontentloaded", timeout=60000)
|
|
735
|
+
if is_session_authenticated(page, base_url):
|
|
736
|
+
print("✅ 已处于登录状态")
|
|
737
|
+
return
|
|
738
|
+
|
|
739
|
+
print(f"🔐 正在登录 Confluence...(第 {attempt}/{max_retries} 次)")
|
|
740
|
+
page.wait_for_selector("#username-field, input[name='os_username']", state="visible", timeout=15000)
|
|
741
|
+
# 等待 Atlassian 登录网关 React 表单可交互,避免偶发「点了但没提交」
|
|
742
|
+
page.wait_for_timeout(400)
|
|
743
|
+
|
|
744
|
+
user_sel = "#username-field" if page.locator("#username-field").count() else "input[name='os_username']"
|
|
745
|
+
pass_sel = "#password-field" if page.locator("#password-field").count() else "input[name='os_password']"
|
|
746
|
+
btn_sel = "#login-button" if page.locator("#login-button").count() else "#login"
|
|
747
|
+
|
|
748
|
+
page.fill(user_sel, username, timeout=8000)
|
|
749
|
+
page.fill(pass_sel, password, timeout=8000)
|
|
750
|
+
page.wait_for_timeout(200)
|
|
751
|
+
|
|
752
|
+
btn = page.locator(btn_sel)
|
|
753
|
+
try:
|
|
754
|
+
# 关键:导航与点击同一事务,避免 click 后 wait_for_url 的竞态(旧逻辑约 30% 失败)
|
|
755
|
+
with page.expect_navigation(timeout=45000, wait_until="domcontentloaded"):
|
|
756
|
+
btn.click()
|
|
757
|
+
except Exception as nav_err:
|
|
758
|
+
last_err = f"点击登录后未发生导航: {nav_err}"
|
|
759
|
+
print(f" ⚠ {last_err}")
|
|
760
|
+
page.wait_for_timeout(2000)
|
|
761
|
+
|
|
762
|
+
page.wait_for_timeout(1500)
|
|
763
|
+
|
|
764
|
+
if is_session_authenticated(page, base_url):
|
|
765
|
+
ru = page.locator('meta[name="ajs-remote-user"]').get_attribute("content") or ""
|
|
766
|
+
print(f"✅ 登录成功(用户: {ru or username})")
|
|
767
|
+
return
|
|
356
768
|
|
|
357
|
-
if is_login_page(page):
|
|
358
769
|
err = ""
|
|
359
770
|
for sel in [".aui-message.error", ".aui-message.aui-message-error", "#login-error"]:
|
|
360
771
|
loc = page.locator(sel)
|
|
361
772
|
if loc.count():
|
|
362
773
|
err = (loc.first.inner_text() or "").strip()
|
|
363
774
|
break
|
|
364
|
-
|
|
365
|
-
|
|
366
|
-
|
|
367
|
-
|
|
775
|
+
last_err = err or f"仍在登录页 ({page.url[:80]})"
|
|
776
|
+
print(f" ⚠ 登录未成功: {last_err}")
|
|
777
|
+
|
|
778
|
+
debug_path = Path.cwd() / f"confluence-login-fail-{attempt}.png"
|
|
779
|
+
try:
|
|
780
|
+
page.screenshot(path=str(debug_path), full_page=True)
|
|
781
|
+
print(f" 已保存登录失败截图: {debug_path}")
|
|
782
|
+
except Exception:
|
|
783
|
+
pass
|
|
784
|
+
|
|
785
|
+
page.context.clear_cookies()
|
|
786
|
+
|
|
787
|
+
raise RuntimeError(
|
|
788
|
+
f"Confluence 登录失败(已重试 {max_retries} 次)。请检查账号密码或 .env 中的 CONFLUENCE_* 配置。{last_err}"
|
|
789
|
+
)
|
|
368
790
|
|
|
369
791
|
|
|
370
792
|
def main():
|
|
@@ -398,8 +820,9 @@ def main():
|
|
|
398
820
|
prefix = os.environ.get("DOC_PRODUCT_PREFIX", "{{DOC_PRODUCT_PREFIX}}").strip() or "{{DOC_PRODUCT_PREFIX}}"
|
|
399
821
|
if prefix.upper() in ("BREAKX", "BX") and not os.environ.get("CONFLUENCE_SPACE"):
|
|
400
822
|
os.environ["CONFLUENCE_SPACE"] = "xhs"
|
|
401
|
-
|
|
402
|
-
|
|
823
|
+
raw_arg = sys.argv[1].strip()
|
|
824
|
+
version = normalize_version(raw_arg)
|
|
825
|
+
keywords = search_keywords(prefix, version, raw_input=raw_arg)
|
|
403
826
|
|
|
404
827
|
# 输出目录默认为仓库 docs/ 下
|
|
405
828
|
default_download = os.environ.get("DOC_DOWNLOAD_DIR")
|
|
@@ -427,6 +850,58 @@ def main():
|
|
|
427
850
|
|
|
428
851
|
print(f"🖥️ 浏览器: headless={headless}, slow_mo={slow_mo}ms(有头模式: CONFLUENCE_HEADED=1)")
|
|
429
852
|
|
|
853
|
+
force_playwright = env_bool("CONFLUENCE_FORCE_PLAYWRIGHT")
|
|
854
|
+
use_rest = env_bool("CONFLUENCE_USE_REST") # 默认关闭;仅显式开启时才走 Token
|
|
855
|
+
skip_playwright = env_bool("CONFLUENCE_SKIP_PLAYWRIGHT")
|
|
856
|
+
|
|
857
|
+
playwright_ok = False
|
|
858
|
+
if not skip_playwright:
|
|
859
|
+
print("\n📄 使用 Playwright 登录并拉取 PRD(主路径)...")
|
|
860
|
+
try:
|
|
861
|
+
playwright_ok = _download_via_playwright(
|
|
862
|
+
base_url=base_url,
|
|
863
|
+
username=username,
|
|
864
|
+
password=password,
|
|
865
|
+
prefix=prefix,
|
|
866
|
+
version=version,
|
|
867
|
+
keywords=keywords,
|
|
868
|
+
docs_root=docs_root,
|
|
869
|
+
version_dir=version_dir,
|
|
870
|
+
slug=slug,
|
|
871
|
+
headless=headless,
|
|
872
|
+
slow_mo=slow_mo,
|
|
873
|
+
)
|
|
874
|
+
except Exception as e:
|
|
875
|
+
print(f"\n⚠ Playwright 拉取失败: {e}")
|
|
876
|
+
|
|
877
|
+
if playwright_ok:
|
|
878
|
+
return
|
|
879
|
+
|
|
880
|
+
if use_rest:
|
|
881
|
+
print("\n🌐 Playwright 未成功,回退 REST API(Bearer Token)...")
|
|
882
|
+
if try_rest_download(base_url, prefix, version, keywords, docs_root, version_dir, slug):
|
|
883
|
+
return
|
|
884
|
+
|
|
885
|
+
print("\n❌ PRD 拉取失败:Playwright 与 REST 均未成功。")
|
|
886
|
+
print(" 排查:CONFLUENCE_HEADED=1 CONFLUENCE_SLOW_MO=500 有头模式观察登录")
|
|
887
|
+
print(" 或直链:CONFLUENCE_PAGE_ID=<pageId> python3 confluence-doc.py <版本>")
|
|
888
|
+
sys.exit(1)
|
|
889
|
+
|
|
890
|
+
|
|
891
|
+
def _download_via_playwright(
|
|
892
|
+
base_url: str,
|
|
893
|
+
username: str,
|
|
894
|
+
password: str,
|
|
895
|
+
prefix: str,
|
|
896
|
+
version: str,
|
|
897
|
+
keywords: list[str],
|
|
898
|
+
docs_root: Path,
|
|
899
|
+
version_dir: Path,
|
|
900
|
+
slug: str,
|
|
901
|
+
headless: bool,
|
|
902
|
+
slow_mo: int,
|
|
903
|
+
) -> bool:
|
|
904
|
+
"""Playwright 主路径:登录 → 搜索/直链 → 写 Markdown。"""
|
|
430
905
|
with sync_playwright() as p:
|
|
431
906
|
launch_opts: dict = {"headless": headless}
|
|
432
907
|
if slow_mo > 0:
|
|
@@ -444,9 +919,13 @@ def main():
|
|
|
444
919
|
ensure_logged_in(page, base_url, username, password)
|
|
445
920
|
print(f"📄 当前页: {page.url[:120]}")
|
|
446
921
|
|
|
447
|
-
direct_url =
|
|
922
|
+
direct_url = resolve_direct_page_url(base_url)
|
|
448
923
|
target_url = direct_url if direct_url else None
|
|
449
|
-
matched_keyword =
|
|
924
|
+
matched_keyword = (
|
|
925
|
+
"CONFLUENCE_PAGE_URL"
|
|
926
|
+
if env("CONFLUENCE_PAGE_URL")
|
|
927
|
+
else ("CONFLUENCE_PAGE_ID" if env("CONFLUENCE_PAGE_ID") else None)
|
|
928
|
+
)
|
|
450
929
|
|
|
451
930
|
matched_title = ""
|
|
452
931
|
if not target_url:
|
|
@@ -463,10 +942,14 @@ def main():
|
|
|
463
942
|
pass
|
|
464
943
|
print(f"❌ 未找到 {prefix} {version} 的需求文档。")
|
|
465
944
|
print(
|
|
466
|
-
" 排查:1)
|
|
945
|
+
" 排查:1) 页面标题是否含 ADI_V2.3.4(下划线,非 ADI-V)及中文后缀"
|
|
946
|
+
)
|
|
947
|
+
print(
|
|
948
|
+
" 2) 依次尝试 ADI_V2.3.4 / ADI-V2.3.4 / ADI_V2.3.4结算 / V2.3.4"
|
|
467
949
|
)
|
|
468
|
-
print(" 3)
|
|
469
|
-
|
|
950
|
+
print(" 3) 直链:CONFLUENCE_PAGE_ID=109609740 或 CONFLUENCE_PAGE_URL=<完整URL>")
|
|
951
|
+
print(" 4) ADI 页面包屑可能无 PRD 字样,脚本已按 ADI_V* 标题规则识别")
|
|
952
|
+
return False
|
|
470
953
|
|
|
471
954
|
print(f"✅ 命中关键词: {matched_keyword}")
|
|
472
955
|
if matched_title:
|
|
@@ -485,52 +968,31 @@ def main():
|
|
|
485
968
|
if not content_elem.count():
|
|
486
969
|
content_elem = page.locator(".content-body")
|
|
487
970
|
content_html = content_elem.inner_html() if content_elem.count() else ""
|
|
971
|
+
if len(content_html.strip()) < 20:
|
|
972
|
+
raise RuntimeError("Playwright 拉取正文为空,请检查登录态或 pageId")
|
|
973
|
+
|
|
974
|
+
token = get_api_token()
|
|
975
|
+
md_path = write_prd_markdown(
|
|
976
|
+
docs_root,
|
|
977
|
+
title,
|
|
978
|
+
matched_keyword,
|
|
979
|
+
target_url,
|
|
980
|
+
content_html,
|
|
981
|
+
prefix,
|
|
982
|
+
base_url,
|
|
983
|
+
token,
|
|
984
|
+
version_dir,
|
|
985
|
+
slug,
|
|
986
|
+
request_get=page.request.get,
|
|
987
|
+
)
|
|
488
988
|
|
|
489
|
-
|
|
490
|
-
for i, img_url in enumerate(img_urls):
|
|
491
|
-
original_url = img_url
|
|
492
|
-
if img_url.startswith("//"):
|
|
493
|
-
img_url = "https:" + img_url
|
|
494
|
-
elif img_url.startswith("/"):
|
|
495
|
-
img_url = base_url.rstrip("/") + img_url
|
|
496
|
-
elif not img_url.startswith("http"):
|
|
497
|
-
img_url = urljoin(base_url, img_url)
|
|
498
|
-
|
|
499
|
-
if img_url.startswith("http") and not img_url.startswith("data:"):
|
|
500
|
-
parsed = urlparse(img_url)
|
|
501
|
-
ext = os.path.splitext(parsed.path)[1] or ".png"
|
|
502
|
-
filename = f"image_{i + 1}{ext}"
|
|
503
|
-
filepath = version_dir / filename
|
|
504
|
-
try:
|
|
505
|
-
resp = page.request.get(img_url)
|
|
506
|
-
if resp.ok:
|
|
507
|
-
filepath.write_bytes(resp.body())
|
|
508
|
-
content_html = content_html.replace(f'src="{original_url}"', f'src="{slug}/{filename}"')
|
|
509
|
-
except Exception:
|
|
510
|
-
pass
|
|
511
|
-
|
|
512
|
-
content_md = parse_html_to_markdown(content_html)
|
|
513
|
-
md_content = f"""# {title}
|
|
514
|
-
|
|
515
|
-
> 版本: {matched_keyword}
|
|
516
|
-
> 来源: {target_url}
|
|
517
|
-
|
|
518
|
-
---
|
|
519
|
-
|
|
520
|
-
{content_md}
|
|
521
|
-
"""
|
|
522
|
-
md_filename = sanitize_filename(title) + ".md"
|
|
523
|
-
if not re.search(rf"{prefix}[-_]?V", md_filename, flags=re.IGNORECASE):
|
|
524
|
-
md_filename = f"{matched_keyword}-{md_filename}"
|
|
525
|
-
md_path = docs_root / md_filename
|
|
526
|
-
md_path.write_text(md_content, encoding="utf-8")
|
|
527
|
-
|
|
528
|
-
print("\n✅ 获取完成!")
|
|
989
|
+
print("\n✅ 获取完成(Playwright)!")
|
|
529
990
|
print(f"📄 主文档: {md_path}")
|
|
530
991
|
print(f"🖼️ 图片目录: {version_dir}")
|
|
992
|
+
return True
|
|
531
993
|
except Exception as e:
|
|
532
|
-
print(f"\n❌ 获取失败: {e}")
|
|
533
|
-
|
|
994
|
+
print(f"\n❌ Playwright 获取失败: {e}")
|
|
995
|
+
return False
|
|
534
996
|
finally:
|
|
535
997
|
if env_bool("CONFLUENCE_DEBUG_PAUSE", default=not headless):
|
|
536
998
|
try:
|
|
@@ -27,6 +27,16 @@ python3 confluence-doc.py <版本号>
|
|
|
27
27
|
|
|
28
28
|
说明:脚本内置默认账号密码(`lixinxin` / `Xin@147258`),可直接执行;如需覆盖,再设置 `CONFLUENCE_USERNAME` / `CONFLUENCE_PASSWORD`。
|
|
29
29
|
|
|
30
|
+
**下载策略(v1.3.4+)**:默认 **Playwright 登录 + 搜索** 拉取 PRD(主路径,不依赖会过期的 Token)。仅当 Playwright 失败且显式设置 `CONFLUENCE_USE_REST=1` 时才回退 REST API。
|
|
31
|
+
|
|
32
|
+
```bash
|
|
33
|
+
# 有头模式排查登录(推荐)
|
|
34
|
+
CONFLUENCE_HEADED=1 CONFLUENCE_SLOW_MO=500 CONFLUENCE_DEBUG_PAUSE=1 python3 confluence-doc.py 2.3.4
|
|
35
|
+
|
|
36
|
+
# 可选:Playwright 失败后回退 Token(不推荐作默认)
|
|
37
|
+
CONFLUENCE_USE_REST=1 python3 confluence-doc.py 2.3.4
|
|
38
|
+
```
|
|
39
|
+
|
|
30
40
|
有头调试(可观察登录与搜索过程):
|
|
31
41
|
|
|
32
42
|
```bash
|
|
@@ -48,8 +58,17 @@ python3 confluence-doc.py {{PRODUCT_PREFIX}}_V1.2.3
|
|
|
48
58
|
2. `{{PRODUCT_PREFIX}}-V<版本>`(如 ADI-V2.3.2)
|
|
49
59
|
3. `{{PRODUCT_PREFIX}}_<版本>`(如 ADI_2.3.2)
|
|
50
60
|
4. `{{PRODUCT_PREFIX}}-<版本>`(如 ADI-2.3.2)
|
|
61
|
+
5. `ADI_V<版本>结算`(ADI 长标题页,如 V2.3.4 结算对接欢聚)
|
|
51
62
|
|
|
52
|
-
|
|
63
|
+
每个关键词都会走 **dosearchsite 全站搜索** + 顶栏搜索;匹配时忽略 `-`/`_` 差异;ADI 页按 `ADI_V*` 标题规则识别(面包屑可无 PRD 字样)。
|
|
64
|
+
|
|
65
|
+
## 直链(跳过搜索)
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
CONFLUENCE_PAGE_ID=109609740 python3 confluence-doc.py 2.3.4
|
|
69
|
+
# 或
|
|
70
|
+
CONFLUENCE_PAGE_URL="https://confluence.huan.tv/spaces/yyzcpt/pages/109609740/..." python3 confluence-doc.py 2.3.4
|
|
71
|
+
```
|
|
53
72
|
|
|
54
73
|
## 输出结构
|
|
55
74
|
|