sino-bc-cli 1.8.23 → 1.8.24
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/index.js +1 -1
- package/dist/skills/custom/biz-line-strategy-generate/SKILL.md +3 -2
- package/dist/skills/custom/biz-line-strategy-generate/references/api-flow.md +1 -1
- package/dist/skills/custom/biz-line-strategy-generate/references/output-excel.md +1 -1
- package/dist/skills/custom/biz-line-strategy-generate/references/runtime.md +3 -3
- package/dist/skills/custom/biz-line-strategy-generate/references/spec.md +9 -2
- package/dist/skills/custom/biz-line-strategy-generate/scripts/build_report.py +225 -79
- package/dist/skills/custom/biz-line-strategy-generate/scripts/cli_json.py +91 -25
- package/dist/skills/custom/biz-line-strategy-generate/scripts/fetch_data.py +8 -5
- package/dist/skills/custom/biz-line-strategy-generate/scripts/summarize.py +588 -400
- package/dist/skills/custom/biz-line-strategy-generate/scripts/test_strategy_rules.py +53 -6
- package/dist/skills/custom/biz-line-strategy-generate/scripts/test_workflow.py +168 -6
- package/package.json +1 -1
|
@@ -28,7 +28,7 @@ metadata:
|
|
|
28
28
|
使用本技能目录的 **`scripts/run_workflow.py`**,不要临时重写拉数、分析或排版脚本。执行前定位技能的绝对路径;Python 只需标准库。首次使用读取 [`references/runtime.md`](references/runtime.md)。
|
|
29
29
|
|
|
30
30
|
1. 用户给名称,传 `--name`;命中多个候选则返回歧义。用户给多个 ID,重复传 `--id`。用户要第一页/前 N 条,先用既有 `biz-line +list --data` 选出真实 ID,再在**一次入口调用**中传入。
|
|
31
|
-
2. 入口默认最多 **3 条业务线并发**,允许 `--concurrency 1` 或 `2`,不能大于 3。同一批次不要并排启动多个入口绕开限制;单条内部 CLI 请求和分页串行。分页回包只落到本次临时目录的 `*.pN.out.json`,由脚本写 `*.pages.json` 索引;**禁止**在对话里重写拉数脚本、把各页 `extend` 进内存或再拼一份 `merged.json`。分析按页迭代,不要把全表读进 list。
|
|
31
|
+
2. 入口默认最多 **3 条业务线并发**,允许 `--concurrency 1` 或 `2`,不能大于 3。同一批次不要并排启动多个入口绕开限制;单条内部 CLI 请求和分页串行。分页回包只落到本次临时目录的 `*.pN.out.json`,由脚本写 `*.pages.json` 索引;**禁止**在对话里重写拉数脚本、把各页 `extend` 进内存或再拼一份 `merged.json`。分析按页迭代,不要把全表读进 list。明细写 `details.ndjson`,出价按广告组落盘外排;不要把明细或出价快照再攒成大 list。
|
|
32
32
|
3. 每条自动执行现拉 → 固定分析 → 一次性排版 → Excel 回读校验 → 发布最终文件 → 清理。默认接口周期;用户给日期时成对传 `--curr-start/--curr-end`、`--prev-start/--prev-end`。
|
|
33
33
|
4. 每条成功后立即交付**该业务线自己的最终 Excel**,并删除本次其他临时产物,包括 JSON、请求/回包、日志副本、图片、临时脚本、缓存、无扩展名文件及临时 Excel,不等全部业务线结束。
|
|
34
34
|
5. 失败任务返回阶段和错误摘要,清理自己的临时文件,其他业务线继续。清理失败须报告残留路径,不能声称已清理。只给实际成功返回的 `output_path` 链接;不用已删除的请求目录作交付物。
|
|
@@ -69,7 +69,8 @@ python3 -B <本技能绝对路径>/scripts/run_workflow.py --id <真实ID1> --id
|
|
|
69
69
|
- 未绑定项目/账号不要编造。多项目仍一行,名称 `\n` 分隔。
|
|
70
70
|
- 文案用中文/`display_*`;词包只写「新增/本期未出现 N 个」+ 最多 3 短词。
|
|
71
71
|
- learning 第 1 层只写消耗 ≥$50 的新增。
|
|
72
|
-
- CLI 回包必须直写文件(`exec_cli_to_file
|
|
72
|
+
- CLI 回包必须直写文件(`exec_cli_to_file` 只返回退出码),`capture_output` 会在约 64KB 截断。失败摘要只读文件头,不要 `read_text` 整页。每页是 `stem.pN.out.json`,拉数成功才写 `stem.pages.json`;不要写全表 `merged.json`,也不要手写翻页把结果攒在内存里。翻页上限用脚本 `PAGE_CAP`(≥80),不要手写 30 页就停。
|
|
73
|
+
- `payload.json` 只记明细路径和条数,不要再嵌入全量 `details` 数组。Excel 明细表按 ndjson 流式写出,校验按行回读,不要先展开成 list 再进 SST。
|
|
73
74
|
- 每条业务线只留一份最终 Excel;禁止按桌面 `业务线策略生成_*.xlsx` glob 删除。
|
|
74
75
|
|
|
75
76
|
## References
|
|
@@ -94,7 +94,7 @@ sino-bc-cli biz-line api biz-line-project-list --data '{"bizLineId": 1251}'
|
|
|
94
94
|
|
|
95
95
|
## 分页与数据量
|
|
96
96
|
|
|
97
|
-
维度/成效接口默认每页 10 条(`size`),脚本显式 `size=1000`,按 `data.total` 翻页,上限 `PAGE_CAP=80`。每一页回包直写 `stem.pN.out.json`,全部页成功后写 `stem.pages.json`;不要把各页拼成内存全表或 `merged.json`。分析按页 yield
|
|
97
|
+
维度/成效接口默认每页 10 条(`size`),脚本显式 `size=1000`,按 `data.total` 翻页,上限 `PAGE_CAP=80`。每一页回包直写 `stem.pN.out.json`,全部页成功后写 `stem.pages.json`;不要把各页拼成内存全表或 `merged.json`。分析按页 yield,占比和比率一次扫描。明细写 `details.ndjson`,出价按广告组落盘外排。TikTok 出价按天可达 4~5 万行,并发多条业务线时尤其不能在 Python 里 `extend` 全量。每次现拉,不要跳过已有页;最终 Excel 校验发布后立即清理该条全部临时文件。
|
|
98
98
|
|
|
99
99
|
## 汇总到 Excel 前的数据整理
|
|
100
100
|
|
|
@@ -35,7 +35,7 @@
|
|
|
35
35
|
|
|
36
36
|
## 输出与校验
|
|
37
37
|
|
|
38
|
-
|
|
38
|
+
先在本条临时目录生成一次最终样式:策略表走共享字符串,明细表按 `details.ndjson` 流式写出(单元格用 inlineStr),不要先把全部明细 intern 进 SST。通过 ZIP、两张表、列名及逐单元格完整性校验后,以独占创建方式发布;明细校验按行回读,再次回读验证成功才交付。最终目录必须在批次临时根外。
|
|
39
39
|
|
|
40
40
|
默认最终目录为桌面 `~/Desktop`,不存在时回退用户主目录,可用 `--output-dir` 指定。文件名为 `业务线策略_<ID>_<名称>_<随机标识>.xlsx`,同名目标拒绝覆盖。
|
|
41
41
|
|
|
@@ -7,9 +7,9 @@
|
|
|
7
7
|
- 默认同时处理至多 3 条业务线,可显式设置 1 或 2。单条从查询到清理占一个名额,清理结束即补位。同一批次不能拆成多个入口绕过上限;不做跨会话全局调度。
|
|
8
8
|
- 单条内部接口、渠道和分页串行;只通过已安装 `sino-bc-cli`,不手写 HTTP。
|
|
9
9
|
- Windows 通过已安装 CLI 对应 Node 入口传参数数组,避免 cmd 重新解释 JSON。权限错误应报告,不当成未安装去自动下载。
|
|
10
|
-
- `--format json`,请求用 `--data`。成功必须同时满足退出码 0 和信封 `ok == true`;stderr 与 stdout
|
|
10
|
+
- `--format json`,请求用 `--data`。成功必须同时满足退出码 0 和信封 `ok == true`;stderr 与 stdout 都直写本次文件。`exec_cli_to_file` 只返回退出码,不得把 stdout 再读成字符串;失败摘要只读文件头。拉数回读用 `drop_rows`,返回值里只留 `total`/行数。
|
|
11
11
|
- 列表行为 `data.result`,项目为 `data` 数组,维度/成效为 `data.data` 和 `data.total`。分页 `size=1000`、`PAGE_CAP=80`;总数未取全、提前空页、失败或总数变化均中止该条,不导出部分成功文件。
|
|
12
|
-
- 维度/成效每一页 MUST 直写 `bl-<id>/<stem>.pN.out.json
|
|
12
|
+
- 维度/成效每一页 MUST 直写 `bl-<id>/<stem>.pN.out.json`。拉数在内存中只保留当前页和行数累计,统计完本页行数后丢掉信封中的行 list,成功后才写 `<stem>.pages.json`(`total` / `pages` / `counts`)。开始前若已有本 stem 的 `pages.json` 必须先删掉。MUST NOT 把各页 `extend` 成全表再写 `merged.json`。汇总按 `pages.json` 逐页 yield,占比/比率/覆盖日期一次扫描,不得为分析再拼全表 list。明细逐条写 `details.ndjson`,`payload.json` 只记路径;出价按广告组落盘外排。无索引却有 `p2` 及之后页时失败。清单校验只对索引上的 `total`/`counts`,不得为校验再整表加载。失败页 MUST NOT 写下完整 `pages.json`。
|
|
13
13
|
|
|
14
14
|
## 目录归属
|
|
15
15
|
|
|
@@ -18,7 +18,7 @@
|
|
|
18
18
|
.strategy-owner.json
|
|
19
19
|
line-<业务线ID>-<随机标识>/
|
|
20
20
|
.strategy-owner.json
|
|
21
|
-
请求体、stderr、分页回包 <stem>.pN.out.json、索引 <stem>.pages.json
|
|
21
|
+
请求体、stderr、分页回包 <stem>.pN.out.json、索引 <stem>.pages.json、details.ndjson、出价外排、临时 Excel
|
|
22
22
|
```
|
|
23
23
|
|
|
24
24
|
每次现拉,新建目录,不复用历史缓存。所有本次临时文件写入本次目录;不向技能源码树写缓存。最终 Excel 写在批次临时根以外,文件名包含业务线 ID 和随机请求标识,独占创建、不覆盖已有文件。
|
|
@@ -37,7 +37,7 @@
|
|
|
37
37
|
|
|
38
38
|
### Requirement: 分页与失败回包必须完整校验
|
|
39
39
|
|
|
40
|
-
每页 CLI 回包 MUST 直写 `<stem>.pN.out.json`
|
|
40
|
+
每页 CLI 回包 MUST 直写 `<stem>.pN.out.json` 并检查退出码、信封和分页总数。`exec_cli_to_file` MUST 只返回退出码,MUST NOT 把 stdout 再读成字符串返回。`run_cli` 在失败摘要中 MUST 只读文件头部,MUST NOT 再 `read_text` 整页。JSON 解析失败 MUST NOT 把文件头之后埋着的对象当成成功回包。拉数回读 MUST 在返回前丢掉信封中的行 list,只保留 `total` 与本页行数。拉数成功后 MUST 写 `<stem>.pages.json`(含 `total`、`pages`、每页 `counts`)。拉数过程 MUST 只在内存保留当前页与行数累计,MUST NOT 将各页累积为全表 list,MUST NOT 再写一份全表 `merged.json`。汇总 MUST 按 `pages.json` 顺序逐页 yield 行;`pages.json` 缺失但存在 `p2` 及之后页文件时 MUST 失败,不得只读第一页。`share_map` / `metric_value` / 覆盖日期 / 出价快照 MUST 一次扫描完成,MUST NOT 为占比或比率把全表物化成 list。新增对象识别 MUST 只为新增行保留明细,持续对象 MUST NOT 再攒成全表。生产分析 MUST 把明细逐条写入 `details.ndjson`,`payload.json` MUST 只记路径与条数,MUST NOT 再嵌入全量明细数组。生产路径的 `bid_changes` MUST 按广告组把日快照落盘并外排序,MUST NOT 把全渠道出价快照同时留在内存 map。开始分析前的清单校验 MUST 只核对该索引与 manifest 的 `total`/`counts`,MUST NOT 为校验整表加载。广告/广告组命名规律 MUST 只用原始展示名,MUST NOT 把空名回退成 ID 再拆词。异常、解析失败、未到总数即遇空页、达到分页上限仍未取全,MUST 阻止该业务线生成成功报告,且 MUST NOT 写下完整 `pages.json`。CLI stderr 错误 MUST 保留到本次任务结果摘要,而非当作无数据。其他业务线 MUST 可以继续执行。
|
|
41
41
|
|
|
42
42
|
#### Scenario: 后续页失败不导出部分结果
|
|
43
43
|
- **GIVEN** 首页成功且声明仍有下一页
|
|
@@ -51,6 +51,8 @@
|
|
|
51
51
|
- **AND** 不存在 `stem.merged.json`
|
|
52
52
|
- **AND** 汇总能从页文件读出这 3 行
|
|
53
53
|
- **AND** `share_map` / `metric_value` 对一次性迭代器也能得到与 list 相同的占比和比率
|
|
54
|
+
- **AND** 生产分析写出 `details.ndjson`,`payload.json` 只含路径而不含明细数组
|
|
55
|
+
- **AND** 同一组出价快照在落盘外排与内存路径下得到相同结果
|
|
54
56
|
|
|
55
57
|
### Requirement: 指标与周期口径固定
|
|
56
58
|
|
|
@@ -84,7 +86,7 @@
|
|
|
84
86
|
|
|
85
87
|
### Requirement: 每条业务线只交付一份校验通过的最终 Excel
|
|
86
88
|
|
|
87
|
-
每条业务线 MUST 独立产出一份包含固定 12 列策略摘要及完整明细的 Excel。分析与排版 MUST 由固定脚本完成;正常流程 MUST 一次生成最终样式,MUST NOT 依赖 Agent
|
|
89
|
+
每条业务线 MUST 独立产出一份包含固定 12 列策略摘要及完整明细的 Excel。分析与排版 MUST 由固定脚本完成;正常流程 MUST 一次生成最终样式,MUST NOT 依赖 Agent 临时生成修订脚本或二次导入重排。导出明细表 MUST 按 ndjson 或等价迭代器流式写入,MUST NOT 先把全部明细展开成 list 再写入共享字符串表。文件发布前 MUST 检查 Excel 容器、两张表、列名及记录完整性;明细表校验 MUST 按行回读对比,MUST NOT 先 `fromstring` 整张明细表。最终路径 MUST 位于待清理工作目录之外,名称 MUST 能区分业务线与请求,MUST NOT 覆盖已存在的其他文件。
|
|
88
90
|
|
|
89
91
|
#### Scenario: 并发生成不覆盖结果
|
|
90
92
|
- **GIVEN** 两条业务线在同一秒完成导出
|
|
@@ -96,6 +98,11 @@
|
|
|
96
98
|
- **WHEN** 执行发布前校验
|
|
97
99
|
- **THEN** 该任务失败,不对外声明最终 Excel 已生成
|
|
98
100
|
|
|
101
|
+
#### Scenario: 明细走 ndjson 导出
|
|
102
|
+
- **GIVEN** 明细是两行 ndjson 文件
|
|
103
|
+
- **WHEN** 导出并校验 Excel
|
|
104
|
+
- **THEN** 明细表行数为 2,且不要求调用方先把文件读成 list
|
|
105
|
+
|
|
99
106
|
### Requirement: 任务结束后按归属清理全部临时产物
|
|
100
107
|
|
|
101
108
|
每条业务线任务完成后,MUST 清理本次任务创建的全部非最终产物,包括但不限于 JSON、请求/响应、日志副本、预览图片、临时脚本、临时 Excel、检查文件及运行时缓存。成功时 MUST 先确认最终文件已发布且验证通过,再清理;失败时 MUST 先保存可返回的错误摘要到内存/输出,再清理其临时和不完整产物。所有任务结束后 MUST 清理本次批次共享临时目录。MUST NOT 保留额外磁盘分析报告或将已删除目录作为交付物。
|
|
@@ -14,16 +14,8 @@ payload.json 结构:
|
|
|
14
14
|
{
|
|
15
15
|
"output_path": "可选,未传时默认写到桌面 业务线策略生成_YYYYMMDD_HHMMSS.xlsx",
|
|
16
16
|
"rows": [ { "...策略摘要行..." } ],
|
|
17
|
-
"details":
|
|
18
|
-
|
|
19
|
-
"bizLine": "业务线",
|
|
20
|
-
"kind": "国家|广告组|出价|兴趣词|关键词|广告",
|
|
21
|
-
"action": "新增|本期未出现|上调|下调|当期",
|
|
22
|
-
"channel": "Meta|Google|TikTok",
|
|
23
|
-
"name": "名称",
|
|
24
|
-
"extra": "补充"
|
|
25
|
-
}
|
|
26
|
-
]
|
|
17
|
+
"details": "details.ndjson 的路径(兼容旧调用时也可以是明细对象数组)",
|
|
18
|
+
"detail_count": 13
|
|
27
19
|
}
|
|
28
20
|
|
|
29
21
|
列固定为:组别、业务线、本期/上期、项目名称、
|
|
@@ -41,6 +33,7 @@ import zipfile
|
|
|
41
33
|
import math
|
|
42
34
|
import unicodedata
|
|
43
35
|
import uuid
|
|
36
|
+
from itertools import chain, zip_longest
|
|
44
37
|
from pathlib import Path
|
|
45
38
|
from xml.etree import ElementTree as ET
|
|
46
39
|
from datetime import datetime
|
|
@@ -160,20 +153,13 @@ def _intern_factory():
|
|
|
160
153
|
return intern, order
|
|
161
154
|
|
|
162
155
|
|
|
163
|
-
def
|
|
164
|
-
"""
|
|
156
|
+
def build_strategy_strings(rows):
|
|
157
|
+
"""只为策略表建 SST。明细走 inlineStr,避免把几万行广告名再进共享字符串。"""
|
|
165
158
|
intern, order = _intern_factory()
|
|
166
159
|
header_indices = [intern(label) for _key, label in COLUMNS]
|
|
167
160
|
row_indices = [
|
|
168
161
|
[intern(str(row.get(key, "") or "")) for key, _label in COLUMNS] for row in rows
|
|
169
162
|
]
|
|
170
|
-
detail_header = [intern(label) for _key, label in DETAIL_COLUMNS]
|
|
171
|
-
detail_rows = []
|
|
172
|
-
for row in details or []:
|
|
173
|
-
detail_rows.append(
|
|
174
|
-
[intern(str(row.get(key, "") or "")) for key, _label in DETAIL_COLUMNS]
|
|
175
|
-
)
|
|
176
|
-
|
|
177
163
|
# XML 会把原始 CR/CRLF 归一化为 LF;用字符引用保留客户文本,保证逐字回读。
|
|
178
164
|
items = "".join(
|
|
179
165
|
f'<si><t xml:space="preserve">{xml_escape(text, {chr(13): " "})}</t></si>' for text in order
|
|
@@ -183,7 +169,13 @@ def build_shared_strings(rows, details=None):
|
|
|
183
169
|
f'<sst xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main" '
|
|
184
170
|
f'count="{len(order)}" uniqueCount="{len(order)}">{items}</sst>'
|
|
185
171
|
)
|
|
186
|
-
return sst_xml, header_indices, row_indices
|
|
172
|
+
return sst_xml, header_indices, row_indices
|
|
173
|
+
|
|
174
|
+
|
|
175
|
+
def build_shared_strings(rows, details=None):
|
|
176
|
+
"""兼容旧调用:策略表走 SST;明细参数忽略,导出不再把明细 intern。"""
|
|
177
|
+
sst_xml, header_indices, row_indices = build_strategy_strings(rows)
|
|
178
|
+
return sst_xml, header_indices, row_indices, [], []
|
|
187
179
|
|
|
188
180
|
|
|
189
181
|
def build_sheet_xml(header_indices, row_indices, columns, content_keys=None, multiline_keys=None, values=None):
|
|
@@ -275,38 +267,148 @@ def chunks(text, width, max_lines=20):
|
|
|
275
267
|
return result + [part] if part or not result else result
|
|
276
268
|
|
|
277
269
|
|
|
278
|
-
def
|
|
279
|
-
"""
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
|
|
270
|
+
def iter_detail_records(details):
|
|
271
|
+
"""明细来源:内存 list 或 ndjson 路径。按行 yield,不把文件读成一张大表。"""
|
|
272
|
+
if details is None:
|
|
273
|
+
return
|
|
274
|
+
if isinstance(details, (str, Path)):
|
|
275
|
+
path = Path(details)
|
|
276
|
+
if not path.exists():
|
|
277
|
+
return
|
|
278
|
+
with path.open(encoding="utf-8") as handle:
|
|
279
|
+
for line in handle:
|
|
280
|
+
if line.strip():
|
|
281
|
+
yield json.loads(line)
|
|
282
|
+
return
|
|
283
|
+
for record in details:
|
|
284
|
+
yield record
|
|
285
|
+
|
|
286
|
+
|
|
287
|
+
def trim_strategy_rows(rows):
|
|
288
|
+
"""主表只留可见摘要;超长原文变成「完整说明」明细。策略行很少,可在内存处理。"""
|
|
289
|
+
trimmed = [dict(row) for row in rows]
|
|
290
|
+
overflow: list[dict] = []
|
|
291
|
+
for row in trimmed:
|
|
283
292
|
for key, label in COLUMNS:
|
|
284
|
-
text = str(row.get(key) or
|
|
285
|
-
width = COL_WIDTH_CONTENT if key in CONTENT_KEYS else (
|
|
293
|
+
text = str(row.get(key) or "")
|
|
294
|
+
width = COL_WIDTH_CONTENT if key in CONTENT_KEYS else (
|
|
295
|
+
COL_WIDTH_MULTILINE if key in MULTILINE_KEYS else COL_WIDTH_DEFAULT
|
|
296
|
+
)
|
|
286
297
|
parts = chunks(text, width, 18)
|
|
287
298
|
if len(parts) > 1:
|
|
288
|
-
|
|
289
|
-
|
|
299
|
+
overflow.append(
|
|
300
|
+
{
|
|
301
|
+
"bizLine": row.get("bizLine", ""),
|
|
302
|
+
"kind": "完整说明",
|
|
303
|
+
"action": label,
|
|
304
|
+
"name": label,
|
|
305
|
+
"extra": text,
|
|
306
|
+
}
|
|
307
|
+
)
|
|
308
|
+
row[key] = parts[0] + "\n(完整内容见明细/完整说明)"
|
|
309
|
+
return trimmed, overflow
|
|
310
|
+
|
|
311
|
+
|
|
312
|
+
def expand_detail_record(record):
|
|
313
|
+
"""单条明细按列宽拆续行。一次只处理一条,供流式写出。"""
|
|
314
|
+
split = {
|
|
315
|
+
key: chunks(
|
|
316
|
+
str(record.get(key) or ""),
|
|
317
|
+
COL_WIDTH_CONTENT if key in ("name", "extra") else COL_WIDTH_DEFAULT,
|
|
318
|
+
18,
|
|
319
|
+
)
|
|
320
|
+
for key, _ in DETAIL_COLUMNS
|
|
321
|
+
}
|
|
322
|
+
count = max(map(len, split.values()))
|
|
290
323
|
expanded = []
|
|
291
|
-
for
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
324
|
+
for index in range(count):
|
|
325
|
+
# 续行有行号标识,身份短字段重复展示便于筛选;长字段逐段保留。
|
|
326
|
+
row = {
|
|
327
|
+
key: (
|
|
328
|
+
parts[index]
|
|
329
|
+
if index < len(parts)
|
|
330
|
+
else (parts[0] if len(parts) == 1 and key not in ("name", "extra") else "")
|
|
331
|
+
)
|
|
332
|
+
for key, parts in split.items()
|
|
333
|
+
}
|
|
334
|
+
if count > 1:
|
|
335
|
+
row["action"] += f" [续 {index + 1}/{count}]"
|
|
336
|
+
expanded.append(row)
|
|
337
|
+
return expanded
|
|
338
|
+
|
|
339
|
+
|
|
340
|
+
def prepare_layout(rows, details):
|
|
341
|
+
"""兼容旧测试:仍返回两张表的 list。生产写 xlsx 走流式路径,不要用这个物化明细。"""
|
|
342
|
+
trimmed, overflow = trim_strategy_rows(rows)
|
|
343
|
+
expanded = []
|
|
344
|
+
for record in chain(overflow, iter_detail_records(details)):
|
|
345
|
+
expanded.extend(expand_detail_record(record))
|
|
346
|
+
return trimmed, expanded
|
|
347
|
+
|
|
348
|
+
|
|
349
|
+
def _inline_t(text: str) -> str:
|
|
350
|
+
return xml_escape(str(text or ""), {chr(13): " "})
|
|
351
|
+
|
|
352
|
+
|
|
353
|
+
def _local_tag(tag: str) -> str:
|
|
354
|
+
return tag.rsplit("}", 1)[-1]
|
|
355
|
+
|
|
356
|
+
|
|
357
|
+
def _detail_row_xml(row_num: int, record: dict, style_id: int = 2) -> str:
|
|
358
|
+
cells = []
|
|
359
|
+
lines = 1
|
|
360
|
+
for i, (key, _) in enumerate(DETAIL_COLUMNS):
|
|
361
|
+
text = str(record.get(key) or "")
|
|
362
|
+
width = COL_WIDTH_CONTENT if key in ("name", "extra") else COL_WIDTH_DEFAULT
|
|
363
|
+
lines = max(lines, visual_lines(text, width))
|
|
364
|
+
ref = f"{col_letter(i)}{row_num}"
|
|
365
|
+
cells.append(
|
|
366
|
+
f'<c r="{ref}" s="{style_id}" t="inlineStr"><is><t xml:space="preserve">{_inline_t(text)}</t></is></c>'
|
|
367
|
+
)
|
|
368
|
+
height = 32 if row_num == 1 else min(409, max(30, lines * 15 + 10))
|
|
369
|
+
return f'<row r="{row_num}" ht="{height}" customHeight="1">{"".join(cells)}</row>'
|
|
370
|
+
|
|
371
|
+
|
|
372
|
+
def write_detail_sheet(records, dest: Path) -> int:
|
|
373
|
+
"""先写数据行再包 worksheet,避免先攒整张明细 XML。"""
|
|
374
|
+
body = dest.with_suffix(".body.xml")
|
|
375
|
+
count = 0
|
|
376
|
+
with body.open("w", encoding="utf-8") as handle:
|
|
377
|
+
for record in records:
|
|
378
|
+
for expanded in expand_detail_record(record):
|
|
379
|
+
count += 1
|
|
380
|
+
handle.write(_detail_row_xml(count + 1, expanded))
|
|
381
|
+
last = count + 1
|
|
382
|
+
dim = f"A1:{col_letter(len(DETAIL_COLUMNS) - 1)}{last}"
|
|
383
|
+
cols = "".join(
|
|
384
|
+
f'<col min="{i + 1}" max="{i + 1}" width="'
|
|
385
|
+
f'{COL_WIDTH_CONTENT if key in ("name", "extra") else COL_WIDTH_DEFAULT}" customWidth="1"/>'
|
|
386
|
+
for i, (key, _) in enumerate(DETAIL_COLUMNS)
|
|
387
|
+
)
|
|
388
|
+
header = _detail_row_xml(1, {key: label for key, label in DETAIL_COLUMNS}, 1)
|
|
389
|
+
with dest.open("w", encoding="utf-8") as out, body.open(encoding="utf-8") as src:
|
|
390
|
+
out.write('<?xml version="1.0" encoding="UTF-8" standalone="yes"?>')
|
|
391
|
+
out.write('<worksheet xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main">')
|
|
392
|
+
out.write(f'<dimension ref="{dim}"/>')
|
|
393
|
+
out.write('<sheetViews><sheetView workbookViewId="0">')
|
|
394
|
+
out.write('<pane ySplit="1" topLeftCell="A2" activePane="bottomLeft" state="frozen"/>')
|
|
395
|
+
out.write(f"</sheetView></sheetViews><cols>{cols}</cols><sheetData>")
|
|
396
|
+
out.write(header)
|
|
397
|
+
while True:
|
|
398
|
+
chunk = src.read(262144)
|
|
399
|
+
if not chunk:
|
|
400
|
+
break
|
|
401
|
+
out.write(chunk)
|
|
402
|
+
out.write(f'</sheetData><autoFilter ref="{dim}"/></worksheet>')
|
|
403
|
+
body.unlink(missing_ok=True)
|
|
404
|
+
return count
|
|
301
405
|
|
|
302
406
|
|
|
303
407
|
def write_xlsx(rows, output_path: str, details=None) -> str:
|
|
304
408
|
if not rows:
|
|
305
|
-
raise ValueError(
|
|
306
|
-
rows,
|
|
307
|
-
sst_xml, header_indices, row_indices
|
|
308
|
-
rows, details
|
|
309
|
-
)
|
|
409
|
+
raise ValueError("策略数据为空")
|
|
410
|
+
rows, overflow = trim_strategy_rows(rows)
|
|
411
|
+
sst_xml, header_indices, row_indices = build_strategy_strings(rows)
|
|
310
412
|
sheet_xml = build_sheet_xml(
|
|
311
413
|
header_indices,
|
|
312
414
|
row_indices,
|
|
@@ -315,50 +417,89 @@ def write_xlsx(rows, output_path: str, details=None) -> str:
|
|
|
315
417
|
multiline_keys=MULTILINE_KEYS,
|
|
316
418
|
values=rows,
|
|
317
419
|
)
|
|
318
|
-
detail_xml = build_sheet_xml(
|
|
319
|
-
detail_header,
|
|
320
|
-
detail_rows,
|
|
321
|
-
DETAIL_COLUMNS,
|
|
322
|
-
content_keys={"name", "extra"},
|
|
323
|
-
multiline_keys=set(),
|
|
324
|
-
values=details,
|
|
325
|
-
)
|
|
326
420
|
|
|
327
421
|
out_dir = os.path.dirname(output_path)
|
|
328
422
|
if out_dir and not os.path.isdir(out_dir):
|
|
329
423
|
os.makedirs(out_dir, exist_ok=True)
|
|
330
424
|
|
|
331
|
-
|
|
332
|
-
|
|
333
|
-
|
|
334
|
-
|
|
335
|
-
|
|
336
|
-
|
|
337
|
-
|
|
338
|
-
|
|
339
|
-
|
|
425
|
+
sheet2_path = Path(output_path).with_name(Path(output_path).name + ".sheet2.xml")
|
|
426
|
+
try:
|
|
427
|
+
write_detail_sheet(chain(overflow, iter_detail_records(details)), sheet2_path)
|
|
428
|
+
with zipfile.ZipFile(output_path, "x", zipfile.ZIP_DEFLATED) as zf:
|
|
429
|
+
zf.writestr("[Content_Types].xml", CONTENT_TYPES_XML)
|
|
430
|
+
zf.writestr("_rels/.rels", ROOT_RELS_XML)
|
|
431
|
+
zf.writestr("xl/workbook.xml", WORKBOOK_XML)
|
|
432
|
+
zf.writestr("xl/_rels/workbook.xml.rels", WORKBOOK_RELS_XML)
|
|
433
|
+
zf.writestr("xl/styles.xml", STYLES_XML)
|
|
434
|
+
zf.writestr("xl/sharedStrings.xml", sst_xml)
|
|
435
|
+
zf.writestr("xl/worksheets/sheet1.xml", sheet_xml)
|
|
436
|
+
zf.write(sheet2_path, "xl/worksheets/sheet2.xml")
|
|
437
|
+
finally:
|
|
438
|
+
sheet2_path.unlink(missing_ok=True)
|
|
439
|
+
sheet2_path.with_suffix(".body.xml").unlink(missing_ok=True)
|
|
340
440
|
|
|
341
441
|
return output_path
|
|
342
442
|
|
|
343
443
|
|
|
444
|
+
def _cell_text(cell, strings, ns) -> str:
|
|
445
|
+
kind = cell.get("t")
|
|
446
|
+
if kind == "inlineStr":
|
|
447
|
+
return "".join(cell.itertext())
|
|
448
|
+
value = cell.find("s:v", ns)
|
|
449
|
+
if value is None or value.text is None:
|
|
450
|
+
return ""
|
|
451
|
+
if kind == "s":
|
|
452
|
+
return strings[int(value.text)]
|
|
453
|
+
return value.text
|
|
454
|
+
|
|
455
|
+
|
|
456
|
+
def _iter_sheet_rows(archive, sheet_name, strings, ns):
|
|
457
|
+
"""按行解析 worksheet,读完一行就丢掉 DOM。"""
|
|
458
|
+
with archive.open(sheet_name) as handle:
|
|
459
|
+
for _event, elem in ET.iterparse(handle, events=("end",)):
|
|
460
|
+
if _local_tag(elem.tag) != "row":
|
|
461
|
+
continue
|
|
462
|
+
cells = [child for child in elem if _local_tag(child.tag) == "c"]
|
|
463
|
+
yield [_cell_text(cell, strings, ns) for cell in cells]
|
|
464
|
+
elem.clear()
|
|
465
|
+
|
|
466
|
+
|
|
344
467
|
def validate_xlsx(path, rows, details):
|
|
345
|
-
"""
|
|
346
|
-
ns = {
|
|
347
|
-
|
|
468
|
+
"""逐单元格回读。策略表可整表比对;明细表按行对比,不先展开全表。"""
|
|
469
|
+
ns = {"s": "http://schemas.openxmlformats.org/spreadsheetml/2006/main"}
|
|
470
|
+
expected_rows, overflow = trim_strategy_rows(rows)
|
|
348
471
|
with zipfile.ZipFile(path) as archive:
|
|
349
472
|
if archive.testzip():
|
|
350
|
-
raise ValueError(
|
|
351
|
-
workbook = ET.fromstring(archive.read(
|
|
352
|
-
if [s.attrib[
|
|
353
|
-
raise ValueError(
|
|
354
|
-
strings = [
|
|
355
|
-
|
|
356
|
-
|
|
357
|
-
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
|
|
473
|
+
raise ValueError("Excel ZIP 校验失败")
|
|
474
|
+
workbook = ET.fromstring(archive.read("xl/workbook.xml"))
|
|
475
|
+
if [s.attrib["name"] for s in workbook.findall("s:sheets/s:sheet", ns)] != ["策略", "明细"]:
|
|
476
|
+
raise ValueError("Excel 必须包含策略和明细两张表")
|
|
477
|
+
strings = ["".join(item.itertext()) for item in ET.fromstring(archive.read("xl/sharedStrings.xml"))]
|
|
478
|
+
actual1 = list(_iter_sheet_rows(archive, "xl/worksheets/sheet1.xml", strings, ns))
|
|
479
|
+
target1 = [[label for _, label in COLUMNS]] + [
|
|
480
|
+
[str(record.get(key) or "") for key, _ in COLUMNS] for record in expected_rows
|
|
481
|
+
]
|
|
482
|
+
if actual1 != target1:
|
|
483
|
+
raise ValueError("Excel 第 1 张表内容或行列数不完整")
|
|
484
|
+
actual2 = _iter_sheet_rows(archive, "xl/worksheets/sheet2.xml", strings, ns)
|
|
485
|
+
header = next(actual2, None)
|
|
486
|
+
if header != [label for _, label in DETAIL_COLUMNS]:
|
|
487
|
+
raise ValueError("Excel 第 2 张表内容或行列数不完整")
|
|
488
|
+
missing = object()
|
|
489
|
+
expected2 = (
|
|
490
|
+
expanded
|
|
491
|
+
for record in chain(overflow, iter_detail_records(details))
|
|
492
|
+
for expanded in expand_detail_record(record)
|
|
493
|
+
)
|
|
494
|
+
count = 0
|
|
495
|
+
for exp, act in zip_longest(expected2, actual2, fillvalue=missing):
|
|
496
|
+
if exp is missing or act is missing:
|
|
497
|
+
raise ValueError("Excel 第 2 张表内容或行列数不完整")
|
|
498
|
+
target = [str(exp.get(key) or "") for key, _ in DETAIL_COLUMNS]
|
|
499
|
+
if act != target:
|
|
500
|
+
raise ValueError("Excel 第 2 张表内容或行列数不完整")
|
|
501
|
+
count += 1
|
|
502
|
+
return {"rows": len(expected_rows), "details": count}
|
|
362
503
|
|
|
363
504
|
|
|
364
505
|
def main() -> int:
|
|
@@ -387,11 +528,16 @@ def main() -> int:
|
|
|
387
528
|
|
|
388
529
|
output_path = args.output or payload.get("output_path") or default_output_path()
|
|
389
530
|
output_path = os.path.abspath(output_path)
|
|
390
|
-
details = payload.get("details")
|
|
391
|
-
if
|
|
531
|
+
details = payload.get("details")
|
|
532
|
+
if isinstance(details, str):
|
|
533
|
+
details = Path(details)
|
|
534
|
+
elif not isinstance(details, list):
|
|
392
535
|
details = []
|
|
393
536
|
|
|
394
537
|
write_xlsx(rows, output_path, details=details)
|
|
538
|
+
detail_count = payload.get("detail_count")
|
|
539
|
+
if not isinstance(detail_count, int):
|
|
540
|
+
detail_count = sum(1 for _ in iter_detail_records(details))
|
|
395
541
|
|
|
396
542
|
print(
|
|
397
543
|
json.dumps(
|
|
@@ -399,7 +545,7 @@ def main() -> int:
|
|
|
399
545
|
"ok": True,
|
|
400
546
|
"output_path": output_path,
|
|
401
547
|
"row_count": len(rows),
|
|
402
|
-
"detail_count":
|
|
548
|
+
"detail_count": detail_count,
|
|
403
549
|
},
|
|
404
550
|
ensure_ascii=False,
|
|
405
551
|
)
|