sino-bc-cli 1.8.21 → 1.8.23

package/dist/skills/custom/biz-line-strategy-generate/SKILL.md

@@ -28,7 +28,7 @@ metadata:
 使用本技能目录的 **`scripts/run_workflow.py`**，不要临时重写拉数、分析或排版脚本。执行前定位技能的绝对路径；Python 只需标准库。首次使用读取 [`references/runtime.md`](references/runtime.md)。
 
 1. 用户给名称，传 `--name`；命中多个候选则返回歧义。用户给多个 ID，重复传 `--id`。用户要第一页/前 N 条，先用既有 `biz-line +list --data` 选出真实 ID，再在**一次入口调用**中传入。
-2. 入口默认最多 **3 条业务线并发**，允许 `--concurrency 1` 或 `2`，不能大于 3。同一批次不要并排启动多个入口绕开限制；单条内部 CLI 请求和分页串行。
+2. 入口默认最多 **3 条业务线并发**，允许 `--concurrency 1` 或 `2`，不能大于 3。同一批次不要并排启动多个入口绕开限制；单条内部 CLI 请求和分页串行。分页回包只落到本次临时目录的 `*.pN.out.json`，由脚本写 `*.pages.json` 索引；**禁止**在对话里重写拉数脚本、把各页 `extend` 进内存或再拼一份 `merged.json`。分析按页迭代，不要把全表读进 list。
 3. 每条自动执行现拉 → 固定分析 → 一次性排版 → Excel 回读校验 → 发布最终文件 → 清理。默认接口周期；用户给日期时成对传 `--curr-start/--curr-end`、`--prev-start/--prev-end`。
 4. 每条成功后立即交付**该业务线自己的最终 Excel**，并删除本次其他临时产物，包括 JSON、请求/回包、日志副本、图片、临时脚本、缓存、无扩展名文件及临时 Excel，不等全部业务线结束。
 5. 失败任务返回阶段和错误摘要，清理自己的临时文件，其他业务线继续。清理失败须报告残留路径，不能声称已清理。只给实际成功返回的 `output_path` 链接；不用已删除的请求目录作交付物。
@@ -69,7 +69,7 @@ python3 -B <本技能绝对路径>/scripts/run_workflow.py --id <真实ID1> --id
 - 未绑定项目/账号不要编造。多项目仍一行，名称 `\n` 分隔。
 - 文案用中文/`display_*`；词包只写「新增/本期未出现 N 个」+ 最多 3 短词。
 - learning 第 1 层只写消耗 ≥$50 的新增。
-- CLI 回包必须直写文件（`exec_cli_to_file`），`capture_output` 会在约 64KB 截断。翻页上限用脚本 `PAGE_CAP`（≥80），不要手写 30 页就停。
+- CLI 回包必须直写文件（`exec_cli_to_file`），`capture_output` 会在约 64KB 截断。每页是 `stem.pN.out.json`，拉数成功才写 `stem.pages.json`；不要写全表 `merged.json`，也不要手写翻页把结果攒在内存里。翻页上限用脚本 `PAGE_CAP`（≥80），不要手写 30 页就停。
 - 每条业务线只留一份最终 Excel；禁止按桌面 `业务线策略生成_*.xlsx` glob 删除。
 
 ## References

package/dist/skills/custom/biz-line-strategy-generate/references/api-flow.md

@@ -94,7 +94,7 @@ sino-bc-cli biz-line api biz-line-project-list --data '{"bizLineId": 1251}'
 
 ## 分页与数据量
 
-维度/成效接口默认每页 10 条（`size`），脚本显式 `size=1000`，按 `data.total` 翻页，上限 `PAGE_CAP=80`（TikTok 出价按天可到 4～5 万行）。回包直写文件。每次现拉，不要跳过已有页；最终 Excel 校验发布后立即清理该条全部临时文件。
+维度/成效接口默认每页 10 条（`size`），脚本显式 `size=1000`，按 `data.total` 翻页，上限 `PAGE_CAP=80`。每一页回包直写 `stem.pN.out.json`，全部页成功后写 `stem.pages.json`；不要把各页拼成内存全表或 `merged.json`。分析按页 yield，占比和比率一次扫描。TikTok 出价按天可达 4～5 万行，并发多条业务线时尤其不能在 Python 里 `extend` 全量。每次现拉，不要跳过已有页；最终 Excel 校验发布后立即清理该条全部临时文件。
 
 ## 汇总到 Excel 前的数据整理
 

package/dist/skills/custom/biz-line-strategy-generate/references/runtime.md

@@ -9,6 +9,7 @@
 - Windows 通过已安装 CLI 对应 Node 入口传参数数组，避免 cmd 重新解释 JSON。权限错误应报告，不当成未安装去自动下载。
 - `--format json`，请求用 `--data`。成功必须同时满足退出码 0 和信封 `ok == true`；stderr 与 stdout 都直写本次文件。
 - 列表行为 `data.result`，项目为 `data` 数组，维度/成效为 `data.data` 和 `data.total`。分页 `size=1000`、`PAGE_CAP=80`；总数未取全、提前空页、失败或总数变化均中止该条，不导出部分成功文件。
+- 维度/成效每一页 MUST 直写 `bl-<id>/<stem>.pN.out.json`。拉数在内存中只保留当前页和行数累计，成功后才写 `<stem>.pages.json`（`total` / `pages` / `counts`）。开始前若已有本 stem 的 `pages.json` 必须先删掉。MUST NOT 把各页 `extend` 成全表再写 `merged.json`。汇总按 `pages.json` 逐页 yield，占比/比率/覆盖日期一次扫描，不得为分析再拼全表 list。无索引却有 `p2` 及之后页时失败。清单校验只对索引上的 `total`/`counts`，不得为校验再整表加载。失败页 MUST NOT 写下完整 `pages.json`。
 
 ## 目录归属
 
@@ -17,7 +18,7 @@
   .strategy-owner.json
   line-<业务线ID>-<随机标识>/
     .strategy-owner.json
-    请求体、回包、stderr、merged JSON、payload、summary、临时 Excel
+    请求体、stderr、分页回包 <stem>.pN.out.json、索引 <stem>.pages.json、临时 Excel
 ```
 
 每次现拉，新建目录，不复用历史缓存。所有本次临时文件写入本次目录；不向技能源码树写缓存。最终 Excel 写在批次临时根以外，文件名包含业务线 ID 和随机请求标识，独占创建、不覆盖已有文件。

package/dist/skills/custom/biz-line-strategy-generate/references/spec.md

@@ -37,12 +37,20 @@
 
 ### Requirement: 分页与失败回包必须完整校验
 
-每页 CLI 回包 MUST 直写文件并检查退出码、信封和分页总数。异常、解析失败、未到总数即遇空页、达到分页上限仍未取全，MUST 阻止该业务线生成成功报告。CLI stderr 错误 MUST 保留到本次任务结果摘要，而非当作无数据。其他业务线 MUST 可以继续执行。
+每页 CLI 回包 MUST 直写 `<stem>.pN.out.json` 并检查退出码、信封和分页总数。拉数成功后 MUST 写 `<stem>.pages.json`（含 `total`、`pages`、每页 `counts`）。拉数过程 MUST 只在内存保留当前页与行数累计，MUST NOT 将各页累积为全表 list，MUST NOT 再写一份全表 `merged.json`。汇总 MUST 按 `pages.json` 顺序逐页 yield 行；`pages.json` 缺失但存在 `p2` 及之后页文件时 MUST 失败，不得只读第一页。`share_map` / `metric_value` / 覆盖日期 / 出价快照 MUST 一次扫描完成，MUST NOT 为占比或比率把全表物化成 list。新增对象识别 MUST 只为新增行保留明细，持续对象 MUST NOT 再攒成全表。开始分析前的清单校验 MUST 只核对该索引与 manifest 的 `total`/`counts`，MUST NOT 为校验整表加载。广告/广告组命名规律 MUST 只用原始展示名，MUST NOT 把空名回退成 ID 再拆词。异常、解析失败、未到总数即遇空页、达到分页上限仍未取全，MUST 阻止该业务线生成成功报告，且 MUST NOT 写下完整 `pages.json`。CLI stderr 错误 MUST 保留到本次任务结果摘要，而非当作无数据。其他业务线 MUST 可以继续执行。
 
 #### Scenario: 后续页失败不导出部分结果
 - **GIVEN** 首页成功且声明仍有下一页
 - **WHEN** 后续页调用失败或返回空页但累计数量不足
-- **THEN** 该业务线失败，不将不完整数据生成最终 Excel，其他业务线继续执行
+- **THEN** 该业务线失败，不将不完整数据生成最终 Excel，不留下完整 `pages.json`，其他业务线继续执行
+
+#### Scenario: 多页成功只落分页文件
+- **GIVEN** 某 stem 两页合计 3 行且 `data.total` 为 3
+- **WHEN** 拉数成功
+- **THEN** 目录中有 `stem.p1.out.json`、`stem.p2.out.json` 与 `stem.pages.json`（`total=3`、`counts=[2,1]` 或等价分页）
+- **AND** 不存在 `stem.merged.json`
+- **AND** 汇总能从页文件读出这 3 行
+- **AND** `share_map` / `metric_value` 对一次性迭代器也能得到与 list 相同的占比和比率
 
 ### Requirement: 指标与周期口径固定
 

package/dist/skills/custom/biz-line-strategy-generate/scripts/cli_json.py

@@ -250,15 +250,23 @@ def optional_num(row, key):
 
 
 def metric_value(rows, metric="CPI"):
-    """同一粒度先合计基础指标再计算，任一缺失不当作零补齐。"""
+    """同一粒度先合计基础指标再计算，任一缺失不当作零补齐。只扫一遍，可接迭代器。"""
     formulas = {"CPI": ("spend_usd", "installs", 1), "CPC": ("spend_usd", "clicks", 1),
                 "CPM": ("spend_usd", "impressions", 1000), "ROAS": ("purchase_value", "spend_usd", 1),
                 "CTR": ("clicks", "impressions", 100)}
     a, b, scale = formulas[metric]
-    if not rows or any(optional_num(r, key) is None for r in rows for key in (a, b)):
+    seen = False
+    num_a = num_b = 0.0
+    for row in rows or ():
+        seen = True
+        left, right = optional_num(row, a), optional_num(row, b)
+        if left is None or right is None:
+            return None
+        num_a += left
+        num_b += right
+    if not seen:
         return None
-    denominator = sum(optional_num(r, b) for r in rows)
-    return sum(optional_num(r, a) for r in rows) / denominator * scale if denominator > 0 else None
+    return num_a / num_b * scale if num_b > 0 else None
 
 
 def core_rule(text):

package/dist/skills/custom/biz-line-strategy-generate/scripts/fetch_data.py

@@ -41,8 +41,17 @@ MET_CMD = {
 }
 
 
-def fetch_paged(shortcut: str, base_body: dict, stem: str, bl_dir: Path) -> list:
-    all_rows: list = []
+def fetch_paged(shortcut: str, base_body: dict, stem: str, bl_dir: Path) -> int:
+    """逐页把 CLI 回包落到 stem.pN.out.json，成功后再写 stem.pages.json。
+
+    只在内存里保留当前页和行数累计，不把全表 extend 成一份 list，也不写 merged.json。
+    开始前删掉本 stem 残留索引，避免同目录重试失败后仍留下上一轮完整 pages.json。
+    """
+    index_path = bl_dir / f"{stem}.pages.json"
+    if index_path.exists():
+        index_path.unlink()
+    accumulated = 0
+    counts: list[int] = []
     page = 1
     total = 0
     while True:
@@ -57,19 +66,22 @@ def fetch_paged(shortcut: str, base_body: dict, stem: str, bl_dir: Path) -> list
         if page > 1 and current_total != total:
             raise RuntimeError(f"{stem}: 分页期间总数变化，请重新拉取")
         total = current_total
-        all_rows.extend(rows)
-        report_progress(f"{stem} page={page} got={len(rows)} acc={len(all_rows)} total={total}")
-        if len(all_rows) > total:
+        got = len(rows)
+        accumulated += got
+        counts.append(got)
+        report_progress(f"{stem} page={page} got={got} acc={accumulated} total={total}")
+        if accumulated > total:
             raise RuntimeError(f"{stem}: 累计行数超过 total")
-        if len(all_rows) == total:
+        if accumulated == total:
             break
         if not rows or page >= PAGE_CAP:
-            raise RuntimeError(f"{stem}: 分页不完整 {len(all_rows)}/{total}")
+            raise RuntimeError(f"{stem}: 分页不完整 {accumulated}/{total}")
         page += 1
-    (bl_dir / f"{stem}.merged.json").write_text(
-        json.dumps(all_rows, ensure_ascii=False), encoding="utf-8"
+    (bl_dir / f"{stem}.pages.json").write_text(
+        json.dumps({"total": total, "pages": len(counts), "counts": counts}, ensure_ascii=False),
+        encoding="utf-8",
     )
-    return all_rows
+    return total
 
 
 CHANNEL_ALIAS = {
@@ -336,7 +348,7 @@ def fetch_business(args):
                     "dimensions": dims,
                     **extra,
                 }
-                manifest[stem] = len(fetch_paged(DIM_CMD[ch], body, stem, bl_dir))
+                manifest[stem] = fetch_paged(DIM_CMD[ch], body, stem, bl_dir)
             for dims in met_dims(ch):
                 stem = f"met_{ch}_{per}_{'+'.join(dims)}"
                 body = {
@@ -346,7 +358,7 @@ def fetch_business(args):
                     "dimensions": dims,
                     "indicators": INDICATORS,
                 }
-                manifest[stem] = len(fetch_paged(MET_CMD[ch], body, stem, bl_dir))
+                manifest[stem] = fetch_paged(MET_CMD[ch], body, stem, bl_dir)
 
     # 只有全部所需数据完成后登记清单，汇总前按清单核对文件和数量。
     (req / 'fetch-manifest.json').write_text(json.dumps(manifest), encoding='utf-8')

package/dist/skills/custom/biz-line-strategy-generate/scripts/summarize.py

@@ -1,6 +1,6 @@
 #!/usr/bin/env python3
 # -*- coding: utf-8 -*-
-"""按固定规则汇总 merged JSON，生成可直接校验及导出的 payload。
+"""按固定规则汇总分页临时文件，生成可直接校验及导出的 payload。
 """
 from __future__ import annotations
 
@@ -42,13 +42,47 @@ CH_LABEL = {"fb": "Meta", "gg": "Google", "tt": "TikTok"}
 DETAIL_LIMIT = 8
 
 
-def load_merged(bl: Path, stem: str) -> list:
-    data = load_json_file(bl / f"{stem}.merged.json")
-    if isinstance(data, list):
-        return data
+def iter_stem_rows(bl: Path, stem: str):
+    """按页 yield 行，不把全表拼成 list。可重复调用，每次重新读盘。"""
+    meta_path = bl / f"{stem}.pages.json"
+    if meta_path.exists():
+        meta = load_json_file(meta_path)
+        if not isinstance(meta, dict) or not isinstance(meta.get("counts"), list):
+            raise ValueError(f"拉数文件不完整：{stem}")
+        accumulated = 0
+        for index, expected in enumerate(meta["counts"], start=1):
+            env = load_json_file(bl / f"{stem}.p{index}.out.json")
+            page_rows, _ = unwrap_page(env or {})
+            if len(page_rows) != expected:
+                raise ValueError(f"拉数文件不完整：{stem}")
+            yield from page_rows
+            accumulated += len(page_rows)
+        if meta.get("total") != accumulated:
+            raise ValueError(f"拉数文件不完整：{stem}")
+        return
+    extras = [p for p in bl.glob(f"{stem}.p*.out.json") if p.name != f"{stem}.p1.out.json"]
+    if extras:
+        raise ValueError(f"拉数文件不完整：{stem}")
     env = load_json_file(bl / f"{stem}.p1.out.json")
-    rows, _ = unwrap_page(env or {})
-    return rows
+    yield from unwrap_page(env or {})[0]
+
+
+def load_merged(bl: Path, stem: str) -> list:
+    """兼容测试与诊断：需要 list 时才物化。分析路径应走 iter_stem_rows。"""
+    return list(iter_stem_rows(bl, stem))
+
+
+def object_name_maps(rows, id_key, name_key) -> tuple[dict[str, str], dict[str, str]]:
+    """id → 明细展示名（空则回退 id），以及原始 display（空名不当作 id，避免命名规律拆 ID）。"""
+    names: dict[str, str] = {}
+    raw: dict[str, str] = {}
+    for row in rows:
+        if row.get(id_key):
+            ident = str(row[id_key])
+            shown = disp(row, name_key)
+            raw[ident] = shown
+            names[ident] = shown or ident
+    return names, raw
 
 
 def labels_of(rows, key) -> set[str]:
@@ -56,10 +90,13 @@ def labels_of(rows, key) -> set[str]:
 
 
 def share_map(rows, dim_key):
-    total = sum(num(r, "spend_usd") for r in rows)
+    """一次扫描累加消耗占比，可接迭代器。"""
+    total = 0.0
     acc = defaultdict(float)
-    for r in rows:
-        acc[disp(r, dim_key) or "未标注"] += num(r, "spend_usd")
+    for row in rows:
+        spend = num(row, "spend_usd")
+        total += spend
+        acc[disp(row, dim_key) or "未标注"] += spend
     if total <= 0:
         return {}, 0.0
     return {k: v / total * 100 for k, v in acc.items()}, total
@@ -90,18 +127,30 @@ def share_moved(curr: dict, prev: dict) -> bool:
 
 
 def bid_changes(rows, fields):
-    """逐个出价字段比较有效日快照，缺失值不补零；保留字段与对象 ID。"""
+    """逐个出价字段比较有效日快照，缺失值不补零；只保留日期与出价字段，可接迭代器。"""
     by_ag = defaultdict(list)
+    names: dict[str, tuple[str, str]] = {}
+    strategies: dict[str, str] = {}
     for row in rows:
-        if row.get("adgroup_id"):
-            by_ag[str(row["adgroup_id"])].append(row)
+        ag = row.get("adgroup_id")
+        if not ag:
+            continue
+        ident = str(ag)
+        date_s = str(row.get("calendar_date") or "")
+        if ident not in names or date_s >= names[ident][0]:
+            names[ident] = (date_s, disp(row, "adgroup_name") or ident)
+            strategies[ident] = disp(row, "bid_strategy")
+        snap = {"calendar_date": str(row.get("calendar_date") or "")}
+        for field in fields:
+            snap[field] = optional_num(row, field)
+        by_ag[ident].append(snap)
     result = []
     for ag, records in by_ag.items():
-        records = sorted(records, key=lambda r: str(r.get("calendar_date") or ""))
+        records = sorted(records, key=lambda r: r["calendar_date"])
         changes = []
         changed_dates = set()
         for field in fields:
-            valid = [(str(r.get("calendar_date") or ""), optional_num(r, field)) for r in records if optional_num(r, field) is not None]
+            valid = [(r["calendar_date"], r[field]) for r in records if r[field] is not None]
             times = sum(a[1] != b[1] for a, b in zip(valid, valid[1:]))
             changed_dates.update(b[0] for a, b in zip(valid, valid[1:]) if a[1] != b[1])
             if times:
@@ -109,28 +158,35 @@ def bid_changes(rows, fields):
         if changes:
             # 多字段变动以首个首末不同的字段展示方向，明细完整保留各字段。
             primary = next((c for c in changes if c["from"] != c["to"]), changes[0])
-            result.append({"id": ag, "name": disp(records[-1], "adgroup_name") or ag,
-                           **primary, "times": len(changed_dates), "fields": changes, "strategy": disp(records[-1], "bid_strategy")})
+            result.append({"id": ag, "name": names[ag][1],
+                           **primary, "times": len(changed_dates), "fields": changes, "strategy": strategies[ag]})
     return sorted(result, key=lambda r: (-r["times"], r["id"]))
 
 
 def new_items(curr_rows, prev_rows, key, id_key=None):
-    """按稳定 ID/维度值聚合基础指标，再识别本期新出现的对象。"""
+    """按稳定 ID/维度值聚合基础指标，再识别本期新出现的对象。只为新增对象保留行。"""
     def identity(row):
         return str(row.get(id_key) or "") if id_key else disp(row, key)
+    previous: dict[str, str] = {}
+    for row in prev_rows:
+        ident = identity(row)
+        if ident:
+            previous[ident] = disp(row, key) or ident
     grouped = defaultdict(list)
+    current_ids: set[str] = set()
     for row in curr_rows:
-        if identity(row):
-            grouped[identity(row)].append(row)
-    previous = {identity(r): r for r in prev_rows if identity(r)}
+        ident = identity(row)
+        if not ident:
+            continue
+        current_ids.add(ident)
+        if ident not in previous:
+            grouped[ident].append(row)
     items = []
     for ident, records in grouped.items():
-        if ident in previous:
-            continue
         items.append({"id": ident if id_key else "", "label": disp(records[0], key) or ident,
                       "spend": sum(num(r, "spend_usd") for r in records), "rows": records,
                       "cpi": metric_value(records), "cpm": metric_value(records, "CPM")})
-    stops = [disp(r, key) or ident for ident, r in previous.items() if ident not in grouped]
+    stops = [label for ident, label in previous.items() if ident not in current_ids]
     return items, stops
 
 
@@ -208,9 +264,14 @@ def summarize_request(req: Path):
     manifest = load_json_file(req / 'fetch-manifest.json')
     if manifest is not None:
         for stem, count in manifest.items():
-            data = load_json_file(bl / f'{stem}.merged.json')
-            if not isinstance(data, list) or len(data) != count:
-                raise ValueError(f'拉数文件不完整：{stem}')
+            meta = load_json_file(bl / f"{stem}.pages.json")
+            if (
+                not isinstance(meta, dict)
+                or meta.get("total") != count
+                or not isinstance(meta.get("counts"), list)
+                or sum(meta["counts"]) != count
+            ):
+                raise ValueError(f"拉数文件不完整：{stem}")
     group = detail_data.get("ssoTeamCodeName") or biz.get("ssoTeamCodeName") or ""
     name = detail_data.get("name") or biz.get("name") or ""
     project = "\n".join(projects.get("projectNames") or detail_data.get("projectNames") or [])
@@ -231,21 +292,33 @@ def summarize_request(req: Path):
     imps = {"curr": {}, "prev": {}}
     inst = {"curr": {}, "prev": {}}
     days = {"curr": {}, "prev": {}}
+    coverage_dates = {"curr": {}, "prev": {}}
     for ch in ("fb", "gg", "tt"):
         for per in ("curr", "prev"):
-            day_rows = load_merged(bl, f"met_{ch}_{per}_calendar_date")
-            acc_rows = load_merged(bl, f"met_{ch}_{per}_account_id")
-            rows = acc_rows or day_rows
-            spend[per][ch] = sum(num(r, "spend_usd") for r in rows)
-            imps[per][ch] = sum(num(r, "impressions") for r in rows)
-            inst[per][ch] = sum(num(r, "installs") for r in acc_rows or rows)
-            days[per][ch] = len(
-                {
-                    disp(r, "calendar_date") or str(r.get("calendar_date") or "")
-                    for r in day_rows
-                    if num(r, "spend_usd") > 0
-                }
-            )
+            day_spend = day_imps = day_inst = 0.0
+            dates_all: set[str] = set()
+            dates_spend: set[str] = set()
+            for row in iter_stem_rows(bl, f"met_{ch}_{per}_calendar_date"):
+                day_spend += num(row, "spend_usd")
+                day_imps += num(row, "impressions")
+                day_inst += num(row, "installs")
+                raw = row.get("calendar_date")
+                if raw:
+                    dates_all.add(str(raw)[:10])
+                if num(row, "spend_usd") > 0:
+                    dates_spend.add(disp(row, "calendar_date") or str(raw or ""))
+            acc_spend = acc_imps = acc_inst = 0.0
+            has_acc = False
+            for row in iter_stem_rows(bl, f"met_{ch}_{per}_account_id"):
+                has_acc = True
+                acc_spend += num(row, "spend_usd")
+                acc_imps += num(row, "impressions")
+                acc_inst += num(row, "installs")
+            spend[per][ch] = acc_spend if has_acc else day_spend
+            imps[per][ch] = acc_imps if has_acc else day_imps
+            inst[per][ch] = acc_inst if has_acc else day_inst
+            days[per][ch] = len(dates_spend)
+            coverage_dates[per][ch] = dates_all
 
     curr_total = sum(spend["curr"].values())
     prev_total = sum(spend["prev"].values())
@@ -262,9 +335,12 @@ def summarize_request(req: Path):
     core_index = str(biz.get("coreIndex") or detail_data.get("coreIndex") or "").strip()
     rule = core_rule(core_index)
     metric = rule["metric"]
-    all_rows = {per: [r for ch in CH_LABEL for r in load_merged(bl, f"met_{ch}_{per}_account_id")] for per in ("curr", "prev")}
-    core_c, core_p = (metric_value(all_rows[per], metric) for per in ("curr", "prev"))
-    cpi_c, cpi_p = (metric_value(all_rows[per], "CPI") for per in ("curr", "prev"))
+    def iter_account_rows(per: str):
+        for ch in CH_LABEL:
+            yield from iter_stem_rows(bl, f"met_{ch}_{per}_account_id")
+
+    core_c, core_p = (metric_value(iter_account_rows(per), metric) for per in ("curr", "prev"))
+    cpi_c, cpi_p = (metric_value(iter_account_rows(per), "CPI") for per in ("curr", "prev"))
     qoq_label, qoq_pct = qoq_axis(core_c, core_p, metric in ("ROAS", "CTR"))
     kpi_usable = rule["usable"]
     spend_verdict = period_verdict(
@@ -289,7 +365,7 @@ def summarize_request(req: Path):
         for ch, accounts in projects.get("accounts", {}).items():
             if not accounts:
                 continue
-            dates = sorted({str(r.get("calendar_date") or "")[:10] for r in load_merged(bl, f"met_{ch}_{per}_calendar_date") if r.get("calendar_date")})
+            dates = sorted(coverage_dates[per].get(ch) or set())
             missing = expected - set(dates)
             label = "本期" if per == "curr" else "上期"
             coverage_notes.append(f"{CH_LABEL[ch]}{label}实际覆盖 {dates[0] if dates else '-'}～{dates[-1] if dates else '-'}，{len(dates)}天/{len(expected)}天" + ("（覆盖不完整）" if missing else ""))
@@ -309,7 +385,7 @@ def summarize_request(req: Path):
         "gg": "dim_gg_curr_calendar_date+adgroup_id+adgroup_name+cpc_bid_amount",
         "tt": "dim_tt_curr_calendar_date+adgroup_id+adgroup_name+bid_amount",
     }
-    bids = {ch: bid_changes(load_merged(bl, bid_stems[ch]), bid_fields[ch]) for ch in bid_fields}
+    bids = {ch: bid_changes(iter_stem_rows(bl, bid_stems[ch]), bid_fields[ch]) for ch in bid_fields}
     bid_stats = {ch: bid_adjust_stats(bids[ch]) for ch in bids}
 
     def bid_text(ch):
@@ -408,8 +484,8 @@ def summarize_request(req: Path):
         segs = []
         moved = False
         for ch in channels:
-            curr, _ = share_map(load_merged(bl, f"met_{ch}_curr_{stem}"), dim_key)
-            prev, _ = share_map(load_merged(bl, f"met_{ch}_prev_{stem}"), dim_key)
+            curr, _ = share_map(iter_stem_rows(bl, f"met_{ch}_curr_{stem}"), dim_key)
+            prev, _ = share_map(iter_stem_rows(bl, f"met_{ch}_prev_{stem}"), dim_key)
             if share_moved(curr, prev):
                 moved = True
             if curr or prev:
@@ -430,12 +506,16 @@ def summarize_request(req: Path):
             # Meta 兴趣的三个配置字段来自同一组合请求，不能找不存在的单字段文件。
             if ch == 'fb' and stem == 'targeting_interests':
                 combined = 'targeting_interests+targeting_flexible_interests+targeting_exclusion_interests'
-                for field in combined.split('+'):
-                    curr |= labels_of(load_merged(bl, f'dim_{ch}_curr_{combined}'), field)
-                    prev |= labels_of(load_merged(bl, f'dim_{ch}_prev_{combined}'), field)
+                fields = combined.split('+')
+                for target, per in ((curr, 'curr'), (prev, 'prev')):
+                    for row in iter_stem_rows(bl, f'dim_{ch}_{per}_{combined}'):
+                        for field in fields:
+                            lab = disp(row, field)
+                            if lab:
+                                target.add(lab)
                 continue
-            curr |= labels_of(load_merged(bl, f"dim_{ch}_curr_{stem}"), key)
-            prev |= labels_of(load_merged(bl, f"dim_{ch}_prev_{stem}"), key)
+            curr |= labels_of(iter_stem_rows(bl, f"dim_{ch}_curr_{stem}"), key)
+            prev |= labels_of(iter_stem_rows(bl, f"dim_{ch}_prev_{stem}"), key)
         return curr, prev
 
     def add_setting_details(kind, curr_set, prev_set, channel=""):
@@ -476,16 +556,18 @@ def summarize_request(req: Path):
         bits = []
         for ch in CH_LABEL:
             stem = f"{id_key}+{name_key}"
-            crows = load_merged(bl, f"dim_{ch}_curr_{stem}")
-            prows = load_merged(bl, f"dim_{ch}_prev_{stem}")
-            c = {str(r[id_key]): r for r in crows if r.get(id_key)}
-            p = {str(r[id_key]): r for r in prows if r.get(id_key)}
+
+            def index_objects(period: str) -> tuple[dict[str, str], dict[str, str]]:
+                return object_name_maps(iter_stem_rows(bl, f"dim_{ch}_{period}_{stem}"), id_key, name_key)
+
+            c, c_raw = index_objects("curr")
+            p, _p_raw = index_objects("prev")
             new, absent = sorted(c.keys() - p.keys()), sorted(p.keys() - c.keys())
-            bits.append(f"{CH_LABEL[ch]}：本期新出现 {len(new)} 个，本期未出现 {len(absent)} 个；命名规律：{naming_patterns([disp(c[i], name_key) for i in new])}")
+            bits.append(f"{CH_LABEL[ch]}：本期新出现 {len(new)} 个，本期未出现 {len(absent)} 个；命名规律：{naming_patterns([c_raw[i] for i in new])}")
             for ident in sorted(c.keys() | p.keys()):
-                record = c.get(ident) or p[ident]
+                record_name = c.get(ident) or p[ident]
                 action = "当期" if first_period else ("新增" if ident in new else ("本期未出现" if ident in absent else "持续"))
-                details.append({"bizLine": name, "kind": kind, "action": action, "channel": CH_LABEL[ch], "name": disp(record, name_key) or ident, "extra": f"ID={ident}；集合差异不等同创建/暂停操作"})
+                details.append({"bizLine": name, "kind": kind, "action": action, "channel": CH_LABEL[ch], "name": record_name, "extra": f"ID={ident}；集合差异不等同创建/暂停操作"})
         return bits
 
     ag_bits = object_strategy("广告组", "adgroup_id", "adgroup_name")
@@ -514,11 +596,11 @@ def summarize_request(req: Path):
         for title, stem, key, channels, id_key in dim_specs:
             items_all, stops_all = [], []
             for ch in channels:
-                curr_rows = load_merged(bl, f"met_{ch}_curr_{stem}")
-                prev_rows = load_merged(bl, f"met_{ch}_prev_{stem}")
-                curr_share_map, _ = share_map(curr_rows, key)
-                prev_share_map, _ = share_map(prev_rows, key)
-                items, stops = new_items(curr_rows, prev_rows, key, id_key)
+                curr_stem = f"met_{ch}_curr_{stem}"
+                prev_stem = f"met_{ch}_prev_{stem}"
+                curr_share_map, _ = share_map(iter_stem_rows(bl, curr_stem), key)
+                prev_share_map, _ = share_map(iter_stem_rows(bl, prev_stem), key)
+                items, stops = new_items(iter_stem_rows(bl, curr_stem), iter_stem_rows(bl, prev_stem), key, id_key)
                 stops = [
                     s
                     for s in stops

package/dist/skills/custom/biz-line-strategy-generate/scripts/test_workflow.py

@@ -53,6 +53,115 @@ class DataRules(unittest.TestCase):
             with self.assertRaises(RuntimeError):
                 fetch_paged("+facebook-metrics", {}, "fixture", Path(tmp))
             self.assertFalse((Path(tmp) / "fixture.merged.json").exists())
+            self.assertFalse((Path(tmp) / "fixture.pages.json").exists())
+
+    def test_paged_success_writes_page_files_not_merged(self):
+        """拉数只把各页回包落到临时文件，不在内存拼全表、不写 merged.json。"""
+        pages = [
+            {"ok": True, "data": {"data": [{"id": 1}, {"id": 2}], "total": 3}},
+            {"ok": True, "data": {"data": [{"id": 3}], "total": 3}},
+        ]
+
+        def api(_command, body, out):
+            env = pages[body["page"] - 1]
+            out.parent.mkdir(parents=True, exist_ok=True)
+            out.write_text(json.dumps(env, ensure_ascii=False), encoding="utf-8")
+            return env
+
+        with tempfile.TemporaryDirectory() as tmp, patch("fetch_data.run_cli", api):
+            root = Path(tmp)
+            total = fetch_paged("+facebook-metrics", {}, "fixture", root)
+            self.assertEqual(total, 3)
+            self.assertFalse((root / "fixture.merged.json").exists())
+            self.assertEqual(
+                json.loads((root / "fixture.pages.json").read_text(encoding="utf-8")),
+                {"total": 3, "pages": 2, "counts": [2, 1]},
+            )
+            self.assertTrue((root / "fixture.p1.out.json").exists())
+            self.assertTrue((root / "fixture.p2.out.json").exists())
+            from summarize import load_merged
+            self.assertEqual(load_merged(root, "fixture"), [{"id": 1}, {"id": 2}, {"id": 3}])
+            from summarize import iter_stem_rows, share_map
+            self.assertEqual(list(iter_stem_rows(root, "fixture")), [{"id": 1}, {"id": 2}, {"id": 3}])
+            self.assertEqual(list(iter_stem_rows(root, "fixture")), [{"id": 1}, {"id": 2}, {"id": 3}])
+
+    def test_share_map_and_metric_value_accept_iterators(self):
+        from summarize import share_map
+        shares, total = share_map(
+            iter([
+                {"spend_usd": 10, "audience_type": "A"},
+                {"spend_usd": 30, "audience_type": "B"},
+            ]),
+            "audience_type",
+        )
+        self.assertEqual(total, 40)
+        self.assertEqual(shares["A"], 25)
+        self.assertEqual(shares["B"], 75)
+        self.assertEqual(
+            metric_value(iter([{"spend_usd": 10, "installs": 1}, {"spend_usd": 90, "installs": 49}])),
+            2,
+        )
+        self.assertIsNone(metric_value(iter([{"spend_usd": 10, "installs": 0}])))
+
+    def test_bid_changes_accepts_iterator_without_full_row_list(self):
+        rows = iter([
+            {"adgroup_id": "1", "adgroup_name": "同名", "calendar_date": "2026-09-01", "bid_amount": 1},
+            {"adgroup_id": "1", "adgroup_name": "同名", "calendar_date": "2026-09-02", "bid_amount": 2},
+            {"adgroup_id": "1", "adgroup_name": "同名", "calendar_date": "2026-09-03", "bid_amount": 1},
+        ])
+        changes = bid_changes(rows, ["bid_amount"])
+        self.assertEqual(changes[0]["times"], 2)
+        self.assertEqual(changes[0]["from"], changes[0]["to"])
+        self.assertEqual(changes[0]["name"], "同名")
+
+    def test_new_items_only_materializes_new_rows(self):
+        from summarize import new_items
+        items, stops = new_items(
+            iter([
+                {"ad_id": "keep", "ad_name": "新", "spend_usd": 10, "installs": 1, "impressions": 1000},
+                {"ad_id": "old", "ad_name": "旧", "spend_usd": 99, "installs": 1, "impressions": 1000},
+            ]),
+            iter([{"ad_id": "old", "ad_name": "旧"}]),
+            "ad_name",
+            "ad_id",
+        )
+        self.assertEqual(len(items), 1)
+        self.assertEqual(items[0]["id"], "keep")
+        self.assertEqual(len(items[0]["rows"]), 1)
+        self.assertEqual(stops, [])
+
+    def test_iter_stem_rows_rejects_extra_pages_without_index(self):
+        from summarize import iter_stem_rows
+        with tempfile.TemporaryDirectory() as tmp:
+            root = Path(tmp)
+            page = {"ok": True, "data": {"data": [{"id": 1}], "total": 2}}
+            (root / "fixture.p1.out.json").write_text(json.dumps(page), encoding="utf-8")
+            (root / "fixture.p2.out.json").write_text(json.dumps(page), encoding="utf-8")
+            with self.assertRaises(ValueError):
+                list(iter_stem_rows(root, "fixture"))
+
+    def test_failed_retry_unlinks_stale_pages_index(self):
+        with tempfile.TemporaryDirectory() as tmp, patch("fetch_data.run_cli", return_value={"ok": False, "error": "失败"}):
+            root = Path(tmp)
+            (root / "fixture.pages.json").write_text(
+                json.dumps({"total": 3, "pages": 2, "counts": [2, 1]}), encoding="utf-8"
+            )
+            with self.assertRaises(RuntimeError):
+                fetch_paged("+facebook-metrics", {}, "fixture", root)
+            self.assertFalse((root / "fixture.pages.json").exists())
+
+    def test_empty_ad_name_stays_out_of_naming_patterns(self):
+        from summarize import object_name_maps
+        from cli_json import naming_patterns
+        names, raw = object_name_maps(
+            iter([{"ad_id": "camp_summer_01", "ad_name": ""}]),
+            "ad_id",
+            "ad_name",
+        )
+        self.assertEqual(names["camp_summer_01"], "camp_summer_01")
+        self.assertEqual(raw["camp_summer_01"], "")
+        self.assertEqual(naming_patterns([raw["camp_summer_01"]]), "命名规律不明显")
+        self.assertIn("camp", naming_patterns([names["camp_summer_01"]]))
 
     def test_short_page_still_fetches_total(self):
         pages = [{"ok": True, "data": {"data": [{"id": 1}], "total": 2}}, {"ok": True, "data": {"data": [], "total": 2}}]
@@ -163,8 +272,10 @@ class DataRules(unittest.TestCase):
                 self.assertIn('ID=2', objects[1]['extra'])
             bid = next(r for r in payload['details'] if r['kind'] == '出价')
             self.assertNotEqual(bid['action'], '下调')
-            # 汇总前损坏已完成文件必须失败，不能重新退回第一页继续。
-            (root / 'bl-1/met_fb_curr_account_id.merged.json').write_text('[]')
+            # 汇总前损坏已完成的分页索引必须失败，不能重新退回第一页继续。
+            (root / "bl-1/met_fb_curr_account_id.pages.json").write_text(
+                json.dumps({"total": 0, "pages": 0, "counts": []}), encoding="utf-8"
+            )
             with self.assertRaises(ValueError):
                 summarize_request(root)