@cloud411716/fancy-webnovel 2.2.0 → 3.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/client.js +201 -15
- package/cordis.patch.yml +5 -0
- package/docs/douyin-plugin.md +118 -0
- package/docs/material-library/README.md +404 -0
- package/docs/material-library/import.js +437 -0
- package/docs/material-library/prompt.md +370 -0
- package/docs/material-library/queries.surql +94 -0
- package/docs/material-library/sample.jsonl +14 -0
- package/docs/material-library/schema.surql +46 -0
- package/docs/mcp-profile.example.yml +24 -0
- package/docs/story-graph-design.md +602 -0
- package/docs/story-graph-validation.md +81 -0
- package/docs/workspace-mcp.md +284 -0
- package/index.js +11 -7
- package/lib/atomic-write.js +109 -0
- package/lib/chapter-order.js +100 -0
- package/lib/confirmation.js +80 -0
- package/lib/db-init.js +123 -0
- package/lib/deployed-state.js +11 -11
- package/lib/direction-labels.js +265 -0
- package/lib/error-codes.js +31 -31
- package/lib/infra.js +29 -4
- package/lib/material-db.js +149 -0
- package/lib/path-guard.js +150 -0
- package/lib/projects.js +400 -0
- package/lib/scraper-utils.js +140 -0
- package/lib/surreal-server.js +338 -0
- package/lib/topic-decision.js +860 -0
- package/lib/topic-fields.js +251 -0
- package/lib/topic-schema.js +225 -0
- package/lib/workspace.js +501 -0
- package/package.json +29 -4
- package/plugins/fancy-analyze/analyze-definitions.md +519 -0
- package/plugins/fancy-analyze/index.js +1128 -77
- package/plugins/fancy-analyze/ingest-coverage.js +107 -0
- package/plugins/fancy-analyze/schema-analyze.md +355 -107
- package/plugins/fancy-analyze/scripts/append-summary-chapter.js +612 -0
- package/plugins/fancy-analyze/summary-progress.js +71 -7
- package/plugins/fancy-analyze/templates.js +437 -139
- package/plugins/fancy-bootstrap/index.js +276 -131
- package/plugins/fancy-bootstrap/scripts/init.js +59 -25
- package/plugins/fancy-bootstrap/templates.js +35 -18
- package/plugins/fancy-db/index.js +172 -0
- package/plugins/fancy-douyin/deploy.js +134 -0
- package/plugins/fancy-douyin/index.js +551 -0
- package/plugins/fancy-douyin/prompts/01-analysis-prompt.md +84 -0
- package/plugins/fancy-douyin/scripts/crawl-note.cjs +194 -0
- package/plugins/fancy-douyin/scripts/lib.cjs +197 -0
- package/plugins/fancy-douyin/scripts/scan-account.cjs +128 -0
- package/plugins/fancy-douyin/scripts/start-chrome.cjs +203 -0
- package/plugins/fancy-douyin/state.js +197 -0
- package/plugins/fancy-douyin/templates.js +127 -0
- package/plugins/fancy-export/exporter.js +805 -0
- package/plugins/fancy-export/index.js +63 -0
- package/plugins/fancy-import/import-driver.js +29 -6
- package/plugins/fancy-import/index.js +13 -5
- package/plugins/fancy-import/prompts/01-import-prompt.md +39 -14
- package/plugins/fancy-import/templates.js +17 -8
- package/plugins/fancy-material/exporter.js +153 -0
- package/plugins/fancy-material/genres.js +256 -0
- package/plugins/fancy-material/index.js +835 -0
- package/plugins/fancy-material/inventory.js +143 -0
- package/plugins/fancy-material/kinds.js +313 -0
- package/plugins/fancy-material/queue.js +55 -0
- package/plugins/fancy-material/scripts/query.js +254 -0
- package/plugins/fancy-material/templates.js +258 -0
- package/plugins/fancy-material/tool-deploy.js +138 -0
- package/plugins/fancy-material/validate.js +149 -0
- package/plugins/fancy-project/index.js +133 -0
- package/plugins/fancy-scan/existing-block.js +100 -0
- package/plugins/fancy-scan/index.js +410 -21
- package/plugins/fancy-scan/prompts/01-analysis-prompt.md +302 -0
- package/plugins/fancy-scan/scripts/append-block.js +725 -0
- package/plugins/fancy-scan/scripts/report-validate.js +824 -0
- package/plugins/fancy-scan/scripts/run-scan.js +169 -22
- package/plugins/fancy-scan/scripts/scrapers/fanqie-rank-scraper.js +145 -16
- package/plugins/fancy-scan/scripts/scrapers/jjwxc-rank-scraper.js +179 -49
- package/plugins/fancy-scan/scripts/scrapers/qimao-rank-scraper.js +38 -20
- package/plugins/fancy-scan/templates/blocks.js +155 -0
- package/plugins/fancy-scan/templates/platforms.js +372 -0
- package/plugins/fancy-scan/templates/report-schema.js +90 -0
- package/plugins/fancy-scan/templates.js +545 -146
- package/plugins/fancy-scan/tool-deploy.js +335 -0
- package/plugins/fancy-scan/wait-turn.js +46 -0
- package/plugins/fancy-story-builder/db/README.md +106 -0
- package/plugins/fancy-story-builder/db/client.js +228 -0
- package/plugins/fancy-story-builder/db/commands.js +145 -0
- package/plugins/fancy-story-builder/db/ids.js +39 -0
- package/plugins/fancy-story-builder/db/import.js +798 -0
- package/plugins/fancy-story-builder/db/project.js +299 -0
- package/plugins/fancy-story-builder/db/query.js +361 -0
- package/plugins/fancy-story-builder/db/schema.js +417 -0
- package/plugins/fancy-story-builder/db/stage-sync.js +238 -0
- package/plugins/fancy-story-builder/flesh-output.js +19 -6
- package/plugins/fancy-story-builder/index.js +4105 -396
- package/plugins/fancy-story-builder/patch-driver.js +390 -0
- package/plugins/fancy-story-builder/prompts/01-skeleton-prompt.md +29 -19
- package/plugins/fancy-story-builder/prompts/02-flesh-prompt.md +72 -10
- package/plugins/fancy-story-builder/prompts/03-rhythm-prompt.md +40 -5
- package/plugins/fancy-story-builder/prompts/04-writing-prompt.md +37 -0
- package/plugins/fancy-story-builder/purpose-template.md +42 -66
- package/plugins/fancy-story-builder/references/formulas-skeleton.json +229 -56
- package/plugins/fancy-story-builder/references/genre-profiles.json +3514 -0
- package/plugins/fancy-story-builder/references/glossary.md +15 -6
- package/plugins/fancy-story-builder/references/index.md +97 -6
- package/plugins/fancy-story-builder/references//350/212/202/345/245/217/346/214/207/345/257/274/00-index.md +1 -1
- package/plugins/fancy-story-builder/references//350/212/202/345/245/217/346/214/207/345/257/274/07-audit.md +25 -7
- package/plugins/fancy-story-builder/references//350/241/200/350/202/211/346/214/207/345/257/274/01-character.md +5 -1
- package/plugins/fancy-story-builder/references//350/241/200/350/202/211/346/214/207/345/257/274/03-relations.md +6 -2
- package/plugins/fancy-story-builder/references//350/241/200/350/202/211/346/214/207/345/257/274/05-audit.md +15 -5
- package/plugins/fancy-story-builder/references//351/252/250/346/236/266/346/214/207/345/257/274/00-index.md +1 -1
- package/plugins/fancy-story-builder/references//351/252/250/346/236/266/346/214/207/345/257/274/01-positioning.md +2 -1
- package/plugins/fancy-story-builder/references//351/252/250/346/236/266/346/214/207/345/257/274/03-entities.md +1 -1
- package/plugins/fancy-story-builder/references//351/252/250/346/236/266/346/214/207/345/257/274/06-audit.md +19 -1
- package/plugins/fancy-story-builder/schema-setting.json +1100 -0
- package/plugins/fancy-story-builder/schema-setting.md +201 -293
- package/plugins/fancy-story-builder/scripts/brief-validate.js +164 -0
- package/plugins/fancy-story-builder/scripts/content-quality.js +4 -1
- package/plugins/fancy-story-builder/scripts/flesh-validate.js +135 -12
- package/plugins/fancy-story-builder/scripts/graph-utils.js +146 -47
- package/plugins/fancy-story-builder/scripts/reference-log.js +110 -0
- package/plugins/fancy-story-builder/scripts/rhythm-validate.js +100 -4
- package/plugins/fancy-story-builder/scripts/schema-pages.js +2668 -0
- package/plugins/fancy-story-builder/scripts/semantic-lint.js +5 -1
- package/plugins/fancy-story-builder/scripts/skeleton-validate.js +158 -17
- package/plugins/fancy-story-builder/scripts/stage-batches.js +36 -2
- package/plugins/fancy-story-builder/scripts/stage-contract.js +12 -0
- package/plugins/fancy-story-builder/scripts/story-artifacts.js +933 -0
- package/plugins/fancy-story-builder/scripts/upstream-query.js +1 -1
- package/plugins/fancy-story-builder/scripts/v2/baseline.js +112 -0
- package/plugins/fancy-story-builder/scripts/v2/cast.js +517 -0
- package/plugins/fancy-story-builder/scripts/v2/context.js +434 -0
- package/plugins/fancy-story-builder/scripts/v2/contracts.js +493 -0
- package/plugins/fancy-story-builder/scripts/v2/core.js +419 -0
- package/plugins/fancy-story-builder/scripts/v2/engine.js +1068 -0
- package/plugins/fancy-story-builder/scripts/v2/export-markdown.js +794 -0
- package/plugins/fancy-story-builder/scripts/v2/genre.js +195 -0
- package/plugins/fancy-story-builder/scripts/v2/knowledge.js +321 -0
- package/plugins/fancy-story-builder/scripts/v2/macro.js +409 -0
- package/plugins/fancy-story-builder/scripts/v2/manifest.js +538 -0
- package/plugins/fancy-story-builder/scripts/v2/migrate.js +137 -0
- package/plugins/fancy-story-builder/scripts/v2/prompts.js +1335 -0
- package/plugins/fancy-story-builder/scripts/v2/rhythm.js +618 -0
- package/plugins/fancy-story-builder/scripts/v2/scale.js +843 -0
- package/plugins/fancy-story-builder/scripts/v2/snapshot.js +121 -0
- package/plugins/fancy-story-builder/scripts/v2/stages.js +273 -0
- package/plugins/fancy-story-builder/scripts/v2/writing.js +216 -0
- package/plugins/fancy-story-builder/storydb/README.md +271 -0
- package/plugins/fancy-story-builder/storydb/cli.js +458 -0
- package/plugins/fancy-story-builder/storydb/client.js +351 -0
- package/plugins/fancy-story-builder/storydb/compile.js +452 -0
- package/plugins/fancy-story-builder/storydb/context.js +288 -0
- package/plugins/fancy-story-builder/storydb/definitions.js +699 -0
- package/plugins/fancy-story-builder/storydb/driver.js +106 -0
- package/plugins/fancy-story-builder/storydb/dsh-tools.js +313 -0
- package/plugins/fancy-story-builder/storydb/export.js +175 -0
- package/plugins/fancy-story-builder/storydb/ids.js +31 -0
- package/plugins/fancy-story-builder/storydb/impact.js +170 -0
- package/plugins/fancy-story-builder/storydb/import.js +559 -0
- package/plugins/fancy-story-builder/storydb/mcp.js +99 -0
- package/plugins/fancy-story-builder/storydb/prompts.js +952 -0
- package/plugins/fancy-story-builder/storydb/stage-command.js +379 -0
- package/plugins/fancy-story-builder/storydb/stage.js +426 -0
- package/plugins/fancy-story-builder/storydb/store.js +891 -0
- package/plugins/fancy-story-builder/storydb/tools.js +592 -0
- package/plugins/fancy-story-builder/storydb/validation.js +183 -0
- package/plugins/fancy-story-builder/templates/v2/10-scale/scale-template.json +144 -0
- package/plugins/fancy-story-builder/templates/v2/20-macro/emotion-template.json +15 -0
- package/plugins/fancy-story-builder/templates/v2/20-macro/endings-template.json +16 -0
- package/plugins/fancy-story-builder/templates/v2/20-macro/lines-template.json +28 -0
- package/plugins/fancy-story-builder/templates/v2/20-macro/roles-template.json +10 -0
- package/plugins/fancy-story-builder/templates/v2/20-macro/story-template.json +19 -0
- package/plugins/fancy-story-builder/templates/v2/20-macro/volumes-template.json +24 -0
- package/plugins/fancy-story-builder/templates/v2/20-macro/world-template.json +24 -0
- package/plugins/fancy-story-builder/templates/v2/30-cast/characters-template.json +49 -0
- package/plugins/fancy-story-builder/templates/v2/30-cast/climaxes-template.json +14 -0
- package/plugins/fancy-story-builder/templates/v2/30-cast/conflicts-template.json +30 -0
- package/plugins/fancy-story-builder/templates/v2/30-cast/factions-template.json +18 -0
- package/plugins/fancy-story-builder/templates/v2/30-cast/foreshadows-template.json +29 -0
- package/plugins/fancy-story-builder/templates/v2/30-cast/information-template.json +29 -0
- package/plugins/fancy-story-builder/templates/v2/30-cast/items-template.json +13 -0
- package/plugins/fancy-story-builder/templates/v2/30-cast/locations-template.json +12 -0
- package/plugins/fancy-story-builder/templates/v2/30-cast/payoffs-template.json +16 -0
- package/plugins/fancy-story-builder/templates/v2/30-cast/relations-template.json +44 -0
- package/plugins/fancy-story-builder/templates/v2/40-rhythm/chapter-map-template.json +81 -0
- package/plugins/fancy-story-builder/templates/v2/40-rhythm/outline-template.json +162 -0
- package/plugins/fancy-story-builder/templates/v2/50-writing/ledger-template.json +46 -0
- package/plugins/fancy-story-builder/templates/v2/60-deslop/report-template.json +10 -0
- package/plugins/fancy-story-builder/templates//350/212/202/345/245/217//346/227/266/345/272/217/346/230/240/345/260/204/346/250/241/346/235/277.json +267 -44
- package/plugins/fancy-story-builder/templates//350/241/200/350/202/211//344/270/226/347/225/214/350/247/202/350/247/204/345/210/231/346/250/241/346/235/277.json +121 -19
- package/plugins/fancy-story-builder/templates//350/241/200/350/202/211//345/205/263/351/224/256/351/201/223/345/205/267/346/250/241/346/235/277.json +129 -21
- package/plugins/fancy-story-builder/templates//350/241/200/350/202/211//345/205/266/344/273/226/345/256/236/344/275/223/346/250/241/346/235/277.json +62 -0
- package/plugins/fancy-story-builder/templates//350/241/200/350/202/211//345/206/262/347/252/201/346/250/241/346/235/277.json +76 -17
- package/plugins/fancy-story-builder/templates//350/241/200/350/202/211//345/211/247/346/203/205/346/250/241/346/235/277.json +56 -8
- package/plugins/fancy-story-builder/templates//350/241/200/350/202/211//345/212/277/345/212/233/346/250/241/346/235/277.json +63 -13
- package/plugins/fancy-story-builder/templates//350/241/200/350/202/211//345/234/260/347/202/271/346/250/241/346/235/277.json +34 -6
- package/plugins/fancy-story-builder/templates//350/241/200/350/202/211//347/273/223/345/261/200/346/250/241/346/235/277.json +81 -18
- package/plugins/fancy-story-builder/templates//350/241/200/350/202/211//350/247/222/350/211/262/346/250/241/346/235/277.json +202 -42
- package/plugins/fancy-story-builder/templates//351/252/250/346/236/266//345/205/263/347/263/273/346/250/241/346/235/277.json +160 -27
- package/plugins/fancy-story-builder/templates//351/252/250/346/236/266//345/256/236/344/275/223/346/250/241/346/235/277.json +217 -41
- package/plugins/fancy-story-builder/templates//351/252/250/346/236/266//346/241/206/346/236/266/346/250/241/346/235/277.json +215 -41
- package/plugins/fancy-story-builder/templates.js +272 -74
- package/plugins/fancy-story-builder/wiki-references.js +429 -93
- package/plugins/fancy-story-builder/writing-driver.js +336 -9
- package/plugins/fancy-topic/index.js +414 -61
- package/plugins/fancy-topic/prompts/01-exploration-prompt.md +115 -0
- package/plugins/fancy-topic/scripts/save-suggestion.js +219 -0
- package/plugins/fancy-topic/scripts/select-topic.js +106 -63
- package/plugins/fancy-topic/templates.js +225 -122
- package/plugins/fancy-topic/tool-deploy.js +200 -0
- package/lib/safe-rm.js +0 -94
|
@@ -18,10 +18,11 @@
|
|
|
18
18
|
*/
|
|
19
19
|
|
|
20
20
|
import fs from 'fs';
|
|
21
|
+
import { atomicWriteFile } from '#atomic-write';
|
|
21
22
|
import path from 'path';
|
|
22
23
|
import { pathToFileURL } from 'url';
|
|
23
24
|
import { chromium } from 'playwright-core';
|
|
24
|
-
import { getArg, localDateStamp, runCli } from '#scraper-utils';
|
|
25
|
+
import { getArg, localDateStamp, runCli, renderIntroLines, sanitizeBookFields, stripThousands } from '#scraper-utils';
|
|
25
26
|
|
|
26
27
|
const BASE_URL = "https://www.jjwxc.net/topten.php";
|
|
27
28
|
|
|
@@ -33,9 +34,66 @@ const RANK_TYPES = [
|
|
|
33
34
|
];
|
|
34
35
|
|
|
35
36
|
const DETAIL_CHUNK = 6;
|
|
36
|
-
const DETAIL_LIMIT =
|
|
37
|
+
const DETAIL_LIMIT = 300;
|
|
37
38
|
const TOP = 10;
|
|
38
39
|
|
|
40
|
+
// ---------------------------------------------------------------------------
|
|
41
|
+
// 详情页解析(Node 侧纯函数,可被单元测试直接调用)
|
|
42
|
+
// ---------------------------------------------------------------------------
|
|
43
|
+
|
|
44
|
+
/**
|
|
45
|
+
* HTML 片段 → 纯文本。
|
|
46
|
+
*
|
|
47
|
+
* 晋江的"文案"是多段的,段间是 `<br>`(不是 `<p>`),要转成换行才能读。
|
|
48
|
+
* 页面里还有 ` `(晋江排版大量用它),不清掉会到处是空白。
|
|
49
|
+
*/
|
|
50
|
+
export function htmlToText(html) {
|
|
51
|
+
return String(html || '')
|
|
52
|
+
.replace(/<br\s*\/?>/gi, '\n')
|
|
53
|
+
.replace(/<\/p>/gi, '\n\n')
|
|
54
|
+
.replace(/<[^>]+>/g, '')
|
|
55
|
+
.replace(/ /gi, ' ')
|
|
56
|
+
.replace(/&/gi, '&')
|
|
57
|
+
.replace(/</gi, '<')
|
|
58
|
+
.replace(/>/gi, '>')
|
|
59
|
+
.replace(/"/gi, '"')
|
|
60
|
+
.replace(/'/gi, "'")
|
|
61
|
+
.replace(/\r\n?/g, '\n')
|
|
62
|
+
.replace(/[ \t]+\n/g, '\n')
|
|
63
|
+
.replace(/\n{3,}/g, '\n\n')
|
|
64
|
+
.trim();
|
|
65
|
+
}
|
|
66
|
+
|
|
67
|
+
/**
|
|
68
|
+
* 从详情页 HTML 里解析**简介**和**内容标签**。
|
|
69
|
+
*
|
|
70
|
+
* 简介:`<div id="novelintro" itemprop="description">…</div>`
|
|
71
|
+
* 文案有多段,段间是 `<br>`(见 htmlToText)。
|
|
72
|
+
*
|
|
73
|
+
* 内容标签:`<a href="//www.jjwxc.net/bookbase.php?bq=32">宫廷侯爵</a>`
|
|
74
|
+
* —— 整页里只有这些链接指向 bookbase。实测 12/12 本,
|
|
75
|
+
* **bq 链接数 == 标签数**(3~6 个/本),不会混进别的东西。
|
|
76
|
+
*
|
|
77
|
+
* 标签是**平台官方内容标签**,题材与元素套路混在一起
|
|
78
|
+
* (如 都市 / 破镜重圆 / 天之骄子 / 高岭之花 / 暗恋),
|
|
79
|
+
* 且**没有"主分类"概念** —— 这点与番茄不同(番茄第一个标签是主分类)。
|
|
80
|
+
*/
|
|
81
|
+
export function parseNovelDetail(html) {
|
|
82
|
+
const out = { intro: '', tags: [] };
|
|
83
|
+
const m = String(html || '').match(/<div[^>]*id="novelintro"[^>]*>([\s\S]*?)<\/div>/i);
|
|
84
|
+
if (m) out.intro = htmlToText(m[1]);
|
|
85
|
+
|
|
86
|
+
const tags = [];
|
|
87
|
+
const re = /<a[^>]*href=["'][^"']*bookbase\.php\?bq=\d+[^"']*["'][^>]*>([^<]{1,30})<\/a>/gi;
|
|
88
|
+
let t;
|
|
89
|
+
while ((t = re.exec(String(html || ''))) !== null) {
|
|
90
|
+
const name = htmlToText(t[1]);
|
|
91
|
+
if (name && !tags.includes(name)) tags.push(name);
|
|
92
|
+
}
|
|
93
|
+
out.tags = tags;
|
|
94
|
+
return out;
|
|
95
|
+
}
|
|
96
|
+
|
|
39
97
|
// ---------------------------------------------------------------------------
|
|
40
98
|
// 命令行参数解析
|
|
41
99
|
// ---------------------------------------------------------------------------
|
|
@@ -76,14 +134,18 @@ async function extractRankData(page) {
|
|
|
76
134
|
var typeCell = cells[3];
|
|
77
135
|
var statusCell = cells[4];
|
|
78
136
|
var wordsCell = cells[5];
|
|
79
|
-
var scoreCell = cells[6];
|
|
137
|
+
var scoreCell = cells[6] || null;
|
|
80
138
|
|
|
81
139
|
var author = (authorCell.innerText || '').trim();
|
|
82
140
|
var title = (titleCell.innerText || '').trim();
|
|
83
141
|
var type = (typeCell.innerText || '').trim();
|
|
84
142
|
var status = (statusCell.innerText || '').trim();
|
|
85
143
|
var words = (wordsCell.innerText || '').trim();
|
|
86
|
-
|
|
144
|
+
// 积分列(第 7 列)可能整列缺失 —— 榜单页偶尔只有 6 列。
|
|
145
|
+
// 之前直接取 cells[6].innerText 会抛 TypeError,
|
|
146
|
+
// page.evaluate 整体 reject → 整榜不落盘。
|
|
147
|
+
// 缺积分的书照样收,积分留空。
|
|
148
|
+
var score = scoreCell ? (scoreCell.innerText || '').trim() : '';
|
|
87
149
|
|
|
88
150
|
// 排除标题行
|
|
89
151
|
if (author === '作者' || title === '作品') continue;
|
|
@@ -112,49 +174,72 @@ async function extractRankData(page) {
|
|
|
112
174
|
});
|
|
113
175
|
}
|
|
114
176
|
|
|
177
|
+
// ---------------------------------------------------------------------------
|
|
178
|
+
// 单本书的字段渲染(纯函数,可被单元测试直接调用)
|
|
179
|
+
// ---------------------------------------------------------------------------
|
|
180
|
+
|
|
181
|
+
/**
|
|
182
|
+
* 一本书 → markdown 字段行(不含 `### #N 书名` 标题行)。
|
|
183
|
+
*
|
|
184
|
+
* 抽出来是为了能单测这两条格式规则 —— 它们原本埋在 scrapeRank 的
|
|
185
|
+
* async 函数里,测不到:
|
|
186
|
+
*
|
|
187
|
+
* 1. **积分不带千分位逗号**:晋江榜单页写的是 "1,099,899,136",
|
|
188
|
+
* 存成纯数字,下游才好比较/求和,模型抄数字时也不会把逗号抄进去
|
|
189
|
+
* ("327,650,784" 和 "327650784" 在模型的算术里不是一回事);
|
|
190
|
+
* 2. **简介是 `- ` 列表项**:单换行用硬换行保留,空行才是段落分隔;
|
|
191
|
+
* 每段首尾无空白,作品页也单独作为一个列表项。
|
|
192
|
+
*
|
|
193
|
+
* @param {object} b - 一本书
|
|
194
|
+
* @returns {string[]} markdown 行
|
|
195
|
+
*/
|
|
196
|
+
export function renderBookLines(raw) {
|
|
197
|
+
const b = sanitizeBookFields(raw);
|
|
198
|
+
const lines = [];
|
|
199
|
+
if (b.author) lines.push(`- **作者:** ${b.author}`);
|
|
200
|
+
if (b.type) lines.push(`- **题材:** ${b.type}`);
|
|
201
|
+
if (b.status) lines.push(`- **状态:** ${b.status}`);
|
|
202
|
+
if (b.words) lines.push(`- **字数:** ${stripThousands(b.words)}`);
|
|
203
|
+
// 积分是 10 亿级的,榜单页必带千分位逗号 —— 写盘前去掉。
|
|
204
|
+
if (b.score) lines.push(`- **积分:** ${stripThousands(b.score)}`);
|
|
205
|
+
if (b.tags && b.tags.length) lines.push(`- **标签:** ${b.tags.join('、')}`);
|
|
206
|
+
// 简介和其他字段一样是 `- ` 列表项;单换行用硬换行,空行才是段落分隔。
|
|
207
|
+
// 之前是「空行 + `**简介**` + 简介整块」—— 没有冒号,且和上下文粘成一段,
|
|
208
|
+
// 而且简介首尾的空白字符原样留在文件里。
|
|
209
|
+
lines.push(...renderIntroLines(b.intro || ''));
|
|
210
|
+
if (b.novelid) lines.push(`- [作品页](https://www.jjwxc.net/onebook.php?novelid=${b.novelid})`);
|
|
211
|
+
return lines;
|
|
212
|
+
}
|
|
213
|
+
|
|
115
214
|
// ---------------------------------------------------------------------------
|
|
116
215
|
// 详情页提取(gb18030 + itemprop 微数据)
|
|
117
216
|
// ---------------------------------------------------------------------------
|
|
118
217
|
|
|
119
218
|
async function fetchDetails(page, ids) {
|
|
120
|
-
//
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
const
|
|
133
|
-
|
|
219
|
+
// 分块抓:一次把 200 本的 HTML 全带过 evaluate 边界会有几十 MB 的
|
|
220
|
+
// 序列化开销,分块后每块才 1MB 上下。
|
|
221
|
+
const map = {};
|
|
222
|
+
for (let i = 0; i < ids.length; i += DETAIL_CHUNK) {
|
|
223
|
+
const chunk = ids.slice(i, i + DETAIL_CHUNK);
|
|
224
|
+
// 浏览器侧只负责"把解码后的原始 HTML 拿回来",
|
|
225
|
+
// **解析放在 Node 侧**(parseNovelDetail)—— 这样解析逻辑能被单元测试直接调用。
|
|
226
|
+
const details = await page.evaluate(async (list) => {
|
|
227
|
+
const results = [];
|
|
228
|
+
for (const id of list) {
|
|
229
|
+
try {
|
|
230
|
+
// redirect: 'error' 防止 onebook.php 重定向到外站造成 SSRF
|
|
231
|
+
const response = await fetch(`/onebook.php?novelid=${id}`, { redirect: 'error' });
|
|
232
|
+
const buffer = await response.arrayBuffer();
|
|
233
|
+
results.push({ id: id, html: new TextDecoder('gb18030').decode(buffer) });
|
|
234
|
+
} catch (e) {
|
|
235
|
+
results.push({ id: id, err: String(e) });
|
|
134
236
|
}
|
|
135
|
-
|
|
136
|
-
const statusMatch = html.match(/itemprop="updataStatus"[^>]*>\s*([^\s<]{1,6})/) || [];
|
|
137
|
-
const status = statusMatch[1] || (html.includes('连载中') ? '连载中' : html.includes('已完结') ? '已完结' : '');
|
|
138
|
-
|
|
139
|
-
results.push({
|
|
140
|
-
id: id,
|
|
141
|
-
collect: prop('collectedCount'),
|
|
142
|
-
nutrition: prop('nutritionCount'),
|
|
143
|
-
score: prop('scoreCount'),
|
|
144
|
-
review: prop('reviewCount'),
|
|
145
|
-
words: prop('wordCount'),
|
|
146
|
-
status: status
|
|
147
|
-
});
|
|
148
|
-
} catch (e) {
|
|
149
|
-
results.push({ id: id, err: String(e) });
|
|
150
237
|
}
|
|
238
|
+
return results;
|
|
239
|
+
}, chunk);
|
|
240
|
+
for (const d of details) {
|
|
241
|
+
map[d.id] = d.err ? { err: d.err } : parseNovelDetail(d.html);
|
|
151
242
|
}
|
|
152
|
-
return results;
|
|
153
|
-
}, ids);
|
|
154
|
-
|
|
155
|
-
const map = {};
|
|
156
|
-
for (const d of details) {
|
|
157
|
-
map[d.id] = d;
|
|
158
243
|
}
|
|
159
244
|
return map;
|
|
160
245
|
}
|
|
@@ -174,6 +259,7 @@ async function scrapeRank(browser, rankTypeId, channelId) {
|
|
|
174
259
|
const chLabel = channelId === "0" ? "全站" : `频道${channelId}`;
|
|
175
260
|
|
|
176
261
|
const page = await browser.newPage();
|
|
262
|
+
const partialReasons = [];
|
|
177
263
|
|
|
178
264
|
try {
|
|
179
265
|
await page.goto(url, { waitUntil: 'networkidle', timeout: 30000 });
|
|
@@ -193,6 +279,36 @@ async function scrapeRank(browser, rankTypeId, channelId) {
|
|
|
193
279
|
}
|
|
194
280
|
});
|
|
195
281
|
console.error(` ✓ 列表:${data.channels.length} 个频道,共 ${totalBooks} 本`);
|
|
282
|
+
// 解析到 0 本 = 页面结构变了 / 选择器失效 / 没加载出来。
|
|
283
|
+
// extractRankData 永远至少返回一个 `{ name: '全站', books: [] }`,
|
|
284
|
+
// 所以上面的 `!data.channels.length` 守卫**永远不触发** ——
|
|
285
|
+
// 结果是写出一个只有 "## 全站 — 0 本" 的空文件,
|
|
286
|
+
// 上层 `ok: scanFiles.length > 0` 还判定成功,用户拿到空报告。
|
|
287
|
+
if (totalBooks === 0) {
|
|
288
|
+
console.error(`❌ 晋江-${rt.label}(${chLabel})采集失败:页面解析到 0 本书(选择器可能失效或页面未加载)。请人工打开 ${url} 确认。`);
|
|
289
|
+
return null;
|
|
290
|
+
}
|
|
291
|
+
|
|
292
|
+
// 逐本进详情页补**简介**和**内容标签**(列表页没有这两个字段)。
|
|
293
|
+
// 此前 fetchDetails 写好了却从没被调用 —— 晋江产出的 md 里因此一直没有
|
|
294
|
+
// 简介和标签,提示词里"提取简介标签""简介标签"全是空的。
|
|
295
|
+
const detailIds = [];
|
|
296
|
+
for (const ch of data.channels) {
|
|
297
|
+
for (const b of ch.books) {
|
|
298
|
+
if (b.novelid && !detailIds.includes(b.novelid)) detailIds.push(b.novelid);
|
|
299
|
+
}
|
|
300
|
+
}
|
|
301
|
+
const detailIdsToFetch = detailIds.slice(0, DETAIL_LIMIT);
|
|
302
|
+
if (detailIds.length > DETAIL_LIMIT) {
|
|
303
|
+
console.error(` ⚠ 详情页只采前 ${DETAIL_LIMIT} 本(共 ${detailIds.length} 本),其余没有简介和标签`);
|
|
304
|
+
}
|
|
305
|
+
let detailMap = {};
|
|
306
|
+
if (!LIST_ONLY && detailIdsToFetch.length) {
|
|
307
|
+
console.error(` → 采详情页 ${detailIdsToFetch.length} 本(简介 + 内容标签)`);
|
|
308
|
+
detailMap = await fetchDetails(page, detailIdsToFetch);
|
|
309
|
+
const okCount = detailIdsToFetch.filter((id) => detailMap[id] && !detailMap[id].err).length;
|
|
310
|
+
console.error(` ✓ 详情页:${okCount}/${detailIdsToFetch.length} 本取到`);
|
|
311
|
+
}
|
|
196
312
|
|
|
197
313
|
// 构建输出内容
|
|
198
314
|
const now = new Date().toISOString();
|
|
@@ -203,22 +319,28 @@ async function scrapeRank(browser, rankTypeId, channelId) {
|
|
|
203
319
|
`- 抓取时间:${now}`,
|
|
204
320
|
`- 频道数:${data.channels.length}`,
|
|
205
321
|
`- 总条目数:${totalBooks}`,
|
|
322
|
+
...(detailIds.length > DETAIL_LIMIT
|
|
323
|
+
? [`- 详情覆盖:${DETAIL_LIMIT} / ${detailIds.length}(其余条目没有简介与标签)`]
|
|
324
|
+
: []),
|
|
206
325
|
"",
|
|
207
326
|
"---",
|
|
208
327
|
"",
|
|
209
328
|
];
|
|
210
329
|
|
|
330
|
+
|
|
211
331
|
for (const ch of data.channels) {
|
|
212
332
|
lines.push(`## ${ch.name} — ${ch.books.length} 本`, "");
|
|
213
333
|
for (let i = 0; i < ch.books.length; i++) {
|
|
214
334
|
const b = ch.books[i];
|
|
335
|
+
const d = b.novelid ? detailMap[b.novelid] : null;
|
|
336
|
+
if (d && !d.err) {
|
|
337
|
+
b.intro = d.intro;
|
|
338
|
+
b.tags = d.tags;
|
|
339
|
+
} else if (d && d.err) {
|
|
340
|
+
partialReasons.push(`详情页取数失败 ${b.novelid}: ${d.err}`);
|
|
341
|
+
}
|
|
215
342
|
lines.push(`### #${i + 1} ${b.title}`);
|
|
216
|
-
|
|
217
|
-
if (b.type) lines.push(`**题材:** ${b.type}`);
|
|
218
|
-
if (b.status) lines.push(`**状态:** ${b.status}`);
|
|
219
|
-
if (b.words) lines.push(`**字数:** ${b.words}`);
|
|
220
|
-
if (b.score) lines.push(`**积分:** ${b.score}`);
|
|
221
|
-
if (b.novelid) lines.push(`[作品页](https://www.jjwxc.net/onebook.php?novelid=${b.novelid})`);
|
|
343
|
+
lines.push(...renderBookLines(b));
|
|
222
344
|
lines.push("");
|
|
223
345
|
}
|
|
224
346
|
lines.push("---", "");
|
|
@@ -226,7 +348,8 @@ async function scrapeRank(browser, rankTypeId, channelId) {
|
|
|
226
348
|
|
|
227
349
|
return {
|
|
228
350
|
content: lines.join("\n"),
|
|
229
|
-
partial:
|
|
351
|
+
partial: partialReasons.length > 0,
|
|
352
|
+
partialReasons,
|
|
230
353
|
};
|
|
231
354
|
} finally {
|
|
232
355
|
await page.close();
|
|
@@ -283,24 +406,31 @@ async function main() {
|
|
|
283
406
|
for (const rt of rankTypes) {
|
|
284
407
|
for (const ch of channels) {
|
|
285
408
|
try {
|
|
409
|
+
// rtInfo 必须在**读它的所有地方之前**声明。
|
|
410
|
+
// 之前它声明在 partialReasons 那个 if 的后面,
|
|
411
|
+
// 而上面的 if 会先读 rtInfo → TDZ ReferenceError
|
|
412
|
+
// → 整榜已经采好的 content 被 catch 吞掉、整榜不落盘。
|
|
413
|
+
// 只要有一本详情页失败就触发,等于"详情页偶发失败 = 整个榜单没了"。
|
|
414
|
+
const rtInfo = RANK_TYPES.find((r) => r.id === rt);
|
|
286
415
|
const result = await scrapeRank(browser, rt, ch);
|
|
287
416
|
if (!result) {
|
|
288
417
|
failed++;
|
|
289
|
-
const rtInfo = RANK_TYPES.find((r) => r.id === rt);
|
|
290
418
|
partialReasons.push(`${rtInfo ? rtInfo.label : rt}: no usable data`);
|
|
291
419
|
continue;
|
|
292
420
|
}
|
|
293
421
|
if (result.partial) {
|
|
294
422
|
partial = true;
|
|
295
423
|
}
|
|
424
|
+
if (result.partialReasons && result.partialReasons.length) {
|
|
425
|
+
for (const r of result.partialReasons) partialReasons.push(`${rtInfo ? rtInfo.label : rt}: ${r}`);
|
|
426
|
+
}
|
|
296
427
|
|
|
297
|
-
const rtInfo = RANK_TYPES.find((r) => r.id === rt);
|
|
298
428
|
const date = localDateStamp();
|
|
299
429
|
const chLabel = ch === "0" ? "全站" : `频道${ch}`;
|
|
300
430
|
const filename = `晋江${rtInfo.label}_${chLabel}_${date}.md`;
|
|
301
431
|
fs.mkdirSync(OUTDIR, { recursive: true });
|
|
302
432
|
const filepath = path.join(OUTDIR, filename);
|
|
303
|
-
|
|
433
|
+
atomicWriteFile(filepath, result.content);
|
|
304
434
|
written++;
|
|
305
435
|
files.push(filepath);
|
|
306
436
|
console.error(` ✓ 已保存: ${filepath}`);
|
|
@@ -14,10 +14,11 @@
|
|
|
14
14
|
*/
|
|
15
15
|
|
|
16
16
|
import fs from 'fs';
|
|
17
|
+
import { atomicWriteFile } from '#atomic-write';
|
|
17
18
|
import path from 'path';
|
|
18
19
|
import { pathToFileURL } from 'url';
|
|
19
20
|
import { chromium } from 'playwright-core';
|
|
20
|
-
import { getArg, localDateStamp, runCli } from '#scraper-utils';
|
|
21
|
+
import { getArg, localDateStamp, runCli, renderIntroLines, sanitizeBookFields, sanitizeMarkdownInline } from '#scraper-utils';
|
|
21
22
|
|
|
22
23
|
const RANK_URL = "https://www.qimao.com/paihang";
|
|
23
24
|
|
|
@@ -395,25 +396,22 @@ function renderMarkdown(ch, rt, period, url, books, rawCount, now = new Date().t
|
|
|
395
396
|
"",
|
|
396
397
|
];
|
|
397
398
|
|
|
398
|
-
for (const
|
|
399
|
+
for (const raw of books) {
|
|
400
|
+
const b = sanitizeBookFields(raw);
|
|
399
401
|
try {
|
|
400
402
|
lines.push(`### #${b.rank} ${b.title}`);
|
|
401
|
-
if (b.author) lines.push(
|
|
402
|
-
if (b.genre) lines.push(
|
|
403
|
-
if (b.subGenre) lines.push(
|
|
404
|
-
if (b.status) lines.push(
|
|
405
|
-
if (b.words) lines.push(
|
|
406
|
-
if (b.heat) lines.push(
|
|
407
|
-
|
|
408
|
-
|
|
409
|
-
|
|
410
|
-
|
|
411
|
-
|
|
412
|
-
|
|
413
|
-
const paras = desc.split(/\n+/).map((p) => p.trim()).filter(Boolean);
|
|
414
|
-
lines.push(paras[0]);
|
|
415
|
-
for (const p of paras.slice(1)) lines.push("", p);
|
|
416
|
-
}
|
|
403
|
+
if (b.author) lines.push(`- **作者:** ${b.author}`);
|
|
404
|
+
if (b.genre) lines.push(`- **题材:** ${b.genre}`);
|
|
405
|
+
if (b.subGenre) lines.push(`- **子分类:** ${b.subGenre}`);
|
|
406
|
+
if (b.status) lines.push(`- **状态:** ${b.status}`);
|
|
407
|
+
if (b.words) lines.push(`- **字数:** ${b.words}`);
|
|
408
|
+
if (b.heat) lines.push(`- **热度:** ${b.heat}`);
|
|
409
|
+
// 简介和其他字段一样是 `- ` 列表项;单换行用硬换行,空行才是段落分隔。
|
|
410
|
+
// 之前是「空行 + `**简介**` + 分段」—— 没有冒号、前后多一个空行,
|
|
411
|
+
// 而且段首尾的空白字符原样留在文件里。
|
|
412
|
+
lines.push(...renderIntroLines(cleanDesc(b.desc)));
|
|
413
|
+
// 作品页链接放在简介之后,与起点/番茄/晋江产物保持同一字段顺序。
|
|
414
|
+
if (b.url) lines.push(`- [作品页](${b.url})`);
|
|
417
415
|
lines.push("", "---", "");
|
|
418
416
|
} catch (bookErr) {
|
|
419
417
|
lines.push("", "---", "");
|
|
@@ -567,7 +565,27 @@ async function main() {
|
|
|
567
565
|
}
|
|
568
566
|
try {
|
|
569
567
|
for (const target of targets) {
|
|
570
|
-
|
|
568
|
+
// 每个 target 单独兜住异常(2026-10-29 审计)。
|
|
569
|
+
//
|
|
570
|
+
// scrapeRank 的主体被自己的 try/catch 包着,普通失败会返回 null;
|
|
571
|
+
// 但 browser.newPage() / page.close() / fetchDetailIntros 这些在
|
|
572
|
+
// try 之外,抛出来会**跳出整个 for 循环** ——
|
|
573
|
+
// 后面所有 target 都不再采集,而用户只看到"采集失败"。
|
|
574
|
+
let result = null;
|
|
575
|
+
try {
|
|
576
|
+
result = await scrapeRank(browser, target.channel, target.rankType, target.period);
|
|
577
|
+
} catch (e) {
|
|
578
|
+
const msg = e && e.message ? e.message : String(e);
|
|
579
|
+
console.error(` ✗ 七猫-${target.channel}/${target.rankType} 采集异常:${msg}`);
|
|
580
|
+
partialReasons.push(`${target.channel}/${target.rankType} 采集异常:${msg}`);
|
|
581
|
+
failed++;
|
|
582
|
+
// 浏览器已经死了就别再试后面的 target —— 每个都会立刻失败、白烧时间
|
|
583
|
+
if (browser.isConnected && !browser.isConnected()) {
|
|
584
|
+
console.error(' ✗ 浏览器已断开,停止剩余榜单的采集');
|
|
585
|
+
break;
|
|
586
|
+
}
|
|
587
|
+
continue;
|
|
588
|
+
}
|
|
571
589
|
if (!result) {
|
|
572
590
|
failed++;
|
|
573
591
|
continue;
|
|
@@ -577,7 +595,7 @@ async function main() {
|
|
|
577
595
|
const filename = outputFilename(target.channel, target.rankType, target.period, date);
|
|
578
596
|
fs.mkdirSync(OUTDIR, { recursive: true });
|
|
579
597
|
const filepath = path.join(OUTDIR, filename);
|
|
580
|
-
|
|
598
|
+
atomicWriteFile(filepath, result.content);
|
|
581
599
|
written++;
|
|
582
600
|
files.push(filepath);
|
|
583
601
|
// 详情页没取到简介的书:原因进 partialReasons,一路带到插件层的失败清单
|
|
@@ -0,0 +1,155 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* blocks.js —— 按平台生成的提示词**动态段落**
|
|
3
|
+
*
|
|
4
|
+
* 这些段落依赖平台的数据形态(平铺 / 分组)、热度指标与标签来源,
|
|
5
|
+
* 不能写死在静态提示词里。它们只负责**拼文本**,不做任何校验 ——
|
|
6
|
+
* 校验在 scripts/report-validate.js 与 scripts/append-block.js 里。
|
|
7
|
+
*
|
|
8
|
+
* 静态的任务契约在 prompts/01-analysis-prompt.md;
|
|
9
|
+
* 本文件生成的内容通过 `${...}` 占位符注入。
|
|
10
|
+
*/
|
|
11
|
+
|
|
12
|
+
/**
|
|
13
|
+
* 3.2 的「数据形态」段 —— **按平台说**(评审 P0-1)。
|
|
14
|
+
*
|
|
15
|
+
* 以前四个平台统一写成"题材组",起点/七猫实际是平铺的。
|
|
16
|
+
*/
|
|
17
|
+
export function dataShapeBlock(ds) {
|
|
18
|
+
if (ds.grouped && ds.genreFrom === 'group') {
|
|
19
|
+
return [
|
|
20
|
+
`- **数据形态**:文件按**题材分组** —— 每个 \`${ds.groupTitle}\` 是一个题材组,`,
|
|
21
|
+
` 组里每本书一个 \`${ds.bookHeading}\`。`,
|
|
22
|
+
` **这是分层样本,不是全市场随机样本**:每组本数相同。`,
|
|
23
|
+
];
|
|
24
|
+
}
|
|
25
|
+
if (ds.grouped) {
|
|
26
|
+
return [
|
|
27
|
+
`- **数据形态**:文件按**频道分组**(${ds.groupTitle})—— 频道是榜单范围,**不是题材**。`,
|
|
28
|
+
` 每个 \`${ds.bookHeading}\` 是一本书;题材取自每本书自己的 \`${ds.genreField}\` 字段。`,
|
|
29
|
+
` 这是**完整频道榜**,不是"每题材前 N 本"的分层样本;各题材样本量可能不均。`,
|
|
30
|
+
];
|
|
31
|
+
}
|
|
32
|
+
return [
|
|
33
|
+
`- **数据形态**:文件是**平铺的书目列表** —— 每个 \`${ds.bookHeading}\` 就是**一本书**,`,
|
|
34
|
+
` **不是题材组**。文件里**没有** \`## 某题材 — N 本\` 这样的分组标题。`,
|
|
35
|
+
` **书名永远不是题材名** —— \`#1 玄鉴仙族\` 里的"玄鉴仙族"是书名。`,
|
|
36
|
+
];
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
/**
|
|
40
|
+
* 3.3 的「题材口径」段 —— 分组平台 vs 字段平台。
|
|
41
|
+
*/
|
|
42
|
+
export function genreRuleBlock(ds, multiList = false) {
|
|
43
|
+
if (ds.genreFrom === 'group') {
|
|
44
|
+
return [
|
|
45
|
+
`- **题材以数据文件里的官方分组为准**(${ds.groupTitle} 里的题材名就是平台给的题材),`,
|
|
46
|
+
` **不要另造题材分类** —— 自造分类与采集端的分组不一致,题材频次会失真。`,
|
|
47
|
+
` 统计题材频次时**只算具体的题材组、跳过"全部"**,否则同一本书会被数两遍、占比加起来超过 100%。`,
|
|
48
|
+
` 同一本书只归入它所在的**那一个**官方题材。`,
|
|
49
|
+
];
|
|
50
|
+
}
|
|
51
|
+
return [
|
|
52
|
+
`- **题材取自每本书自己的 \`${ds.genreField}\` 字段**(如 \`${ds.genreExample}\`),`,
|
|
53
|
+
` **不是**从标题推断的 —— 标题里只有书名。`,
|
|
54
|
+
...(ds.grouped ? [
|
|
55
|
+
` **频道/分组标题只是榜单范围,不是题材** —— 不要把"全站"之类当题材。`,
|
|
56
|
+
] : []),
|
|
57
|
+
` **不要另造题材分类**:自造分类与采集端给的题材值不一致,题材频次会失真。`,
|
|
58
|
+
...(multiList ? [
|
|
59
|
+
` 同一本书在多个榜单里出现时,**它的题材字段应当一致**;若发现不一致,`,
|
|
60
|
+
` 在"一、数据与方法"里写明你采信了哪一个、为什么。`,
|
|
61
|
+
] : []),
|
|
62
|
+
` **子分类只作细分**,主表按大类聚合(细分可能有几十个、每个只有 1-2 本)。`,
|
|
63
|
+
];
|
|
64
|
+
}
|
|
65
|
+
|
|
66
|
+
/**
|
|
67
|
+
* 3.2 的「标签」规则 —— 没有官方标签字段的平台**不做标签频次**(评审 P0-4)。
|
|
68
|
+
*
|
|
69
|
+
* 实测(起点 2026-10-29):9 个文件 0 个 `**标签:**`,
|
|
70
|
+
* 简介里的 `【…】` 有出版说明(`【本作品简体实体书由次元书馆出版】`)、
|
|
71
|
+
* 营销语(`【均订2万,放心追看!】`),分隔符也不是 `+` 而是 `、`。
|
|
72
|
+
* 照旧规则统计,会把"均订2万"算成标签。
|
|
73
|
+
*/
|
|
74
|
+
export function tagRuleBlock(ds) {
|
|
75
|
+
if (ds.hasTags) {
|
|
76
|
+
return [
|
|
77
|
+
` - **标签**:用 \`**标签:**\` 行(平台官方分类,比从简介猜准);`,
|
|
78
|
+
` 那一行没有就记"无标签",**不要根据书名猜标签**`,
|
|
79
|
+
];
|
|
80
|
+
}
|
|
81
|
+
return [
|
|
82
|
+
` - **标签**:本平台的数据文件里**没有** \`**标签:**\` 字段 ——`,
|
|
83
|
+
` **不要做标签频次分析**(没有数据源)。`,
|
|
84
|
+
` 如果确实要用简介里的 \`【…】\`,那只算**推断**,且必须:`,
|
|
85
|
+
` - 只认**用 \`、\` 或 \`,\` 分隔**的标签串(多个 \`【】\` 要逐个看);`,
|
|
86
|
+
` - \`【本作品简体实体书由…出版】\`、\`【均订2万,放心追看!】\` 这类是**说明/营销语**,不是标签;`,
|
|
87
|
+
` - **标注"推断"**并报出覆盖率(如"X/N 本出现可识别的标签串");`,
|
|
88
|
+
` - 覆盖率太低就**直接删掉标签这一项分析**,不要硬凑`,
|
|
89
|
+
];
|
|
90
|
+
}
|
|
91
|
+
|
|
92
|
+
/**
|
|
93
|
+
* 3.3 的「竞争度指标」段 —— 样本形态 + 指标口径 + 高/中/低阈值。
|
|
94
|
+
*
|
|
95
|
+
* - P0-2:平铺数据的样本量极不均匀,"前 3 ÷ 总量"在 1-3 本的题材上恒为 1.000;
|
|
96
|
+
* - P1-3:高/中/低必须有阈值,否则不同日期的报告不可比;
|
|
97
|
+
* - P1-4:同质化必须"与众数比",两两比会恒为 100%。
|
|
98
|
+
*/
|
|
99
|
+
export function competitionBlock(ds, h, multiList = false) {
|
|
100
|
+
const sampleNote = ds.stratified
|
|
101
|
+
? [
|
|
102
|
+
` 这份数据是"每题材前 N 本"的分层样本,**每组本数相同**,`,
|
|
103
|
+
` 因此"书目数占比"和"头部集中度(前 N ÷ N)"在这种数据上**恒等于常数、没有区分度**:`,
|
|
104
|
+
` **不要拿它们当指标**(这类表在所有题材上并列,得不出差异)。`,
|
|
105
|
+
]
|
|
106
|
+
: [
|
|
107
|
+
` 这份数据是**榜单样本**,**各题材的样本量极不均匀**`,
|
|
108
|
+
` (同一次扫榜里,有的题材有 20 本,有的只有 1 本)。`,
|
|
109
|
+
` **样本量不均的直接后果**:`,
|
|
110
|
+
` - 只有 1-3 本的题材,"前 3 名 ÷ 总量"恒等于 1.000、"中位数"就是那一本的值 —— **没有区分度**;`,
|
|
111
|
+
...(multiList ? [
|
|
112
|
+
` - 某题材出现在**越多榜单**里,它的书目数与${h.label}总量就机械性越大 ——`,
|
|
113
|
+
` 这是**榜单选择偏差**,不是市场需求,**不能**读成"某题材需求是另一个的 N 倍"。`,
|
|
114
|
+
] : []),
|
|
115
|
+
];
|
|
116
|
+
const shareName = ds.grouped ? '需求份额' : '样本内份额';
|
|
117
|
+
const shareFormula = ds.grouped
|
|
118
|
+
? ` - **${shareName}** = 该题材${h.label}总量 ÷ 全部题材${h.label}总量(给出百分比)`
|
|
119
|
+
: ` - **${shareName}** = 该题材${h.label}总量 ÷ **样本内**全部题材${h.label}总量(给出百分比);`;
|
|
120
|
+
return [
|
|
121
|
+
`- **竞争度指标(5 项)—— 按下面的口径算,不要凭感觉打分**:`,
|
|
122
|
+
...sampleNote,
|
|
123
|
+
` **每个指标都要带样本量**,并按样本量分档:`,
|
|
124
|
+
` - **样本量 ≥5 本** → 正常打分(高/中/低);`,
|
|
125
|
+
` - **3-4 本** → 头部集中度 / 同质化程度可写数值或“只描述”,差异化空间写“只描述”;上述三项都不打高/中/低;`,
|
|
126
|
+
` - **<3 本** → 头部集中度 / 同质化程度 / 差异化空间三项标“样本不足”,这三项不参与排序;`,
|
|
127
|
+
` - **份额列不受样本量分档影响**:始终按总量算百分比(可带高/中/低档),样本不足只影响其余三项打分。`,
|
|
128
|
+
` **改用${h.label}数据**:`,
|
|
129
|
+
` - **需求规模** = 该题材**有${h.label}值的书**的总量,并给出**均值**、**中位数**与**样本量**`,
|
|
130
|
+
...(ds.grouped
|
|
131
|
+
? [shareFormula]
|
|
132
|
+
: [shareFormula,
|
|
133
|
+
` **这是榜单样本内部的份额,不是市场份额** —— 报告里要写"样本内份额",不许简写成"份额"`]),
|
|
134
|
+
` - **头部集中度** = 该题材**前 3 名**${h.label}之和 ÷ 该题材${h.label}总量`,
|
|
135
|
+
` - **同质化程度** = 该题材里,与**该题材众数组合**在`,
|
|
136
|
+
` "金手指 / 主角身份 / **简介核心冲突**"三项中**至少两项一致**的书目数 ÷ N`,
|
|
137
|
+
` (只一致一项或零项 → 计 0)。`,
|
|
138
|
+
` **是"与众数比",不是"两两比"** —— 两两比的话几乎每本都能找到同类,结果恒为 100%。`,
|
|
139
|
+
` **先定编码规则再统计**:同一维度里机制相近的归一类(如"系统/签到/模拟器/游戏面板"算一类),`,
|
|
140
|
+
` 差别大的分开算;在"一、数据与方法"里把你用的类别列出来,并写出**众数组合**与**匹配本数**`,
|
|
141
|
+
` ("简介核心冲突"的类别自定,但要在方法节写清,如 复仇/生存/争霸/日常经营/探案规则/恋爱/其他)`,
|
|
142
|
+
` - **差异化空间** = 该题材**还没被满足**的读者需求。`,
|
|
143
|
+
` **数据里没有评论,也没有第一章** —— 只能从**简介标签、核心卖点、同题材书目的空白处**推断,`,
|
|
144
|
+
` **必须标注"推断"**;不许声称知道评论里说什么`,
|
|
145
|
+
` **高/中/低必须用统一阈值**(每次凭感觉的话,不同日期的报告没法比):`,
|
|
146
|
+
` - **${shareName} / 需求规模**:用**本次快照内的分位数** —— 前 ⌊n/3⌋ 个为高、后 ⌊n/3⌋ 个为低、其余为中`,
|
|
147
|
+
` (n 是题材数;n<3 时不分档。份额列**必须写百分比**,不能写成"3 本仅描述")`,
|
|
148
|
+
` - **头部集中度**:≥0.8 高 / 0.5-0.8 中 / <0.5 低;**<3 本的题材标"样本不足",3-4 本只描述、不打分**。`,
|
|
149
|
+
` 表里写 **0–1 的小数**(0.56、0.4286),不要写百分比`,
|
|
150
|
+
` - **同质化程度**:≥0.7 高 / 0.4-0.7 中 / <0.4 低(前提是先定好编码类别);同样写 0–1 的小数`,
|
|
151
|
+
` - **差异化空间**:3 级 rubric —— **高** = 题材内 ≥3 本共享同一组合、且至少 2 个合理组合缺席;`,
|
|
152
|
+
` **中** = 有主导组合但仍有可见变体空间;**低** = 头部书已覆盖多种组合;`,
|
|
153
|
+
` 3-4 本只描述、不打分;<3 本标“样本不足”`,
|
|
154
|
+
];
|
|
155
|
+
}
|