@litou.cjs/stock-data-downloader 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +74 -0
- package/README.md +151 -0
- package/dist/cli/index.d.ts +7 -0
- package/dist/cli/index.d.ts.map +1 -0
- package/dist/cli/index.js +207 -0
- package/dist/cli/index.js.map +1 -0
- package/dist/core/downloader.d.ts +54 -0
- package/dist/core/downloader.d.ts.map +1 -0
- package/dist/core/downloader.js +149 -0
- package/dist/core/downloader.js.map +1 -0
- package/dist/core/fuzzySearch.d.ts +32 -0
- package/dist/core/fuzzySearch.d.ts.map +1 -0
- package/dist/core/fuzzySearch.js +85 -0
- package/dist/core/fuzzySearch.js.map +1 -0
- package/dist/datasources/cninfo-stable.d.ts +33 -0
- package/dist/datasources/cninfo-stable.d.ts.map +1 -0
- package/dist/datasources/cninfo-stable.js +99 -0
- package/dist/datasources/cninfo-stable.js.map +1 -0
- package/dist/datasources/cninfo.d.ts +71 -0
- package/dist/datasources/cninfo.d.ts.map +1 -0
- package/dist/datasources/cninfo.js +242 -0
- package/dist/datasources/cninfo.js.map +1 -0
- package/dist/datasources/orgid.d.ts +48 -0
- package/dist/datasources/orgid.d.ts.map +1 -0
- package/dist/datasources/orgid.js +106 -0
- package/dist/datasources/orgid.js.map +1 -0
- package/dist/types/index.d.ts +83 -0
- package/dist/types/index.d.ts.map +1 -0
- package/dist/types/index.js +5 -0
- package/dist/types/index.js.map +1 -0
- package/dist/utils/fileUtils.d.ts +23 -0
- package/dist/utils/fileUtils.d.ts.map +1 -0
- package/dist/utils/fileUtils.js +51 -0
- package/dist/utils/fileUtils.js.map +1 -0
- package/dist/utils/logger.d.ts +14 -0
- package/dist/utils/logger.d.ts.map +1 -0
- package/dist/utils/logger.js +40 -0
- package/dist/utils/logger.js.map +1 -0
- package/dist/utils/progress.d.ts +11 -0
- package/dist/utils/progress.d.ts.map +1 -0
- package/dist/utils/progress.js +31 -0
- package/dist/utils/progress.js.map +1 -0
- package/package.json +74 -0
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
# Changelog | 更新日志
|
|
2
|
+
|
|
3
|
+
All notable changes to this project will be documented in this file.
|
|
4
|
+
|
|
5
|
+
本文档记录项目的所有重要更新。
|
|
6
|
+
|
|
7
|
+
## [0.1.0] - 2026-03-25
|
|
8
|
+
|
|
9
|
+
### Added | 新增
|
|
10
|
+
|
|
11
|
+
- ✨ **核心功能**
|
|
12
|
+
- 支持 A 股财务报告下载(年报、半年报、季报)
|
|
13
|
+
- 集成巨潮资讯网数据源
|
|
14
|
+
- 自动识别上交所、深交所股票
|
|
15
|
+
|
|
16
|
+
- 🔍 **智能搜索**
|
|
17
|
+
- 股票代码精确匹配
|
|
18
|
+
- 公司名称模糊搜索
|
|
19
|
+
- 拼音搜索(完整拼音 + 首字母)
|
|
20
|
+
- 搜索结果评分排序
|
|
21
|
+
|
|
22
|
+
- ⚡ **下载功能**
|
|
23
|
+
- 可配置并发下载(默认 3 个)
|
|
24
|
+
- 自动重试机制(3 次,指数退避)
|
|
25
|
+
- 实时进度条显示
|
|
26
|
+
- 自动跳过已存在文件
|
|
27
|
+
- PDF 文件完整性验证
|
|
28
|
+
|
|
29
|
+
- 📝 **日志系统**
|
|
30
|
+
- 控制台彩色日志输出
|
|
31
|
+
- 文件日志记录
|
|
32
|
+
- 可配置日志级别(info/debug/warn/error)
|
|
33
|
+
|
|
34
|
+
- 🛡️ **健壮性**
|
|
35
|
+
- 网络健康预检
|
|
36
|
+
- 速率限制保护
|
|
37
|
+
- 错误分类处理
|
|
38
|
+
- 详细错误报告
|
|
39
|
+
|
|
40
|
+
- 📦 **CLI 界面**
|
|
41
|
+
- 交互式模式
|
|
42
|
+
- 命令行参数模式
|
|
43
|
+
- 完整的帮助文档
|
|
44
|
+
|
|
45
|
+
### Technical | 技术实现
|
|
46
|
+
|
|
47
|
+
- TypeScript 5.9.3
|
|
48
|
+
- ES Module 支持
|
|
49
|
+
- Node.js >= 18.0.0
|
|
50
|
+
|
|
51
|
+
### Documentation | 文档
|
|
52
|
+
|
|
53
|
+
- README.md(中英文)
|
|
54
|
+
- 用户使用指南
|
|
55
|
+
- 技术规格说明书
|
|
56
|
+
- 数据源研究报告
|
|
57
|
+
|
|
58
|
+
## [Unreleased] | 计划中
|
|
59
|
+
|
|
60
|
+
### 未来版本计划
|
|
61
|
+
|
|
62
|
+
- 支持港股、美股市场
|
|
63
|
+
- 批量下载多个股票
|
|
64
|
+
- 可选择性下载特定类型报告
|
|
65
|
+
- 更完整的股票列表数据
|
|
66
|
+
- 报告内容解析功能
|
|
67
|
+
- GUI 桌面应用
|
|
68
|
+
- 数据导出(Excel、CSV)
|
|
69
|
+
- 自动更新检查
|
|
70
|
+
|
|
71
|
+
---
|
|
72
|
+
|
|
73
|
+
格式基于 [Keep a Changelog](https://keepachangelog.com/)
|
|
74
|
+
版本遵循 [Semantic Versioning](https://semver.org/)
|
package/README.md
ADDED
|
@@ -0,0 +1,151 @@
|
|
|
1
|
+
# Stock Data Downloader | A股财务报告下载工具
|
|
2
|
+
|
|
3
|
+
[](https://www.npmjs.com/package/@litou.cjs/stock-data-downloader)
|
|
4
|
+
[](https://www.npmjs.com/package/@litou.cjs/stock-data-downloader)
|
|
5
|
+
[](https://nodejs.org/)
|
|
6
|
+
|
|
7
|
+
一个命令行工具,用于批量下载 A 股上市公司的财务报告(年报、半年报、季报)。
|
|
8
|
+
|
|
9
|
+
## 功能特性
|
|
10
|
+
|
|
11
|
+
- ✅ **多种报告类型**: 支持年报、半年报、季报下载
|
|
12
|
+
- 🔍 **智能搜索**: 支持股票代码、公司名称、拼音模糊匹配
|
|
13
|
+
- ⚡ **并发下载**: 可配置并发数,默认 3 个并发
|
|
14
|
+
- 🔄 **自动重试**: 失败自动重试,支持指数退避
|
|
15
|
+
- 📊 **进度显示**: 实时进度条显示下载状态
|
|
16
|
+
- 📝 **日志记录**: 支持控制台和文件双路日志输出
|
|
17
|
+
- ✅ **文件验证**: 自动验证 PDF 文件完整性
|
|
18
|
+
- 🚫 **去重处理**: 自动跳过已下载的文件
|
|
19
|
+
|
|
20
|
+
## 安装
|
|
21
|
+
|
|
22
|
+
```bash
|
|
23
|
+
npm install -g @litou.cjs/stock-data-downloader
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
## 快速开始
|
|
27
|
+
|
|
28
|
+
### 交互式模式
|
|
29
|
+
|
|
30
|
+
```bash
|
|
31
|
+
stock-data-downloader
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
程序会提示你输入股票代码或公司名称,然后展示匹配结果供选择。
|
|
35
|
+
|
|
36
|
+
### 命令行模式
|
|
37
|
+
|
|
38
|
+
```bash
|
|
39
|
+
# 下载贵州茅台最近10年的报告
|
|
40
|
+
stock-data-downloader --stock 600519
|
|
41
|
+
|
|
42
|
+
# 下载平安银行最近5年的报告
|
|
43
|
+
stock-data-downloader --stock 平安银行 --years 5
|
|
44
|
+
|
|
45
|
+
# 指定输出路径和并发数
|
|
46
|
+
stock-data-downloader --stock 600519 --output ./reports --concurrency 5
|
|
47
|
+
|
|
48
|
+
# 详细日志模式
|
|
49
|
+
stock-data-downloader --stock 600519 --verbose
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
## 命令行选项
|
|
53
|
+
|
|
54
|
+
```
|
|
55
|
+
Options:
|
|
56
|
+
-s, --stock <code> 股票代码或公司名称
|
|
57
|
+
-y, --years <number> 下载年份数量 (默认: 10)
|
|
58
|
+
-o, --output <path> 输出路径 (默认: ./downloads)
|
|
59
|
+
-c, --concurrency <number> 并发下载数 (默认: 3)
|
|
60
|
+
--timeout <ms> 超时时间(毫秒) (默认: 30000)
|
|
61
|
+
-v, --verbose 详细日志模式
|
|
62
|
+
-V, --version 显示版本号
|
|
63
|
+
-h, --help 显示帮助信息
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
## 输出文件结构
|
|
67
|
+
|
|
68
|
+
```
|
|
69
|
+
downloads/
|
|
70
|
+
└── 600519_贵州茅台/
|
|
71
|
+
├── 600519_贵州茅台_2023_年报.pdf
|
|
72
|
+
├── 600519_贵州茅台_2023_半年报.pdf
|
|
73
|
+
├── 600519_贵州茅台_2023_Q1报.pdf
|
|
74
|
+
├── 600519_贵州茅台_2023_Q3报.pdf
|
|
75
|
+
├── 600519_贵州茅台_2022_年报.pdf
|
|
76
|
+
└── ...
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
## 数据源
|
|
80
|
+
|
|
81
|
+
- **主数据源**: 巨潮资讯网 (cninfo.com.cn)
|
|
82
|
+
- **官方、权威、完整**的 A 股上市公司公告数据
|
|
83
|
+
|
|
84
|
+
## 开发
|
|
85
|
+
|
|
86
|
+
### 项目结构
|
|
87
|
+
|
|
88
|
+
```
|
|
89
|
+
src/
|
|
90
|
+
├── cli/ # CLI 入口和命令行处理
|
|
91
|
+
│ └── index.ts
|
|
92
|
+
├── core/ # 核心逻辑
|
|
93
|
+
│ ├── downloader.ts # 下载引擎(并发、重试、进度)
|
|
94
|
+
│ └── fuzzySearch.ts # 模糊搜索功能
|
|
95
|
+
├── datasources/ # 数据源实现
|
|
96
|
+
│ └── cninfo.ts # 巨潮资讯网接口
|
|
97
|
+
├── utils/ # 工具函数
|
|
98
|
+
│ ├── logger.ts # 日志系统
|
|
99
|
+
│ ├── progress.ts # 进度条
|
|
100
|
+
│ └── fileUtils.ts # 文件工具
|
|
101
|
+
└── types/ # TypeScript 类型定义
|
|
102
|
+
└── index.ts
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
## 技术栈
|
|
106
|
+
|
|
107
|
+
- **语言**: TypeScript
|
|
108
|
+
- **运行时**: Node.js >= 18.0.0
|
|
109
|
+
- **主要依赖**:
|
|
110
|
+
- `axios`: HTTP 请求
|
|
111
|
+
- `commander`: CLI 框架
|
|
112
|
+
- `inquirer`: 交互式命令行
|
|
113
|
+
- `p-queue`: 并发控制
|
|
114
|
+
- `cli-progress`: 进度条
|
|
115
|
+
- `winston`: 日志系统
|
|
116
|
+
- `pinyin-pro`: 拼音转换
|
|
117
|
+
- `fs-extra`: 文件操作
|
|
118
|
+
|
|
119
|
+
## 常见问题
|
|
120
|
+
|
|
121
|
+
### Q: 为什么有些报告下载失败?
|
|
122
|
+
|
|
123
|
+
A: 可能的原因:
|
|
124
|
+
1. 网络不稳定:程序会自动重试3次
|
|
125
|
+
2. 数据源限流:程序内置了速率限制机制
|
|
126
|
+
3. 文件不存在:部分公司某些季度可能未发布报告
|
|
127
|
+
|
|
128
|
+
### Q: 支持哪些股票市场?
|
|
129
|
+
|
|
130
|
+
A: 当前版本仅支持 A 股(上交所、深交所)。未来版本计划支持港股和美股。
|
|
131
|
+
|
|
132
|
+
### Q: 如何处理重复下载?
|
|
133
|
+
|
|
134
|
+
A: 程序会自动检查文件是否已存在,已存在的文件会被跳过。
|
|
135
|
+
|
|
136
|
+
## License
|
|
137
|
+
|
|
138
|
+
MIT
|
|
139
|
+
|
|
140
|
+
## Author
|
|
141
|
+
|
|
142
|
+
陈建生
|
|
143
|
+
|
|
144
|
+
## Changelog
|
|
145
|
+
|
|
146
|
+
### v0.1.0 (2026-03-25)
|
|
147
|
+
|
|
148
|
+
- 初始 MVP 版本发布
|
|
149
|
+
- 支持 A 股年报、半年报、季报下载
|
|
150
|
+
- 实现模糊搜索功能
|
|
151
|
+
- 支持并发下载和自动重试
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/cli/index.ts"],"names":[],"mappings":";AACA;;;GAGG"}
|
|
@@ -0,0 +1,207 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
/**
|
|
3
|
+
* CLI 入口
|
|
4
|
+
* 支持交互式和命令行参数两种模式
|
|
5
|
+
*/
|
|
6
|
+
import { Command } from 'commander';
|
|
7
|
+
import inquirer from 'inquirer';
|
|
8
|
+
import * as path from 'path';
|
|
9
|
+
import CninfoDataSource from '../datasources/cninfo.js';
|
|
10
|
+
import { Downloader } from '../core/downloader.js';
|
|
11
|
+
import { FuzzySearch, loadStockList } from '../core/fuzzySearch.js';
|
|
12
|
+
import { Logger } from '../utils/logger.js';
|
|
13
|
+
import { ProgressBar } from '../utils/progress.js';
|
|
14
|
+
import { FileUtils } from '../utils/fileUtils.js';
|
|
15
|
+
const DEFAULT_YEARS = 10;
|
|
16
|
+
const DEFAULT_OUTPUT_PATH = './downloads';
|
|
17
|
+
const DEFAULT_CONCURRENCY = 3;
|
|
18
|
+
const DEFAULT_TIMEOUT = 30000;
|
|
19
|
+
const DEFAULT_RETRY_COUNT = 3;
|
|
20
|
+
const RETRY_DELAYS = [1000, 2000, 4000];
|
|
21
|
+
async function main() {
|
|
22
|
+
const program = new Command();
|
|
23
|
+
program
|
|
24
|
+
.name('stock-data-downloader')
|
|
25
|
+
.description('下载 A 股上市公司财务报告(年报、半年报、季报)')
|
|
26
|
+
.version('0.1.0')
|
|
27
|
+
.option('-s, --stock <code>', '股票代码或公司名称')
|
|
28
|
+
.option('-y, --years <number>', '下载年份数量', String(DEFAULT_YEARS))
|
|
29
|
+
.option('-o, --output <path>', '输出路径', DEFAULT_OUTPUT_PATH)
|
|
30
|
+
.option('-c, --concurrency <number>', '并发下载数', String(DEFAULT_CONCURRENCY))
|
|
31
|
+
.option('--timeout <ms>', '超时时间(毫秒)', String(DEFAULT_TIMEOUT))
|
|
32
|
+
.option('-v, --verbose', '详细日志模式')
|
|
33
|
+
.parse(process.argv);
|
|
34
|
+
const options = program.opts();
|
|
35
|
+
const logger = new Logger(path.join(options.output || DEFAULT_OUTPUT_PATH, 'download.log'), options.verbose ? 'debug' : 'info');
|
|
36
|
+
logger.info('='.repeat(60));
|
|
37
|
+
logger.info('Stock Data Downloader - A 股财务报告下载工具');
|
|
38
|
+
logger.info('='.repeat(60));
|
|
39
|
+
try {
|
|
40
|
+
const stockList = await loadStockList();
|
|
41
|
+
const fuzzySearch = new FuzzySearch(stockList);
|
|
42
|
+
let selectedStock;
|
|
43
|
+
if (options.stock) {
|
|
44
|
+
const searchResults = fuzzySearch.search(options.stock);
|
|
45
|
+
if (searchResults.length === 0) {
|
|
46
|
+
logger.error(`❌ 未找到匹配的股票: ${options.stock}`);
|
|
47
|
+
process.exit(12);
|
|
48
|
+
}
|
|
49
|
+
if (searchResults.length === 1) {
|
|
50
|
+
selectedStock = searchResults[0];
|
|
51
|
+
logger.info(`✓ 找到股票: ${selectedStock.code} - ${selectedStock.name}`);
|
|
52
|
+
}
|
|
53
|
+
else {
|
|
54
|
+
const choices = searchResults.map(stock => ({
|
|
55
|
+
name: `${stock.code} - ${stock.name} (${stock.market === 'shanghai' ? '上交所' : '深交所'})`,
|
|
56
|
+
value: stock
|
|
57
|
+
}));
|
|
58
|
+
const answer = await inquirer.prompt([
|
|
59
|
+
{
|
|
60
|
+
type: 'list',
|
|
61
|
+
name: 'stock',
|
|
62
|
+
message: `找到 ${searchResults.length} 个匹配结果,请选择:`,
|
|
63
|
+
choices
|
|
64
|
+
}
|
|
65
|
+
]);
|
|
66
|
+
selectedStock = answer.stock;
|
|
67
|
+
}
|
|
68
|
+
}
|
|
69
|
+
else {
|
|
70
|
+
const answer = await inquirer.prompt([
|
|
71
|
+
{
|
|
72
|
+
type: 'input',
|
|
73
|
+
name: 'stockInput',
|
|
74
|
+
message: '请输入股票代码或公司名称:',
|
|
75
|
+
validate: (input) => input.trim().length > 0 || '输入不能为空'
|
|
76
|
+
}
|
|
77
|
+
]);
|
|
78
|
+
const searchResults = fuzzySearch.search(answer.stockInput);
|
|
79
|
+
if (searchResults.length === 0) {
|
|
80
|
+
logger.error(`❌ 未找到匹配的股票: ${answer.stockInput}`);
|
|
81
|
+
process.exit(12);
|
|
82
|
+
}
|
|
83
|
+
const choices = searchResults.map(stock => ({
|
|
84
|
+
name: `${stock.code} - ${stock.name}`,
|
|
85
|
+
value: stock
|
|
86
|
+
}));
|
|
87
|
+
const selectAnswer = await inquirer.prompt([
|
|
88
|
+
{
|
|
89
|
+
type: 'list',
|
|
90
|
+
name: 'stock',
|
|
91
|
+
message: '请选择股票:',
|
|
92
|
+
choices
|
|
93
|
+
}
|
|
94
|
+
]);
|
|
95
|
+
selectedStock = selectAnswer.stock;
|
|
96
|
+
}
|
|
97
|
+
const confirmAnswer = await inquirer.prompt([
|
|
98
|
+
{
|
|
99
|
+
type: 'confirm',
|
|
100
|
+
name: 'confirm',
|
|
101
|
+
message: `确认下载 ${selectedStock.code} - ${selectedStock.name} 的财务报告?`,
|
|
102
|
+
default: true
|
|
103
|
+
}
|
|
104
|
+
]);
|
|
105
|
+
if (!confirmAnswer.confirm) {
|
|
106
|
+
logger.info('已取消');
|
|
107
|
+
process.exit(3);
|
|
108
|
+
}
|
|
109
|
+
const years = parseInt(options.years || DEFAULT_YEARS);
|
|
110
|
+
const outputPath = options.output || DEFAULT_OUTPUT_PATH;
|
|
111
|
+
const concurrency = parseInt(options.concurrency || DEFAULT_CONCURRENCY);
|
|
112
|
+
logger.info('');
|
|
113
|
+
logger.info(`股票: ${selectedStock.code} - ${selectedStock.name}`);
|
|
114
|
+
logger.info(`时间范围: 最近 ${years} 年`);
|
|
115
|
+
logger.info(`输出路径: ${outputPath}`);
|
|
116
|
+
logger.info(`并发数: ${concurrency}`);
|
|
117
|
+
logger.info('');
|
|
118
|
+
logger.info('正在检查网络连接...');
|
|
119
|
+
const dataSource = new CninfoDataSource();
|
|
120
|
+
const isHealthy = await dataSource.healthCheck();
|
|
121
|
+
if (!isHealthy) {
|
|
122
|
+
logger.error('❌ 网络连接失败或数据源不可用');
|
|
123
|
+
process.exit(11);
|
|
124
|
+
}
|
|
125
|
+
logger.info('✓ 网络连接正常');
|
|
126
|
+
const downloader = new Downloader({
|
|
127
|
+
concurrency,
|
|
128
|
+
timeout: DEFAULT_TIMEOUT,
|
|
129
|
+
retryCount: DEFAULT_RETRY_COUNT,
|
|
130
|
+
retryDelays: RETRY_DELAYS,
|
|
131
|
+
outputPath
|
|
132
|
+
}, dataSource);
|
|
133
|
+
logger.info('');
|
|
134
|
+
logger.info('正在查询可用报告...');
|
|
135
|
+
// 改为顺序查询,减少对API的压力
|
|
136
|
+
logger.info(' 正在查询年报...');
|
|
137
|
+
const annualReports = await dataSource.queryReports(selectedStock.code, 'annual', years);
|
|
138
|
+
logger.info(' 正在查询半年报...');
|
|
139
|
+
const semiannualReports = await dataSource.queryReports(selectedStock.code, 'semiannual', years);
|
|
140
|
+
logger.info(' 正在查询季报...');
|
|
141
|
+
const quarterlyReports = await dataSource.queryReports(selectedStock.code, 'quarterly', years);
|
|
142
|
+
const filteredAnnual = dataSource.filterReports(annualReports, 'annual');
|
|
143
|
+
const filteredSemiannual = dataSource.filterReports(semiannualReports, 'semiannual');
|
|
144
|
+
const filteredQuarterly = dataSource.filterReports(quarterlyReports, 'quarterly');
|
|
145
|
+
const totalReports = filteredAnnual.length + filteredSemiannual.length + filteredQuarterly.length;
|
|
146
|
+
logger.info(`✓ 找到 ${totalReports} 份报告`);
|
|
147
|
+
logger.info(` - 年报: ${filteredAnnual.length} 份`);
|
|
148
|
+
logger.info(` - 半年报: ${filteredSemiannual.length} 份`);
|
|
149
|
+
logger.info(` - 季报: ${filteredQuarterly.length} 份`);
|
|
150
|
+
if (totalReports === 0) {
|
|
151
|
+
logger.warn('⚠️ 未找到任何报告');
|
|
152
|
+
process.exit(0);
|
|
153
|
+
}
|
|
154
|
+
logger.info('');
|
|
155
|
+
logger.info('开始下载...');
|
|
156
|
+
logger.info('');
|
|
157
|
+
const progressBar = new ProgressBar();
|
|
158
|
+
progressBar.start(totalReports);
|
|
159
|
+
let currentProgress = 0;
|
|
160
|
+
downloader.onProgress((current, total, fileName) => {
|
|
161
|
+
currentProgress = current;
|
|
162
|
+
progressBar.update(current, fileName);
|
|
163
|
+
});
|
|
164
|
+
const [annualResult, semiannualResult, quarterlyResult] = await Promise.all([
|
|
165
|
+
downloader.downloadReports(filteredAnnual, '年报', selectedStock.code, selectedStock.name),
|
|
166
|
+
downloader.downloadReports(filteredSemiannual, '半年报', selectedStock.code, selectedStock.name),
|
|
167
|
+
downloader.downloadReports(filteredQuarterly, '季报', selectedStock.code, selectedStock.name)
|
|
168
|
+
]);
|
|
169
|
+
progressBar.stop();
|
|
170
|
+
const totalDownloaded = annualResult.downloaded + semiannualResult.downloaded + quarterlyResult.downloaded;
|
|
171
|
+
const totalFailed = annualResult.failed + semiannualResult.failed + quarterlyResult.failed;
|
|
172
|
+
const totalSkipped = annualResult.skipped + semiannualResult.skipped + quarterlyResult.skipped;
|
|
173
|
+
const totalTime = Math.max(annualResult.totalTime, semiannualResult.totalTime, quarterlyResult.totalTime);
|
|
174
|
+
logger.info('');
|
|
175
|
+
logger.info('='.repeat(60));
|
|
176
|
+
logger.info('✅ 下载完成!');
|
|
177
|
+
logger.info('='.repeat(60));
|
|
178
|
+
logger.info(`成功: ${totalDownloaded} 份`);
|
|
179
|
+
if (totalSkipped > 0) {
|
|
180
|
+
logger.info(`跳过: ${totalSkipped} 份(已存在)`);
|
|
181
|
+
}
|
|
182
|
+
if (totalFailed > 0) {
|
|
183
|
+
logger.warn(`失败: ${totalFailed} 份`);
|
|
184
|
+
const allErrors = [
|
|
185
|
+
...annualResult.errors,
|
|
186
|
+
...semiannualResult.errors,
|
|
187
|
+
...quarterlyResult.errors
|
|
188
|
+
];
|
|
189
|
+
allErrors.forEach(error => {
|
|
190
|
+
logger.error(` ✗ ${error.report.announcementTitle}: ${error.reason}`);
|
|
191
|
+
});
|
|
192
|
+
}
|
|
193
|
+
logger.info(`用时: ${FileUtils.formatDuration(totalTime)}`);
|
|
194
|
+
logger.info(`文件保存在: ${path.join(outputPath, selectedStock.code + '_' + selectedStock.name)}`);
|
|
195
|
+
logger.info('='.repeat(60));
|
|
196
|
+
process.exit(totalFailed > 0 ? 1 : 0);
|
|
197
|
+
}
|
|
198
|
+
catch (error) {
|
|
199
|
+
logger.error('❌ 程序执行失败: ' + error.message);
|
|
200
|
+
if (options.verbose) {
|
|
201
|
+
console.error(error);
|
|
202
|
+
}
|
|
203
|
+
process.exit(2);
|
|
204
|
+
}
|
|
205
|
+
}
|
|
206
|
+
main().catch(console.error);
|
|
207
|
+
//# sourceMappingURL=index.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/cli/index.ts"],"names":[],"mappings":";AACA;;;GAGG;AAEH,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AACpC,OAAO,QAAQ,MAAM,UAAU,CAAC;AAChC,OAAO,KAAK,IAAI,MAAM,MAAM,CAAC;AAC7B,OAAO,gBAAgB,MAAM,0BAA0B,CAAC;AACxD,OAAO,EAAE,UAAU,EAAE,MAAM,uBAAuB,CAAC;AACnD,OAAO,EAAE,WAAW,EAAE,aAAa,EAAE,MAAM,wBAAwB,CAAC;AACpE,OAAO,EAAE,MAAM,EAAE,MAAM,oBAAoB,CAAC;AAC5C,OAAO,EAAE,WAAW,EAAE,MAAM,sBAAsB,CAAC;AACnD,OAAO,EAAE,SAAS,EAAE,MAAM,uBAAuB,CAAC;AAGlD,MAAM,aAAa,GAAG,EAAE,CAAC;AACzB,MAAM,mBAAmB,GAAG,aAAa,CAAC;AAC1C,MAAM,mBAAmB,GAAG,CAAC,CAAC;AAC9B,MAAM,eAAe,GAAG,KAAK,CAAC;AAC9B,MAAM,mBAAmB,GAAG,CAAC,CAAC;AAC9B,MAAM,YAAY,GAAG,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,CAAC,CAAC;AAExC,KAAK,UAAU,IAAI;IACjB,MAAM,OAAO,GAAG,IAAI,OAAO,EAAE,CAAC;IAE9B,OAAO;SACJ,IAAI,CAAC,uBAAuB,CAAC;SAC7B,WAAW,CAAC,2BAA2B,CAAC;SACxC,OAAO,CAAC,OAAO,CAAC;SAChB,MAAM,CAAC,oBAAoB,EAAE,WAAW,CAAC;SACzC,MAAM,CAAC,sBAAsB,EAAE,QAAQ,EAAE,MAAM,CAAC,aAAa,CAAC,CAAC;SAC/D,MAAM,CAAC,qBAAqB,EAAE,MAAM,EAAE,mBAAmB,CAAC;SAC1D,MAAM,CAAC,4BAA4B,EAAE,OAAO,EAAE,MAAM,CAAC,mBAAmB,CAAC,CAAC;SAC1E,MAAM,CAAC,gBAAgB,EAAE,UAAU,EAAE,MAAM,CAAC,eAAe,CAAC,CAAC;SAC7D,MAAM,CAAC,eAAe,EAAE,QAAQ,CAAC;SACjC,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;IAEvB,MAAM,OAAO,GAAG,OAAO,CAAC,IAAI,EAAE,CAAC;IAE/B,MAAM,MAAM,GAAG,IAAI,MAAM,CACvB,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,MAAM,IAAI,mBAAmB,EAAE,cAAc,CAAC,EAChE,OAAO,CAAC,OAAO,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CACnC,CAAC;IAEF,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC;IAC5B,MAAM,CAAC,IAAI,CAAC,qCAAqC,CAAC,CAAC;IACnD,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC;IAE5B,IAAI,CAAC;QACH,MAAM,SAAS,GAAG,MAAM,aAAa,EAAE,CAAC;QACxC,MAAM,WAAW,GAAG,IAAI,WAAW,CAAC,SAAS,CAAC,CAAC;QAE/C,IAAI,aAAa,CAAC;QAElB,IAAI,OAAO,CAAC,KAAK,EAAE,CAAC;YAClB,MAAM,aAAa,GAAG,WAAW,CAAC,MAAM,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC;YAExD,IAAI,aAAa,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;gBAC/B,MAAM,CAAC,KAAK,CAAC,eAAe,OAAO,CAAC,KAAK,EAAE,CAAC,CAAC;gBAC7C,OAAO,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;YACnB,CAAC;YAED,IAAI,aAAa,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;gBAC/B,aAAa,GAAG,aAAa,CAAC,CAAC,CAAC,CAAC;gBACjC,MAAM,CAAC,IAAI,CAAC,WAAW,aAAa,CAAC,IAAI,MAAM,aAAa,CAAC,IAAI,EAAE,CAAC,CAAC;YACvE,CAAC;iBAAM,CAAC;gBACN,MAAM,OAAO,GAAG,aAAa,CAAC,GAAG,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;oBAC1C,IAAI,EAAE,GAAG,KAAK,CAAC,IAAI,MAAM,KAAK,CAAC,IAAI,KAAK,KAAK,CAAC,MAAM,KAAK,UAAU,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG;oBACtF,KAAK,EAAE,KAAK;iBACb,CAAC,CAAC,CAAC;gBAEJ,MAAM,MAAM,GAAG,MAAM,QAAQ,CAAC,MAAM,CAAC;oBACnC;wBACE,IAAI,EAAE,MAAM;wBACZ,IAAI,EAAE,OAAO;wBACb,OAAO,EAAE,MAAM,aAAa,CAAC,MAAM,aAAa;wBAChD,OAAO;qBACR;iBACF,CAAC,CAAC;gBAEH,aAAa,GAAG,MAAM,CAAC,KAAK,CAAC;YAC/B,CAAC;QACH,CAAC;aAAM,CAAC;YACN,MAAM,MAAM,GAAG,MAAM,QAAQ,CAAC,MAAM,CAAC;gBACnC;oBACE,IAAI,EAAE,OAAO;oBACb,IAAI,EAAE,YAAY;oBAClB,OAAO,EAAE,eAAe;oBACxB,QAAQ,EAAE,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC,MAAM,GAAG,CAAC,IAAI,QAAQ;iBACzD;aACF,CAAC,CAAC;YAEH,MAAM,aAAa,GAAG,WAAW,CAAC,MAAM,CAAC,MAAM,CAAC,UAAU,CAAC,CAAC;YAE5D,IAAI,aAAa,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;gBAC/B,MAAM,CAAC,KAAK,CAAC,eAAe,MAAM,CAAC,UAAU,EAAE,CAAC,CAAC;gBACjD,OAAO,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;YACnB,CAAC;YAED,MAAM,OAAO,GAAG,aAAa,CAAC,GAAG,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;gBAC1C,IAAI,EAAE,GAAG,KAAK,CAAC,IAAI,MAAM,KAAK,CAAC,IAAI,EAAE;gBACrC,KAAK,EAAE,KAAK;aACb,CAAC,CAAC,CAAC;YAEJ,MAAM,YAAY,GAAG,MAAM,QAAQ,CAAC,MAAM,CAAC;gBACzC;oBACE,IAAI,EAAE,MAAM;oBACZ,IAAI,EAAE,OAAO;oBACb,OAAO,EAAE,QAAQ;oBACjB,OAAO;iBACR;aACF,CAAC,CAAC;YAEH,aAAa,GAAG,YAAY,CAAC,KAAK,CAAC;QACrC,CAAC;QAED,MAAM,aAAa,GAAG,MAAM,QAAQ,CAAC,MAAM,CAAC;YAC1C;gBACE,IAAI,EAAE,SAAS;gBACf,IAAI,EAAE,SAAS;gBACf,OAAO,EAAE,QAAQ,aAAa,CAAC,IAAI,MAAM,aAAa,CAAC,IAAI,SAAS;gBACpE,OAAO,EAAE,IAAI;aACd;SACF,CAAC,CAAC;QAEH,IAAI,CAAC,aAAa,CAAC,OAAO,EAAE,CAAC;YAC3B,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;YACnB,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QAClB,CAAC;QAED,MAAM,KAAK,GAAG,QAAQ,CAAC,OAAO,CAAC,KAAK,IAAI,aAAa,CAAC,CAAC;QACvD,MAAM,UAAU,GAAG,OAAO,CAAC,MAAM,IAAI,mBAAmB,CAAC;QACzD,MAAM,WAAW,GAAG,QAAQ,CAAC,OAAO,CAAC,WAAW,IAAI,mBAAmB,CAAC,CAAC;QAEzE,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;QAChB,MAAM,CAAC,IAAI,CAAC,OAAO,aAAa,CAAC,IAAI,MAAM,aAAa,CAAC,IAAI,EAAE,CAAC,CAAC;QACjE,MAAM,CAAC,IAAI,CAAC,YAAY,KAAK,IAAI,CAAC,CAAC;QACnC,MAAM,CAAC,IAAI,CAAC,SAAS,UAAU,EAAE,CAAC,CAAC;QACnC,MAAM,CAAC,IAAI,CAAC,QAAQ,WAAW,EAAE,CAAC,CAAC;QACnC,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;QAEhB,MAAM,CAAC,IAAI,CAAC,aAAa,CAAC,CAAC;QAC3B,MAAM,UAAU,GAAG,IAAI,gBAAgB,EAAE,CAAC;QAC1C,MAAM,SAAS,GAAG,MAAM,UAAU,CAAC,WAAW,EAAE,CAAC;QAEjD,IAAI,CAAC,SAAS,EAAE,CAAC;YACf,MAAM,CAAC,KAAK,CAAC,iBAAiB,CAAC,CAAC;YAChC,OAAO,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;QACnB,CAAC;QAED,MAAM,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC;QAExB,MAAM,UAAU,GAAG,IAAI,UAAU,CAC/B;YACE,WAAW;YACX,OAAO,EAAE,eAAe;YACxB,UAAU,EAAE,mBAAmB;YAC/B,WAAW,EAAE,YAAY;YACzB,UAAU;SACX,EACD,UAAU,CACX,CAAC;QAEF,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;QAChB,MAAM,CAAC,IAAI,CAAC,aAAa,CAAC,CAAC;QAE3B,mBAAmB;QACnB,MAAM,CAAC,IAAI,CAAC,aAAa,CAAC,CAAC;QAC3B,MAAM,aAAa,GAAG,MAAM,UAAU,CAAC,YAAY,CAAC,aAAa,CAAC,IAAI,EAAE,QAAQ,EAAE,KAAK,CAAC,CAAC;QAEzF,MAAM,CAAC,IAAI,CAAC,cAAc,CAAC,CAAC;QAC5B,MAAM,iBAAiB,GAAG,MAAM,UAAU,CAAC,YAAY,CAAC,aAAa,CAAC,IAAI,EAAE,YAAY,EAAE,KAAK,CAAC,CAAC;QAEjG,MAAM,CAAC,IAAI,CAAC,aAAa,CAAC,CAAC;QAC3B,MAAM,gBAAgB,GAAG,MAAM,UAAU,CAAC,YAAY,CAAC,aAAa,CAAC,IAAI,EAAE,WAAW,EAAE,KAAK,CAAC,CAAC;QAE/F,MAAM,cAAc,GAAG,UAAU,CAAC,aAAa,CAAC,aAAa,EAAE,QAAQ,CAAC,CAAC;QACzE,MAAM,kBAAkB,GAAG,UAAU,CAAC,aAAa,CAAC,iBAAiB,EAAE,YAAY,CAAC,CAAC;QACrF,MAAM,iBAAiB,GAAG,UAAU,CAAC,aAAa,CAAC,gBAAgB,EAAE,WAAW,CAAC,CAAC;QAElF,MAAM,YAAY,GAAG,cAAc,CAAC,MAAM,GAAG,kBAAkB,CAAC,MAAM,GAAG,iBAAiB,CAAC,MAAM,CAAC;QAElG,MAAM,CAAC,IAAI,CAAC,QAAQ,YAAY,MAAM,CAAC,CAAC;QACxC,MAAM,CAAC,IAAI,CAAC,WAAW,cAAc,CAAC,MAAM,IAAI,CAAC,CAAC;QAClD,MAAM,CAAC,IAAI,CAAC,YAAY,kBAAkB,CAAC,MAAM,IAAI,CAAC,CAAC;QACvD,MAAM,CAAC,IAAI,CAAC,WAAW,iBAAiB,CAAC,MAAM,IAAI,CAAC,CAAC;QAErD,IAAI,YAAY,KAAK,CAAC,EAAE,CAAC;YACvB,MAAM,CAAC,IAAI,CAAC,aAAa,CAAC,CAAC;YAC3B,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QAClB,CAAC;QAED,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;QAChB,MAAM,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;QACvB,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;QAEhB,MAAM,WAAW,GAAG,IAAI,WAAW,EAAE,CAAC;QACtC,WAAW,CAAC,KAAK,CAAC,YAAY,CAAC,CAAC;QAEhC,IAAI,eAAe,GAAG,CAAC,CAAC;QACxB,UAAU,CAAC,UAAU,CAAC,CAAC,OAAO,EAAE,KAAK,EAAE,QAAQ,EAAE,EAAE;YACjD,eAAe,GAAG,OAAO,CAAC;YAC1B,WAAW,CAAC,MAAM,CAAC,OAAO,EAAE,QAAQ,CAAC,CAAC;QACxC,CAAC,CAAC,CAAC;QAEH,MAAM,CAAC,YAAY,EAAE,gBAAgB,EAAE,eAAe,CAAC,GAAG,MAAM,OAAO,CAAC,GAAG,CAAC;YAC1E,UAAU,CAAC,eAAe,CAAC,cAAc,EAAE,IAAI,EAAE,aAAa,CAAC,IAAI,EAAE,aAAa,CAAC,IAAI,CAAC;YACxF,UAAU,CAAC,eAAe,CAAC,kBAAkB,EAAE,KAAK,EAAE,aAAa,CAAC,IAAI,EAAE,aAAa,CAAC,IAAI,CAAC;YAC7F,UAAU,CAAC,eAAe,CAAC,iBAAiB,EAAE,IAAI,EAAE,aAAa,CAAC,IAAI,EAAE,aAAa,CAAC,IAAI,CAAC;SAC5F,CAAC,CAAC;QAEH,WAAW,CAAC,IAAI,EAAE,CAAC;QAEnB,MAAM,eAAe,GAAG,YAAY,CAAC,UAAU,GAAG,gBAAgB,CAAC,UAAU,GAAG,eAAe,CAAC,UAAU,CAAC;QAC3G,MAAM,WAAW,GAAG,YAAY,CAAC,MAAM,GAAG,gBAAgB,CAAC,MAAM,GAAG,eAAe,CAAC,MAAM,CAAC;QAC3F,MAAM,YAAY,GAAG,YAAY,CAAC,OAAO,GAAG,gBAAgB,CAAC,OAAO,GAAG,eAAe,CAAC,OAAO,CAAC;QAC/F,MAAM,SAAS,GAAG,IAAI,CAAC,GAAG,CAAC,YAAY,CAAC,SAAS,EAAE,gBAAgB,CAAC,SAAS,EAAE,eAAe,CAAC,SAAS,CAAC,CAAC;QAE1G,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;QAChB,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC;QAC5B,MAAM,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;QACvB,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC;QAC5B,MAAM,CAAC,IAAI,CAAC,OAAO,eAAe,IAAI,CAAC,CAAC;QACxC,IAAI,YAAY,GAAG,CAAC,EAAE,CAAC;YACrB,MAAM,CAAC,IAAI,CAAC,OAAO,YAAY,SAAS,CAAC,CAAC;QAC5C,CAAC;QACD,IAAI,WAAW,GAAG,CAAC,EAAE,CAAC;YACpB,MAAM,CAAC,IAAI,CAAC,OAAO,WAAW,IAAI,CAAC,CAAC;YAEpC,MAAM,SAAS,GAAG;gBAChB,GAAG,YAAY,CAAC,MAAM;gBACtB,GAAG,gBAAgB,CAAC,MAAM;gBAC1B,GAAG,eAAe,CAAC,MAAM;aAC1B,CAAC;YAEF,SAAS,CAAC,OAAO,CAAC,KAAK,CAAC,EAAE;gBACxB,MAAM,CAAC,KAAK,CAAC,OAAO,KAAK,CAAC,MAAM,CAAC,iBAAiB,KAAK,KAAK,CAAC,MAAM,EAAE,CAAC,CAAC;YACzE,CAAC,CAAC,CAAC;QACL,CAAC;QACD,MAAM,CAAC,IAAI,CAAC,OAAO,SAAS,CAAC,cAAc,CAAC,SAAS,CAAC,EAAE,CAAC,CAAC;QAC1D,MAAM,CAAC,IAAI,CAAC,UAAU,IAAI,CAAC,IAAI,CAAC,UAAU,EAAE,aAAa,CAAC,IAAI,GAAG,GAAG,GAAG,aAAa,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;QAC9F,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC;QAE5B,OAAO,CAAC,IAAI,CAAC,WAAW,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;IAExC,CAAC;IAAC,OAAO,KAAU,EAAE,CAAC;QACpB,MAAM,CAAC,KAAK,CAAC,YAAY,GAAG,KAAK,CAAC,OAAO,CAAC,CAAC;QAC3C,IAAI,OAAO,CAAC,OAAO,EAAE,CAAC;YACpB,OAAO,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC;QACvB,CAAC;QACD,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;IAClB,CAAC;AACH,CAAC;AAED,IAAI,EAAE,CAAC,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC"}
|
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* 核心下载引擎
|
|
3
|
+
* 负责并发控制、重试机制、进度跟踪
|
|
4
|
+
*/
|
|
5
|
+
import type { Report, DownloadResult } from '../types/index.js';
|
|
6
|
+
import CninfoDataSource from '../datasources/cninfo.js';
|
|
7
|
+
export interface DownloaderConfig {
|
|
8
|
+
concurrency: number;
|
|
9
|
+
timeout: number;
|
|
10
|
+
retryCount: number;
|
|
11
|
+
retryDelays: number[];
|
|
12
|
+
outputPath: string;
|
|
13
|
+
}
|
|
14
|
+
export declare class Downloader {
|
|
15
|
+
private config;
|
|
16
|
+
private dataSource;
|
|
17
|
+
private queue;
|
|
18
|
+
private progressCallback?;
|
|
19
|
+
constructor(config: DownloaderConfig, dataSource: CninfoDataSource);
|
|
20
|
+
/**
|
|
21
|
+
* 设置进度回调
|
|
22
|
+
*/
|
|
23
|
+
onProgress(callback: (current: number, total: number, fileName: string) => void): void;
|
|
24
|
+
/**
|
|
25
|
+
* 生成文件名(按规格说明格式)
|
|
26
|
+
*/
|
|
27
|
+
private generateFileName;
|
|
28
|
+
/**
|
|
29
|
+
* 从标题提取年份
|
|
30
|
+
*/
|
|
31
|
+
private extractYear;
|
|
32
|
+
/**
|
|
33
|
+
* 从标题提取季度
|
|
34
|
+
*/
|
|
35
|
+
private extractQuarter;
|
|
36
|
+
/**
|
|
37
|
+
* 清洗文件名中的特殊字符
|
|
38
|
+
*/
|
|
39
|
+
private sanitizeFileName;
|
|
40
|
+
/**
|
|
41
|
+
* 检查文件是否已存在
|
|
42
|
+
*/
|
|
43
|
+
private fileExists;
|
|
44
|
+
/**
|
|
45
|
+
* 下载单个文件(带重试)
|
|
46
|
+
*/
|
|
47
|
+
private downloadSingle;
|
|
48
|
+
/**
|
|
49
|
+
* 批量下载报告
|
|
50
|
+
*/
|
|
51
|
+
downloadReports(reports: Report[], reportTypeName: string, stockCode: string, stockName: string): Promise<DownloadResult>;
|
|
52
|
+
}
|
|
53
|
+
export default Downloader;
|
|
54
|
+
//# sourceMappingURL=downloader.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"downloader.d.ts","sourceRoot":"","sources":["../../src/core/downloader.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAKH,OAAO,KAAK,EAAE,MAAM,EAAE,cAAc,EAAiB,MAAM,mBAAmB,CAAC;AAC/E,OAAO,gBAAgB,MAAM,0BAA0B,CAAC;AAExD,MAAM,WAAW,gBAAgB;IAC/B,WAAW,EAAE,MAAM,CAAC;IACpB,OAAO,EAAE,MAAM,CAAC;IAChB,UAAU,EAAE,MAAM,CAAC;IACnB,WAAW,EAAE,MAAM,EAAE,CAAC;IACtB,UAAU,EAAE,MAAM,CAAC;CACpB;AAED,qBAAa,UAAU;IACrB,OAAO,CAAC,MAAM,CAAmB;IACjC,OAAO,CAAC,UAAU,CAAmB;IACrC,OAAO,CAAC,KAAK,CAAS;IACtB,OAAO,CAAC,gBAAgB,CAAC,CAA6D;gBAE1E,MAAM,EAAE,gBAAgB,EAAE,UAAU,EAAE,gBAAgB;IAMlE;;OAEG;IACH,UAAU,CAAC,QAAQ,EAAE,CAAC,OAAO,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,QAAQ,EAAE,MAAM,KAAK,IAAI,GAAG,IAAI;IAItF;;OAEG;IACH,OAAO,CAAC,gBAAgB;IAgBxB;;OAEG;IACH,OAAO,CAAC,WAAW;IAKnB;;OAEG;IACH,OAAO,CAAC,cAAc;IAMtB;;OAEG;IACH,OAAO,CAAC,gBAAgB;IAIxB;;OAEG;YACW,UAAU;IASxB;;OAEG;YACW,cAAc;IAsC5B;;OAEG;IACG,eAAe,CACnB,OAAO,EAAE,MAAM,EAAE,EACjB,cAAc,EAAE,MAAM,EACtB,SAAS,EAAE,MAAM,EACjB,SAAS,EAAE,MAAM,GAChB,OAAO,CAAC,cAAc,CAAC;CAyD3B;AAED,eAAe,UAAU,CAAC"}
|
|
@@ -0,0 +1,149 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* 核心下载引擎
|
|
3
|
+
* 负责并发控制、重试机制、进度跟踪
|
|
4
|
+
*/
|
|
5
|
+
import * as path from 'path';
|
|
6
|
+
import fs from 'fs-extra';
|
|
7
|
+
import PQueue from 'p-queue';
|
|
8
|
+
export class Downloader {
|
|
9
|
+
constructor(config, dataSource) {
|
|
10
|
+
this.config = config;
|
|
11
|
+
this.dataSource = dataSource;
|
|
12
|
+
this.queue = new PQueue({ concurrency: config.concurrency });
|
|
13
|
+
}
|
|
14
|
+
/**
|
|
15
|
+
* 设置进度回调
|
|
16
|
+
*/
|
|
17
|
+
onProgress(callback) {
|
|
18
|
+
this.progressCallback = callback;
|
|
19
|
+
}
|
|
20
|
+
/**
|
|
21
|
+
* 生成文件名(按规格说明格式)
|
|
22
|
+
*/
|
|
23
|
+
generateFileName(report, reportTypeName) {
|
|
24
|
+
const title = report.announcementTitle;
|
|
25
|
+
const year = this.extractYear(title);
|
|
26
|
+
const quarter = this.extractQuarter(title);
|
|
27
|
+
let reportLabel = reportTypeName;
|
|
28
|
+
if (quarter) {
|
|
29
|
+
reportLabel = `Q${quarter}报`;
|
|
30
|
+
}
|
|
31
|
+
const sanitizedName = this.sanitizeFileName(report.secName);
|
|
32
|
+
const fileName = `${report.secCode}_${sanitizedName}_${year}_${reportLabel}.pdf`;
|
|
33
|
+
return fileName;
|
|
34
|
+
}
|
|
35
|
+
/**
|
|
36
|
+
* 从标题提取年份
|
|
37
|
+
*/
|
|
38
|
+
extractYear(title) {
|
|
39
|
+
const match = title.match(/(\d{4})年/);
|
|
40
|
+
return match ? parseInt(match[1]) : new Date().getFullYear();
|
|
41
|
+
}
|
|
42
|
+
/**
|
|
43
|
+
* 从标题提取季度
|
|
44
|
+
*/
|
|
45
|
+
extractQuarter(title) {
|
|
46
|
+
if (title.includes('第一季度') || title.includes('Q1'))
|
|
47
|
+
return 1;
|
|
48
|
+
if (title.includes('第三季度') || title.includes('Q3'))
|
|
49
|
+
return 3;
|
|
50
|
+
return null;
|
|
51
|
+
}
|
|
52
|
+
/**
|
|
53
|
+
* 清洗文件名中的特殊字符
|
|
54
|
+
*/
|
|
55
|
+
sanitizeFileName(name) {
|
|
56
|
+
return name.replace(/[*?"<>|:/\\]/g, '_');
|
|
57
|
+
}
|
|
58
|
+
/**
|
|
59
|
+
* 检查文件是否已存在
|
|
60
|
+
*/
|
|
61
|
+
async fileExists(filePath) {
|
|
62
|
+
try {
|
|
63
|
+
await fs.access(filePath);
|
|
64
|
+
return true;
|
|
65
|
+
}
|
|
66
|
+
catch {
|
|
67
|
+
return false;
|
|
68
|
+
}
|
|
69
|
+
}
|
|
70
|
+
/**
|
|
71
|
+
* 下载单个文件(带重试)
|
|
72
|
+
*/
|
|
73
|
+
async downloadSingle(report, reportTypeName, stockFolder) {
|
|
74
|
+
const fileName = this.generateFileName(report, reportTypeName);
|
|
75
|
+
const filePath = path.join(stockFolder, fileName);
|
|
76
|
+
if (await this.fileExists(filePath)) {
|
|
77
|
+
return { success: true, skipped: true };
|
|
78
|
+
}
|
|
79
|
+
for (let attempt = 1; attempt <= this.config.retryCount; attempt++) {
|
|
80
|
+
try {
|
|
81
|
+
const buffer = await this.dataSource.downloadReport(report, stockFolder);
|
|
82
|
+
if (!this.dataSource.validatePdf(buffer)) {
|
|
83
|
+
throw new Error('PDF 文件验证失败');
|
|
84
|
+
}
|
|
85
|
+
await fs.ensureDir(stockFolder);
|
|
86
|
+
await fs.writeFile(filePath, buffer);
|
|
87
|
+
return { success: true };
|
|
88
|
+
}
|
|
89
|
+
catch (error) {
|
|
90
|
+
if (attempt < this.config.retryCount) {
|
|
91
|
+
const delay = this.config.retryDelays[attempt - 1] || 1000;
|
|
92
|
+
await new Promise(resolve => setTimeout(resolve, delay));
|
|
93
|
+
}
|
|
94
|
+
else {
|
|
95
|
+
return { success: false, error: error.message };
|
|
96
|
+
}
|
|
97
|
+
}
|
|
98
|
+
}
|
|
99
|
+
return { success: false, error: '下载失败(已重试)' };
|
|
100
|
+
}
|
|
101
|
+
/**
|
|
102
|
+
* 批量下载报告
|
|
103
|
+
*/
|
|
104
|
+
async downloadReports(reports, reportTypeName, stockCode, stockName) {
|
|
105
|
+
const startTime = Date.now();
|
|
106
|
+
const stockFolder = path.join(this.config.outputPath, `${stockCode}_${this.sanitizeFileName(stockName)}`);
|
|
107
|
+
await fs.ensureDir(stockFolder);
|
|
108
|
+
const results = {
|
|
109
|
+
success: true,
|
|
110
|
+
downloaded: 0,
|
|
111
|
+
failed: 0,
|
|
112
|
+
skipped: 0,
|
|
113
|
+
warnings: [],
|
|
114
|
+
errors: [],
|
|
115
|
+
totalTime: 0
|
|
116
|
+
};
|
|
117
|
+
let completed = 0;
|
|
118
|
+
const total = reports.length;
|
|
119
|
+
const downloadTasks = reports.map(report => async () => {
|
|
120
|
+
const fileName = this.generateFileName(report, reportTypeName);
|
|
121
|
+
if (this.progressCallback) {
|
|
122
|
+
this.progressCallback(completed, total, fileName);
|
|
123
|
+
}
|
|
124
|
+
const result = await this.downloadSingle(report, reportTypeName, stockFolder);
|
|
125
|
+
completed++;
|
|
126
|
+
if (result.skipped) {
|
|
127
|
+
results.skipped++;
|
|
128
|
+
}
|
|
129
|
+
else if (result.success) {
|
|
130
|
+
results.downloaded++;
|
|
131
|
+
}
|
|
132
|
+
else {
|
|
133
|
+
results.failed++;
|
|
134
|
+
results.errors.push({
|
|
135
|
+
report,
|
|
136
|
+
reason: result.error || '未知错误',
|
|
137
|
+
attempts: this.config.retryCount
|
|
138
|
+
});
|
|
139
|
+
}
|
|
140
|
+
await this.dataSource.rateLimit();
|
|
141
|
+
});
|
|
142
|
+
await this.queue.addAll(downloadTasks);
|
|
143
|
+
results.totalTime = Date.now() - startTime;
|
|
144
|
+
results.success = results.failed === 0;
|
|
145
|
+
return results;
|
|
146
|
+
}
|
|
147
|
+
}
|
|
148
|
+
export default Downloader;
|
|
149
|
+
//# sourceMappingURL=downloader.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"downloader.js","sourceRoot":"","sources":["../../src/core/downloader.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH,OAAO,KAAK,IAAI,MAAM,MAAM,CAAC;AAC7B,OAAO,EAAE,MAAM,UAAU,CAAC;AAC1B,OAAO,MAAM,MAAM,SAAS,CAAC;AAY7B,MAAM,OAAO,UAAU;IAMrB,YAAY,MAAwB,EAAE,UAA4B;QAChE,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC;QACrB,IAAI,CAAC,UAAU,GAAG,UAAU,CAAC;QAC7B,IAAI,CAAC,KAAK,GAAG,IAAI,MAAM,CAAC,EAAE,WAAW,EAAE,MAAM,CAAC,WAAW,EAAE,CAAC,CAAC;IAC/D,CAAC;IAED;;OAEG;IACH,UAAU,CAAC,QAAoE;QAC7E,IAAI,CAAC,gBAAgB,GAAG,QAAQ,CAAC;IACnC,CAAC;IAED;;OAEG;IACK,gBAAgB,CAAC,MAAc,EAAE,cAAsB;QAC7D,MAAM,KAAK,GAAG,MAAM,CAAC,iBAAiB,CAAC;QACvC,MAAM,IAAI,GAAG,IAAI,CAAC,WAAW,CAAC,KAAK,CAAC,CAAC;QACrC,MAAM,OAAO,GAAG,IAAI,CAAC,cAAc,CAAC,KAAK,CAAC,CAAC;QAE3C,IAAI,WAAW,GAAG,cAAc,CAAC;QACjC,IAAI,OAAO,EAAE,CAAC;YACZ,WAAW,GAAG,IAAI,OAAO,GAAG,CAAC;QAC/B,CAAC;QAED,MAAM,aAAa,GAAG,IAAI,CAAC,gBAAgB,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;QAC5D,MAAM,QAAQ,GAAG,GAAG,MAAM,CAAC,OAAO,IAAI,aAAa,IAAI,IAAI,IAAI,WAAW,MAAM,CAAC;QAEjF,OAAO,QAAQ,CAAC;IAClB,CAAC;IAED;;OAEG;IACK,WAAW,CAAC,KAAa;QAC/B,MAAM,KAAK,GAAG,KAAK,CAAC,KAAK,CAAC,UAAU,CAAC,CAAC;QACtC,OAAO,KAAK,CAAC,CAAC,CAAC,QAAQ,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,IAAI,EAAE,CAAC,WAAW,EAAE,CAAC;IAC/D,CAAC;IAED;;OAEG;IACK,cAAc,CAAC,KAAa;QAClC,IAAI,KAAK,CAAC,QAAQ,CAAC,MAAM,CAAC,IAAI,KAAK,CAAC,QAAQ,CAAC,IAAI,CAAC;YAAE,OAAO,CAAC,CAAC;QAC7D,IAAI,KAAK,CAAC,QAAQ,CAAC,MAAM,CAAC,IAAI,KAAK,CAAC,QAAQ,CAAC,IAAI,CAAC;YAAE,OAAO,CAAC,CAAC;QAC7D,OAAO,IAAI,CAAC;IACd,CAAC;IAED;;OAEG;IACK,gBAAgB,CAAC,IAAY;QACnC,OAAO,IAAI,CAAC,OAAO,CAAC,eAAe,EAAE,GAAG,CAAC,CAAC;IAC5C,CAAC;IAED;;OAEG;IACK,KAAK,CAAC,UAAU,CAAC,QAAgB;QACvC,IAAI,CAAC;YACH,MAAM,EAAE,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;YAC1B,OAAO,IAAI,CAAC;QACd,CAAC;QAAC,MAAM,CAAC;YACP,OAAO,KAAK,CAAC;QACf,CAAC;IACH,CAAC;IAED;;OAEG;IACK,KAAK,CAAC,cAAc,CAC1B,MAAc,EACd,cAAsB,EACtB,WAAmB;QAEnB,MAAM,QAAQ,GAAG,IAAI,CAAC,gBAAgB,CAAC,MAAM,EAAE,cAAc,CAAC,CAAC;QAC/D,MAAM,QAAQ,GAAG,IAAI,CAAC,IAAI,CAAC,WAAW,EAAE,QAAQ,CAAC,CAAC;QAElD,IAAI,MAAM,IAAI,CAAC,UAAU,CAAC,QAAQ,CAAC,EAAE,CAAC;YACpC,OAAO,EAAE,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,IAAI,EAAE,CAAC;QAC1C,CAAC;QAED,KAAK,IAAI,OAAO,GAAG,CAAC,EAAE,OAAO,IAAI,IAAI,CAAC,MAAM,CAAC,UAAU,EAAE,OAAO,EAAE,EAAE,CAAC;YACnE,IAAI,CAAC;gBACH,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,UAAU,CAAC,cAAc,CAAC,MAAM,EAAE,WAAW,CAAC,CAAC;gBAEzE,IAAI,CAAC,IAAI,CAAC,UAAU,CAAC,WAAW,CAAC,MAAM,CAAC,EAAE,CAAC;oBACzC,MAAM,IAAI,KAAK,CAAC,YAAY,CAAC,CAAC;gBAChC,CAAC;gBAED,MAAM,EAAE,CAAC,SAAS,CAAC,WAAW,CAAC,CAAC;gBAChC,MAAM,EAAE,CAAC,SAAS,CAAC,QAAQ,EAAE,MAAM,CAAC,CAAC;gBAErC,OAAO,EAAE,OAAO,EAAE,IAAI,EAAE,CAAC;YAE3B,CAAC;YAAC,OAAO,KAAU,EAAE,CAAC;gBACpB,IAAI,OAAO,GAAG,IAAI,CAAC,MAAM,CAAC,UAAU,EAAE,CAAC;oBACrC,MAAM,KAAK,GAAG,IAAI,CAAC,MAAM,CAAC,WAAW,CAAC,OAAO,GAAG,CAAC,CAAC,IAAI,IAAI,CAAC;oBAC3D,MAAM,IAAI,OAAO,CAAC,OAAO,CAAC,EAAE,CAAC,UAAU,CAAC,OAAO,EAAE,KAAK,CAAC,CAAC,CAAC;gBAC3D,CAAC;qBAAM,CAAC;oBACN,OAAO,EAAE,OAAO,EAAE,KAAK,EAAE,KAAK,EAAE,KAAK,CAAC,OAAO,EAAE,CAAC;gBAClD,CAAC;YACH,CAAC;QACH,CAAC;QAED,OAAO,EAAE,OAAO,EAAE,KAAK,EAAE,KAAK,EAAE,WAAW,EAAE,CAAC;IAChD,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,eAAe,CACnB,OAAiB,EACjB,cAAsB,EACtB,SAAiB,EACjB,SAAiB;QAEjB,MAAM,SAAS,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;QAC7B,MAAM,WAAW,GAAG,IAAI,CAAC,IAAI,CAC3B,IAAI,CAAC,MAAM,CAAC,UAAU,EACtB,GAAG,SAAS,IAAI,IAAI,CAAC,gBAAgB,CAAC,SAAS,CAAC,EAAE,CACnD,CAAC;QAEF,MAAM,EAAE,CAAC,SAAS,CAAC,WAAW,CAAC,CAAC;QAEhC,MAAM,OAAO,GAAG;YACd,OAAO,EAAE,IAAI;YACb,UAAU,EAAE,CAAC;YACb,MAAM,EAAE,CAAC;YACT,OAAO,EAAE,CAAC;YACV,QAAQ,EAAE,EAAc;YACxB,MAAM,EAAE,EAAqB;YAC7B,SAAS,EAAE,CAAC;SACb,CAAC;QAEF,IAAI,SAAS,GAAG,CAAC,CAAC;QAClB,MAAM,KAAK,GAAG,OAAO,CAAC,MAAM,CAAC;QAE7B,MAAM,aAAa,GAAG,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC,KAAK,IAAI,EAAE;YACrD,MAAM,QAAQ,GAAG,IAAI,CAAC,gBAAgB,CAAC,MAAM,EAAE,cAAc,CAAC,CAAC;YAE/D,IAAI,IAAI,CAAC,gBAAgB,EAAE,CAAC;gBAC1B,IAAI,CAAC,gBAAgB,CAAC,SAAS,EAAE,KAAK,EAAE,QAAQ,CAAC,CAAC;YACpD,CAAC;YAED,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,cAAc,CAAC,MAAM,EAAE,cAAc,EAAE,WAAW,CAAC,CAAC;YAE9E,SAAS,EAAE,CAAC;YAEZ,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC;gBACnB,OAAO,CAAC,OAAO,EAAE,CAAC;YACpB,CAAC;iBAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC;gBAC1B,OAAO,CAAC,UAAU,EAAE,CAAC;YACvB,CAAC;iBAAM,CAAC;gBACN,OAAO,CAAC,MAAM,EAAE,CAAC;gBACjB,OAAO,CAAC,MAAM,CAAC,IAAI,CAAC;oBAClB,MAAM;oBACN,MAAM,EAAE,MAAM,CAAC,KAAK,IAAI,MAAM;oBAC9B,QAAQ,EAAE,IAAI,CAAC,MAAM,CAAC,UAAU;iBACjC,CAAC,CAAC;YACL,CAAC;YAED,MAAM,IAAI,CAAC,UAAU,CAAC,SAAS,EAAE,CAAC;QACpC,CAAC,CAAC,CAAC;QAEH,MAAM,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,aAAa,CAAC,CAAC;QAEvC,OAAO,CAAC,SAAS,GAAG,IAAI,CAAC,GAAG,EAAE,GAAG,SAAS,CAAC;QAC3C,OAAO,CAAC,OAAO,GAAG,OAAO,CAAC,MAAM,KAAK,CAAC,CAAC;QAEvC,OAAO,OAAO,CAAC;IACjB,CAAC;CAEF;AAED,eAAe,UAAU,CAAC"}
|