gangtise-openapi-cli 0.22.1 → 0.24.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +48 -38
- package/dist/src/cli.js +90 -8
- package/dist/src/core/args.js +12 -0
- package/dist/src/core/client.js +32 -86
- package/dist/src/core/commandBodies.js +1 -1
- package/dist/src/core/config.js +1 -1
- package/dist/src/core/endpoints.js +31 -2
- package/dist/src/core/errors.js +6 -1
- package/dist/src/core/normalize.js +0 -5
- package/dist/src/core/output.js +5 -3
- package/dist/src/core/printer.js +16 -0
- package/dist/src/core/quoteSharding.js +66 -11
- package/dist/src/core/transport.js +39 -2
- package/dist/src/version.js +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -4,6 +4,38 @@
|
|
|
4
4
|
|
|
5
5
|
## Changelog
|
|
6
6
|
|
|
7
|
+
### v0.24.0 — 2026-07-07
|
|
8
|
+
|
|
9
|
+
**新增**
|
|
10
|
+
- `raw list` — 列出所有已注册的 endpoint key(含 method / path / description),配合 `raw call <key>` 使用,不必再翻文档记 key;支持 `--format`(默认 table)/ `--output`
|
|
11
|
+
- AI 同步生成端点内置 120s 超时下限(`one-pager` / `investment-logic` / `peer-comparison` / `theme-tracking` / `research-outline` / `management-discuss-announcement` / `management-discuss-earnings-call`)——生成耗时长不再撞 30s 默认超时触发重试,**不必再手动前缀 `GANGTISE_TIMEOUT_MS`**;显式设更大值仍生效(取 max)
|
|
12
|
+
- 429 响应尊重 `Retry-After`(秒或 HTTP-date;覆盖 JSON、非 JSON、下载三类错误路径),优先于默认指数退避,封顶 60s 防挂死
|
|
13
|
+
- 超大结果(≥5 万行且走非流式渲染:table/json/markdown,或 jsonl/csv 未带 `--output`)在 stderr 提示改用 `--format jsonl --output <path>` 流式落盘
|
|
14
|
+
|
|
15
|
+
**性能**
|
|
16
|
+
- JSON 请求启用 gzip(`accept-encoding: gzip` + 本地解压)——实测 `reference constant-list` 2110B→586B(3.6x),K 线类高重复大 JSON 收益更高;下载二进制路径不变
|
|
17
|
+
- 全市场按日分片(`quote fund-flow` / `day-kline` / `day-kline-us`,均 1 天/片)自动跳过周六日(A/港/美股周末闭市必空),省 ~28% 请求与每日调用配额;多日分片(`day-kline-hk` 2 天、`index-day-kline` 30 天)不受影响
|
|
18
|
+
|
|
19
|
+
**修复**
|
|
20
|
+
- 表格(table/markdown)显示宽度纳入 emoji 码位区(0x1F000–0x1FAFF),含 emoji 的微信群名/消息不再错位
|
|
21
|
+
- `fundamental earning-forecast` 默认 `--end-date`("today")改用运行机器本地日期;此前用 UTC 日期,CST 凌晨 0–8 点会算成"昨天"
|
|
22
|
+
|
|
23
|
+
**文档 / 工程**(不影响已发布 CLI 行为)
|
|
24
|
+
- `insight announcement`(A 股公告)时间过滤时区说明:`--start-time`/`--end-time` 按运行机器时区换算,跨机器精确边界改传 13 位毫秒时间戳
|
|
25
|
+
- CI 测试矩阵增加 Node 24(此前仅 20;发布用 24)
|
|
26
|
+
|
|
27
|
+
### v0.23.0 — 2026-07-05
|
|
28
|
+
|
|
29
|
+
**行为变更(注意)**
|
|
30
|
+
- ⚠️ 默认 API 域名迁移:`https://open.gangtise.com` → `https://openapi.gangtise.com`。旧域名仍可用,CLI 只是切换了默认值(新旧域名多接口实测等价);如需固定旧域名设 `GANGTISE_BASE_URL=https://open.gangtise.com`
|
|
31
|
+
- `vault wechat-chatroom-list`:服务端接口改版为返回 `{ total, list }`(此前无 `total`、列名 `chatRoomList`)。CLI 相应改为按 `total` 并发翻页(不再串行翻页);省略 `--size` 仍拉全量、传 `--size N` 取前 N 条,`Total:` 提示恢复
|
|
32
|
+
- 无翻页的行情端点(`quote fund-flow` / `minute-kline` / 显式多标的的日 K:`day-kline`·`-hk`·`-us`·`index-day-kline`)返回行数撞上单次 `--limit` 时标 `partial`(退出码 3)+ stderr 警告,避免被静默截断;`--limit` 现本地校验 ≤ 10000(撞服务端上限也不漏标)。K 线 `--security all` 仍走日期分片自动补全,不受影响
|
|
33
|
+
|
|
34
|
+
**新增**
|
|
35
|
+
- `quote fund-flow` — A股个股日资金流向(沪深京),含小/中/大/特大单流入流出金额及占比、主力净流入等字段;`--security`(或 `aShares` 全市场)、`--start-date` / `--end-date`、`--limit`(默认 6000,上限 10000)、`--field` 指定返回字段;无积分消耗(单只证券无翻页,撞 `--limit` 时的截断处理见上「行为变更」;**`aShares` 全市场须显式传 `--start-date`/`--end-date`,CLI 按日自动分片并发合并——缺日期会本地报错**)
|
|
36
|
+
- `reference institution-search` — 机构 ID 搜索,输入机构名/简称返回 `institutionId` 及适用接口参数(`usageScopes`);`--keyword`(必填)、`--category`(`domesticBroker`/`foreignInstitution`/`leadInstitution`/`opinionInstitution`/`foreignOpinionInstitution`,可重复)、`--top`(默认 10,上限 10);免费。覆盖既有 `--broker`/`--institution` 全部机构类(research/foreign-report/opinion/foreign-opinion/summary/roadshow/site-visit/strategy/my-conference)
|
|
37
|
+
- `vault my-conference-list` 新增 `--source` — 按录制来源筛选(`1`=企微会议助理 `2`=会议服务微信群,可重复;不传返回全部)
|
|
38
|
+
|
|
7
39
|
### v0.22.1 — 2026-07-03
|
|
8
40
|
|
|
9
41
|
**修复**
|
|
@@ -81,38 +113,6 @@
|
|
|
81
113
|
**修复(indicator 适配 EDE 后台新结构)**
|
|
82
114
|
- `indicator cross-section` / `time-series` 适配后台改版的返回结构(字段名加 `List` 后缀 `securityCodeList/indicatorCodeList/…`、截面 `values` 改二维 `[指标][证券]`):此前后台改结构后 CLI 拍平失配、退化成原始矩阵,现恢复 `{date, security, name, 指标:值}` 宽表。配合后台同步变化——无数据从 `999999` 报错改为返回 `null`(截面不再 500、不丢行),缺必填参数从笼统 `410106` 改为直接指明缺哪个参数
|
|
83
115
|
|
|
84
|
-
### v0.19.0 — 2026-06-24
|
|
85
|
-
|
|
86
|
-
**新增接口(Indicator · 证券级数据指标 EDE)**
|
|
87
|
-
- `indicator search` — 按名称搜索证券级数据指标,返回 `indicatorCode` 及可传参数 `parameterList`(含 `required` 必填标记与枚举);取数前必先 search 拿 code,绝不猜编码
|
|
88
|
-
- `indicator cross-section` — 指标截面数据(多指标 × 多证券,单日快照):`--indicator` / `--security`(均可重复)/ `--date` / `--currency` / `--scale` / `--indicator-param`
|
|
89
|
-
- `indicator time-series` — 指标时间序列(多指标 × 单证券 或 单指标 × 多证券,按区间):另有 `--start-date` / `--end-date` / `--calendar-type`(`ND`/`TD`/`WD`)
|
|
90
|
-
- 复权等指标专属参数用 `--indicator-param "code:key=value"`,参数 key 与取值以 search 的 `parameterList` 为准(行情复权键为 `adjustmentType`:`1` 不复权 / `2` 前复权 / `3` 后复权)
|
|
91
|
-
- 很多指标有必填参数,默认调用会报 `410106`(缺必填参数):N 期统计补 `periodNum`、区间/周期类补 `startDate`、年度/分红类补 `fiscalYear`;`999999` 多为「该证券公司类型/报告期无数据」而非系统故障。详见 `gangtise-openapi/references/commands/indicator.md`
|
|
92
|
-
|
|
93
|
-
**修复**
|
|
94
|
-
- `vault stock-pool-stocks --pool-id <id>` 过滤失效:此前因选项默认值 `["all"]` 泄漏,传具体 pool id 仍返回全部股票池证券;现已修复——传 id 精确过滤,省略则默认全量
|
|
95
|
-
- `auth` 缺凭证报错补充跨 shell(bash/zsh/fish)的 `export` 提示
|
|
96
|
-
|
|
97
|
-
**文档**
|
|
98
|
-
- README / SKILL 补充 indicator 命令组与取数最佳实践;`official-account` 命令文档补全
|
|
99
|
-
|
|
100
|
-
### v0.18.0 — 2026-06-17
|
|
101
|
-
|
|
102
|
-
**新增接口(Insight · 产业公众号资讯)**
|
|
103
|
-
- `insight official-account list` — 查询公众号资讯列表:支持 `--keyword`(需用数据中的具体词,非整句白话)/ `--account-id`(公众号 ID)/ `--security` / `--category`(文章类型枚举:`news`/`law`/`report`/`view`/`data`/`event`/`meeting`/`notice`/`recruit`/`investEdu`/`brand`/`notes`/`other`)/ `--industry`(`citicIndustry`/`swIndustry` 行业 ID)/ `--search-type`(`1` 标题 / `2` 全文)/ `--rank-type`(`1` 综合 / `2` 时间倒序);返回含模型生成摘要 `summary` 及关联行业/题材/证券列表
|
|
104
|
-
- `insight official-account download --article-id <id>` — 下载公众号文章:`--file-type 1` txt(默认)/ `2` HTML
|
|
105
|
-
|
|
106
|
-
### v0.17.0 — 2026-06-15
|
|
107
|
-
|
|
108
|
-
**接口变更(Breaking)**
|
|
109
|
-
- 日程类命令(`roadshow` / `site-visit` / `strategy` / `forum` list)改为各自只暴露 API spec 支持的筛选选项,移除原先一刀切多出的无效选项:`strategy` 仅保留 `--institution` / `--location`;`forum` 仅保留 `--research-area` / `--location`;`site-visit` 移除 `--participant-role` / `--broker-type`;`roadshow` 移除 `--object`。传不支持的选项现由 commander 直接报 `unknown option`(此前会静默发送、服务端返回空结果)
|
|
110
|
-
- `insight announcement list` 移除无效的 `--announcement-type`(服务端忽略、恒返全量);A 股公告分类筛选用 `--category`(`aShareAnnouncementCategory` 常量 ID)
|
|
111
|
-
|
|
112
|
-
**说明 / 修正**
|
|
113
|
-
- `--industry` 用 `citicIndustry` 码(`1008001xx`,全命令通用);`--research-area` 用 `gangtiseIndustry` 码(行业 `1008001xx` + 宏观/策略/固收/金工/海外等方向 `122000xxx`)。详见 `gangtise-openapi/references/commands/reference-and-lookup.md`
|
|
114
|
-
- 日程类 `--location`(domesticCity)服务端过滤已生效(v0.16.0 时曾未生效)
|
|
115
|
-
|
|
116
116
|
> 更早版本及完整更新历史见 [CHANGELOG.md](CHANGELOG.md)。
|
|
117
117
|
|
|
118
118
|
## 首次安装
|
|
@@ -149,7 +149,7 @@ npm run dev -- --help
|
|
|
149
149
|
```bash
|
|
150
150
|
export GANGTISE_ACCESS_KEY="your-ak"
|
|
151
151
|
export GANGTISE_SECRET_KEY="your-sk"
|
|
152
|
-
export GANGTISE_BASE_URL="https://
|
|
152
|
+
export GANGTISE_BASE_URL="https://openapi.gangtise.com"
|
|
153
153
|
export GANGTISE_TOKEN="Bearer xxx"
|
|
154
154
|
|
|
155
155
|
# 性能/调试可选项
|
|
@@ -227,7 +227,7 @@ cp -r gangtise-openapi ~/.hermes/skills/gangtise-openapi
|
|
|
227
227
|
| 模块 | 子命令 | 说明 |
|
|
228
228
|
|------|--------|------|
|
|
229
229
|
| **Auth** | `login` / `status` | 认证登录、状态查询 |
|
|
230
|
-
| **Lookup** | `broker-org list` / `meeting-org list` |
|
|
230
|
+
| **Lookup** | `broker-org list` / `meeting-org list` | 券商/会议机构本地全量枚举表(按名称找 ID 优先 `reference institution-search`;行业/区域/公告分类/题材/申万码已改用 Reference 接口) |
|
|
231
231
|
| **Insight** | `opinion list` | 内资机构观点 |
|
|
232
232
|
| | `summary list` / `download` | 纪要(含下载,支持 `--file-type` 选原始/HTML) |
|
|
233
233
|
| | `roadshow list` | 路演 |
|
|
@@ -244,6 +244,7 @@ cp -r gangtise-openapi ~/.hermes/skills/gangtise-openapi
|
|
|
244
244
|
| | `official-account list` / `download` | 产业公众号资讯(含 txt/HTML 下载) |
|
|
245
245
|
| **Reference** | `securities-search` | GTS Code 搜索(按名称/代码/拼音匹配) |
|
|
246
246
|
| | `chiefs-search` | 首席分析师 ID 搜索(按姓名/机构/团队匹配) |
|
|
247
|
+
| | `institution-search` | 机构 ID 搜索(内资券商/外资/牵头/观点机构,按名称匹配) |
|
|
247
248
|
| | `constant-category` | 常量分类列表(含各分类适用的接口与参数) |
|
|
248
249
|
| | `constant-list` | 按分类导出常量值全量列表(行业/城市/公告分类/区域等) |
|
|
249
250
|
| | `concept-search` | 题材 ID 搜索(名称/拼音/分组名匹配) |
|
|
@@ -253,6 +254,7 @@ cp -r gangtise-openapi ~/.hermes/skills/gangtise-openapi
|
|
|
253
254
|
| | `index-day-kline` | 沪深京指数日K线 |
|
|
254
255
|
| | `minute-kline` | A股分钟K线 |
|
|
255
256
|
| | `realtime` | 个股实时行情快照(A股/港股/美股) |
|
|
257
|
+
| | `fund-flow` | A股个股日资金流向(沪深京;小/中/大/特大单 + 主力净流入) |
|
|
256
258
|
| **Fundamental** | `income-statement` / `balance-sheet` / `cash-flow` | A股三大财务报表(累计) |
|
|
257
259
|
| | `income-statement-quarterly` / `cash-flow-quarterly` | A股利润表/现金流量表(单季度) |
|
|
258
260
|
| | `income-statement-hk` / `balance-sheet-hk` / `cash-flow-hk` | 港股三大财务报表(中国会计准则) |
|
|
@@ -301,7 +303,7 @@ cp -r gangtise-openapi ~/.hermes/skills/gangtise-openapi
|
|
|
301
303
|
- `gangtise indicator ...`
|
|
302
304
|
- `gangtise alternative ...`
|
|
303
305
|
- `gangtise reference ...`
|
|
304
|
-
- `gangtise raw call ...`
|
|
306
|
+
- `gangtise raw call ...` / `gangtise raw list`
|
|
305
307
|
|
|
306
308
|
## 推荐工作流
|
|
307
309
|
|
|
@@ -362,7 +364,7 @@ gangtise ai knowledge-batch --query 比亚迪 --query 最近热门概念
|
|
|
362
364
|
- `vault record-list`
|
|
363
365
|
- `vault my-conference-list`
|
|
364
366
|
- `vault wechat-message-list`
|
|
365
|
-
- `vault wechat-chatroom-list
|
|
367
|
+
- `vault wechat-chatroom-list`
|
|
366
368
|
- `ai hot-topic`
|
|
367
369
|
|
|
368
370
|
规则:
|
|
@@ -373,7 +375,6 @@ gangtise ai knowledge-batch --query 比亚迪 --query 最近热门概念
|
|
|
373
375
|
- 安全上限:自动翻页最多 1000 页,防止异常循环
|
|
374
376
|
- 部分页失败、或服务端实际返回行数与 `total` 矛盾(提前短页)时,不丢弃已取到的数据:结果带 `partial: true`(页失败时另有 `failedPages`;K线分片为 `failedShards`;`--format json` 可见),stderr 输出警告,**进程退出码为 3**(完整成功为 0)
|
|
375
377
|
- 分页结果中 `total` 字段会被保留(json 格式输出 `{total, list}`);其他格式下 stderr 输出 `Total: N, showing: M`(json 格式不输出该行)
|
|
376
|
-
- `vault wechat-chatroom-list` 是特例:接口不返回 `total`,CLI 改为串行翻页——省略 `--size` 拉全量、传 `--size N` 取前 N 条,单页 50,无 `Total:` 提示
|
|
377
378
|
|
|
378
379
|
## 智能文件命名
|
|
379
380
|
|
|
@@ -462,6 +463,8 @@ gangtise reference securities-search --keyword "银行" --category stock --categ
|
|
|
462
463
|
# 首席分析师 ID 搜索(按姓名/机构/团队;拿 chiefId 供 insight opinion list --chief 使用)
|
|
463
464
|
gangtise reference chiefs-search --keyword 东吴证券 --top 3 --format json
|
|
464
465
|
gangtise reference chiefs-search --keyword 芦哲 --format json
|
|
466
|
+
# 机构 ID 搜索(--category: domesticBroker/foreignInstitution/leadInstitution/opinionInstitution)
|
|
467
|
+
gangtise reference institution-search --keyword 招商证券 --category domesticBroker --top 3 --format json
|
|
465
468
|
|
|
466
469
|
# 常量查询:先看分类,再按分类导出全量常量值
|
|
467
470
|
gangtise reference constant-category --format json
|
|
@@ -502,6 +505,8 @@ gangtise quote minute-kline --security 600519.SH --start-time "2026-04-15 09:30:
|
|
|
502
505
|
gangtise quote realtime --security 600519.SH --security 00700.HK --security AAPL.O --field securityCode --field tradeTime --field latestPrice --field pctChange --field volume --format json
|
|
503
506
|
# 实时行情:全市场批量(建议配合 --field 精简字段)
|
|
504
507
|
gangtise quote realtime --security aShares --field securityCode --field latestPrice --field pctChange --field volume --format json
|
|
508
|
+
# A股个股日资金流向(沪深京;--security aShares 全市场;--limit 上限 10000,超限缩短日期区间分批)
|
|
509
|
+
gangtise quote fund-flow --security 600519.SH --security 000001.SZ --start-date 2026-06-01 --end-date 2026-06-05 --field mainNetInflow --field largeInflow --field xlargeInflow --format json
|
|
505
510
|
```
|
|
506
511
|
|
|
507
512
|
> **历史 vs 实时**:`day-kline*` 仅返回历史数据(当日数据入库时间:A 股 ~15:30 / 港股 ~16:30 / 美股 ~07:00 北京时间)。盘中需要最新成交价、振幅等实时字段必须走 `quote realtime`。
|
|
@@ -598,8 +603,9 @@ gangtise vault record-list --keyword 晨会 --category upload --category mobile
|
|
|
598
603
|
# 录音速记下载(--content-type: original/asr/summary)
|
|
599
604
|
gangtise vault record-download --record-id 49412 --content-type summary
|
|
600
605
|
|
|
601
|
-
#
|
|
606
|
+
# 我的会议列表(--source 录制来源:1=企微会议助理 2=会议服务微信群,可重复;不传返回全部)
|
|
602
607
|
gangtise vault my-conference-list --keyword AI --category earningsCall --institution C100000027
|
|
608
|
+
gangtise vault my-conference-list --source 2 --category earningsCall --size 20
|
|
603
609
|
# 我的会议下载(--content-type: asr/summary)
|
|
604
610
|
gangtise vault my-conference-download --conference-id 43319 --content-type asr
|
|
605
611
|
|
|
@@ -678,6 +684,10 @@ gangtise alternative concept-securities --concept-id 121000130 --format json
|
|
|
678
684
|
### Raw
|
|
679
685
|
|
|
680
686
|
```bash
|
|
687
|
+
# 先列出所有 endpoint key(配合 raw call,不必翻文档记 key)
|
|
688
|
+
gangtise raw list
|
|
689
|
+
gangtise raw list --format json # key / method / path / description
|
|
690
|
+
|
|
681
691
|
gangtise raw call insight.opinion.list --body '{"from":0,"size":120}'
|
|
682
692
|
```
|
|
683
693
|
|
package/dist/src/cli.js
CHANGED
|
@@ -2,13 +2,13 @@
|
|
|
2
2
|
import { Command, Option } from "commander";
|
|
3
3
|
import { checkAsyncContent, pollAsyncContent, POLL_MAX_ATTEMPTS } from "./core/asyncContent.js";
|
|
4
4
|
import { readTokenCache, redactTokenCache } from "./core/auth.js";
|
|
5
|
-
import { collectKeyValue, collectList, collectNumberList, maybeArray, parseFrom, parseNumberOption, parseOptionalNumberOption, parseSize, parseTimestamp13 } from "./core/args.js";
|
|
5
|
+
import { collectKeyValue, collectList, collectNumberList, localDateString, maybeArray, parseFrom, parseNumberOption, parseOptionalNumberOption, parseSize, parseTimestamp13 } from "./core/args.js";
|
|
6
6
|
import { buildIndicatorCrossSectionBody, buildIndicatorTimeSeriesBody, buildQuoteKlineBody, buildStockPoolStocksBody, buildWechatChatroomListBody, buildWechatMessageListBody } from "./core/commandBodies.js";
|
|
7
7
|
import { flattenCrossSection, flattenTimeSeries, unwrapIndicatorData } from "./core/indicatorMatrix.js";
|
|
8
|
-
import { callKlineWithSharding } from "./core/quoteSharding.js";
|
|
8
|
+
import { callKlineWithSharding, isAllMarket, isFullMarket } from "./core/quoteSharding.js";
|
|
9
9
|
import { loadConfig } from "./core/config.js";
|
|
10
10
|
import { resolveTitle, saveDownloadResult, uniquePath } from "./core/download.js";
|
|
11
|
-
import { ENDPOINTS } from "./core/endpoints.js";
|
|
11
|
+
import { ENDPOINTS, listEndpoints } from "./core/endpoints.js";
|
|
12
12
|
import { ApiError, ConfigError, ValidationError } from "./core/errors.js";
|
|
13
13
|
import { normalizeRows } from "./core/normalize.js";
|
|
14
14
|
import { parseOutputFormat } from "./core/output.js";
|
|
@@ -35,6 +35,33 @@ async function emit(options, produce, cache) {
|
|
|
35
35
|
async function withClient(fn) {
|
|
36
36
|
await fn(await createClient());
|
|
37
37
|
}
|
|
38
|
+
/**
|
|
39
|
+
* Server-side default row cap shared by the limit-capped, non-paginated quote endpoints
|
|
40
|
+
* (fund-flow, minute-kline, day/index kline — all default to 6000 per the API docs). We
|
|
41
|
+
* send it EXPLICITLY when `--limit` is omitted (rather than letting the server apply its
|
|
42
|
+
* own default) so the request limit and the truncation `cap` below are always the same
|
|
43
|
+
* number — never a guess about the server's default that can drift out of sync.
|
|
44
|
+
*/
|
|
45
|
+
const DEFAULT_QUOTE_LIMIT = 6000;
|
|
46
|
+
/**
|
|
47
|
+
* Limit-capped, non-paginated endpoints (fund-flow, kline) report `total` as the
|
|
48
|
+
* RETURNED row count, not the true total, so a full page (rows == the limit we sent) is
|
|
49
|
+
* the only truncation signal. Flag the result partial (printData → exit 3) + warn so a
|
|
50
|
+
* capped export isn't mistaken for the full set. `cap` MUST be the exact limit the caller
|
|
51
|
+
* sent on the request; `--limit` is validated to <= 10000 so `cap` can't exceed the
|
|
52
|
+
* server ceiling and hide a truncation.
|
|
53
|
+
*/
|
|
54
|
+
function flagIfLimitTruncated(data, cap, label) {
|
|
55
|
+
if (!data || typeof data !== "object" || Array.isArray(data))
|
|
56
|
+
return;
|
|
57
|
+
const rec = data;
|
|
58
|
+
if (rec.partial === true)
|
|
59
|
+
return;
|
|
60
|
+
if (Array.isArray(rec.list) && rec.list.length >= cap) {
|
|
61
|
+
rec.partial = true;
|
|
62
|
+
process.stderr.write(`[gangtise] warning: ${label} returned ${rec.list.length} rows = the ${cap}-row limit; results are likely truncated (this endpoint has no pagination). Narrow --start-date/--end-date or raise --limit (max 10000), fetching in date batches.\n`);
|
|
63
|
+
}
|
|
64
|
+
}
|
|
38
65
|
/**
|
|
39
66
|
* Run a download. If `output` is set we already know the destination, so the
|
|
40
67
|
* client streams the body straight to disk (no in-memory Uint8Array copy);
|
|
@@ -307,13 +334,63 @@ const addKlineCommand = (name, endpointKey, securityHelp, shardDays) => quote.co
|
|
|
307
334
|
.option("--field <field>", "Field", collectList, [])
|
|
308
335
|
.option("--format <format>", "Output format", "table")
|
|
309
336
|
.option("--output <path>")
|
|
310
|
-
.action((options) =>
|
|
337
|
+
.action((options) => withClient(async (client) => {
|
|
338
|
+
const format = parseOutputFormat(options.format);
|
|
339
|
+
const body = buildQuoteKlineBody(options);
|
|
340
|
+
if (isAllMarket(body)) {
|
|
341
|
+
// `--security all` is date-sharded: callKlineWithSharding lifts the limit to the
|
|
342
|
+
// API max and owns completeness (partial / failedShards), so leave `limit` unset
|
|
343
|
+
// and skip the single-request truncation guard.
|
|
344
|
+
const data = await callKlineWithSharding(client, endpointKey, body, { shardDays });
|
|
345
|
+
await printData(data, format, options.output);
|
|
346
|
+
return;
|
|
347
|
+
}
|
|
348
|
+
// Explicit securities go out as one request: pin the limit to the known default so
|
|
349
|
+
// the sent limit and the truncation cap are the same number by construction.
|
|
350
|
+
const limit = body.limit ?? DEFAULT_QUOTE_LIMIT;
|
|
351
|
+
const data = await callKlineWithSharding(client, endpointKey, { ...body, limit }, { shardDays });
|
|
352
|
+
flagIfLimitTruncated(data, limit, name);
|
|
353
|
+
await printData(data, format, options.output);
|
|
354
|
+
}));
|
|
311
355
|
addKlineCommand("day-kline", "quote.day-kline", "Security code (A-share: .SH/.SZ/.BJ, or 'all' for full market)", 1);
|
|
312
356
|
addKlineCommand("day-kline-hk", "quote.day-kline-hk", "Security code (HK stock: .HK, or 'all' for full market)", 2);
|
|
313
357
|
addKlineCommand("day-kline-us", "quote.day-kline-us", "Security code (US stock: e.g. AAPL.O, or 'all' for full market)", 1);
|
|
314
358
|
addKlineCommand("index-day-kline", "quote.index-day-kline", "Index code (.SH/.SZ/.BJ, or 'all' for full market)", 30);
|
|
315
|
-
quote.command("minute-kline").option("--security <code>", "Security code (A-share only: .SH/.SZ/.BJ)").option("--start-time <datetime>", "Start time (yyyy-MM-dd HH:mm:ss)").option("--end-time <datetime>", "End time (yyyy-MM-dd HH:mm:ss)").option("--limit <number>", "Max rows per request (default:
|
|
359
|
+
quote.command("minute-kline").option("--security <code>", "Security code (A-share only: .SH/.SZ/.BJ)").option("--start-time <datetime>", "Start time (yyyy-MM-dd HH:mm:ss)").option("--end-time <datetime>", "End time (yyyy-MM-dd HH:mm:ss)").option("--limit <number>", "Max rows per request (default: 6000, max: 10000)").option("--field <field>", "Field", collectList, []).option("--format <format>", "Output format", "table").option("--output <path>").action((options) => withClient(async (client) => {
|
|
360
|
+
const format = parseOutputFormat(options.format);
|
|
361
|
+
const limit = parseOptionalNumberOption(options.limit, "--limit", { integer: true, min: 1, max: 10000 }) ?? DEFAULT_QUOTE_LIMIT;
|
|
362
|
+
const data = await client.call("quote.minute-kline", { securityCode: options.security, startTime: options.startTime, endTime: options.endTime, limit, fieldList: maybeArray(options.field) });
|
|
363
|
+
flagIfLimitTruncated(data, limit, "minute-kline");
|
|
364
|
+
await printData(data, format, options.output);
|
|
365
|
+
}));
|
|
316
366
|
quote.command("realtime").description("Realtime quote snapshot (A-share / HK / US)").option("--security <code>", "Security code (e.g. 600519.SH / 00700.HK / AAPL.O), or market keyword: aShares / hkStocks / usStocks", collectList, []).option("--field <field>", "Field", collectList, []).option("--format <format>", "Output format", "table").option("--output <path>").action((options) => emit(options, (client) => client.call("quote.realtime", { securityList: maybeArray(options.security), fieldList: maybeArray(options.field) })));
|
|
367
|
+
quote.command("fund-flow").description("A-share daily fund flow (SH/SZ/BJ)").option("--security <code>", "Security code (e.g. 600519.SH / 872931.BJ), or 'aShares' for full A-share market — auto-sharded by day (repeat)", collectList, []).option("--start-date <date>", "Start date yyyy-MM-dd (default: endDate minus 1 year)").option("--end-date <date>", "End date yyyy-MM-dd (default: latest trading day)").option("--limit <number>", "Max rows per request (default: 6000, max: 10000; single-security cap — aShares auto-shards by day)").option("--field <field>", "Field, e.g. mainNetInflow/largeInflow/xlargeOutflow (repeat); omit for all", collectList, []).option("--format <format>", "Output format", "table").option("--output <path>").action((options) => withClient(async (client) => {
|
|
368
|
+
const format = parseOutputFormat(options.format);
|
|
369
|
+
const body = {
|
|
370
|
+
securityList: maybeArray(options.security),
|
|
371
|
+
startDate: options.startDate,
|
|
372
|
+
endDate: options.endDate,
|
|
373
|
+
limit: parseOptionalNumberOption(options.limit, "--limit", { integer: true, min: 1, max: 10000 }),
|
|
374
|
+
fieldList: maybeArray(options.field),
|
|
375
|
+
};
|
|
376
|
+
if (isFullMarket(body, "aShares")) {
|
|
377
|
+
// Full-market fund-flow: the server errors (430012/430013) instead of truncating when
|
|
378
|
+
// a single request exceeds the row cap, so date-shard by day (~5.4k A-share rows/day,
|
|
379
|
+
// under the lifted API cap) and merge — same mechanism as `--security all` kline.
|
|
380
|
+
// Sharding needs an explicit range; without both dates it would fall back to one
|
|
381
|
+
// doomed full-market request, so require the range up front with a clear message.
|
|
382
|
+
if (!body.startDate || !body.endDate) {
|
|
383
|
+
throw new ValidationError("quote fund-flow --security aShares requires both --start-date and --end-date (the full market is fetched via per-day shards)");
|
|
384
|
+
}
|
|
385
|
+
const data = await callKlineWithSharding(client, "quote.fund-flow", body, { shardDays: 1, fullMarketValue: "aShares" });
|
|
386
|
+
await printData(data, format, options.output);
|
|
387
|
+
return;
|
|
388
|
+
}
|
|
389
|
+
const limit = body.limit ?? DEFAULT_QUOTE_LIMIT;
|
|
390
|
+
const data = await client.call("quote.fund-flow", { ...body, limit });
|
|
391
|
+
flagIfLimitTruncated(data, limit, "fund-flow");
|
|
392
|
+
await printData(data, format, options.output);
|
|
393
|
+
}));
|
|
317
394
|
program.addCommand(quote);
|
|
318
395
|
const fundamental = new Command("fundamental").description("Fundamental APIs");
|
|
319
396
|
const addFinancialReport = (name, endpointKey, periodHelp = "Period") => fundamental.command(name).requiredOption("--security-code <code>").option("--start-date <date>").option("--end-date <date>").option("--fiscal-year <year>", "Fiscal year", collectList, []).option("--period <period>", periodHelp, collectList, []).option("--report-type <type>", "Report type", collectList, []).option("--field <field>", "Field", collectList, []).option("--format <format>", "Output format", "table").option("--output <path>").action((options) => emit(options, (client) => client.call(endpointKey, { securityCode: options.securityCode, startDate: options.startDate, endDate: options.endDate, fiscalYear: maybeArray(options.fiscalYear), period: options.period.length ? options.period : undefined, reportType: options.reportType.length ? options.reportType : undefined, fieldList: maybeArray(options.field) })));
|
|
@@ -351,7 +428,7 @@ fundamental.command("valuation-analysis").requiredOption("--security-code <code>
|
|
|
351
428
|
}));
|
|
352
429
|
fundamental.command("top-holders").requiredOption("--security-code <code>").addOption(new Option("--holder-type <type>", "Holder type: top10/top10Float").choices(["top10", "top10Float"]).makeOptionMandatory()).option("--start-date <date>").option("--end-date <date>").option("--fiscal-year <year>", "Fiscal year", collectList, []).option("--period <period>", "Period: q1/interim/q3/annual/latest", collectList, []).option("--format <format>", "Output format", "table").option("--output <path>").action((options) => emit(options, (client) => client.call("fundamental.top-holders", { securityCode: options.securityCode, holderType: options.holderType, startDate: options.startDate, endDate: options.endDate, fiscalYear: maybeArray(options.fiscalYear), period: options.period.length ? options.period : undefined })));
|
|
353
430
|
fundamental.command("earning-forecast").requiredOption("--security-code <code>").option("--start-date <date>", "Start date (default: 1 year before end-date)").option("--end-date <date>", "End date (default: today)").option("--consensus <name>", "Consensus indicator: netIncome/netIncomeYoy/eps/pe/bps/pb/peg/roe/ps", collectList, []).option("--format <format>", "Output format", "table").option("--output <path>").action((options) => emit(options, (client) => {
|
|
354
|
-
const endDate = options.endDate ?? new Date()
|
|
431
|
+
const endDate = options.endDate ?? localDateString(new Date());
|
|
355
432
|
// Anchor the default window to endDate (as the help text promises), not to today —
|
|
356
433
|
// a historical --end-date without --start-date should mean "the year before it".
|
|
357
434
|
const startDate = options.startDate ?? new Date(new Date(`${endDate}T00:00:00Z`).getTime() - 365 * 24 * 60 * 60 * 1000).toISOString().slice(0, 10);
|
|
@@ -479,13 +556,18 @@ reference.command("chiefs-search").requiredOption("--keyword <text>", "Search ke
|
|
|
479
556
|
keyword: options.keyword,
|
|
480
557
|
top: parseNumberOption(options.top, "--top", { integer: true, min: 1 }),
|
|
481
558
|
})));
|
|
559
|
+
reference.command("institution-search").requiredOption("--keyword <text>", "Search keyword (institution name / abbreviation)").option("--category <name>", "Category: domesticBroker/foreignInstitution/leadInstitution/opinionInstitution/foreignOpinionInstitution (repeat); omit for all", collectList, []).option("--top <number>", "Max results (default: 10, max: 10)", "10").option("--format <format>", "Output format", "table").option("--output <path>").action((options) => emit(options, (client) => client.call("reference.institution-search", {
|
|
560
|
+
keyword: options.keyword,
|
|
561
|
+
categoryList: maybeArray(options.category),
|
|
562
|
+
top: parseNumberOption(options.top, "--top", { integer: true, min: 1 }),
|
|
563
|
+
})));
|
|
482
564
|
program.addCommand(reference);
|
|
483
565
|
const vault = new Command("vault").description("Vault APIs");
|
|
484
566
|
vault.command("drive-list").option("--from <number>", "Starting offset", "0").option("--size <number>", "Total rows to return; omit to fetch all").option("--start-time <datetime>").option("--end-time <datetime>").option("--keyword <text>").option("--file-type <number>", "File type", collectNumberList, []).option("--space-type <number>", "Space type", collectNumberList, []).option("--format <format>", "Output format", "table").option("--output <path>").action((options) => emit(options, (client) => client.call("vault.drive.list", { from: parseFrom(options.from), size: parseSize(options.size), startTime: options.startTime, endTime: options.endTime, keyword: options.keyword, fileTypeList: options.fileType.length ? options.fileType : undefined, spaceTypeList: options.spaceType.length ? options.spaceType : undefined }), { endpointKey: "vault.drive.list", idField: "fileId" }));
|
|
485
567
|
addDownloadCommand(vault, { endpointKey: "vault.drive.download", name: "drive-download", idOption: "--file-id", idField: "fileId", fallbackPrefix: "file", titleListEndpoint: "vault.drive.list" });
|
|
486
568
|
vault.command("record-list").option("--from <number>", "Starting offset", "0").option("--size <number>", "Total rows to return; omit to fetch all").option("--start-time <datetime>").option("--end-time <datetime>").option("--keyword <text>").option("--category <name>", "Recording type: upload/link/mobile/gtNote/pc/share", collectList, []).option("--space-type <number>", "Space type: 1=my records / 2=tenant records", collectNumberList, []).option("--format <format>", "Output format", "table").option("--output <path>").action((options) => emit(options, (client) => client.call("vault.record.list", { from: parseFrom(options.from), size: parseSize(options.size), startTime: options.startTime, endTime: options.endTime, keyword: options.keyword, categoryList: maybeArray(options.category), spaceTypeList: options.spaceType.length ? options.spaceType : undefined }), { endpointKey: "vault.record.list", idField: "recordId" }));
|
|
487
569
|
addDownloadCommand(vault, { endpointKey: "vault.record.download", name: "record-download", idOption: "--record-id", idField: "recordId", fallbackPrefix: "record", contentTypeDescription: "Content type: original/asr/summary", titleListEndpoint: "vault.record.list" });
|
|
488
|
-
vault.command("my-conference-list").option("--from <number>", "Starting offset", "0").option("--size <number>", "Total rows to return; omit to fetch all").option("--start-time <datetime>").option("--end-time <datetime>").option("--keyword <text>").option("--research-area <id>", "Research area ID", collectList, []).option("--security <code>", "Security code", collectList, []).option("--institution <id>", "Institution ID", collectList, []).option("--category <name>", "Conference category: earningsCall/strategyMeeting/fundRoadshow/shareholdersMeeting/maMeeting/specialMeeting/companyAnalysis/industryAnalysis/other", collectList, []).option("--format <format>", "Output format", "table").option("--output <path>").action((options) => emit(options, (client) => client.call("vault.my-conference.list", { from: parseFrom(options.from), size: parseSize(options.size), startTime: options.startTime, endTime: options.endTime, keyword: options.keyword, researchAreaList: maybeArray(options.researchArea), securityList: maybeArray(options.security), institutionList: maybeArray(options.institution), categoryList: maybeArray(options.category) }), { endpointKey: "vault.my-conference.list", idField: "conferenceId" }));
|
|
570
|
+
vault.command("my-conference-list").option("--from <number>", "Starting offset", "0").option("--size <number>", "Total rows to return; omit to fetch all").option("--start-time <datetime>").option("--end-time <datetime>").option("--keyword <text>").option("--research-area <id>", "Research area ID", collectList, []).option("--security <code>", "Security code", collectList, []).option("--institution <id>", "Institution ID", collectList, []).option("--category <name>", "Conference category: earningsCall/strategyMeeting/fundRoadshow/shareholdersMeeting/maMeeting/specialMeeting/companyAnalysis/industryAnalysis/other", collectList, []).option("--source <number>", "Recording source: 1=企微会议助理 2=会议服务微信群 (repeat)", collectNumberList, []).option("--format <format>", "Output format", "table").option("--output <path>").action((options) => emit(options, (client) => client.call("vault.my-conference.list", { from: parseFrom(options.from), size: parseSize(options.size), startTime: options.startTime, endTime: options.endTime, keyword: options.keyword, researchAreaList: maybeArray(options.researchArea), securityList: maybeArray(options.security), institutionList: maybeArray(options.institution), categoryList: maybeArray(options.category), sourceList: options.source.length ? options.source : undefined }), { endpointKey: "vault.my-conference.list", idField: "conferenceId" }));
|
|
489
571
|
addDownloadCommand(vault, { endpointKey: "vault.my-conference.download", name: "my-conference-download", idOption: "--conference-id", idField: "conferenceId", fallbackPrefix: "conference", contentTypeDescription: "Content type: asr/summary", titleListEndpoint: "vault.my-conference.list" });
|
|
490
572
|
vault.command("wechat-message-list").option("--from <number>", "Starting offset", "0").option("--size <number>", "Total rows to return; omit to fetch all").option("--start-time <datetime>").option("--end-time <datetime>").option("--keyword <text>").option("--security <code>", "Security code (e.g. 000001.SZ)", collectList, []).option("--wechat-group-id <id>", "WeChat group ID", collectList, []).option("--industry <id>", "Industry ID", collectList, []).option("--category <name>", "Message type: text/image/documents/url", collectList, []).option("--tag <name>", "Tag: roadShow/research/strategyMeeting/meetingSummary/industryComment/companyComment/earningsReview", collectList, []).option("--format <format>", "Output format", "table").option("--output <path>").action((options) => emit(options, (client) => client.call("vault.wechat-message.list", buildWechatMessageListBody(options))));
|
|
491
573
|
vault.command("wechat-chatroom-list").option("--from <number>", "Starting offset", "0").option("--size <number>", "Total rows to return; omit to fetch all").option("--room-name <name>", "WeChat group name; repeat or comma-separate for multiple names", collectList, []).option("--format <format>", "Output format", "table").option("--output <path>").action((options) => emit(options, (client) => client.call("vault.wechat-chatroom.list", buildWechatChatroomListBody(options))));
|
|
@@ -571,7 +653,7 @@ program.command("raw").description("Raw API calls").addCommand(new Command("call
|
|
|
571
653
|
}
|
|
572
654
|
const data = await client.call(endpointKey, body);
|
|
573
655
|
await printData(data, format, options.output);
|
|
574
|
-
}));
|
|
656
|
+
})).addCommand(new Command("list").description("List all registered endpoint keys (for use with 'raw call')").option("--format <format>", "Output format", "table").option("--output <path>").action((options) => printData(listEndpoints(), parseOutputFormat(options.format), options.output)));
|
|
575
657
|
async function checkForUpdate(timeoutMs = 2000) {
|
|
576
658
|
try {
|
|
577
659
|
const response = await fetch("https://registry.npmjs.org/gangtise-openapi-cli/latest", { signal: AbortSignal.timeout(timeoutMs) });
|
package/dist/src/core/args.js
CHANGED
|
@@ -24,6 +24,9 @@ export function parseNumberOption(value, optionName, config = {}) {
|
|
|
24
24
|
if (config.min !== undefined && parsed < config.min) {
|
|
25
25
|
throw new ValidationError(`Invalid ${optionName}: expected a number >= ${config.min}`);
|
|
26
26
|
}
|
|
27
|
+
if (config.max !== undefined && parsed > config.max) {
|
|
28
|
+
throw new ValidationError(`Invalid ${optionName}: expected a number <= ${config.max}`);
|
|
29
|
+
}
|
|
27
30
|
return parsed;
|
|
28
31
|
}
|
|
29
32
|
export function parseOptionalNumberOption(value, optionName, config = {}) {
|
|
@@ -91,6 +94,15 @@ export function parseTimestamp13(value, optionName) {
|
|
|
91
94
|
}
|
|
92
95
|
return parsed;
|
|
93
96
|
}
|
|
97
|
+
/** Machine-local calendar date as `yyyy-MM-dd`, for CLI "default: today" options.
|
|
98
|
+
* `new Date().toISOString().slice(0,10)` renders the UTC day — for CST users a
|
|
99
|
+
* pre-08:00 "today" resolves to yesterday. Anchoring to local components matches
|
|
100
|
+
* toTimestamp13's local-midnight convention. */
|
|
101
|
+
export function localDateString(d) {
|
|
102
|
+
const month = String(d.getMonth() + 1).padStart(2, "0");
|
|
103
|
+
const day = String(d.getDate()).padStart(2, "0");
|
|
104
|
+
return `${d.getFullYear()}-${month}-${day}`;
|
|
105
|
+
}
|
|
94
106
|
// Parse repeatable `--indicator-param "code:key=value"` specs into the nested
|
|
95
107
|
// indicatorParamList the EDE cross-section / time-series endpoints expect.
|
|
96
108
|
// Multiple specs for the same code accumulate into one group, first-seen order.
|
package/dist/src/core/client.js
CHANGED
|
@@ -5,9 +5,9 @@ import { pipeline } from "node:stream/promises";
|
|
|
5
5
|
import { request } from "undici";
|
|
6
6
|
import { isTokenCacheValid, normalizeToken, readTokenCache, requireAccessCredentials, writeTokenCache } from "./auth.js";
|
|
7
7
|
import { ApiError, ValidationError } from "./errors.js";
|
|
8
|
-
import { ENDPOINTS } from "./endpoints.js";
|
|
8
|
+
import { ENDPOINTS, resolveTimeoutMs } from "./endpoints.js";
|
|
9
9
|
import { getLookupData } from "./lookupData/index.js";
|
|
10
|
-
import { getDispatcher, isVerbose, logTiming, markRetryable, PAGE_CONCURRENCY, runWithConcurrency, withRetry } from "./transport.js";
|
|
10
|
+
import { decodeResponseBody, getDispatcher, isVerbose, logTiming, markRetryable, PAGE_CONCURRENCY, parseRetryAfterMs, runWithConcurrency, withRetry } from "./transport.js";
|
|
11
11
|
// Auth errors that warrant a forced re-login + one replay. 8000014/8000015 are
|
|
12
12
|
// AK/SK errors; 0000001008 is a server-side token invalidation (the token still
|
|
13
13
|
// looks valid by local expiry, so only a forced refresh recovers it).
|
|
@@ -119,12 +119,12 @@ export class GangtiseClient {
|
|
|
119
119
|
return false;
|
|
120
120
|
return 'msg' in obj || 'data' in obj || 'success' in obj || 'status' in obj;
|
|
121
121
|
}
|
|
122
|
-
throwHttpError(parsed, statusCode) {
|
|
122
|
+
throwHttpError(parsed, statusCode, retryAfterMs) {
|
|
123
123
|
if (this.isEnvelope(parsed)) {
|
|
124
124
|
const code = parsed.code === undefined ? undefined : String(parsed.code);
|
|
125
|
-
throw new ApiError(parsed.msg || `API request failed (HTTP ${statusCode})`, code, statusCode, parsed);
|
|
125
|
+
throw new ApiError(parsed.msg || `API request failed (HTTP ${statusCode})`, code, statusCode, parsed, retryAfterMs);
|
|
126
126
|
}
|
|
127
|
-
throw new ApiError(`API request failed (HTTP ${statusCode})`, undefined, statusCode, parsed);
|
|
127
|
+
throw new ApiError(`API request failed (HTTP ${statusCode})`, undefined, statusCode, parsed, retryAfterMs);
|
|
128
128
|
}
|
|
129
129
|
unwrapEnvelope(parsed, statusCode) {
|
|
130
130
|
if (!this.isEnvelope(parsed)) {
|
|
@@ -294,75 +294,6 @@ export class GangtiseClient {
|
|
|
294
294
|
}
|
|
295
295
|
return out;
|
|
296
296
|
}
|
|
297
|
-
/**
|
|
298
|
-
* Sequential pagination for endpoints that page by offset but return NO `total`
|
|
299
|
-
* and use a non-standard list key (e.g. wechat chatroom's `chatRoomList`). We
|
|
300
|
-
* can't fan out like requestPaginated (no total ⇒ unknown page count), so page
|
|
301
|
-
* serially until a short page (fewer rows than requested) signals the end.
|
|
302
|
-
* Returns `{ [listKey]: rows }` so normalize/printer treat it like any list.
|
|
303
|
-
*/
|
|
304
|
-
async requestSequentialPaginated(endpoint, body) {
|
|
305
|
-
const initialBody = body && typeof body === 'object' ? { ...body } : {};
|
|
306
|
-
if ('from' in initialBody && (typeof initialBody.from !== 'number' || !Number.isFinite(initialBody.from) || initialBody.from < 0)) {
|
|
307
|
-
throw new ValidationError('Invalid from: expected a non-negative number');
|
|
308
|
-
}
|
|
309
|
-
if ('size' in initialBody && initialBody.size !== undefined && (typeof initialBody.size !== 'number' || !Number.isFinite(initialBody.size) || initialBody.size <= 0)) {
|
|
310
|
-
throw new ValidationError('Invalid size: expected a positive number');
|
|
311
|
-
}
|
|
312
|
-
const listKey = endpoint.pagination?.listKey ?? 'list';
|
|
313
|
-
const maxPageSize = endpoint.pagination?.maxPageSize ?? 50;
|
|
314
|
-
const startFrom = typeof initialBody.from === 'number' && Number.isFinite(initialBody.from) ? initialBody.from : 0;
|
|
315
|
-
const requestedSize = typeof initialBody.size === 'number' && Number.isFinite(initialBody.size) ? initialBody.size : undefined;
|
|
316
|
-
const extractList = (page) => {
|
|
317
|
-
if (!page || typeof page !== 'object')
|
|
318
|
-
return null;
|
|
319
|
-
const arr = page[listKey];
|
|
320
|
-
return Array.isArray(arr) ? arr : null;
|
|
321
|
-
};
|
|
322
|
-
const collected = [];
|
|
323
|
-
let firstPage = null;
|
|
324
|
-
let from = startFrom;
|
|
325
|
-
const MAX_PAGES = 1000;
|
|
326
|
-
let truncatedByPageCap = false;
|
|
327
|
-
let partialShape = false;
|
|
328
|
-
for (let page = 0;; page++) {
|
|
329
|
-
const remaining = requestedSize === undefined ? maxPageSize : requestedSize - collected.length;
|
|
330
|
-
if (requestedSize !== undefined && remaining <= 0)
|
|
331
|
-
break;
|
|
332
|
-
const size = Math.min(maxPageSize, remaining);
|
|
333
|
-
const pageData = await this.requestJson(endpoint, { ...initialBody, from, size });
|
|
334
|
-
if (firstPage === null)
|
|
335
|
-
firstPage = pageData;
|
|
336
|
-
const list = extractList(pageData);
|
|
337
|
-
if (list === null) {
|
|
338
|
-
// First response isn't a paginated-list shape → return it untouched. But a
|
|
339
|
-
// LATER page losing shape must NOT discard the rows already collected (mirrors
|
|
340
|
-
// requestPaginated's fail-soft fan-out): stop, keep them, and warn loudly.
|
|
341
|
-
if (page === 0)
|
|
342
|
-
return firstPage;
|
|
343
|
-
partialShape = true;
|
|
344
|
-
process.stderr.write(`[gangtise] warning: a page response had unexpected shape; results are partial — ${collected.length} rows fetched.\n`);
|
|
345
|
-
break;
|
|
346
|
-
}
|
|
347
|
-
for (const item of list)
|
|
348
|
-
collected.push(item);
|
|
349
|
-
if (list.length < size)
|
|
350
|
-
break; // short page ⇒ no more rows
|
|
351
|
-
if (page + 1 >= MAX_PAGES) {
|
|
352
|
-
truncatedByPageCap = true;
|
|
353
|
-
break;
|
|
354
|
-
}
|
|
355
|
-
from += list.length;
|
|
356
|
-
}
|
|
357
|
-
if (truncatedByPageCap) {
|
|
358
|
-
process.stderr.write(`[gangtise] warning: hit the ${MAX_PAGES}-page safety cap; fetched ${collected.length} rows. Pass --size to fetch a bounded subset.\n`);
|
|
359
|
-
}
|
|
360
|
-
const rows = requestedSize === undefined ? collected : collected.slice(0, requestedSize);
|
|
361
|
-
const out = { [listKey]: rows };
|
|
362
|
-
if (partialShape)
|
|
363
|
-
out.partial = true;
|
|
364
|
-
return out;
|
|
365
|
-
}
|
|
366
297
|
async login() {
|
|
367
298
|
const authorization = await this.getAuthorizationHeader();
|
|
368
299
|
const cache = await readTokenCache(this.config.tokenCachePath);
|
|
@@ -378,9 +309,13 @@ export class GangtiseClient {
|
|
|
378
309
|
const dispatcher = getDispatcher();
|
|
379
310
|
const url = this.buildUrl(endpoint.path);
|
|
380
311
|
const authState = { retried: false };
|
|
312
|
+
const timeoutMs = resolveTimeoutMs(this.config.timeoutMs, endpoint);
|
|
381
313
|
return withRetry(async () => {
|
|
382
314
|
const headers = {
|
|
383
315
|
'content-type': 'application/json',
|
|
316
|
+
// undici does not auto-decompress; decodeResponseBody gunzips below. Server
|
|
317
|
+
// gzip cuts JSON payloads ~3-10x (measured 3.6x on constant-list).
|
|
318
|
+
'accept-encoding': 'gzip',
|
|
384
319
|
};
|
|
385
320
|
// Keep the header we actually sent: the self-heal check compares it against
|
|
386
321
|
// the current memoCache token to tell "stale token" from "fresh token died".
|
|
@@ -394,12 +329,22 @@ export class GangtiseClient {
|
|
|
394
329
|
method: endpoint.method,
|
|
395
330
|
headers,
|
|
396
331
|
body: endpoint.method === 'GET' ? undefined : JSON.stringify(body ?? {}),
|
|
397
|
-
headersTimeout:
|
|
398
|
-
bodyTimeout:
|
|
332
|
+
headersTimeout: timeoutMs,
|
|
333
|
+
bodyTimeout: timeoutMs,
|
|
399
334
|
dispatcher,
|
|
400
335
|
});
|
|
401
|
-
|
|
336
|
+
// Only buffer + gunzip when the server actually compressed; an unencoded
|
|
337
|
+
// response reads as text directly (and keeps existing behavior on that path).
|
|
338
|
+
const encoding = response.headers['content-encoding'];
|
|
339
|
+
const gzipped = (Array.isArray(encoding) ? encoding[0] : encoding)?.toLowerCase().trim() === 'gzip';
|
|
340
|
+
const text = gzipped
|
|
341
|
+
? decodeResponseBody(new Uint8Array(await response.body.arrayBuffer()), encoding)
|
|
342
|
+
: await response.body.text();
|
|
402
343
|
logTiming(`${endpoint.method} ${endpoint.path}`, Date.now() - startedAt, `${response.statusCode}, ${text.length}B`);
|
|
344
|
+
// Parse Retry-After once so every error path below (JSON parse failure AND the
|
|
345
|
+
// envelope/HTTP-error throw) carries it — a non-JSON 429/503 must still honor
|
|
346
|
+
// the server's rate window instead of falling back to default backoff.
|
|
347
|
+
const retryAfterMs = parseRetryAfterMs(response.headers['retry-after'], Date.now());
|
|
403
348
|
let parsed;
|
|
404
349
|
try {
|
|
405
350
|
parsed = JSON.parse(text);
|
|
@@ -408,13 +353,13 @@ export class GangtiseClient {
|
|
|
408
353
|
const message = response.statusCode >= 400
|
|
409
354
|
? `API request failed (HTTP ${response.statusCode})`
|
|
410
355
|
: 'Failed to parse API response';
|
|
411
|
-
throw new ApiError(message, undefined, response.statusCode, text.slice(0, 500));
|
|
356
|
+
throw new ApiError(message, undefined, response.statusCode, text.slice(0, 500), retryAfterMs);
|
|
412
357
|
}
|
|
413
358
|
try {
|
|
414
359
|
// Auth errors can arrive as HTTP 4xx or as a 200-wrapped error envelope;
|
|
415
360
|
// both routes must reach the self-heal check below.
|
|
416
361
|
if (response.statusCode >= 400) {
|
|
417
|
-
this.throwHttpError(parsed, response.statusCode);
|
|
362
|
+
this.throwHttpError(parsed, response.statusCode, retryAfterMs);
|
|
418
363
|
}
|
|
419
364
|
return this.unwrapEnvelope(parsed, response.statusCode);
|
|
420
365
|
}
|
|
@@ -473,6 +418,9 @@ export class GangtiseClient {
|
|
|
473
418
|
});
|
|
474
419
|
}
|
|
475
420
|
const contentType = Array.isArray(response.headers['content-type']) ? response.headers['content-type'][0] : response.headers['content-type'];
|
|
421
|
+
// From the final (post-redirect) response, so a rate-limited download honors
|
|
422
|
+
// Retry-After too — every error branch below passes it into the ApiError.
|
|
423
|
+
const retryAfterMs = parseRetryAfterMs(response.headers['retry-after'], Date.now());
|
|
476
424
|
if (contentType?.includes('application/json')) {
|
|
477
425
|
const text = await response.body.text();
|
|
478
426
|
logTiming(`GET ${endpoint.path} (json)`, Date.now() - startedAt, `${response.statusCode}, ${text.length}B`);
|
|
@@ -482,14 +430,14 @@ export class GangtiseClient {
|
|
|
482
430
|
}
|
|
483
431
|
catch {
|
|
484
432
|
if (response.statusCode >= 400) {
|
|
485
|
-
throw new ApiError('Download failed', undefined, response.statusCode, text);
|
|
433
|
+
throw new ApiError('Download failed', undefined, response.statusCode, text, retryAfterMs);
|
|
486
434
|
}
|
|
487
435
|
return { text, contentType };
|
|
488
436
|
}
|
|
489
437
|
let data;
|
|
490
438
|
try {
|
|
491
439
|
if (response.statusCode >= 400) {
|
|
492
|
-
this.throwHttpError(parsed, response.statusCode);
|
|
440
|
+
this.throwHttpError(parsed, response.statusCode, retryAfterMs);
|
|
493
441
|
}
|
|
494
442
|
data = this.unwrapEnvelope(parsed, response.statusCode);
|
|
495
443
|
}
|
|
@@ -506,13 +454,13 @@ export class GangtiseClient {
|
|
|
506
454
|
const text = await response.body.text();
|
|
507
455
|
logTiming(`GET ${endpoint.path} (text)`, Date.now() - startedAt, `${response.statusCode}, ${text.length}B`);
|
|
508
456
|
if (response.statusCode >= 400) {
|
|
509
|
-
throw new ApiError('Download failed', undefined, response.statusCode, text);
|
|
457
|
+
throw new ApiError('Download failed', undefined, response.statusCode, text, retryAfterMs);
|
|
510
458
|
}
|
|
511
459
|
return { text, contentType };
|
|
512
460
|
}
|
|
513
461
|
if (response.statusCode >= 400) {
|
|
514
462
|
const text = await response.body.text();
|
|
515
|
-
throw new ApiError('Download failed', undefined, response.statusCode, text);
|
|
463
|
+
throw new ApiError('Download failed', undefined, response.statusCode, text, retryAfterMs);
|
|
516
464
|
}
|
|
517
465
|
const contentDisposition = response.headers['content-disposition'];
|
|
518
466
|
const filenameMatch = Array.isArray(contentDisposition)
|
|
@@ -572,9 +520,7 @@ export class GangtiseClient {
|
|
|
572
520
|
return this.download(endpoint, query ?? {}, options);
|
|
573
521
|
}
|
|
574
522
|
if (endpoint.kind === 'json' && endpoint.pagination?.enabled) {
|
|
575
|
-
return endpoint
|
|
576
|
-
? this.requestSequentialPaginated(endpoint, body)
|
|
577
|
-
: this.requestPaginated(endpoint, body);
|
|
523
|
+
return this.requestPaginated(endpoint, body);
|
|
578
524
|
}
|
|
579
525
|
return this.requestJson(endpoint, body);
|
|
580
526
|
}
|
|
@@ -4,7 +4,7 @@ export function buildQuoteKlineBody(options) {
|
|
|
4
4
|
securityList: maybeArray(options.security),
|
|
5
5
|
startDate: options.startDate,
|
|
6
6
|
endDate: options.endDate,
|
|
7
|
-
limit: parseOptionalNumberOption(options.limit, "--limit", { integer: true, min: 1 }),
|
|
7
|
+
limit: parseOptionalNumberOption(options.limit, "--limit", { integer: true, min: 1, max: 10000 }),
|
|
8
8
|
fieldList: maybeArray(options.field),
|
|
9
9
|
};
|
|
10
10
|
}
|
package/dist/src/core/config.js
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import os from "node:os";
|
|
2
2
|
import path from "node:path";
|
|
3
|
-
export const DEFAULT_BASE_URL = "https://
|
|
3
|
+
export const DEFAULT_BASE_URL = "https://openapi.gangtise.com";
|
|
4
4
|
export const DEFAULT_TIMEOUT_MS = 30_000;
|
|
5
5
|
export const DEFAULT_TOKEN_CACHE_PATH = path.join(os.homedir(), ".config", "gangtise", "token.json");
|
|
6
6
|
export function loadConfig() {
|
|
@@ -1,3 +1,8 @@
|
|
|
1
|
+
/** Effective request timeout: the endpoint's floor, or the config timeout if higher
|
|
2
|
+
* (a user who raised GANGTISE_TIMEOUT_MS keeps their value). */
|
|
3
|
+
export function resolveTimeoutMs(configTimeoutMs, endpoint) {
|
|
4
|
+
return Math.max(configTimeoutMs, endpoint.timeoutMs ?? 0);
|
|
5
|
+
}
|
|
1
6
|
// Registry entries omit `key`: it is derived from the record key when ENDPOINTS is
|
|
2
7
|
// built below, so the two can never drift.
|
|
3
8
|
const ENDPOINT_DEFS = {
|
|
@@ -181,6 +186,12 @@ const ENDPOINT_DEFS = {
|
|
|
181
186
|
kind: "json",
|
|
182
187
|
description: "Search chief analyst IDs by name / institution / team",
|
|
183
188
|
},
|
|
189
|
+
"reference.institution-search": {
|
|
190
|
+
method: "POST",
|
|
191
|
+
path: "/application/open-reference/institutions/search",
|
|
192
|
+
kind: "json",
|
|
193
|
+
description: "Search institution IDs by keyword (domestic broker / foreign / lead / opinion institution)",
|
|
194
|
+
},
|
|
184
195
|
"reference.constant-category": {
|
|
185
196
|
method: "GET",
|
|
186
197
|
path: "/application/open-reference/constants/category",
|
|
@@ -248,6 +259,12 @@ const ENDPOINT_DEFS = {
|
|
|
248
259
|
kind: "json",
|
|
249
260
|
description: "Query realtime quote snapshot (A-share / HK / US)",
|
|
250
261
|
},
|
|
262
|
+
"quote.fund-flow": {
|
|
263
|
+
method: "POST",
|
|
264
|
+
path: "/application/open-quote/fund-flow/daily",
|
|
265
|
+
kind: "json",
|
|
266
|
+
description: "Query A-share daily fund flow (SH/SZ/BJ; small/medium/large/xlarge orders + main net inflow)",
|
|
267
|
+
},
|
|
251
268
|
// ─── fundamental ───
|
|
252
269
|
"fundamental.income-statement": {
|
|
253
270
|
method: "POST",
|
|
@@ -370,18 +387,21 @@ const ENDPOINT_DEFS = {
|
|
|
370
387
|
path: "/application/open-ai/agent/one-pager",
|
|
371
388
|
kind: "json",
|
|
372
389
|
description: "Generate one pager",
|
|
390
|
+
timeoutMs: 120_000,
|
|
373
391
|
},
|
|
374
392
|
"ai.investment-logic": {
|
|
375
393
|
method: "POST",
|
|
376
394
|
path: "/application/open-ai/agent/investment-logic",
|
|
377
395
|
kind: "json",
|
|
378
396
|
description: "Generate investment logic",
|
|
397
|
+
timeoutMs: 120_000,
|
|
379
398
|
},
|
|
380
399
|
"ai.peer-comparison": {
|
|
381
400
|
method: "POST",
|
|
382
401
|
path: "/application/open-ai/agent/peer-comparison",
|
|
383
402
|
kind: "json",
|
|
384
403
|
description: "Generate peer comparison",
|
|
404
|
+
timeoutMs: 120_000,
|
|
385
405
|
},
|
|
386
406
|
"ai.earnings-review.get-id": {
|
|
387
407
|
method: "POST",
|
|
@@ -400,12 +420,14 @@ const ENDPOINT_DEFS = {
|
|
|
400
420
|
path: "/application/open-ai/agent/theme-tracking",
|
|
401
421
|
kind: "json",
|
|
402
422
|
description: "Get theme tracking daily report",
|
|
423
|
+
timeoutMs: 120_000,
|
|
403
424
|
},
|
|
404
425
|
"ai.research-outline": {
|
|
405
426
|
method: "POST",
|
|
406
427
|
path: "/application/open-ai/agent/research-outline",
|
|
407
428
|
kind: "json",
|
|
408
429
|
description: "Get company research outline",
|
|
430
|
+
timeoutMs: 120_000,
|
|
409
431
|
},
|
|
410
432
|
"ai.hot-topic": {
|
|
411
433
|
method: "POST",
|
|
@@ -419,12 +441,14 @@ const ENDPOINT_DEFS = {
|
|
|
419
441
|
path: "/application/open-ai/management-discuss/from-announcement",
|
|
420
442
|
kind: "json",
|
|
421
443
|
description: "Management discussion from financial reports (half-year/annual)",
|
|
444
|
+
timeoutMs: 120_000,
|
|
422
445
|
},
|
|
423
446
|
"ai.management-discuss-earnings-call": {
|
|
424
447
|
method: "POST",
|
|
425
448
|
path: "/application/open-ai/management-discuss/from-earningsCall",
|
|
426
449
|
kind: "json",
|
|
427
450
|
description: "Management discussion from earnings calls",
|
|
451
|
+
timeoutMs: 120_000,
|
|
428
452
|
},
|
|
429
453
|
"ai.viewpoint-debate.get-id": {
|
|
430
454
|
method: "POST",
|
|
@@ -490,8 +514,8 @@ const ENDPOINT_DEFS = {
|
|
|
490
514
|
path: "/application/open-vault/wechatgroupmsg/chatroomId",
|
|
491
515
|
kind: "json",
|
|
492
516
|
description: "List WeChat group chatroom IDs",
|
|
493
|
-
//
|
|
494
|
-
pagination: { enabled: true, maxPageSize: 50
|
|
517
|
+
// Response is `{ total, list }` (server caps size at 50); auto-paginate by total.
|
|
518
|
+
pagination: { enabled: true, maxPageSize: 50 },
|
|
495
519
|
},
|
|
496
520
|
"vault.stock-pool.list": {
|
|
497
521
|
method: "POST",
|
|
@@ -551,3 +575,8 @@ const ENDPOINT_DEFS = {
|
|
|
551
575
|
},
|
|
552
576
|
};
|
|
553
577
|
export const ENDPOINTS = Object.fromEntries(Object.entries(ENDPOINT_DEFS).map(([key, def]) => [key, { key, ...def }]));
|
|
578
|
+
/** Flat catalog of every registered endpoint, for `raw list` discoverability
|
|
579
|
+
* (so `raw call` doesn't require memorizing endpoint keys). */
|
|
580
|
+
export function listEndpoints() {
|
|
581
|
+
return Object.values(ENDPOINTS).map(({ key, method, path, description }) => ({ key, method, path, description }));
|
|
582
|
+
}
|
package/dist/src/core/errors.js
CHANGED
|
@@ -34,12 +34,17 @@ export class ApiError extends CliError {
|
|
|
34
34
|
code;
|
|
35
35
|
statusCode;
|
|
36
36
|
details;
|
|
37
|
+
retryAfterMs;
|
|
37
38
|
hint;
|
|
38
|
-
constructor(message, code, statusCode, details
|
|
39
|
+
constructor(message, code, statusCode, details,
|
|
40
|
+
/** Server-specified Retry-After (ms), set on 429 responses so the transport
|
|
41
|
+
* backoff can honor it instead of the default exponential schedule. */
|
|
42
|
+
retryAfterMs) {
|
|
39
43
|
super(message);
|
|
40
44
|
this.code = code;
|
|
41
45
|
this.statusCode = statusCode;
|
|
42
46
|
this.details = details;
|
|
47
|
+
this.retryAfterMs = retryAfterMs;
|
|
43
48
|
this.hint = code ? ERROR_HINTS[code] : undefined;
|
|
44
49
|
}
|
|
45
50
|
}
|
|
@@ -24,11 +24,6 @@ export function normalizeRows(value) {
|
|
|
24
24
|
const hasMeta = Object.keys(meta).length > 0;
|
|
25
25
|
return hasMeta ? { ...meta, list } : list;
|
|
26
26
|
}
|
|
27
|
-
if (Array.isArray(record.chatRoomList)) {
|
|
28
|
-
const { chatRoomList, ...meta } = record;
|
|
29
|
-
const hasMeta = Object.keys(meta).length > 0;
|
|
30
|
-
return hasMeta ? { ...meta, list: chatRoomList } : chatRoomList;
|
|
31
|
-
}
|
|
32
27
|
if (Array.isArray(record.constants)) {
|
|
33
28
|
const { constants, ...meta } = record;
|
|
34
29
|
const hasMeta = Object.keys(meta).length > 0;
|
package/dist/src/core/output.js
CHANGED
|
@@ -14,8 +14,9 @@ export function parseOutputFormat(value) {
|
|
|
14
14
|
function sanitizeCell(value) {
|
|
15
15
|
return value.replace(/[\r\n]+/g, " ").replace(/[\u0000-\u001f\u007f]/g, "");
|
|
16
16
|
}
|
|
17
|
-
/** Terminal display width: CJK/fullwidth chars occupy 2 columns — padEnd
|
|
18
|
-
* UTF-16 code units and misaligns every table containing Chinese text
|
|
17
|
+
/** Terminal display width: CJK/fullwidth chars and emoji occupy 2 columns — padEnd
|
|
18
|
+
* counts UTF-16 code units and misaligns every table containing Chinese text or emoji
|
|
19
|
+
* (e.g. WeChat group names). */
|
|
19
20
|
function displayWidth(value) {
|
|
20
21
|
let width = 0;
|
|
21
22
|
for (const ch of value) {
|
|
@@ -23,7 +24,8 @@ function displayWidth(value) {
|
|
|
23
24
|
const wide = (cp >= 0x1100 && cp <= 0x115f) || (cp >= 0x2e80 && cp <= 0xa4cf)
|
|
24
25
|
|| (cp >= 0xac00 && cp <= 0xd7a3) || (cp >= 0xf900 && cp <= 0xfaff)
|
|
25
26
|
|| (cp >= 0xfe30 && cp <= 0xfe4f) || (cp >= 0xff00 && cp <= 0xff60)
|
|
26
|
-
|| (cp >= 0xffe0 && cp <= 0xffe6) || (cp >=
|
|
27
|
+
|| (cp >= 0xffe0 && cp <= 0xffe6) || (cp >= 0x1f000 && cp <= 0x1faff)
|
|
28
|
+
|| (cp >= 0x20000 && cp <= 0x3fffd);
|
|
27
29
|
width += wide ? 2 : 1;
|
|
28
30
|
}
|
|
29
31
|
return width;
|
package/dist/src/core/printer.js
CHANGED
|
@@ -1,6 +1,18 @@
|
|
|
1
1
|
import { normalizeRows } from "./normalize.js";
|
|
2
2
|
import { pickList, renderOutput, saveOutputIfNeeded, streamOutputToFile } from "./output.js";
|
|
3
3
|
import { extractTitles, writeTitleCache } from "./titleCache.js";
|
|
4
|
+
/** Rows above which renderOutput's single in-memory string risks high memory / the V8
|
|
5
|
+
* max-string-length cap. Well above normal result sizes, so it only fires on huge exports. */
|
|
6
|
+
const LARGE_RESULT_ROWS = 50_000;
|
|
7
|
+
/** Warn when we're about to renderOutput a huge result. Called only on the paths that
|
|
8
|
+
* actually render — never after streamOutputToFile streamed — so it can't misfire on a
|
|
9
|
+
* genuinely streamed export, and it DOES fire on the all-scalar-csv list that streaming
|
|
10
|
+
* declines (which then falls back to a full in-memory string). */
|
|
11
|
+
function warnIfLargeInMemory(items, format) {
|
|
12
|
+
if (items && items.length >= LARGE_RESULT_ROWS) {
|
|
13
|
+
process.stderr.write(`[gangtise] note: ${items.length} rows in '${format}' is built entirely in memory; stream large exports to a file with --format jsonl --output <path> (or csv).\n`);
|
|
14
|
+
}
|
|
15
|
+
}
|
|
4
16
|
export async function printData(data, format, output, cache) {
|
|
5
17
|
const normalized = normalizeRows(data);
|
|
6
18
|
const items = pickList(normalized);
|
|
@@ -28,6 +40,9 @@ export async function printData(data, format, output, cache) {
|
|
|
28
40
|
process.stdout.write(`${output}\n`);
|
|
29
41
|
return;
|
|
30
42
|
}
|
|
43
|
+
// streamOutputToFile declined (non-stream format, or an all-scalar csv list) \u2192 we
|
|
44
|
+
// fall back to renderOutput, which builds the whole result as one string.
|
|
45
|
+
warnIfLargeInMemory(items, format);
|
|
31
46
|
const content = renderOutput(normalized, format);
|
|
32
47
|
// CSV files get a BOM so Excel double-click decodes Chinese as UTF-8 (stdout
|
|
33
48
|
// stays BOM-free for pipes).
|
|
@@ -35,5 +50,6 @@ export async function printData(data, format, output, cache) {
|
|
|
35
50
|
process.stdout.write(`${output}\n`);
|
|
36
51
|
return;
|
|
37
52
|
}
|
|
53
|
+
warnIfLargeInMemory(items, format);
|
|
38
54
|
process.stdout.write(`${renderOutput(normalized, format)}\n`);
|
|
39
55
|
}
|
|
@@ -16,11 +16,21 @@ function parseDate(value) {
|
|
|
16
16
|
function formatDate(d) {
|
|
17
17
|
return d.toISOString().slice(0, 10);
|
|
18
18
|
}
|
|
19
|
-
function
|
|
19
|
+
export function isFullMarket(body, fullMarketValue) {
|
|
20
20
|
const list = body.securityList;
|
|
21
21
|
if (!Array.isArray(list) || list.length !== 1)
|
|
22
22
|
return false;
|
|
23
|
-
return list[0] ===
|
|
23
|
+
return list[0] === fullMarketValue;
|
|
24
|
+
}
|
|
25
|
+
/** Kline uses `all` as its whole-market keyword; fund-flow uses `aShares`. Thin wrapper
|
|
26
|
+
* so kline call sites read naturally. */
|
|
27
|
+
export function isAllMarket(body) {
|
|
28
|
+
return isFullMarket(body, "all");
|
|
29
|
+
}
|
|
30
|
+
/** Sat/Sun in UTC — shard dates are formatted from UTC midnight (see parseDate). */
|
|
31
|
+
function isWeekendUtc(d) {
|
|
32
|
+
const day = d.getUTCDay();
|
|
33
|
+
return day === 0 || day === 6;
|
|
24
34
|
}
|
|
25
35
|
function buildShards(start, end, shardDays) {
|
|
26
36
|
const shards = [];
|
|
@@ -28,10 +38,17 @@ function buildShards(start, end, shardDays) {
|
|
|
28
38
|
const endTime = end.getTime();
|
|
29
39
|
while (cursor <= endTime) {
|
|
30
40
|
const shardEnd = Math.min(cursor + (shardDays - 1) * DAY_MS, endTime);
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
41
|
+
// Per-day sharding (shardDays===1): a lone weekend day always returns empty (A/HK/US
|
|
42
|
+
// markets closed) — skip it to save ~28% of requests and daily quota. This covers
|
|
43
|
+
// every 1-day-sharded full-market query: fund-flow AND day-kline / day-kline-us
|
|
44
|
+
// (both shardDays 1 in cli.ts). Multi-day shards (day-kline-hk=2, index=30) always
|
|
45
|
+
// straddle weekdays, so they're never dropped.
|
|
46
|
+
if (!(shardDays === 1 && isWeekendUtc(new Date(cursor)))) {
|
|
47
|
+
shards.push({
|
|
48
|
+
startDate: formatDate(new Date(cursor)),
|
|
49
|
+
endDate: formatDate(new Date(shardEnd)),
|
|
50
|
+
});
|
|
51
|
+
}
|
|
35
52
|
cursor = shardEnd + DAY_MS;
|
|
36
53
|
}
|
|
37
54
|
return shards;
|
|
@@ -43,7 +60,8 @@ function buildShards(start, end, shardDays) {
|
|
|
43
60
|
* single-security queries this is a no-op.
|
|
44
61
|
*/
|
|
45
62
|
export async function callKlineWithSharding(client, endpointKey, body, config) {
|
|
46
|
-
|
|
63
|
+
const fullMarketValue = config.fullMarketValue ?? "all";
|
|
64
|
+
if (!isFullMarket(body, fullMarketValue)) {
|
|
47
65
|
return client.call(endpointKey, body);
|
|
48
66
|
}
|
|
49
67
|
// `--security all` returns thousands of rows per day; lift the default 6000-row
|
|
@@ -51,19 +69,47 @@ export async function callKlineWithSharding(client, endpointKey, body, config) {
|
|
|
51
69
|
// must apply even when a date is missing (no sharding possible then, but the
|
|
52
70
|
// single request still needs the lifted cap).
|
|
53
71
|
const allMarketBody = { ...body, limit: body.limit ?? ALL_MARKET_LIMIT };
|
|
72
|
+
const perShardLimit = allMarketBody.limit ?? ALL_MARKET_LIMIT;
|
|
73
|
+
// A shard maxes out for two reasons with different fixes: a user-set low --limit is
|
|
74
|
+
// raisable; hitting the API cap itself is not (only a smaller internal shardDays window
|
|
75
|
+
// would help). Word the truncation warning accordingly.
|
|
76
|
+
const truncationHint = perShardLimit < ALL_MARKET_LIMIT
|
|
77
|
+
? "raise or omit --limit to fetch the full market"
|
|
78
|
+
: `a single ${config.shardDays}-day window exceeds the ${ALL_MARKET_LIMIT}-row API cap`;
|
|
79
|
+
// A full-market response whose row count reaches the per-request limit was itself
|
|
80
|
+
// capped (a low user --limit, or a single day exceeding the API row cap) — its slice
|
|
81
|
+
// is incomplete, so the result must be flagged partial rather than shown as complete.
|
|
82
|
+
const isTruncated = (rec) => Array.isArray(rec.list) && rec.list.length >= perShardLimit;
|
|
83
|
+
// A single full-market request (missing/unparseable dates, or a range that fits one
|
|
84
|
+
// shard) skips the merge loop below, so it needs the same truncation check inline — or
|
|
85
|
+
// a low --limit / oversized day slips through as a silent exit-0 success.
|
|
86
|
+
const callSingle = async () => {
|
|
87
|
+
const single = await client.call(endpointKey, allMarketBody);
|
|
88
|
+
if (single && typeof single === "object" && !Array.isArray(single) && isTruncated(single)) {
|
|
89
|
+
;
|
|
90
|
+
single.partial = true;
|
|
91
|
+
process.stderr.write(`[gangtise] warning: full-market request hit the ${perShardLimit}-row limit and was likely truncated; results are partial — ${truncationHint}.\n`);
|
|
92
|
+
}
|
|
93
|
+
return single;
|
|
94
|
+
};
|
|
54
95
|
if (!body.startDate || !body.endDate) {
|
|
55
|
-
return
|
|
96
|
+
return callSingle();
|
|
56
97
|
}
|
|
57
98
|
const start = parseDate(body.startDate);
|
|
58
99
|
const end = parseDate(body.endDate);
|
|
59
100
|
if (!start || !end || end < start) {
|
|
60
|
-
return
|
|
101
|
+
return callSingle();
|
|
61
102
|
}
|
|
62
103
|
const totalDays = Math.floor((end.getTime() - start.getTime()) / DAY_MS) + 1;
|
|
63
104
|
if (totalDays <= config.shardDays) {
|
|
64
|
-
return
|
|
105
|
+
return callSingle();
|
|
65
106
|
}
|
|
66
107
|
const shards = buildShards(start, end, config.shardDays);
|
|
108
|
+
// A range that's entirely weekends (per-day sharding) filters down to nothing — no
|
|
109
|
+
// trading days to fetch. Return empty rather than falling into the all-shards-failed
|
|
110
|
+
// check below, which would misread 0 shards as a total outage and throw.
|
|
111
|
+
if (shards.length === 0)
|
|
112
|
+
return { list: [] };
|
|
67
113
|
// isVerbose() (not a direct env read) so the global --verbose flag reaches
|
|
68
114
|
// shard logging too — cli.ts enables it via setVerbose in a preAction hook.
|
|
69
115
|
if (isVerbose()) {
|
|
@@ -89,6 +135,7 @@ export async function callKlineWithSharding(client, endpointKey, body, config) {
|
|
|
89
135
|
let fieldList;
|
|
90
136
|
let header = null;
|
|
91
137
|
const merged = [];
|
|
138
|
+
let truncatedShards = 0;
|
|
92
139
|
for (const r of results) {
|
|
93
140
|
if (!(r && typeof r === "object"))
|
|
94
141
|
continue;
|
|
@@ -97,6 +144,8 @@ export async function callKlineWithSharding(client, endpointKey, body, config) {
|
|
|
97
144
|
header = rec;
|
|
98
145
|
if (!fieldList && Array.isArray(rec.fieldList))
|
|
99
146
|
fieldList = rec.fieldList;
|
|
147
|
+
if (isTruncated(rec))
|
|
148
|
+
truncatedShards++;
|
|
100
149
|
// Append one-by-one rather than push(...list): a future higher row cap could
|
|
101
150
|
// make a single shard's list large enough to overflow the stack via spread.
|
|
102
151
|
if (Array.isArray(rec.list))
|
|
@@ -110,7 +159,9 @@ export async function callKlineWithSharding(client, endpointKey, body, config) {
|
|
|
110
159
|
}
|
|
111
160
|
if (!header)
|
|
112
161
|
return { list: [] };
|
|
113
|
-
|
|
162
|
+
// `total` on a shard is that shard's own row count; overwrite it with the merged count
|
|
163
|
+
// so the JSON `total` and the `Total:` stderr line reflect the whole combined result.
|
|
164
|
+
const out = { ...header, total: merged.length, list: merged };
|
|
114
165
|
if (fieldList)
|
|
115
166
|
out.fieldList = fieldList;
|
|
116
167
|
if (failedShards.length > 0) {
|
|
@@ -118,5 +169,9 @@ export async function callKlineWithSharding(client, endpointKey, body, config) {
|
|
|
118
169
|
out.failedShards = failedShards;
|
|
119
170
|
process.stderr.write(`[gangtise] warning: ${failedShards.length}/${shards.length} shards failed; results are partial (see failedShards)\n`);
|
|
120
171
|
}
|
|
172
|
+
if (truncatedShards > 0) {
|
|
173
|
+
out.partial = true;
|
|
174
|
+
process.stderr.write(`[gangtise] warning: ${truncatedShards}/${shards.length} shard(s) hit the ${perShardLimit}-row limit and were likely truncated; results are partial — ${truncationHint}.\n`);
|
|
175
|
+
}
|
|
121
176
|
return out;
|
|
122
177
|
}
|
|
@@ -1,5 +1,38 @@
|
|
|
1
|
+
import { gunzipSync } from "node:zlib";
|
|
1
2
|
import { Agent } from "undici";
|
|
2
3
|
import { ApiError } from "./errors.js";
|
|
4
|
+
/** Decode an HTTP response body honoring Content-Encoding. undici does not
|
|
5
|
+
* auto-decompress, so requestJson advertises `accept-encoding: gzip` and gunzips
|
|
6
|
+
* here; an unencoded body is read as-is. */
|
|
7
|
+
export function decodeResponseBody(buf, contentEncoding) {
|
|
8
|
+
const enc = (Array.isArray(contentEncoding) ? contentEncoding[0] : contentEncoding)?.toLowerCase().trim();
|
|
9
|
+
const b = Buffer.from(buf.buffer, buf.byteOffset, buf.byteLength);
|
|
10
|
+
return enc === "gzip" ? gunzipSync(b).toString("utf8") : b.toString("utf8");
|
|
11
|
+
}
|
|
12
|
+
/** A Retry-After delay we'll honor even if it exceeds maxDelay — but never past this
|
|
13
|
+
* ceiling, so a hostile/misconfigured header can't hang the CLI for minutes. */
|
|
14
|
+
const RETRY_AFTER_CEILING_MS = 60_000;
|
|
15
|
+
/** Parse a Retry-After header (delta-seconds or an HTTP-date) into a delay in ms.
|
|
16
|
+
* Returns undefined when absent or unparseable. `nowMs` is injected for testing. */
|
|
17
|
+
export function parseRetryAfterMs(value, nowMs) {
|
|
18
|
+
const raw = (Array.isArray(value) ? value[0] : value)?.trim();
|
|
19
|
+
if (!raw)
|
|
20
|
+
return undefined;
|
|
21
|
+
if (/^\d+$/.test(raw))
|
|
22
|
+
return Number(raw) * 1000;
|
|
23
|
+
const dateMs = Date.parse(raw);
|
|
24
|
+
return Number.isNaN(dateMs) ? undefined : Math.max(0, dateMs - nowMs);
|
|
25
|
+
}
|
|
26
|
+
/** A retryable error may carry a server-specified Retry-After (attached by the
|
|
27
|
+
* client on a 429); prefer it over the computed backoff. */
|
|
28
|
+
function retryAfterFromError(error) {
|
|
29
|
+
if (error && typeof error === "object") {
|
|
30
|
+
const v = error.retryAfterMs;
|
|
31
|
+
if (typeof v === "number" && Number.isFinite(v) && v >= 0)
|
|
32
|
+
return v;
|
|
33
|
+
}
|
|
34
|
+
return undefined;
|
|
35
|
+
}
|
|
3
36
|
let cachedDispatcher = null;
|
|
4
37
|
export function getDispatcher() {
|
|
5
38
|
if (!cachedDispatcher) {
|
|
@@ -70,8 +103,12 @@ export async function withRetry(fn, options = {}) {
|
|
|
70
103
|
catch (error) {
|
|
71
104
|
if (attempt >= retries || !isRetryableError(error))
|
|
72
105
|
throw error;
|
|
73
|
-
|
|
74
|
-
|
|
106
|
+
// A server-sent Retry-After (429) wins over exponential backoff, but is capped
|
|
107
|
+
// so it can't stall the CLI; otherwise fall back to jittered exponential backoff.
|
|
108
|
+
const retryAfter = retryAfterFromError(error);
|
|
109
|
+
const delay = retryAfter !== undefined
|
|
110
|
+
? Math.min(retryAfter, RETRY_AFTER_CEILING_MS)
|
|
111
|
+
: Math.min(maxDelay, baseDelay * 2 ** attempt + Math.random() * baseDelay);
|
|
75
112
|
options.onRetry?.(attempt + 1, error, delay);
|
|
76
113
|
await new Promise(resolve => setTimeout(resolve, delay));
|
|
77
114
|
attempt++;
|
package/dist/src/version.js
CHANGED
|
@@ -1,2 +1,2 @@
|
|
|
1
1
|
// Auto-generated — DO NOT EDIT
|
|
2
|
-
export const CLI_VERSION = "0.
|
|
2
|
+
export const CLI_VERSION = "0.24.0";
|