@armadra/agent 0.7.4 → 0.7.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +11 -0
- package/CHANGELOG.zh-CN.md +11 -0
- package/dist/ai/apis/anthropic-request.js +6 -1
- package/dist/ai/apis/cache-params.js +2 -1
- package/dist/ai/apis/google-request.js +2 -1
- package/dist/ai/apis/openai-request.js +3 -1
- package/dist/ai/apis/openai-responses-request.js +2 -1
- package/dist/ai/json-body.d.ts +33 -1
- package/dist/ai/json-body.js +96 -18
- package/dist/bundle/ama.cjs +699 -352
- package/dist/cli/bootstrap.js +13 -5
- package/dist/cli/deps.d.ts +1 -1
- package/dist/modes/acp/acp-server.js +17 -15
- package/dist/modes/acp/acp-sessions.d.ts +14 -1
- package/dist/modes/acp/acp-sessions.js +44 -1
- package/dist/session/manager.d.ts +16 -0
- package/dist/session/manager.js +70 -14
- package/dist/session/offload.d.ts +51 -0
- package/dist/session/offload.js +195 -0
- package/dist/session/store.d.ts +12 -4
- package/dist/session/store.js +18 -8
- package/dist/tools/read-lines.d.ts +39 -0
- package/dist/tools/read-lines.js +163 -86
- package/dist/tools/read.js +3 -3
- package/docs/acp.md +6 -3
- package/docs/en/acp.md +2 -2
- package/docs/session-format.md +10 -0
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,17 @@ English · [简体中文](CHANGELOG.zh-CN.md)
|
|
|
5
5
|
> This file is in English starting with 0.6.0. Release notes for 0.1 through 0.5.1 are in Chinese in
|
|
6
6
|
> [CHANGELOG.zh-CN.md](CHANGELOG.zh-CN.md). New entries go into both files.
|
|
7
7
|
|
|
8
|
+
## 0.7.5 (2026-10-10)
|
|
9
|
+
|
|
10
|
+
### Memory footprint
|
|
11
|
+
|
|
12
|
+
- **`read` no longer blocks the event loop on large files**: the byte-window path for text files over 1 MiB now reads 64 KiB chunks asynchronously, awaiting once per chunk, so TUI/RPC/ACP stay responsive while a 256 MB file is scanned (longest event-loop stall about 170–320 ms → under 20 ms). Output is byte-for-byte identical (the sync and async readers share one line parser); peak memory is unchanged (still one chunk buffer).
|
|
13
|
+
- **Request bodies with images no longer copy each image**: image data goes from the transcript straight into the request body instead of first being joined into a new data-URL string on every request, its escape check runs once per image, and the body is encoded in chunks of at most 256 Ki characters instead of one whole image at a time. Mock 300-step session with 15 images: peak RSS about 630–690 → 390–400 MB, peak heapUsed about 270 → 85 MB. Request bytes are unchanged; SDK `onPayload` callbacks now see large images as objects with `toJSON` (`JSON.stringify` gives the same text).
|
|
14
|
+
- **Old images released after downgrade**: once `context_edit` (for example `image_budget`) replaces a message on the active branch, its image base64 is no longer kept in memory — only the line's position in the session file — and is read back on demand: `get_entries` and fork return the original, and moving the leaf back before the edit (`/tree`, rewind) restores the images. Resuming a session also strips those images while reading, so they no longer count toward the peak (55 MB session with 10 downgraded 3 MB images: peak RSS about 254 → 210 MB). The JSONL format is unchanged; in `entries()` / `branch()` / SDK `session.entries` such image blocks now have `data: ""`.
|
|
15
|
+
- **Closed ACP sessions free the startup session too**: `Runtime.session` and `Runtime.sessionManager` now follow the current session instead of staying on the one created at startup, so after `session/close` its transcript and images are garbage-collected like any other session's (8 sessions × 4 rounds, all closed: external 16.3 → 7.0 MB). SDK code that reads `runtime.session` after a session switch now gets the new session.
|
|
16
|
+
- **ACP `session/prompt` usage covers only its own turn**: a prompt that waited for a background subagent's notification turn no longer counts that turn's tokens in `usage`.
|
|
17
|
+
- **`session/close` during a notification turn**: it used to fail with -32603 and keep the session in memory; now running background subagents are stopped, the running turn (notification turns included) is interrupted, already queued notifications no longer start a turn, and the session is released with no further `session/update`. Closing stdin quiets sessions the same way.
|
|
18
|
+
|
|
8
19
|
## 0.7.4 (2026-10-10)
|
|
9
20
|
|
|
10
21
|
- **Parallel worktree isolation**: two isolated tasks starting together could fail with `could not read .git/worktrees/<id>/commondir`, because `git worktree add` read the other task's half-created admin directory. Worktree add / remove and branch deletion now run one at a time per repository, and the same transient errors from another ama process are retried briefly.
|
package/CHANGELOG.zh-CN.md
CHANGED
|
@@ -4,6 +4,17 @@
|
|
|
4
4
|
|
|
5
5
|
> 从 0.6.0 起 [CHANGELOG.md](CHANGELOG.md) 为英文,本文件保留中文记录(0.1–0.5.1 的完整历史在此)。新条目两份都要加。
|
|
6
6
|
|
|
7
|
+
## 0.7.5(2026-10-10)
|
|
8
|
+
|
|
9
|
+
### 内存占用
|
|
10
|
+
|
|
11
|
+
- **`read` 读大文件不再阻塞事件循环**:超过 1 MiB 的文本走的字节窗口改为异步按 64 KiB 分块读,每块一次 `await`,扫描 256 MB 文件期间 TUI / RPC / ACP 仍能响应(事件循环最长停顿约 170–320 ms → 20 ms 以内)。输出逐字节不变(同步与异步共用同一个行解析器),峰值内存不变(仍只有一块缓冲)。
|
|
12
|
+
- **带图请求体不再复制每张图**:图片数据从转录直接写入请求体,不再每个请求先拼出一份新的 data URL 字符串;转义检查每张图只做一次;请求体按每块最多 256 Ki 码元编码,不再一次编码一整张图。mock 300 步带 15 张图:峰值 RSS 约 630–690 → 390–400 MB,heapUsed 峰值约 270 → 85 MB。请求字节不变;SDK 的 `onPayload` 回调里大图变为带 `toJSON` 的对象(`JSON.stringify` 结果相同)。
|
|
13
|
+
- **降级后释放旧图**:活动分支上的消息被 `context_edit`(如 `image_budget`)改写后,图片 base64 不再常驻内存,只记该行在会话文件中的位置,需要时按位置读回:`get_entries` 与 fork 返回原文,换叶子回到编辑之前(`/tree`、rewind)时恢复图片。续会话读文件时随读随剥,这些旧图不再计入峰值(55 MB、含 10 张被降级的 3 MB 图的会话:峰值 RSS 约 254 → 210 MB)。JSONL 格式不变;`entries()` / `branch()` / SDK `session.entries` 里这类图片块的 `data` 为空串。
|
|
14
|
+
- **ACP 关掉的启动会话也能回收**:`Runtime.session` 与 `Runtime.sessionManager` 改为跟随当前会话(原先固定指向启动时建的会话),`session/close` 之后它的转录与图片和其它会话一样被回收(8 会话 × 4 轮全部关闭后 external 16.3 → 7.0 MB)。SDK 代码在切换会话后读 `runtime.session` 得到的是新会话。
|
|
15
|
+
- **ACP `session/prompt` 的 usage 只算本回合**:等过后台子 Agent 通知回合的提示,`usage` 不再计入那一回合的 token。
|
|
16
|
+
- **通知回合在跑时 `session/close`**:原先以 -32603 失败且会话留在内存;现在先停在跑的后台子 Agent、中断在跑的回合(含通知回合)、已排着的通知不再开回合,然后释放会话,之后不再有 `session/update`。stdin 关闭时同样这样收尾。
|
|
17
|
+
|
|
7
18
|
## 0.7.4(2026-10-10)
|
|
8
19
|
|
|
9
20
|
- **并行 worktree 隔离**:两个隔离任务同时开始时,可能报 `无法读取 .git/worktrees/<id>/commondir`——`git worktree add` 读到了另一个任务建了一半的管理目录。现在同一仓库的 worktree 增删与删分支按顺序执行,另一个 ama 进程造成的同类瞬时错误短暂重试。
|
|
@@ -17,6 +17,7 @@
|
|
|
17
17
|
* 除非 `supportsTemperatureWithThinking`。`toolChoice: "none"`(有工具时)→ `tool_choice:{type:"none"}`。
|
|
18
18
|
*/
|
|
19
19
|
import { contentText, normalizeContext, sanitizeText } from "../context.js";
|
|
20
|
+
import { largeString } from "../json-body.js";
|
|
20
21
|
import { budgetedMaxTokens, clampThinkingLevel, mappedThinkingValue, thinkingBudget, } from "../thinking.js";
|
|
21
22
|
import { detectAnthropicCompat } from "./anthropic-compat.js";
|
|
22
23
|
import { effectiveRetention, resolveCacheRetention, resolvePromptCacheCompat, } from "./cache-params.js";
|
|
@@ -76,7 +77,11 @@ function convertBlocks(content) {
|
|
|
76
77
|
else {
|
|
77
78
|
blocks.push({
|
|
78
79
|
type: "image",
|
|
79
|
-
source: {
|
|
80
|
+
source: {
|
|
81
|
+
type: "base64",
|
|
82
|
+
media_type: block.mimeType,
|
|
83
|
+
data: largeString("", block, block.data),
|
|
84
|
+
},
|
|
80
85
|
});
|
|
81
86
|
}
|
|
82
87
|
}
|
|
@@ -14,6 +14,7 @@
|
|
|
14
14
|
* 并提示一次(实测:某中转的 Responses 对 DeepSeek 拒收 prompt_cache_retention / _options)。
|
|
15
15
|
*/
|
|
16
16
|
import { HttpError, postJson } from "../http.js";
|
|
17
|
+
import { LargeString } from "../json-body.js";
|
|
17
18
|
import { msg } from "../../i18n/index.js";
|
|
18
19
|
const DEFAULT_BASE_URLS = {
|
|
19
20
|
"anthropic-messages": "https://api.anthropic.com",
|
|
@@ -133,7 +134,7 @@ export function isCacheParamRejection(error) {
|
|
|
133
134
|
function stripInto(value, removed) {
|
|
134
135
|
if (Array.isArray(value))
|
|
135
136
|
return value.map((item) => stripInto(item, removed));
|
|
136
|
-
if (typeof value !== "object" || value === null)
|
|
137
|
+
if (typeof value !== "object" || value === null || value instanceof LargeString)
|
|
137
138
|
return value;
|
|
138
139
|
const out = {};
|
|
139
140
|
for (const [key, inner] of Object.entries(value)) {
|
|
@@ -13,6 +13,7 @@
|
|
|
13
13
|
* - `toolChoice: "none"`(有工具时)→ `toolConfig.functionCallingConfig.mode: "NONE"`。
|
|
14
14
|
*/
|
|
15
15
|
import { contentText, normalizeContext, sanitizeText } from "../context.js";
|
|
16
|
+
import { largeString } from "../json-body.js";
|
|
16
17
|
import { DEFAULT_THINKING_BUDGETS, clampBudgetToAnswerRoom, clampThinkingLevel, discreteThinkingLevel, } from "../thinking.js";
|
|
17
18
|
export const GOOGLE_BASE_URL = "https://generativelanguage.googleapis.com/v1beta";
|
|
18
19
|
const GOOGLE_COMPAT_KEYS = [
|
|
@@ -63,7 +64,7 @@ function validSignature(signature) {
|
|
|
63
64
|
BASE64.test(signature));
|
|
64
65
|
}
|
|
65
66
|
function inlineData(block) {
|
|
66
|
-
return { inlineData: { mimeType: block.mimeType, data: block.data } };
|
|
67
|
+
return { inlineData: { mimeType: block.mimeType, data: largeString("", block, block.data) } };
|
|
67
68
|
}
|
|
68
69
|
function convertUser(message) {
|
|
69
70
|
if (typeof message.content === "string") {
|
|
@@ -19,12 +19,14 @@ import { contentText, normalizeContext, normalizeContextInline, sanitizeText, }
|
|
|
19
19
|
import { DEFAULT_THINKING_BUDGETS, clampBudgetToAnswerRoom, clampThinkingLevel, mappedThinkingValue, } from "../thinking.js";
|
|
20
20
|
import { cacheControlFor } from "./anthropic-request.js";
|
|
21
21
|
import { effectiveRetention, resolveCacheRetention, resolvePromptCacheCompat, } from "./cache-params.js";
|
|
22
|
+
import { largeString } from "../json-body.js";
|
|
22
23
|
import { detectCompat } from "./openai-compat.js";
|
|
23
24
|
/** 思考文本回放用的字段名;thinkingSignature 记录来源字段,回放时原样放回。 */
|
|
24
25
|
export const REASONING_FIELDS = ["reasoning_content", "reasoning", "reasoning_text"];
|
|
25
26
|
export const ASSISTANT_BRIDGE_TEXT = "Tool results received.";
|
|
26
27
|
function imagePart(block) {
|
|
27
|
-
|
|
28
|
+
const url = largeString(`data:${block.mimeType};base64,`, block, block.data);
|
|
29
|
+
return { type: "image_url", image_url: { url } };
|
|
28
30
|
}
|
|
29
31
|
function convertUser(message) {
|
|
30
32
|
if (typeof message.content === "string") {
|
|
@@ -21,6 +21,7 @@
|
|
|
21
21
|
* - [W6-O] compat `chatgptBackend`(ChatGPT 订阅后端):最后经 chatgpt-backend.ts 按后端白名单改写。
|
|
22
22
|
*/
|
|
23
23
|
import { contentText, normalizeContext, sanitizeText } from "../context.js";
|
|
24
|
+
import { largeString } from "../json-body.js";
|
|
24
25
|
import { clampThinkingLevel, mappedThinkingValue } from "../thinking.js";
|
|
25
26
|
import { resolveCacheRetention, resolvePromptCacheCompat } from "./cache-params.js";
|
|
26
27
|
import { transformChatGptBody } from "./chatgpt-backend.js";
|
|
@@ -110,7 +111,7 @@ function imagePart(block) {
|
|
|
110
111
|
return {
|
|
111
112
|
type: "input_image",
|
|
112
113
|
detail: "auto",
|
|
113
|
-
image_url: `data:${block.mimeType};base64
|
|
114
|
+
image_url: largeString(`data:${block.mimeType};base64,`, block, block.data),
|
|
114
115
|
};
|
|
115
116
|
}
|
|
116
117
|
function convertUser(message) {
|
package/dist/ai/json-body.d.ts
CHANGED
|
@@ -11,9 +11,36 @@
|
|
|
11
11
|
* 键、`[`)与已验证无需转义的大字符串(需要转义的大字符串整个交给原生);`toJSON`、`undefined` /
|
|
12
12
|
* 函数 / symbol 的跳过与 `null`、数字格式、转义全部由原生处理(带 `toJSON` 的值经单键容器序列化,
|
|
13
13
|
* 保证 `toJSON(key)` 收到与原生相同的键)。
|
|
14
|
+
*
|
|
15
|
+
* 图片由转换器包成 {@link LargeString}(`prefix + text`):片段路径直接写转录里的原字符串,不经模板串
|
|
16
|
+
* 拼接后的扁平化;无需转义的结论与字节数按 ImageBlock 缓存,同一张图每进程只扫一次。流式发送时大片段
|
|
17
|
+
* 按 {@link STREAM_CHUNK_CHARS} 切块编码。
|
|
14
18
|
*/
|
|
15
19
|
/** 达到这个长度(UTF-16 码元)的字符串才走片段路径。 */
|
|
16
20
|
export declare const LARGE_STRING_BYTES: number;
|
|
21
|
+
/** `isPlain` 的调用次数(测试用:同一块图片第二次序列化不应再扫)。 */
|
|
22
|
+
export declare const plainScans: {
|
|
23
|
+
count: number;
|
|
24
|
+
};
|
|
25
|
+
/**
|
|
26
|
+
* 请求体里的大字符串叶子 `prefix + text`(data URL 或裸 base64),用来代替模板串拼接:拼接得到的
|
|
27
|
+
* cons string 在每次序列化时都会被扁平化成一份新的整图字符串。片段路径直接写 `prefix` 与 `text`
|
|
28
|
+
* (`text` 是转录里的原字符串,不拷贝);`text` 无需转义的结论按 `key`(转录里身份稳定的 ImageBlock)
|
|
29
|
+
* 缓存,同一块每进程只扫一次。约定:同一 `key` 的 `text` 不变。其它遍历者看到的是一个带 `toJSON`
|
|
30
|
+
* 的对象,原生 `JSON.stringify` 得到相同字符串。由 {@link largeString} 构造。
|
|
31
|
+
*/
|
|
32
|
+
export declare class LargeString {
|
|
33
|
+
readonly prefix: string;
|
|
34
|
+
readonly key: object;
|
|
35
|
+
readonly text: string;
|
|
36
|
+
constructor(prefix: string, key: object, text: string);
|
|
37
|
+
toJSON(): string;
|
|
38
|
+
}
|
|
39
|
+
/**
|
|
40
|
+
* `prefix + text` 作为请求体的值:`text` 够大且 `prefix` 自身无需转义时给 {@link LargeString},
|
|
41
|
+
* 否则就是拼好的字符串(小图与以前完全相同)。
|
|
42
|
+
*/
|
|
43
|
+
export declare function largeString(prefix: string, key: object, text: string): string | LargeString;
|
|
17
44
|
/** 与 `Buffer.from(JSON.stringify(body), "utf8")` 逐字节相同;大字符串不经中间字符串。 */
|
|
18
45
|
export declare function serializeJsonBody(body: unknown): Buffer;
|
|
19
46
|
/** `postJson` 交给 fetch 的请求体。 */
|
|
@@ -22,10 +49,15 @@ export interface JsonFetchBody {
|
|
|
22
49
|
/** 流式请求体的字节数:调用方须作为 `content-length` 头发出(不走 chunked)。 */
|
|
23
50
|
contentLength?: number;
|
|
24
51
|
}
|
|
52
|
+
/**
|
|
53
|
+
* 流式请求体每块最多编码的码元数:大片段(一整张图)按游标切片(V8 切片串不拷贝)逐块编码,
|
|
54
|
+
* 发送期间只多出一块的 UTF-8 字节,而不是一整张图。
|
|
55
|
+
*/
|
|
56
|
+
export declare const STREAM_CHUNK_CHARS: number;
|
|
25
57
|
/**
|
|
26
58
|
* 给 fetch 的请求体。不含大字符串时就是 `JSON.stringify(body)`(与以前完全相同);含大字符串时是
|
|
27
59
|
* 逐片段编码的流,并给出总字节数。不交给 fetch 一个 Buffer:Node 的 fetch 会把 BufferSource
|
|
28
60
|
* 请求体再复制两份(实测 mock 300 步带图峰值反而从约 1.0 GB 升到 1.85 GB),流则按片段边读边发,
|
|
29
|
-
*
|
|
61
|
+
* 同一时刻只多出一块(≤ {@link STREAM_CHUNK_CHARS} 码元)的 UTF-8 字节。字节序列与 `serializeJsonBody` 相同。
|
|
30
62
|
*/
|
|
31
63
|
export declare function jsonFetchBody(body: unknown): JsonFetchBody;
|
package/dist/ai/json-body.js
CHANGED
|
@@ -11,6 +11,10 @@
|
|
|
11
11
|
* 键、`[`)与已验证无需转义的大字符串(需要转义的大字符串整个交给原生);`toJSON`、`undefined` /
|
|
12
12
|
* 函数 / symbol 的跳过与 `null`、数字格式、转义全部由原生处理(带 `toJSON` 的值经单键容器序列化,
|
|
13
13
|
* 保证 `toJSON(key)` 收到与原生相同的键)。
|
|
14
|
+
*
|
|
15
|
+
* 图片由转换器包成 {@link LargeString}(`prefix + text`):片段路径直接写转录里的原字符串,不经模板串
|
|
16
|
+
* 拼接后的扁平化;无需转义的结论与字节数按 ImageBlock 缓存,同一张图每进程只扫一次。流式发送时大片段
|
|
17
|
+
* 按 {@link STREAM_CHUNK_CHARS} 切块编码。
|
|
14
18
|
*/
|
|
15
19
|
/** 达到这个长度(UTF-16 码元)的字符串才走片段路径。 */
|
|
16
20
|
export const LARGE_STRING_BYTES = 64 * 1024;
|
|
@@ -23,6 +27,7 @@ const CHECK_CHUNK = 16 * 1024;
|
|
|
23
27
|
* 每个大字符串每次序列化只查这一次。
|
|
24
28
|
*/
|
|
25
29
|
function isPlain(value) {
|
|
30
|
+
plainScans.count++;
|
|
26
31
|
for (let i = 0; i < value.length; i += CHECK_CHUNK) {
|
|
27
32
|
const chunk = value.slice(i, i + CHECK_CHUNK);
|
|
28
33
|
if (JSON.stringify(chunk).length !== chunk.length + 2)
|
|
@@ -30,6 +35,54 @@ function isPlain(value) {
|
|
|
30
35
|
}
|
|
31
36
|
return true;
|
|
32
37
|
}
|
|
38
|
+
/** 短字符串无需转义(`LargeString.prefix` 用)。 */
|
|
39
|
+
const isShortPlain = (value) => JSON.stringify(value).length === value.length + 2;
|
|
40
|
+
/** `isPlain` 的调用次数(测试用:同一块图片第二次序列化不应再扫)。 */
|
|
41
|
+
export const plainScans = { count: 0 };
|
|
42
|
+
/**
|
|
43
|
+
* 请求体里的大字符串叶子 `prefix + text`(data URL 或裸 base64),用来代替模板串拼接:拼接得到的
|
|
44
|
+
* cons string 在每次序列化时都会被扁平化成一份新的整图字符串。片段路径直接写 `prefix` 与 `text`
|
|
45
|
+
* (`text` 是转录里的原字符串,不拷贝);`text` 无需转义的结论按 `key`(转录里身份稳定的 ImageBlock)
|
|
46
|
+
* 缓存,同一块每进程只扫一次。约定:同一 `key` 的 `text` 不变。其它遍历者看到的是一个带 `toJSON`
|
|
47
|
+
* 的对象,原生 `JSON.stringify` 得到相同字符串。由 {@link largeString} 构造。
|
|
48
|
+
*/
|
|
49
|
+
export class LargeString {
|
|
50
|
+
prefix;
|
|
51
|
+
key;
|
|
52
|
+
text;
|
|
53
|
+
constructor(prefix, key, text) {
|
|
54
|
+
this.prefix = prefix;
|
|
55
|
+
this.key = key;
|
|
56
|
+
this.text = text;
|
|
57
|
+
}
|
|
58
|
+
toJSON() {
|
|
59
|
+
return this.prefix + this.text;
|
|
60
|
+
}
|
|
61
|
+
}
|
|
62
|
+
/**
|
|
63
|
+
* `prefix + text` 作为请求体的值:`text` 够大且 `prefix` 自身无需转义时给 {@link LargeString},
|
|
64
|
+
* 否则就是拼好的字符串(小图与以前完全相同)。
|
|
65
|
+
*/
|
|
66
|
+
export function largeString(prefix, key, text) {
|
|
67
|
+
if (text.length < LARGE_STRING_BYTES || !isShortPlain(prefix))
|
|
68
|
+
return prefix + text;
|
|
69
|
+
return new LargeString(prefix, key, text);
|
|
70
|
+
}
|
|
71
|
+
/** 已确认无需转义的 `LargeString.text`:键是 `key`,值是当时的 UTF-8 字节数(同时省掉再数一遍)。 */
|
|
72
|
+
const plainBytes = new WeakMap();
|
|
73
|
+
/** `LargeString.text` 可原样写入时返回其 UTF-8 字节数,否则 undefined。 */
|
|
74
|
+
function plainLargeBytes(value) {
|
|
75
|
+
if (!isShortPlain(value.prefix))
|
|
76
|
+
return undefined;
|
|
77
|
+
const known = plainBytes.get(value.key);
|
|
78
|
+
if (known !== undefined && known.length === value.text.length)
|
|
79
|
+
return known.bytes;
|
|
80
|
+
if (!isPlain(value.text))
|
|
81
|
+
return undefined;
|
|
82
|
+
const bytes = Buffer.byteLength(value.text, "utf8");
|
|
83
|
+
plainBytes.set(value.key, { length: value.text.length, bytes });
|
|
84
|
+
return bytes;
|
|
85
|
+
}
|
|
33
86
|
/** 结构遍历的深度上限:更深(或有环)时整棵子树交给原生,环由原生照常抛 TypeError。 */
|
|
34
87
|
const MAX_DEPTH = 64;
|
|
35
88
|
const isLargeString = (value) => typeof value === "string" && value.length >= LARGE_STRING_BYTES;
|
|
@@ -48,6 +101,8 @@ function isContainer(value) {
|
|
|
48
101
|
function collectLarge(value, marked, depth) {
|
|
49
102
|
if (isLargeString(value))
|
|
50
103
|
return true;
|
|
104
|
+
if (value instanceof LargeString)
|
|
105
|
+
return value.text.length >= LARGE_STRING_BYTES;
|
|
51
106
|
if (depth >= MAX_DEPTH || !isContainer(value))
|
|
52
107
|
return false;
|
|
53
108
|
let found = false;
|
|
@@ -81,18 +136,24 @@ function nativeValue(key, value) {
|
|
|
81
136
|
/** 片段:已拼好的 JSON 文本与大字符串交替;连续的小文本先在字符串里累积。 */
|
|
82
137
|
class Pieces {
|
|
83
138
|
parts = [];
|
|
139
|
+
/** 已推入 `parts` 的 UTF-8 字节数。 */
|
|
140
|
+
bytes = 0;
|
|
84
141
|
pending = "";
|
|
85
142
|
text(chunk) {
|
|
86
143
|
this.pending += chunk;
|
|
87
144
|
}
|
|
88
|
-
/**
|
|
89
|
-
large(value) {
|
|
90
|
-
|
|
145
|
+
/** 大字符串原样写入(引号与 `head` 进前一个小片段);`bytes` 已知时不再数一遍。 */
|
|
146
|
+
large(value, head = "", bytes = Buffer.byteLength(value, "utf8")) {
|
|
147
|
+
const before = `${this.pending}"${head}`;
|
|
148
|
+
this.parts.push(before, value);
|
|
149
|
+
this.bytes += Buffer.byteLength(before, "utf8") + bytes;
|
|
91
150
|
this.pending = '"';
|
|
92
151
|
}
|
|
93
152
|
finish() {
|
|
94
|
-
if (this.pending.length > 0)
|
|
153
|
+
if (this.pending.length > 0) {
|
|
95
154
|
this.parts.push(this.pending);
|
|
155
|
+
this.bytes += Buffer.byteLength(this.pending, "utf8");
|
|
156
|
+
}
|
|
96
157
|
this.pending = "";
|
|
97
158
|
return this.parts;
|
|
98
159
|
}
|
|
@@ -109,6 +170,15 @@ function writeEntry(key, value, marked, out, prefix, depth) {
|
|
|
109
170
|
}
|
|
110
171
|
return true;
|
|
111
172
|
}
|
|
173
|
+
if (value instanceof LargeString && value.text.length >= LARGE_STRING_BYTES) {
|
|
174
|
+
const bytes = plainLargeBytes(value);
|
|
175
|
+
out.text(prefix);
|
|
176
|
+
if (bytes !== undefined)
|
|
177
|
+
out.large(value.text, value.prefix, bytes);
|
|
178
|
+
else
|
|
179
|
+
out.text(JSON.stringify(value.toJSON())); // 需要转义:交给原生
|
|
180
|
+
return true;
|
|
181
|
+
}
|
|
112
182
|
// 深度与 collectLarge 一致;有环时越过上限交给原生,由它抛 TypeError
|
|
113
183
|
if (depth < MAX_DEPTH && isContainer(value) && marked.has(value)) {
|
|
114
184
|
out.text(prefix);
|
|
@@ -150,18 +220,9 @@ function jsonParts(body) {
|
|
|
150
220
|
if (!collectLarge(body, marked, 0))
|
|
151
221
|
return undefined;
|
|
152
222
|
const out = new Pieces();
|
|
153
|
-
|
|
154
|
-
if (!isPlain(body))
|
|
155
|
-
return undefined;
|
|
156
|
-
out.large(body);
|
|
157
|
-
}
|
|
158
|
-
else
|
|
159
|
-
writeContainer(body, marked, out, 0);
|
|
223
|
+
writeEntry("", body, marked, out, "", 0); // 顶层:键为 "",与原生调用 toJSON 的键相同
|
|
160
224
|
const parts = out.finish();
|
|
161
|
-
|
|
162
|
-
for (const part of parts)
|
|
163
|
-
length += Buffer.byteLength(part, "utf8");
|
|
164
|
-
return { parts, length };
|
|
225
|
+
return { parts, length: out.bytes };
|
|
165
226
|
}
|
|
166
227
|
/** 与 `Buffer.from(JSON.stringify(body), "utf8")` 逐字节相同;大字符串不经中间字符串。 */
|
|
167
228
|
export function serializeJsonBody(body) {
|
|
@@ -178,11 +239,16 @@ export function serializeJsonBody(body) {
|
|
|
178
239
|
}
|
|
179
240
|
return buf;
|
|
180
241
|
}
|
|
242
|
+
/**
|
|
243
|
+
* 流式请求体每块最多编码的码元数:大片段(一整张图)按游标切片(V8 切片串不拷贝)逐块编码,
|
|
244
|
+
* 发送期间只多出一块的 UTF-8 字节,而不是一整张图。
|
|
245
|
+
*/
|
|
246
|
+
export const STREAM_CHUNK_CHARS = 256 * 1024;
|
|
181
247
|
/**
|
|
182
248
|
* 给 fetch 的请求体。不含大字符串时就是 `JSON.stringify(body)`(与以前完全相同);含大字符串时是
|
|
183
249
|
* 逐片段编码的流,并给出总字节数。不交给 fetch 一个 Buffer:Node 的 fetch 会把 BufferSource
|
|
184
250
|
* 请求体再复制两份(实测 mock 300 步带图峰值反而从约 1.0 GB 升到 1.85 GB),流则按片段边读边发,
|
|
185
|
-
*
|
|
251
|
+
* 同一时刻只多出一块(≤ {@link STREAM_CHUNK_CHARS} 码元)的 UTF-8 字节。字节序列与 `serializeJsonBody` 相同。
|
|
186
252
|
*/
|
|
187
253
|
export function jsonFetchBody(body) {
|
|
188
254
|
const split = jsonParts(body);
|
|
@@ -190,6 +256,7 @@ export function jsonFetchBody(body) {
|
|
|
190
256
|
return { body: JSON.stringify(body) };
|
|
191
257
|
const { parts } = split;
|
|
192
258
|
let next = 0;
|
|
259
|
+
let offset = 0; // 当前片段已发出的码元数
|
|
193
260
|
const stream = new ReadableStream({
|
|
194
261
|
pull(controller) {
|
|
195
262
|
const part = parts[next];
|
|
@@ -197,8 +264,19 @@ export function jsonFetchBody(body) {
|
|
|
197
264
|
controller.close();
|
|
198
265
|
return;
|
|
199
266
|
}
|
|
200
|
-
|
|
201
|
-
|
|
267
|
+
let end = Math.min(part.length, offset + STREAM_CHUNK_CHARS);
|
|
268
|
+
// 不把成对代理项切开(片段里只有成对的:孤立代理项已被原生转义)
|
|
269
|
+
const last = part.charCodeAt(end - 1);
|
|
270
|
+
if (end < part.length && last >= 0xd800 && last <= 0xdbff)
|
|
271
|
+
end--;
|
|
272
|
+
const piece = offset === 0 && end === part.length ? part : part.slice(offset, end);
|
|
273
|
+
if (end === part.length) {
|
|
274
|
+
parts[next++] = ""; // 发完即放手,片段(拼出来的小文本)可尽早回收
|
|
275
|
+
offset = 0;
|
|
276
|
+
}
|
|
277
|
+
else
|
|
278
|
+
offset = end;
|
|
279
|
+
controller.enqueue(Buffer.from(piece, "utf8"));
|
|
202
280
|
},
|
|
203
281
|
cancel() {
|
|
204
282
|
parts.length = 0;
|