dsh-awesome-optimizer 3.1.0 → 3.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +26 -20
- package/README.zh.md +26 -20
- package/cache-diagnostics.md +19 -0
- package/cordis.patch.yml +4 -0
- package/outline-benchmark.md +37 -0
- package/outline-benchmark.windows.json +184 -0
- package/package.json +11 -2
- package/scripts/benchmark-outline.mjs +100 -0
- package/src/ast-outline-config.yml +1 -0
- package/src/ast-outline.js +143 -0
- package/src/cache-diagnostics.js +218 -0
- package/src/code-lexical.js +131 -0
- package/src/code-outline.js +126 -153
- package/src/composite-tools.d.ts +8 -0
- package/src/composite-tools.js +105 -162
- package/src/context-compaction.d.ts +34 -0
- package/src/context-compaction.js +111 -0
- package/src/convergence.js +2 -2
- package/src/custom-prompt.js +3 -3
- package/src/edit-recovery.js +4 -1
- package/src/find-read-output.js +175 -0
- package/src/find-read-ranking.js +63 -0
- package/src/index.d.ts +8 -0
- package/src/index.js +10 -2
- package/src/outline-cache.js +36 -0
- package/src/persistent-eval/daemon-pool.js +21 -0
- package/src/persistent-eval/index.js +18 -10
- package/src/persistent-eval/stdio-runner.js +46 -14
- package/src/persistent-eval/workers/node_worker.mjs +44 -18
- package/src/persistent-eval/workers/py_worker.py +44 -2
- package/src/read-guard.d.ts +28 -0
- package/src/read-guard.js +152 -38
- package/src/telemetry.d.ts +4 -0
- package/src/telemetry.js +20 -12
package/README.md
CHANGED
|
@@ -5,29 +5,20 @@ English | [中文](README.zh.md)
|
|
|
5
5
|
[](https://www.npmjs.com/package/dsh-awesome-optimizer)
|
|
6
6
|
[](LICENSE)
|
|
7
7
|
[](https://github.com/Jaylor-Wang/dsh-awesome-optimizer/releases)
|
|
8
|
-
[](test/)
|
|
9
9
|
[](src/)
|
|
10
10
|
|
|
11
11
|
High-performance runtime policy guards, persistent Python & Node evaluation daemons, PTC-backed composite batch tooling, standalone refined system prompt engine, and objective A/B turn telemetry plugin for the official [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) (`@deepseek-ai/dsh-agent-loop`).
|
|
12
12
|
|
|
13
13
|
---
|
|
14
14
|
|
|
15
|
-
## Highlights in v3.
|
|
15
|
+
## Highlights in v3.5.0
|
|
16
16
|
|
|
17
|
-
- **
|
|
18
|
-
- **
|
|
19
|
-
- **
|
|
20
|
-
- **
|
|
21
|
-
|
|
22
|
-
## Highlights in v3.0.0
|
|
23
|
-
|
|
24
|
-
- **Precision Search-and-Slice (`find_and_read`, replacing `read_batch`)**:
|
|
25
|
-
- Removes `read_batch` (which overlapped with DSH native parallel `read` and encouraged speculative full-file reads) and replaces it with **`find_and_read`** over `ToolExecutionPipeline`.
|
|
26
|
-
- Runs 1–5 regex patterns in parallel through `grep`, deduplicates and ranks matched files by pattern coverage, merges overlapping `±contextLines` windows, and reads only targeted line slices within a strict `24KB` byte budget (overflowing extras to a compact `path:line` index).
|
|
27
|
-
- **Zero-Schema Native `read` Protection & Blind-Read Tail Outline (`readGuard`)**:
|
|
28
|
-
- Hooks `tools/post-execute` on direct `read` calls to truncate oversized minified lines (`maxLineChars: 1500`) and automatically inject a bounded ≤20-line structural symbol outline of unread lines whenever a blind `read` hits truncation—zero extra tool schema overhead.
|
|
29
|
-
- **Zero-Cost `edit` & `edit_batch` Context Recovery (`editRecovery`)**:
|
|
30
|
-
- Automatically attaches exact line-numbered nearby context on `edit` / `edit_batch` match failures (handling CRLF/indentation mismatches, multi-match line lists, rare-anchor alignment, and token-rarity fallback) while strictly skipping policy/sandbox/permission/`ENOENT` denials to avoid content leakage.
|
|
17
|
+
- **Sandbox-authoritative file inspection**: Edit recovery and batch preflight never fall back to local disk when the host filesystem service denies, fails, or cancels a read.
|
|
18
|
+
- **Byte-bounded persistent workers**: Python and Node output limits are enforced in UTF-8 bytes, and external stderr accumulation is bounded.
|
|
19
|
+
- **Unified worker lifecycle**: Session disposal, restart, and plugin shutdown clean up persistent workers consistently and prevent unintended worker recreation.
|
|
20
|
+
- **Concurrent session coverage**: Tests cover isolated multi-session execution, cancellation, unexpected worker termination, and disposal races.
|
|
21
|
+
- **Verification**: 167 tests passed; public declaration type checking passed. No new package dependencies.
|
|
31
22
|
|
|
32
23
|
---
|
|
33
24
|
|
|
@@ -36,7 +27,7 @@ High-performance runtime policy guards, persistent Python & Node evaluation daem
|
|
|
36
27
|
### 1. Refined System Prompt & File-Driven Hot-Reload (`customPrompt`)
|
|
37
28
|
|
|
38
29
|
- **Standalone Mode (`placement: "replace-all"`)**:
|
|
39
|
-
Hooks into the DSH `system-prompt/assemble` waterfall to strip all official verbose sections (`harness:*`, `deployment:*`, `tool:*`, `deliverable:*`, `web:*`), preserving convergence guidance companion sections while presenting your refined `SYSTEM.md` as the standalone authority.
|
|
30
|
+
Hooks into the DSH `system-prompt/assemble` waterfall to strip all official verbose sections (`harness:*`, `deployment:*`, `tool:*`, `deliverable:*`, `web:*`), preserving convergence guidance companion sections while presenting your refined `SYSTEM.md` as the standalone authority. This intentionally abandons those official prompt sections; add any instructions you still need to `SYSTEM.md`.
|
|
40
31
|
- **Dual-Layer Markdown Convention**:
|
|
41
32
|
Automatically discovers global `~/.dsh/SYSTEM.md` (or `$DSH_HOME/SYSTEM.md`) and project workspace `.dsh/SYSTEM.md` (or `SYSTEM.md`), cleanly separating developer system defaults from repo-specific instructions.
|
|
42
33
|
- **Millisecond Hot-Reload**:
|
|
@@ -55,7 +46,22 @@ High-performance runtime policy guards, persistent Python & Node evaluation daem
|
|
|
55
46
|
### 2. PTC Native Composite Batch Tooling (`compositeTools`)
|
|
56
47
|
|
|
57
48
|
- **`find_and_read`**:
|
|
58
|
-
Searches 1–5 regex patterns in parallel
|
|
49
|
+
Searches 1–5 regex patterns in parallel without changing regex whitespace, merges and splits adjacent matches, and dispatches budget-aware batches of native `read` calls in one round-trip. The complete returned text, including headers, errors, indexes, and omission notices, fits `maxTotalBytes` (24KB by default). Matched rows take priority; undisplayed locations become bounded indexes with explicit omission counts. Upstream truncation, failed patterns, totals, and recovery locators that fit are reported.
|
|
50
|
+
- `maxWindowLines: 120`: maximum requested lines per slice, preventing dense chained matches from becoming whole-file reads.
|
|
51
|
+
- `maxReadLines: 600`: maximum total requested lines across all files and windows in one call.
|
|
52
|
+
- `readConcurrency: 3`: maximum simultaneous reads per batch; remaining output space is checked before the next batch. Matches hidden by native read truncation remain in the recovery index.
|
|
53
|
+
- Windows prioritize uncovered regex patterns, then diminishing returns for repeated patterns, line cost, and file diversity. Canonical paths and line numbers break ties deterministically. `maxFiles` bounds selected files without excluding valuable windows in other files before ranking.
|
|
54
|
+
- These bounds cover requested lines and returned bytes, not a fixed number of filesystem bytes or tokens.
|
|
55
|
+
- **`readGuard` inspection preflight**:
|
|
56
|
+
Only supported outline languages trigger extra inspection. The fs service resolves and stats the target before reading, preferring bounded `readBytes` or text streaming. Oversized files (`maxInspectBytes`, default 512KB), cancellation, absent files, and service denials skip the outline while preserving the native read result. Service failures never fall back to local disk. Older readText-only services are supported after size preflight, subject to file-growth races between stat and read.
|
|
57
|
+
- Default `astOutline: "auto"` keeps ordinary files on the lexical path and invokes ast-grep on demand for JSX/TSX, TS decorators/exported namespaces, or declaration-like files with empty lexical results. Set `"off"` to disable or `"always"` to force AST for supported languages. Both levels prioritize top-level exports and sample across the unread tail.
|
|
58
|
+
- The CLI receives only the fs-authorized snapshot on stdin, never the target path. It uses fixed plugin configuration, disables custom language libraries, and runs without a shell. Defaults `astTimeoutMs: 750` and `astMaxOutputBytes: 262144` bound time/output. Cancellation terminates the child; missing CLI, timeout, and errors fall back to lexical results. `astGrepCommand` accepts an executable name or absolute path; the CLI must support `outline --stdin --json`.
|
|
59
|
+
- Current bundled ast-grep rules expose TS namespaces without children; lexical child declarations inside AST-confirmed namespace ranges are retained for navigation. AST successes and lexical fallbacks share the version LRU, preventing repeated launches for the same cached file version. Version changes, mutations, or eviction permit another attempt. Inspect `astAttempts/astSucceeded/astFallbacks/astTimeouts/outlineCacheHits` counters.
|
|
60
|
+
- Run `npm run benchmark:outline -- outline-benchmark.windows.json` for real-CLI first-invocation, repeated uncached, lexical, and version-cache timings with 20 repeated samples. See `outline-benchmark.md` for methodology and local Windows results.
|
|
61
|
+
- Parsed-symbol caches are isolated by session, fs backend, resolved identity, language, and version. Every lookup still stats the file; misses validate the version again after reading. Defaults: `maxCacheEntries: 32`, `maxCacheBytes: 262144`. Cached entries are filtered for the current truncation offset. Unversioned targets are not cached; mutation/code-execution tools and session disposal clear the cache. Set `maxCacheEntries: 0` to disable it. The byte cap measures serialized payload, not JS heap usage.
|
|
62
|
+
- The official standalone `session/disposed` event releases that session's backend caches, retained references, and statistics. Plugin disposal clears remaining caches. Inspections in flight cannot repopulate caches or inject late outlines after disposal; other sessions remain isolated.
|
|
63
|
+
- `getStats(session)` returns a detached per-turn snapshot, also exposed as telemetry `summary.readGuard` (`null` when disabled). `outlineCacheLookups = outlineCacheHits + outlineCacheMisses`; `outlineCacheHitRate = hits / lookups` is in [0, 1], or `null` without lookups. Only enabled caches with valid identity/version after successful preflight count as lookups. Absent, stale, or evicted entries count as misses even if the subsequent read or parse fails. Disabled, unidentified, unversioned, and standalone local backends count as `outlineCacheBypasses`; rejected preflights count as neither. Ordinary and targeted reads do not inflate the lookup denominator.
|
|
64
|
+
- `outlineCacheInvalidations` counts mutation/bash/eval events that clear an existing session cache group, including in-flight inspections. No group means no invalidation; version mismatches count as misses, and disposal cleanup does not count as invalidation. Counters reset on `turn/start` while caches survive turns. These metrics describe local outline caching separately from provider input caching.
|
|
59
65
|
- **`edit_batch`**:
|
|
60
66
|
Sequentially applies up to 10 targeted literal text replacements across single or multiple files in one round trip, with pre-flight same-file simulation, `atomic` abort support, and automatic line-numbered recovery hints on mismatch.
|
|
61
67
|
- **Inherited Security & Sandboxing**:
|
|
@@ -140,7 +146,7 @@ With `placement: "replace-all"`, place your directives in `~/.dsh/SYSTEM.md` (Wi
|
|
|
140
146
|
|
|
141
147
|
You are a master autonomous software engineer powered by DeepSeek Harness.
|
|
142
148
|
|
|
143
|
-
-
|
|
149
|
+
- MUST act directly rather than speculate: investigate via concurrent tools (fs, bash, skills) without orchestration overhead.
|
|
144
150
|
- NEVER emit pre-action narration. Deliver surgical edits and high-signal conclusions.
|
|
145
151
|
- Irreversible operations (`npm publish`): STRICTLY conservative and non-autonomous. NEVER bump versions or mutate artifacts to bypass failures; registry validation causes transient 404 or stale views—NEVER treat as failure or retry publish. MUST verify remote state with read-only commands before declaring status.
|
|
146
152
|
|
|
@@ -178,7 +184,7 @@ You are a master autonomous software engineer powered by DeepSeek Harness.
|
|
|
178
184
|
- Modification & execution:
|
|
179
185
|
- File ops: MUST use `edit_batch` for 2+ edits (whether in one file or across multiple files); use `edit` for a single edit, and `write` for new files or ad-hoc Node/Python scripts.
|
|
180
186
|
- Staged refactor: preview, then apply — symbol renames via `lsp rename` (dry-run → `--apply`); structural rewrites via `ast-grep run -p ... -r ...` (then `-U`).
|
|
181
|
-
- Structured data & calculations: Prioritize persistent kernels (`eval_python`,
|
|
187
|
+
- Structured data & calculations: Prioritize persistent kernels (`eval_python`/`eval_node`, key stdlib pre-imported) for stateful or exploratory work; return compact summaries, never raw dumps. Deliverables/pipelines: `write` to disk, run by path; never inline code.
|
|
182
188
|
- Shell execution: run heavy tasks (`npm test`, `npm run build`, `vitest`, `npm publish`) standalone to prevent Windows deadlocks; chain lightweight commands via `&&` (`git add ... && git commit ... && git push`) to eliminate round trips.
|
|
183
189
|
- Reporting: diffs and evidence first; zero filler. Routine status ≤ 300 Chinese characters. Answer/explain/review/diagnose/plan: no length cap. Compact locators, lists, and deltas only.
|
|
184
190
|
|
package/README.zh.md
CHANGED
|
@@ -5,29 +5,20 @@
|
|
|
5
5
|
[](https://www.npmjs.com/package/dsh-awesome-optimizer)
|
|
6
6
|
[](LICENSE)
|
|
7
7
|
[](https://github.com/Jaylor-Wang/dsh-awesome-optimizer/releases)
|
|
8
|
-
[](test/)
|
|
9
9
|
[](src/)
|
|
10
10
|
|
|
11
11
|
围绕官方 [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) (`@deepseek-ai/dsh-agent-loop`) 打造的高性能运行时策略防护、持久化 Python/Node 执行内核、PTC 复合批处理工具、全新独立系统提示词接管与客观 A/B 轮次度量插件。
|
|
12
12
|
|
|
13
13
|
---
|
|
14
14
|
|
|
15
|
-
## v3.
|
|
15
|
+
## v3.5.0 Worker 字节限制与生命周期加固
|
|
16
16
|
|
|
17
|
-
-
|
|
18
|
-
- **Python
|
|
19
|
-
-
|
|
20
|
-
-
|
|
21
|
-
|
|
22
|
-
## v3.0.0 架构减法与精准定位升级
|
|
23
|
-
|
|
24
|
-
- **精准检索切片工具(`find_and_read`,正式剔除 `read_batch`)**:
|
|
25
|
-
- 彻底移除与 DSH 原生并行 `read` 重叠且易诱导盲读整文件的 `read_batch`,以基于 `ToolExecutionPipeline` 的 **`find_and_read`** 接管探索阶段。
|
|
26
|
-
- 单次支持 1~5 个正则并行 `grep` 检索,按模式覆盖度去重排序文件,自动合并相邻/重叠的 `±contextLines` 行区间再派发 `read` 切片,严格受 `24KB` 字节预算控制(超出部分降级为紧凑 `path:line` 索引)。
|
|
27
|
-
- **原生 `read` 零工具位护栏与盲读尾部骨架(`readGuard`)**:
|
|
28
|
-
- 通过 `tools/post-execute` 无感保护所有原生 `read` 调用:截断超长压缩单行(`maxLineChars: 1500`),并在未指定 `offset/limit` 盲读大文件被截断时,于末尾自动附带剩余行(≤20 条)的轻量符号骨架,零额外 Tool Schema 开销。
|
|
29
|
-
- **零成本 `edit` / `edit_batch` 匹配失败上下文自愈(`editRecovery`)**:
|
|
30
|
-
- 当 `edit` 或 `edit_batch` 因缩进/CRLF/局部过期导致匹配失败时,自动回显带精确行号的附近源码(支持多处命中行号清单、缩进差异诊断、稀有锚点与 Token 滑窗兜底),且严格跳过权限/沙箱/`NO_PROGRESS`/`ENOENT` 拦截以防止内容泄露。
|
|
17
|
+
- **文件服务遵循沙箱权限**:Edit recovery 与批量编辑预检在宿主文件服务拒绝、读取失败或取消时,不会回退到本地磁盘。
|
|
18
|
+
- **Worker 输出按字节限制**:Python 与 Node 的输出上限按 UTF-8 字节执行;外部 stderr 累积也有明确内存上限。
|
|
19
|
+
- **统一 Worker 生命周期**:session disposal、显式 session 重启及插件关闭时一致清理持久 Worker,并防止意外重建。
|
|
20
|
+
- **并发与异常场景覆盖**:测试涵盖多 session 隔离并发、取消、Worker 意外退出和 disposal 竞态。
|
|
21
|
+
- **验证**:167 项测试通过,公开声明类型检查通过,无新增包依赖。
|
|
31
22
|
|
|
32
23
|
---
|
|
33
24
|
|
|
@@ -36,7 +27,7 @@
|
|
|
36
27
|
### 1. 系统提示词精简与文件驱动热重载 (`customPrompt`)
|
|
37
28
|
|
|
38
29
|
- **独立接管模式 (`placement: "replace-all"`)**:
|
|
39
|
-
在 DSH 官方 `system-prompt/assemble` 组装瀑布流中,精确剥除官方 verbose 冗余章节(包括 `harness:*`、`deployment:*`、`tool:*`、`deliverable:*`、`web:*` 等),并保留会话收敛指导等协同章节,将系统提示词基线精简至约 380 tokens
|
|
30
|
+
在 DSH 官方 `system-prompt/assemble` 组装瀑布流中,精确剥除官方 verbose 冗余章节(包括 `harness:*`、`deployment:*`、`tool:*`、`deliverable:*`、`web:*` 等),并保留会话收敛指导等协同章节,将系统提示词基线精简至约 380 tokens 的轻量核心指令。此模式会主动放弃这些官方 prompt sections;仍需保留的指令应自行补入 `SYSTEM.md`。
|
|
40
31
|
- **双层 Markdown 规约**:
|
|
41
32
|
自动寻找全局配置 `~/.dsh/SYSTEM.md`(或 `$DSH_HOME/SYSTEM.md`)与项目级工作区配置 `.dsh/SYSTEM.md`(或 `SYSTEM.md`)。权责清晰,与面向角色的 `AGENTS.md` 解耦。
|
|
42
33
|
- **毫秒级防抖热重载(Hot-Reload)**:
|
|
@@ -55,7 +46,22 @@
|
|
|
55
46
|
### 2. PTC 原生复合批处理工具 (`compositeTools`)
|
|
56
47
|
|
|
57
48
|
- **`find_and_read`**:
|
|
58
|
-
单次并行执行 1~5 条正则 `grep
|
|
49
|
+
单次并行执行 1~5 条正则 `grep`,保留正则原始空白语义,自动合并并拆分邻近命中区间,经原生管线分批 `read`,一跳替代 `grep → 等待 → read`。完整返回文本(含标题、错误、索引和省略说明)受 `maxTotalBytes` 硬限制,默认 24KB;优先保留命中行,未展示位置降级为索引,索引也放不下时明确计数。上游截断、模式失败、匹配总数及可容纳的恢复定位会显式保留。
|
|
50
|
+
- `maxWindowLines: 120`:单次切片最多请求的行数,防止密集命中连锁合并为整文件读取。
|
|
51
|
+
- `maxReadLines: 600`:单次调用累计请求行数上限,包含多个文件及窗口。
|
|
52
|
+
- `readConcurrency: 3`:每批最多同时读取的窗口数;每批完成后重新核算输出空间,空间不足时停止后续读取。原生读取自身截断所隐藏的命中仍进入索引。
|
|
53
|
+
- 窗口按尚未覆盖的正则数优先选择,再考虑重复模式的边际收益、行数成本及文件多样性;同分时按规范路径与行号稳定排序。`maxFiles` 限制实际选中的文件,不提前排除其他文件中的高价值窗口。
|
|
54
|
+
- 以上是行数与输出字节上限,不代表底层读取字节数或 Token 的固定换算。
|
|
55
|
+
- **`readGuard` 读取前检查**:
|
|
56
|
+
仅对支持的大纲语言执行附加检查;先经 fs 服务解析路径、检查文件类型与大小,再优先使用有界 `readBytes` 或文本流。超过 `maxInspectBytes`(默认 512KB)、取消、文件不存在或服务拒绝访问时跳过大纲,不改写原生读取结果。服务失败后不回退到本地磁盘;旧版仅有 `readText` 的服务在大小检查通过后兼容读取,仍受文件检查后变大的竞态限制。
|
|
57
|
+
- 默认 `astOutline: "auto"`:普通文件用轻量词法扫描;JSX/TSX、TS 装饰器/导出命名空间以及存在声明但词法结果为空的文件按需调用 ast-grep。可设 `"off"` 禁用或 `"always"` 对支持语言强制使用 AST。词法扫描支持 JS/TS/Python/Go/Rust 跨行声明、局部箭头与 Unicode 标识符;两级结果均优先顶层导出并在尾部采样。
|
|
58
|
+
- AST 只从 stdin 接收 fs 已校验的内容快照,不让 CLI 读取目标路径;使用插件内固定配置、禁用自定义语言库且不经 shell。`astTimeoutMs: 750`、`astMaxOutputBytes: 262144` 限制耗时与输出;取消会终止子进程,缺少 CLI/超时/失败时回退词法结果。`astGrepCommand` 可指定可执行文件名或绝对路径;需要支持 `outline --stdin --json` 的 ast-grep。
|
|
59
|
+
- 当前 ast-grep 内置规则只提供 TS 命名空间本身;命名空间内部保留词法识别的子声明,避免丢失导航位置。AST 成功结果及失败后的词法结果一起进入版本 LRU,同一文件版本的后续盲读不会再次启动 CLI;版本变化、写入或缓存淘汰后可重新尝试。可通过 `astAttempts/astSucceeded/astFallbacks/astTimeouts/outlineCacheHits` 统计验证。
|
|
60
|
+
- Windows 测量可运行 `npm run benchmark:outline -- outline-benchmark.windows.json`:真实 CLI、20 次重复样本,分别报告首次调用、未缓存重复解析、词法路径与版本缓存命中;详细方法和本机结果见 `outline-benchmark.md`。
|
|
61
|
+
- 大纲缓存按会话、fs 后端、解析后的文件身份、语言与版本隔离,每次仍执行 stat,首次读取后再次验证版本。默认 `maxCacheEntries: 32`、`maxCacheBytes: 262144`,缓存符号条目并按当前截断位置筛选;无可靠版本时不缓存,写入/编辑/执行代码及会话销毁会清除缓存,`maxCacheEntries: 0` 可关闭缓存。字节上限按序列化缓存载荷计,不等于 JS 堆占用。
|
|
62
|
+
- 通过官方独立的 `session/disposed` 事件释放该会话所有后端缓存、强引用与统计;插件卸载清空剩余缓存。释放期间的检查不会重新填充缓存或注入迟到的大纲,其他会话不受影响。
|
|
63
|
+
- `getStats(session)` 返回独立的每轮统计快照,统一 telemetry 的 `summary.readGuard` 同步提供,禁用 readGuard 时为 `null`。`outlineCacheLookups = outlineCacheHits + outlineCacheMisses`,`outlineCacheHitRate = hits / lookups`(0–1,无查询时为 `null`)。只有预检成功、缓存启用且文件身份/版本有效才计入查询;不存在、过期、被淘汰的条目均算 miss,后续读取或解析失败仍保留 miss。禁用、无身份/版本或本地独立后端计为 `outlineCacheBypasses`,预检拒绝不计查询或 bypass;普通/定向 read 不进入大纲查询分母。
|
|
64
|
+
- `outlineCacheInvalidations` 统计清除已有会话缓存组(包括进行中的检查)的写入/编辑/bash/eval 事件次数;无缓存组时不计,版本不匹配计 miss,释放清理不计失效。统计在 `turn/start` 重置,缓存可跨轮复用。这些指标只描述本地大纲缓存,与模型输入缓存命中率分别计算。
|
|
59
65
|
- **`edit_batch`**:
|
|
60
66
|
支持单次顺序执行最多 10 处跨单个或多个文件的字面量精准文本替换,内置同文件内存预演校验、`atomic` 原子中止及匹配失败行号自愈提示。
|
|
61
67
|
- **原生沙箱与安全事务继承**:
|
|
@@ -140,7 +146,7 @@ pnpm add dsh-awesome-optimizer
|
|
|
140
146
|
|
|
141
147
|
You are a master autonomous software engineer powered by DeepSeek Harness.
|
|
142
148
|
|
|
143
|
-
-
|
|
149
|
+
- MUST act directly rather than speculate: investigate via concurrent tools (fs, bash, skills) without orchestration overhead.
|
|
144
150
|
- NEVER emit pre-action narration. Deliver surgical edits and high-signal conclusions.
|
|
145
151
|
- Irreversible operations (`npm publish`): STRICTLY conservative and non-autonomous. NEVER bump versions or mutate artifacts to bypass failures; registry validation causes transient 404 or stale views—NEVER treat as failure or retry publish. MUST verify remote state with read-only commands before declaring status.
|
|
146
152
|
|
|
@@ -178,7 +184,7 @@ You are a master autonomous software engineer powered by DeepSeek Harness.
|
|
|
178
184
|
- Modification & execution:
|
|
179
185
|
- File ops: MUST use `edit_batch` for 2+ edits (whether in one file or across multiple files); use `edit` for a single edit, and `write` for new files or ad-hoc Node/Python scripts.
|
|
180
186
|
- Staged refactor: preview, then apply — symbol renames via `lsp rename` (dry-run → `--apply`); structural rewrites via `ast-grep run -p ... -r ...` (then `-U`).
|
|
181
|
-
- Structured data & calculations: Prioritize persistent kernels (`eval_python`,
|
|
187
|
+
- Structured data & calculations: Prioritize persistent kernels (`eval_python`/`eval_node`, key stdlib pre-imported) for stateful or exploratory work; return compact summaries, never raw dumps. Deliverables/pipelines: `write` to disk, run by path; never inline code.
|
|
182
188
|
- Shell execution: run heavy tasks (`npm test`, `npm run build`, `vitest`, `npm publish`) standalone to prevent Windows deadlocks; chain lightweight commands via `&&` (`git add ... && git commit ... && git push`) to eliminate round trips.
|
|
183
189
|
- Reporting: diffs and evidence first; zero filler. Routine status ≤ 300 Chinese characters. Answer/explain/review/diagnose/plan: no length cap. Compact locators, lists, and deltas only.
|
|
184
190
|
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# HTTP cache diagnostics
|
|
2
|
+
|
|
3
|
+
Set `telemetry.cacheDiagnosticsPath` to an absolute JSONL filename and reload the plugin or restart DSH. Omit the setting to disable. The observer uses AsyncLocalStorage around LLM stream iteration and a shared, reference-counted global fetch wrapper; calls outside an enabled session scope are untouched. No installed DSH files are modified.
|
|
4
|
+
|
|
5
|
+
Supported capture: OpenAI-compatible `/chat/completions` requests with a string JSON body using global fetch. Other protocols, custom transports, Request-object bodies and requests without session IDs are not captured. Absence of a log record is not evidence of cache stability.
|
|
6
|
+
|
|
7
|
+
Request records contain SHA-256 digests for each serialized message, tools, cache key, the remaining request configuration and endpoint; session IDs are hashed. The previous request link and first differing message index compare the same session/endpoint/model across turns. The first request has unknown stability. Message-level equality does not prove the provider's internal token prefix or caching policy. Configuration comparison includes stream/generation settings, so a change is evidence to inspect, not necessarily a cache cause.
|
|
8
|
+
|
|
9
|
+
Response records contain status and allowlisted request IDs. Usage records contain only allowlisted numeric upstream counters and constrained response/model/system-fingerprint identifiers. A missing cached_tokens field remains missing; zero remains zero. Prompt text, generated text, image bytes, credentials and arbitrary headers are never logged. Original response chunks pass through a bounded TransformStream without a cloned-response buffer. Cancellation and transport errors propagate. Do not treat an incomplete/unsupported capture as zero cache usage.
|
|
10
|
+
|
|
11
|
+
Limits: 8 MiB per log file, a 256 KiB asynchronous write queue, 128 previous session/route entries, and 1 MiB per buffered SSE event or JSON response. Reaching either log/queue limit stops capture and emits one host warning; use a new log path and reload to resume. Queued writes drain on disposal. Write failure stops capture with one warning; initialization failure disables diagnostics while preserving telemetry. Oversized events emit capture-gap and stop parsing that response. Response URL, redirected/type metadata and clone metadata are preserved. No automatic retry or inference warmup occurs. A restart/reload is required after source changes; configuration on disk alone does not prove the running process loaded the observer.
|
|
12
|
+
|
|
13
|
+
Interpretation:
|
|
14
|
+
- prefixStable=false: inspect the first changed message and request conversion.
|
|
15
|
+
- Stable messages/config plus absent cache counters: investigate gateway usage reporting.
|
|
16
|
+
- Stable messages/config plus explicit zero: investigate provider routing, retention and eviction; this alone does not prove a provider defect.
|
|
17
|
+
- Model/endpoint changes start a new comparison baseline.
|
|
18
|
+
|
|
19
|
+
The current Web profile is linked to this checkout. Local diagnostics should be stored outside the repository, e.g. `C:/Users/29507/.dsh/diagnostics/cache-http.jsonl`. After restart, verify a request/response/usage sequence during normal usage before relying on the capture. No paid inference requests are needed for unit verification.
|
package/cordis.patch.yml
CHANGED
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
# AST 大纲 Windows 测量
|
|
2
|
+
|
|
3
|
+
## 复现
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
npm run benchmark:outline -- outline-benchmark.windows.json
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
需要 PATH 上提供支持 `outline --stdin --json` 的 ast-grep;本次使用会话中安装的 ast-grep 0.51+。脚本调用实际 readGuard 钩子和实际 CLI,不经过网络或模型。每组先记录第一次调用,再采集 20 个重复样本;所有样本均检查输出包含预期符号,最后断言 AST 次数、内容读取次数和缓存命中次数。
|
|
10
|
+
|
|
11
|
+
原始结果保存在 `outline-benchmark.windows.json`。环境为 Windows x64(10.0.26300)、Node.js v24.21.0。时间戳取宿主系统时钟,不代表测试时钟经过独立校准。
|
|
12
|
+
|
|
13
|
+
## 实测结果
|
|
14
|
+
|
|
15
|
+
| 路径 | 首次调用 ms | 重复中位数 ms | 重复 p95 ms | 21 次调用的内容读取 / AST 启动 |
|
|
16
|
+
| --- | ---: | ---: | ---: | ---: |
|
|
17
|
+
| 2,912B TSX,关闭缓存,每次 AST | 67.468 | 51.269 | 56.480 | 21 / 21 |
|
|
18
|
+
| 同一 TSX,关闭 AST 和缓存 | 0.211 | 0.159 | 0.207 | 21 / 0 |
|
|
19
|
+
| 2,628B 普通 TS,auto、关闭缓存 | 0.352 | 0.185 | 0.225 | 21 / 0 |
|
|
20
|
+
| 2,912B TSX,auto、启用版本缓存 | 50.113 | 0.009 | 0.019 | 1 / 1 |
|
|
21
|
+
| 142,112B、2,444 行 TSX,auto、启用版本缓存 | 58.648 | 0.055 | 0.068 | 1 / 1 |
|
|
22
|
+
|
|
23
|
+
启用缓存的两组均记录了 20 次版本缓存命中。每次命中仍执行 resolve/stat;首次内容读取后再执行一次 stat,21 次调用共 22 次 stat。
|
|
24
|
+
|
|
25
|
+
## 方法与限制
|
|
26
|
+
|
|
27
|
+
- 首组首次调用是本次 benchmark Node 进程的第一次 ast-grep 启动。未清空 Windows 文件缓存,也没有重启系统,因此不是系统冷启动或冷磁盘测量。
|
|
28
|
+
- 未缓存的重复 AST 样本每次启动新的 CLI,是操作系统已热身后的进程启动加解析成本,没有持久 AST worker。
|
|
29
|
+
- fs 使用内存中的版本化内容来隔离 CLI/解析成本。缓存命中包含 readGuard 处理、路径解析、内存 stat、符号筛选和渲染;真实磁盘、远程 fs、网络 stat 会增加延迟。
|
|
30
|
+
- 大样本通过注释填充扩大快照,代表读取/传输体积变化,不代表复杂 AST 或大量符号的最坏情况。CLI 默认输出上限 256KiB,达到上限会回退词法结果。
|
|
31
|
+
- 这些数据不能换算成真实模型任务的固定加速比例或 Token 节省比例;词法与 AST 的信息质量也不完全等价。AST 的 JSX 字符串误报排除、成员行号等由回归测试另行验证。
|
|
32
|
+
|
|
33
|
+
## 配置结论
|
|
34
|
+
|
|
35
|
+
默认 `astOutline: "auto"`:普通文件保留词法路径,已知词法薄弱场景按需使用 AST。同一可缓存文件版本只需启动一次 CLI。默认 750ms 截止时间高于本次观测值,为 Windows 调度留出余量;超时、CLI 不可用、无效输出或输出超限均回退词法结果,失败结果同样按文件版本缓存以避免重复启动。`astOutline: "off"` 可完全关闭,`"always"` 可要求所有支持语言使用 AST。
|
|
36
|
+
|
|
37
|
+
当前 ast-grep 内置 outline 规则不返回 TS 命名空间子声明;实现保留 AST 确认的命名空间范围内的词法子条目。该部分属于组合导航结果,不能声称所有条目都经过 AST 独立提取。
|
|
@@ -0,0 +1,184 @@
|
|
|
1
|
+
{
|
|
2
|
+
"measuredAt": "2026-11-16T13:27:33.111Z",
|
|
3
|
+
"platform": "win32",
|
|
4
|
+
"architecture": "x64",
|
|
5
|
+
"osRelease": "10.0.26300",
|
|
6
|
+
"node": "v24.21.0",
|
|
7
|
+
"sourceBytes": 2912,
|
|
8
|
+
"sourceLines": 44,
|
|
9
|
+
"ordinarySourceBytes": 2628,
|
|
10
|
+
"ordinarySourceLines": 41,
|
|
11
|
+
"methodology": "Real ast-grep CLI over stdin; in-memory versioned fs isolates parsing/spawn cost. First invocation is process-cold, not a flushed OS disk cache. Repeated uncached samples launch a fresh CLI each time. Cache hits include resolve/stat and rendering but no content read or process. No network or model calls.",
|
|
12
|
+
"astUncached": {
|
|
13
|
+
"firstMs": 67.468,
|
|
14
|
+
"firstCounts": {
|
|
15
|
+
"contentReads": 1,
|
|
16
|
+
"statCalls": 2,
|
|
17
|
+
"inspected": 1,
|
|
18
|
+
"linesTruncated": 0,
|
|
19
|
+
"outlinesInjected": 1,
|
|
20
|
+
"outlineCacheHits": 0,
|
|
21
|
+
"astAttempts": 1,
|
|
22
|
+
"astSucceeded": 1,
|
|
23
|
+
"astFallbacks": 0,
|
|
24
|
+
"astTimeouts": 0
|
|
25
|
+
},
|
|
26
|
+
"repeated": {
|
|
27
|
+
"samples": 20,
|
|
28
|
+
"minMs": 48.91,
|
|
29
|
+
"medianMs": 51.269,
|
|
30
|
+
"p95Ms": 56.48,
|
|
31
|
+
"maxMs": 57.3
|
|
32
|
+
},
|
|
33
|
+
"finalCounts": {
|
|
34
|
+
"contentReads": 21,
|
|
35
|
+
"statCalls": 42,
|
|
36
|
+
"inspected": 21,
|
|
37
|
+
"linesTruncated": 0,
|
|
38
|
+
"outlinesInjected": 21,
|
|
39
|
+
"outlineCacheHits": 0,
|
|
40
|
+
"astAttempts": 21,
|
|
41
|
+
"astSucceeded": 21,
|
|
42
|
+
"astFallbacks": 0,
|
|
43
|
+
"astTimeouts": 0
|
|
44
|
+
}
|
|
45
|
+
},
|
|
46
|
+
"lexical": {
|
|
47
|
+
"firstMs": 0.211,
|
|
48
|
+
"firstCounts": {
|
|
49
|
+
"contentReads": 1,
|
|
50
|
+
"statCalls": 2,
|
|
51
|
+
"inspected": 1,
|
|
52
|
+
"linesTruncated": 0,
|
|
53
|
+
"outlinesInjected": 1,
|
|
54
|
+
"outlineCacheHits": 0,
|
|
55
|
+
"astAttempts": 0,
|
|
56
|
+
"astSucceeded": 0,
|
|
57
|
+
"astFallbacks": 0,
|
|
58
|
+
"astTimeouts": 0
|
|
59
|
+
},
|
|
60
|
+
"repeated": {
|
|
61
|
+
"samples": 20,
|
|
62
|
+
"minMs": 0.112,
|
|
63
|
+
"medianMs": 0.159,
|
|
64
|
+
"p95Ms": 0.207,
|
|
65
|
+
"maxMs": 0.429
|
|
66
|
+
},
|
|
67
|
+
"finalCounts": {
|
|
68
|
+
"contentReads": 21,
|
|
69
|
+
"statCalls": 42,
|
|
70
|
+
"inspected": 21,
|
|
71
|
+
"linesTruncated": 0,
|
|
72
|
+
"outlinesInjected": 21,
|
|
73
|
+
"outlineCacheHits": 0,
|
|
74
|
+
"astAttempts": 0,
|
|
75
|
+
"astSucceeded": 0,
|
|
76
|
+
"astFallbacks": 0,
|
|
77
|
+
"astTimeouts": 0
|
|
78
|
+
}
|
|
79
|
+
},
|
|
80
|
+
"autoOrdinary": {
|
|
81
|
+
"firstMs": 0.352,
|
|
82
|
+
"firstCounts": {
|
|
83
|
+
"contentReads": 1,
|
|
84
|
+
"statCalls": 2,
|
|
85
|
+
"inspected": 1,
|
|
86
|
+
"linesTruncated": 0,
|
|
87
|
+
"outlinesInjected": 1,
|
|
88
|
+
"outlineCacheHits": 0,
|
|
89
|
+
"astAttempts": 0,
|
|
90
|
+
"astSucceeded": 0,
|
|
91
|
+
"astFallbacks": 0,
|
|
92
|
+
"astTimeouts": 0
|
|
93
|
+
},
|
|
94
|
+
"repeated": {
|
|
95
|
+
"samples": 20,
|
|
96
|
+
"minMs": 0.097,
|
|
97
|
+
"medianMs": 0.185,
|
|
98
|
+
"p95Ms": 0.225,
|
|
99
|
+
"maxMs": 0.273
|
|
100
|
+
},
|
|
101
|
+
"finalCounts": {
|
|
102
|
+
"contentReads": 21,
|
|
103
|
+
"statCalls": 42,
|
|
104
|
+
"inspected": 21,
|
|
105
|
+
"linesTruncated": 0,
|
|
106
|
+
"outlinesInjected": 21,
|
|
107
|
+
"outlineCacheHits": 0,
|
|
108
|
+
"astAttempts": 0,
|
|
109
|
+
"astSucceeded": 0,
|
|
110
|
+
"astFallbacks": 0,
|
|
111
|
+
"astTimeouts": 0
|
|
112
|
+
}
|
|
113
|
+
},
|
|
114
|
+
"astCached": {
|
|
115
|
+
"firstMs": 50.113,
|
|
116
|
+
"firstCounts": {
|
|
117
|
+
"contentReads": 1,
|
|
118
|
+
"statCalls": 2,
|
|
119
|
+
"inspected": 1,
|
|
120
|
+
"linesTruncated": 0,
|
|
121
|
+
"outlinesInjected": 1,
|
|
122
|
+
"outlineCacheHits": 0,
|
|
123
|
+
"astAttempts": 1,
|
|
124
|
+
"astSucceeded": 1,
|
|
125
|
+
"astFallbacks": 0,
|
|
126
|
+
"astTimeouts": 0
|
|
127
|
+
},
|
|
128
|
+
"repeated": {
|
|
129
|
+
"samples": 20,
|
|
130
|
+
"minMs": 0.008,
|
|
131
|
+
"medianMs": 0.009,
|
|
132
|
+
"p95Ms": 0.019,
|
|
133
|
+
"maxMs": 0.034
|
|
134
|
+
},
|
|
135
|
+
"finalCounts": {
|
|
136
|
+
"contentReads": 1,
|
|
137
|
+
"statCalls": 22,
|
|
138
|
+
"inspected": 21,
|
|
139
|
+
"linesTruncated": 0,
|
|
140
|
+
"outlinesInjected": 21,
|
|
141
|
+
"outlineCacheHits": 20,
|
|
142
|
+
"astAttempts": 1,
|
|
143
|
+
"astSucceeded": 1,
|
|
144
|
+
"astFallbacks": 0,
|
|
145
|
+
"astTimeouts": 0
|
|
146
|
+
}
|
|
147
|
+
},
|
|
148
|
+
"astLarge": {
|
|
149
|
+
"sourceBytes": 142112,
|
|
150
|
+
"sourceLines": 2444,
|
|
151
|
+
"firstMs": 58.648,
|
|
152
|
+
"firstCounts": {
|
|
153
|
+
"contentReads": 1,
|
|
154
|
+
"statCalls": 2,
|
|
155
|
+
"inspected": 1,
|
|
156
|
+
"linesTruncated": 0,
|
|
157
|
+
"outlinesInjected": 1,
|
|
158
|
+
"outlineCacheHits": 0,
|
|
159
|
+
"astAttempts": 1,
|
|
160
|
+
"astSucceeded": 1,
|
|
161
|
+
"astFallbacks": 0,
|
|
162
|
+
"astTimeouts": 0
|
|
163
|
+
},
|
|
164
|
+
"repeated": {
|
|
165
|
+
"samples": 20,
|
|
166
|
+
"minMs": 0.05,
|
|
167
|
+
"medianMs": 0.055,
|
|
168
|
+
"p95Ms": 0.068,
|
|
169
|
+
"maxMs": 0.082
|
|
170
|
+
},
|
|
171
|
+
"finalCounts": {
|
|
172
|
+
"contentReads": 1,
|
|
173
|
+
"statCalls": 22,
|
|
174
|
+
"inspected": 21,
|
|
175
|
+
"linesTruncated": 0,
|
|
176
|
+
"outlinesInjected": 21,
|
|
177
|
+
"outlineCacheHits": 20,
|
|
178
|
+
"astAttempts": 1,
|
|
179
|
+
"astSucceeded": 1,
|
|
180
|
+
"astFallbacks": 0,
|
|
181
|
+
"astTimeouts": 0
|
|
182
|
+
}
|
|
183
|
+
}
|
|
184
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dsh-awesome-optimizer",
|
|
3
|
-
"version": "3.
|
|
3
|
+
"version": "3.5.0",
|
|
4
4
|
"description": "Runtime policy guards, composite batch tooling, persistent Python & Node eval daemons, soft convergence guidance, reactive custom prompt engine, and A/B telemetry plugin for the official DeepSeek Harness (dsh) agent loop.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "src/index.js",
|
|
@@ -30,6 +30,10 @@
|
|
|
30
30
|
"types": "./src/read-guard.d.ts",
|
|
31
31
|
"default": "./src/read-guard.js"
|
|
32
32
|
},
|
|
33
|
+
"./context-compaction": {
|
|
34
|
+
"types": "./src/context-compaction.d.ts",
|
|
35
|
+
"default": "./src/context-compaction.js"
|
|
36
|
+
},
|
|
33
37
|
"./composite-tools": {
|
|
34
38
|
"types": "./src/composite-tools.d.ts",
|
|
35
39
|
"default": "./src/composite-tools.js"
|
|
@@ -59,8 +63,12 @@
|
|
|
59
63
|
"locale/*.json",
|
|
60
64
|
"scripts/typecheck.mjs",
|
|
61
65
|
"scripts/benchmark.mjs",
|
|
66
|
+
"scripts/benchmark-outline.mjs",
|
|
62
67
|
"README.md",
|
|
63
68
|
"README.zh.md",
|
|
69
|
+
"cache-diagnostics.md",
|
|
70
|
+
"outline-benchmark.md",
|
|
71
|
+
"outline-benchmark.windows.json",
|
|
64
72
|
"LICENSE"
|
|
65
73
|
],
|
|
66
74
|
"dsh": {
|
|
@@ -75,7 +83,8 @@
|
|
|
75
83
|
"setup:local": "node scripts/setup-local.mjs",
|
|
76
84
|
"test": "node --test --test-concurrency=1 --test-timeout=10000 test/*.test.js",
|
|
77
85
|
"typecheck": "node scripts/typecheck.mjs",
|
|
78
|
-
"benchmark": "node scripts/benchmark.mjs"
|
|
86
|
+
"benchmark": "node scripts/benchmark.mjs",
|
|
87
|
+
"benchmark:outline": "node scripts/benchmark-outline.mjs"
|
|
79
88
|
},
|
|
80
89
|
"author": "Jaylor-Wang",
|
|
81
90
|
"license": "MIT",
|
|
@@ -0,0 +1,100 @@
|
|
|
1
|
+
import assert from 'node:assert/strict';
|
|
2
|
+
import fs from 'node:fs/promises';
|
|
3
|
+
import os from 'node:os';
|
|
4
|
+
import { applyReadGuard } from '../src/read-guard.js';
|
|
5
|
+
|
|
6
|
+
const samples = 20;
|
|
7
|
+
const source = [
|
|
8
|
+
'import React from "react";',
|
|
9
|
+
...Array.from({ length: 40 }, (_, i) => `export const Panel${i} = () => <section data-id="${i}">Panel ${i}</section>;`),
|
|
10
|
+
'export class Controller {',
|
|
11
|
+
' async render(value: string) { return value; }',
|
|
12
|
+
'}'
|
|
13
|
+
].join('\n');
|
|
14
|
+
|
|
15
|
+
const ordinarySource = [
|
|
16
|
+
'export interface Value { id: string; }',
|
|
17
|
+
...Array.from({ length: 40 }, (_, i) => `export function value${i}(item: Value): string { return item.id; }`)
|
|
18
|
+
].join('\n');
|
|
19
|
+
|
|
20
|
+
function host(astOutline, maxCacheEntries, content) {
|
|
21
|
+
const hooks = new Map();
|
|
22
|
+
const session = {};
|
|
23
|
+
let reads = 0;
|
|
24
|
+
let stats = 0;
|
|
25
|
+
const filesystem = {
|
|
26
|
+
resolve: async () => ({ targetKey: 'benchmark:panel.tsx' }),
|
|
27
|
+
stat: async () => { stats++; return { type: 'file', size: Buffer.byteLength(content), version: 'fixture-v1' }; },
|
|
28
|
+
readBytes: async () => { reads++; return Buffer.from(content); }
|
|
29
|
+
};
|
|
30
|
+
const guard = applyReadGuard({
|
|
31
|
+
on(name, callback) { hooks.set(name, callback); },
|
|
32
|
+
get(name) { return name === 'fs' ? filesystem : undefined; }
|
|
33
|
+
}, { astOutline, maxCacheEntries, astTimeoutMs: 2000 });
|
|
34
|
+
return {
|
|
35
|
+
async inspect(filePath = 'panel.tsx') {
|
|
36
|
+
const result = { content: [{ type: 'text', text: `<content>\n1: import\n\n(Showing lines 1-1 of ${content.split('\n').length}. Use offset=2 to continue.)\n</content>` }] };
|
|
37
|
+
const started = performance.now();
|
|
38
|
+
const decision = await hooks.get('tools/post-execute')({ name: 'read', arguments: { file_path: filePath }, agent: { session } }, result, async () => undefined);
|
|
39
|
+
const durationMs = performance.now() - started;
|
|
40
|
+
const text = decision?.content?.[0]?.text ?? '';
|
|
41
|
+
assert.match(text, /Remaining file outline/);
|
|
42
|
+
assert.match(text, /Panel0|value0/);
|
|
43
|
+
return durationMs;
|
|
44
|
+
},
|
|
45
|
+
counts: () => ({ contentReads: reads, statCalls: stats, ...guard.getStats(session) }),
|
|
46
|
+
dispose() { hooks.get('dispose')(); }
|
|
47
|
+
};
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
function summarize(durations) {
|
|
51
|
+
const sorted = [...durations].sort((a, b) => a - b);
|
|
52
|
+
const rounded = number => Number(number.toFixed(3));
|
|
53
|
+
return {
|
|
54
|
+
samples: sorted.length, minMs: rounded(sorted[0]),
|
|
55
|
+
medianMs: rounded(sorted[Math.floor(sorted.length / 2)]),
|
|
56
|
+
p95Ms: rounded(sorted[Math.ceil(sorted.length * 0.95) - 1]),
|
|
57
|
+
maxMs: rounded(sorted.at(-1))
|
|
58
|
+
};
|
|
59
|
+
}
|
|
60
|
+
|
|
61
|
+
async function measure(astOutline, cacheEntries, filePath, content = source) {
|
|
62
|
+
const fixture = host(astOutline, cacheEntries, content);
|
|
63
|
+
try {
|
|
64
|
+
const firstMs = await fixture.inspect(filePath);
|
|
65
|
+
const firstCounts = fixture.counts();
|
|
66
|
+
const durations = [];
|
|
67
|
+
for (let i = 0; i < samples; i++) durations.push(await fixture.inspect(filePath));
|
|
68
|
+
return { firstMs: Number(firstMs.toFixed(3)), firstCounts, repeated: summarize(durations), finalCounts: fixture.counts() };
|
|
69
|
+
} finally { fixture.dispose(); }
|
|
70
|
+
}
|
|
71
|
+
|
|
72
|
+
// First measurement is the first ast-grep launch in this benchmark process.
|
|
73
|
+
const astUncached = await measure('always', 0, 'panel.tsx');
|
|
74
|
+
const lexical = await measure('off', 0, 'panel.tsx');
|
|
75
|
+
const autoOrdinary = await measure('auto', 0, 'ordinary.ts', ordinarySource);
|
|
76
|
+
const astCached = await measure('auto', 32, 'panel.tsx');
|
|
77
|
+
const largeSource = `${'// source padding to measure a larger authorized snapshot\n'.repeat(2400)}${source}`;
|
|
78
|
+
const astLarge = await measure('auto', 32, 'large-panel.tsx', largeSource);
|
|
79
|
+
assert.equal(astUncached.finalCounts.astSucceeded, samples + 1);
|
|
80
|
+
assert.equal(astUncached.finalCounts.contentReads, samples + 1);
|
|
81
|
+
assert.equal(lexical.finalCounts.astAttempts, 0);
|
|
82
|
+
assert.equal(autoOrdinary.finalCounts.astAttempts, 0);
|
|
83
|
+
assert.equal(astCached.finalCounts.astSucceeded, 1);
|
|
84
|
+
assert.equal(astCached.finalCounts.contentReads, 1);
|
|
85
|
+
assert.equal(astCached.finalCounts.outlineCacheHits, samples);
|
|
86
|
+
assert.equal(astLarge.finalCounts.astSucceeded, 1);
|
|
87
|
+
assert.equal(astLarge.finalCounts.outlineCacheHits, samples);
|
|
88
|
+
const report = {
|
|
89
|
+
measuredAt: new Date().toISOString(),
|
|
90
|
+
platform: process.platform, architecture: process.arch,
|
|
91
|
+
osRelease: os.release(), node: process.version,
|
|
92
|
+
sourceBytes: Buffer.byteLength(source), sourceLines: source.split('\n').length,
|
|
93
|
+
ordinarySourceBytes: Buffer.byteLength(ordinarySource), ordinarySourceLines: ordinarySource.split('\n').length,
|
|
94
|
+
methodology: 'Real ast-grep CLI over stdin; in-memory versioned fs isolates parsing/spawn cost. First invocation is process-cold, not a flushed OS disk cache. Repeated uncached samples launch a fresh CLI each time. Cache hits include resolve/stat and rendering but no content read or process. No network or model calls.',
|
|
95
|
+
astUncached, lexical, autoOrdinary, astCached,
|
|
96
|
+
astLarge: { sourceBytes: Buffer.byteLength(largeSource), sourceLines: largeSource.split('\n').length, ...astLarge }
|
|
97
|
+
};
|
|
98
|
+
const outputPath = process.argv[2];
|
|
99
|
+
if (outputPath) await fs.writeFile(outputPath, JSON.stringify(report, null, 2) + '\n', 'utf8');
|
|
100
|
+
console.log(JSON.stringify(report, null, 2));
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
ruleDirs: []
|