chivgent 0.14.0 → 0.15.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +46 -5
- package/README.zh-CN.md +39 -5
- package/dist/agent.d.ts +4 -0
- package/dist/agent.d.ts.map +1 -1
- package/dist/agent.js +18 -1
- package/dist/agent.js.map +1 -1
- package/dist/context/compaction.d.ts +11 -2
- package/dist/context/compaction.d.ts.map +1 -1
- package/dist/context/compaction.js +15 -3
- package/dist/context/compaction.js.map +1 -1
- package/dist/context/context-manager.d.ts +3 -0
- package/dist/context/context-manager.d.ts.map +1 -1
- package/dist/context/context-manager.js +2 -1
- package/dist/context/context-manager.js.map +1 -1
- package/dist/evals/report.d.ts.map +1 -1
- package/dist/evals/report.js +21 -3
- package/dist/evals/report.js.map +1 -1
- package/dist/evals/runner.d.ts +3 -0
- package/dist/evals/runner.d.ts.map +1 -1
- package/dist/evals/runner.js +3 -0
- package/dist/evals/runner.js.map +1 -1
- package/dist/events.d.ts +6 -0
- package/dist/events.d.ts.map +1 -1
- package/dist/events.js.map +1 -1
- package/dist/llm.d.ts +22 -0
- package/dist/llm.d.ts.map +1 -1
- package/dist/llm.js.map +1 -1
- package/dist/providers/openai-compatible-chat.d.ts.map +1 -1
- package/dist/providers/openai-compatible-chat.js +13 -3
- package/dist/providers/openai-compatible-chat.js.map +1 -1
- package/dist/providers/openai.d.ts.map +1 -1
- package/dist/providers/openai.js +3 -0
- package/dist/providers/openai.js.map +1 -1
- package/dist/providers/usage.d.ts +22 -0
- package/dist/providers/usage.d.ts.map +1 -0
- package/dist/providers/usage.js +90 -0
- package/dist/providers/usage.js.map +1 -0
- package/dist/repl.d.ts.map +1 -1
- package/dist/repl.js +21 -0
- package/dist/repl.js.map +1 -1
- package/dist/session.d.ts +4 -0
- package/dist/session.d.ts.map +1 -1
- package/dist/session.js +10 -0
- package/dist/session.js.map +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -40,6 +40,7 @@ to study before adding production-harness complexity.
|
|
|
40
40
|
- Ranged `read_file` output with continuation hints and bounded tool results.
|
|
41
41
|
- Read-only by default; `--allow-writes` adds `write_file` and `edit_file`.
|
|
42
42
|
- An opt-in `bash` tool behind `--allow-shell`, with streamed output.
|
|
43
|
+
- Token usage from the Provider, per turn, per run, per eval task.
|
|
43
44
|
- An eval suite that measures whether the agent completes tasks, not just runs.
|
|
44
45
|
- Remote sessions: one process holds a session, others attach over a local socket.
|
|
45
46
|
- Several clients can watch one session; any of them can interrupt the run.
|
|
@@ -411,6 +412,7 @@ src/
|
|
|
411
412
|
write.ts Atomic whole-file writes and exact edits
|
|
412
413
|
providers/
|
|
413
414
|
registry.ts Provider registry
|
|
415
|
+
usage.ts Reading and adding up Provider token counts
|
|
414
416
|
deferred-client.ts A client whose Provider can arrive later
|
|
415
417
|
definitions.ts Built-in Provider declarations
|
|
416
418
|
client.ts Credential resolution into an LLM client
|
|
@@ -499,15 +501,20 @@ npm run eval -- --allow-writes --allow-shell --json report.json
|
|
|
499
501
|
```
|
|
500
502
|
|
|
501
503
|
```text
|
|
502
|
-
task pass turns tools p50
|
|
503
|
-
find-auth-logic 4/5 2.0 list_files,search_text,... 6.1s
|
|
504
|
-
rename-symbol 3/5 5.4 read_file,edit_file 9.8s
|
|
504
|
+
task pass turns tokens tools p50
|
|
505
|
+
find-auth-logic 4/5 2.0 3.4k list_files,search_text,... 6.1s
|
|
506
|
+
rename-symbol 3/5 5.4 12.1k read_file,edit_file 9.8s
|
|
505
507
|
|
|
506
|
-
overall 7/10 (70%)
|
|
508
|
+
overall 7/10 (70%) 58.3k tokens
|
|
507
509
|
|
|
508
510
|
rename-symbol attempt 2,5: not-used-tool name="write_file" — called write_file 1 time(s)
|
|
509
511
|
```
|
|
510
512
|
|
|
513
|
+
The `tokens` column is what the Provider reported, so a change can be judged on
|
|
514
|
+
both axes at once: a prompt that lifts the pass rate from 70% to 75% while
|
|
515
|
+
tripling the tokens is usually a bad trade, and without the second number that
|
|
516
|
+
looks like a pure win.
|
|
517
|
+
|
|
511
518
|
**An eval is not a test.** A model is nondeterministic, so one run of a task is a
|
|
512
519
|
coin flip: passing and failing are properties of the model-and-harness
|
|
513
520
|
combination, not of that run. The unit is therefore a pass rate over N attempts,
|
|
@@ -542,6 +549,38 @@ right contents, and an eval that scores results alone would give it full marks.
|
|
|
542
549
|
For a coding agent, tool misuse is the more common and the more interesting
|
|
543
550
|
failure, so tasks assert on how the work was done.
|
|
544
551
|
|
|
552
|
+
## Token usage
|
|
553
|
+
|
|
554
|
+
Providers report what each call cost, and chivgent now keeps it: per turn on
|
|
555
|
+
`message_end`, per run on `agent_end` and the run result, per attempt in the
|
|
556
|
+
eval report, and cumulatively in `/session`.
|
|
557
|
+
|
|
558
|
+
```text
|
|
559
|
+
› /session
|
|
560
|
+
id: 2026-09-07T...
|
|
561
|
+
workspace: /home/me/project
|
|
562
|
+
prompts: 4
|
|
563
|
+
messages: 18
|
|
564
|
+
tokens: 38.2k total, 35.9k in, 2.3k out, 12.0k cached
|
|
565
|
+
```
|
|
566
|
+
|
|
567
|
+
**Tokens only, never money.** Converting to a currency needs a model-to-price
|
|
568
|
+
table, and that table goes stale silently when a Provider changes its rates —
|
|
569
|
+
producing a figure that looks precise and is wrong, which is worse than no
|
|
570
|
+
figure because nobody questions a number with a decimal point in it. Token
|
|
571
|
+
counts are what the Provider itself reported and do not expire.
|
|
572
|
+
|
|
573
|
+
A total is marked incomplete when some call reported nothing, rather than
|
|
574
|
+
counting it as zero: a missing figure stays visibly missing instead of quietly
|
|
575
|
+
understating the total. Compaction's own summarising call is included, since it
|
|
576
|
+
trades a call now against smaller inputs later and that trade cannot be judged
|
|
577
|
+
with half of it hidden.
|
|
578
|
+
|
|
579
|
+
Streamed Chat Completions only report usage when asked, so chivgent sends
|
|
580
|
+
`stream_options: { include_usage: true }`. A self-hosted OpenAI-compatible
|
|
581
|
+
endpoint that predates that field may reject the request; `--no-stream` is the
|
|
582
|
+
fallback.
|
|
583
|
+
|
|
545
584
|
## Remote sessions
|
|
546
585
|
|
|
547
586
|
A session normally lives and dies with the terminal that started it. `--serve`
|
|
@@ -732,7 +771,8 @@ project.
|
|
|
732
771
|
- [x] Extensions and project trust
|
|
733
772
|
- [x] Remote sessions over a local socket
|
|
734
773
|
- [x] An eval suite with deterministic graders and pass rates
|
|
735
|
-
- [
|
|
774
|
+
- [x] Token usage from the Provider, aggregated per run and per eval
|
|
775
|
+
- [ ] TUI
|
|
736
776
|
|
|
737
777
|
Per-command confirmation prompts and command allowlists are deliberately not
|
|
738
778
|
planned. Once a shell tool exists, `bash` can do anything `write_file` can and
|
|
@@ -754,6 +794,7 @@ session-level switches; the real boundary is a container.
|
|
|
754
794
|
- [Stage 9: Extensions and project trust](docs/stage-9-extensions.md)
|
|
755
795
|
- [Stage 10: Remote sessions](docs/stage-10-remote-sessions.md)
|
|
756
796
|
- [Stage 11: Evals](docs/stage-11-evals.md)
|
|
797
|
+
- [Stage 12: Token usage](docs/stage-12-usage.md)
|
|
757
798
|
- [Release process](docs/releasing.md)
|
|
758
799
|
|
|
759
800
|
## Contributing
|
package/README.zh-CN.md
CHANGED
|
@@ -37,6 +37,7 @@
|
|
|
37
37
|
- 支持带续读提示的分段 `read_file`，所有工具结果都有容量上限。
|
|
38
38
|
- 默认只读；`--allow-writes` 才会启用 `write_file` 和 `edit_file`。
|
|
39
39
|
- 通过 `--allow-shell` 选择性开启的 `bash` 工具，输出边跑边显示。
|
|
40
|
+
- 采集 Provider 报告的 token 用量：按轮、按次运行、按 eval 任务分别统计。
|
|
40
41
|
- Eval 套件：衡量 Agent 能不能把活干成，而不只是能不能跑起来。
|
|
41
42
|
- 远程会话：一个进程持有会话，其他进程通过本地 socket 接上来。
|
|
42
43
|
- 多个客户端可以同时观察同一个会话，其中任何一个都能中断当前运行。
|
|
@@ -390,6 +391,7 @@ src/
|
|
|
390
391
|
write.ts 原子整文件写入与精确编辑
|
|
391
392
|
providers/
|
|
392
393
|
registry.ts Provider 注册表
|
|
394
|
+
usage.ts Provider token 用量的解析与累加
|
|
393
395
|
deferred-client.ts Provider 可以稍后到位的客户端
|
|
394
396
|
definitions.ts 内置 Provider 声明
|
|
395
397
|
client.ts 凭据解析并构造 LLM Client
|
|
@@ -478,15 +480,19 @@ npm run eval -- --allow-writes --allow-shell --json report.json
|
|
|
478
480
|
```
|
|
479
481
|
|
|
480
482
|
```text
|
|
481
|
-
task pass turns tools p50
|
|
482
|
-
find-auth-logic 4/5 2.0 list_files,search_text,... 6.1s
|
|
483
|
-
rename-symbol 3/5 5.4 read_file,edit_file 9.8s
|
|
483
|
+
task pass turns tokens tools p50
|
|
484
|
+
find-auth-logic 4/5 2.0 3.4k list_files,search_text,... 6.1s
|
|
485
|
+
rename-symbol 3/5 5.4 12.1k read_file,edit_file 9.8s
|
|
484
486
|
|
|
485
|
-
overall 7/10 (70%)
|
|
487
|
+
overall 7/10 (70%) 58.3k tokens
|
|
486
488
|
|
|
487
489
|
rename-symbol attempt 2,5: not-used-tool name="write_file" — called write_file 1 time(s)
|
|
488
490
|
```
|
|
489
491
|
|
|
492
|
+
`tokens` 那一列是 Provider 自己报的数字，于是一次改动可以同时看两个维度：
|
|
493
|
+
一个把通过率从 70% 提到 75%、token 却翻三倍的 prompt 改动通常是笔坏买卖，
|
|
494
|
+
而少了第二个数字它看起来是纯赚。
|
|
495
|
+
|
|
490
496
|
**Eval 不是测试。** 模型是不确定的，所以一个任务跑一次就是抛硬币：通过与失败不是这次
|
|
491
497
|
运行的属性，而是"模型 + harness"这个组合的属性。因此 eval 的基本单位是 **N 次尝试的
|
|
492
498
|
通过率**；也因此 **evals 绝不进 CI 门禁**——让 CI 花钱、并且以六成概率变红，两周内就会
|
|
@@ -517,6 +523,32 @@ rename-symbol attempt 2,5: not-used-tool name="write_file" — called write_file
|
|
|
517
523
|
对编码 Agent 来说，**工具误用是比答错更常见、也更值得测的失败**，所以任务不仅判定
|
|
518
524
|
"做对了没有"，还判定"是怎么做的"。
|
|
519
525
|
|
|
526
|
+
## Token 用量
|
|
527
|
+
|
|
528
|
+
Provider 每次调用都会报告代价，chivgent 现在会把它留下来：每轮在 `message_end` 上、
|
|
529
|
+
整次运行在 `agent_end` 和运行结果上、每次 eval 尝试在报告里、以及 `/session` 的累计。
|
|
530
|
+
|
|
531
|
+
```text
|
|
532
|
+
› /session
|
|
533
|
+
id: 2026-09-07T...
|
|
534
|
+
workspace: /home/me/project
|
|
535
|
+
prompts: 4
|
|
536
|
+
messages: 18
|
|
537
|
+
tokens: 38.2k total, 35.9k in, 2.3k out, 12.0k cached
|
|
538
|
+
```
|
|
539
|
+
|
|
540
|
+
**只报 token，不报钱。** 换算成金额需要一张"模型 → 单价"的表，而那张表会在 Provider
|
|
541
|
+
调价时**悄悄过期**，产出一个看起来精确、实际错误的金额——那比没有数字更糟，因为没人
|
|
542
|
+
会去质疑一个带小数点的数。token 数是 Provider 自己报的事实，不会过期。
|
|
543
|
+
|
|
544
|
+
某次调用没报用量时，合计会被**标记为不完整**，而不是当成零：缺失的数字要看得见地
|
|
545
|
+
缺着，而不是悄悄把合计压小。压缩自己那次摘要调用**计入**合计——它拿"现在多花一次
|
|
546
|
+
调用"换"后续输入更小"，少算一半就没法判断这笔交易划不划算。
|
|
547
|
+
|
|
548
|
+
流式的 Chat Completions 默认不报用量，必须显式索取，所以 chivgent 会发送
|
|
549
|
+
`stream_options: { include_usage: true }`。某些较早的自建 OpenAI 兼容端点可能不认识
|
|
550
|
+
这个字段而拒绝请求，此时 `--no-stream` 是退路。
|
|
551
|
+
|
|
520
552
|
## 远程会话
|
|
521
553
|
|
|
522
554
|
会话通常和启动它的那个终端同生共死。`--serve` 让它留在一个进程里，其他进程可以接上来：
|
|
@@ -674,7 +706,8 @@ CI 任务或管道运行永远不会自作主张去执行一个 clone 里的代
|
|
|
674
706
|
- [x] 扩展系统与 Project Trust
|
|
675
707
|
- [x] 基于本地 socket 的远程会话
|
|
676
708
|
- [x] 带确定性判定器和通过率的 Eval 套件
|
|
677
|
-
- [
|
|
709
|
+
- [x] Provider Token 用量采集与聚合
|
|
710
|
+
- [ ] TUI
|
|
678
711
|
|
|
679
712
|
逐条命令确认和命令白名单是**主动放弃**的方向。有了 Shell 工具之后，`bash` 能做的
|
|
680
713
|
事是 `write_file` 的超集，任何白名单都能被一行 `sh -c` 绕开，而逐条弹确认只会训练
|
|
@@ -694,6 +727,7 @@ CI 任务或管道运行永远不会自作主张去执行一个 clone 里的代
|
|
|
694
727
|
- [Stage 9：扩展系统与 Project Trust](docs/stage-9-extensions.md)
|
|
695
728
|
- [Stage 10：远程会话](docs/stage-10-remote-sessions.md)
|
|
696
729
|
- [Stage 11：Evals](docs/stage-11-evals.md)
|
|
730
|
+
- [Stage 12：Token 用量](docs/stage-12-usage.md)
|
|
697
731
|
- [发布流程](docs/releasing.md)
|
|
698
732
|
|
|
699
733
|
## 参与贡献
|
package/dist/agent.d.ts
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
import { type AgentEventListener } from "./events.js";
|
|
2
2
|
import { type LLMClient } from "./llm.js";
|
|
3
|
+
import { type UsageTotal } from "./providers/usage.js";
|
|
3
4
|
import type { AssistantMessage, Message } from "./messages.js";
|
|
4
5
|
import type { Tool } from "./tools/tool.js";
|
|
5
6
|
import type { Workspace } from "./workspace.js";
|
|
@@ -33,10 +34,13 @@ export type AgentRunResult = {
|
|
|
33
34
|
readonly finalMessage: AssistantMessage;
|
|
34
35
|
readonly messages: readonly Message[];
|
|
35
36
|
readonly turnCount: number;
|
|
37
|
+
/** What the run cost, when the Provider reported it. */
|
|
38
|
+
readonly usage?: UsageTotal;
|
|
36
39
|
} | {
|
|
37
40
|
readonly status: "max_turns" | "aborted";
|
|
38
41
|
readonly messages: readonly Message[];
|
|
39
42
|
readonly turnCount: number;
|
|
43
|
+
readonly usage?: UsageTotal;
|
|
40
44
|
};
|
|
41
45
|
export declare class AgentProtocolError extends Error {
|
|
42
46
|
constructor(message: string);
|
package/dist/agent.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"agent.d.ts","sourceRoot":"","sources":["../src/agent.ts"],"names":[],"mappings":"AAAA,OAAO,EAEL,KAAK,kBAAkB,EAExB,MAAM,aAAa,CAAC;AACrB,OAAO,EAAgB,KAAK,SAAS,EAAwB,MAAM,UAAU,CAAC;AAC9E,OAAO,KAAK,EACV,gBAAgB,EAChB,OAAO,EAGR,MAAM,eAAe,CAAC;AACvB,OAAO,KAAK,EAAE,IAAI,EAA8B,MAAM,iBAAiB,CAAC;AACxE,OAAO,KAAK,EAAE,SAAS,EAAE,MAAM,gBAAgB,CAAC;AAChD,OAAO,KAAK,EAEV,cAAc,EACf,MAAM,8BAA8B,CAAC;AAEtC,MAAM,WAAW,YAAY;IAC3B,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAC;IAC9B,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,GAAG,EAAE,SAAS,CAAC;IACxB,QAAQ,CAAC,KAAK,EAAE,SAAS,IAAI,EAAE,CAAC;IAChC,QAAQ,CAAC,SAAS,EAAE,SAAS,CAAC;IAC9B,gEAAgE;IAChE,QAAQ,CAAC,OAAO,CAAC,EAAE,kBAAkB,CAAC;IACtC,6EAA6E;IAC7E,QAAQ,CAAC,SAAS,CAAC,EAAE,OAAO,CAAC;IAC7B;;OAEG;IACH,QAAQ,CAAC,cAAc,CAAC,EAAE,cAAc,CAAC;CAC1C;AAED,MAAM,WAAW,eAAe;IAC9B,gFAAgF;IAChF,QAAQ,CAAC,MAAM,CAAC,EAAE,WAAW,CAAC;IAC9B;;;OAGG;IACH,QAAQ,CAAC,OAAO,CAAC,EAAE,SAAS,OAAO,EAAE,CAAC;CACvC;AAED,MAAM,MAAM,cAAc,GACtB;IACE,QAAQ,CAAC,MAAM,EAAE,WAAW,CAAC;IAC7B,QAAQ,CAAC,YAAY,EAAE,gBAAgB,CAAC;IACxC,QAAQ,CAAC,QAAQ,EAAE,SAAS,OAAO,EAAE,CAAC;IACtC,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;
|
|
1
|
+
{"version":3,"file":"agent.d.ts","sourceRoot":"","sources":["../src/agent.ts"],"names":[],"mappings":"AAAA,OAAO,EAEL,KAAK,kBAAkB,EAExB,MAAM,aAAa,CAAC;AACrB,OAAO,EAAgB,KAAK,SAAS,EAAwB,MAAM,UAAU,CAAC;AAC9E,OAAO,EAAY,KAAK,UAAU,EAAE,MAAM,sBAAsB,CAAC;AACjE,OAAO,KAAK,EACV,gBAAgB,EAChB,OAAO,EAGR,MAAM,eAAe,CAAC;AACvB,OAAO,KAAK,EAAE,IAAI,EAA8B,MAAM,iBAAiB,CAAC;AACxE,OAAO,KAAK,EAAE,SAAS,EAAE,MAAM,gBAAgB,CAAC;AAChD,OAAO,KAAK,EAEV,cAAc,EACf,MAAM,8BAA8B,CAAC;AAEtC,MAAM,WAAW,YAAY;IAC3B,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAC;IAC9B,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,GAAG,EAAE,SAAS,CAAC;IACxB,QAAQ,CAAC,KAAK,EAAE,SAAS,IAAI,EAAE,CAAC;IAChC,QAAQ,CAAC,SAAS,EAAE,SAAS,CAAC;IAC9B,gEAAgE;IAChE,QAAQ,CAAC,OAAO,CAAC,EAAE,kBAAkB,CAAC;IACtC,6EAA6E;IAC7E,QAAQ,CAAC,SAAS,CAAC,EAAE,OAAO,CAAC;IAC7B;;OAEG;IACH,QAAQ,CAAC,cAAc,CAAC,EAAE,cAAc,CAAC;CAC1C;AAED,MAAM,WAAW,eAAe;IAC9B,gFAAgF;IAChF,QAAQ,CAAC,MAAM,CAAC,EAAE,WAAW,CAAC;IAC9B;;;OAGG;IACH,QAAQ,CAAC,OAAO,CAAC,EAAE,SAAS,OAAO,EAAE,CAAC;CACvC;AAED,MAAM,MAAM,cAAc,GACtB;IACE,QAAQ,CAAC,MAAM,EAAE,WAAW,CAAC;IAC7B,QAAQ,CAAC,YAAY,EAAE,gBAAgB,CAAC;IACxC,QAAQ,CAAC,QAAQ,EAAE,SAAS,OAAO,EAAE,CAAC;IACtC,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B,wDAAwD;IACxD,QAAQ,CAAC,KAAK,CAAC,EAAE,UAAU,CAAC;CAC7B,GACD;IACE,QAAQ,CAAC,MAAM,EAAE,WAAW,GAAG,SAAS,CAAC;IACzC,QAAQ,CAAC,QAAQ,EAAE,SAAS,OAAO,EAAE,CAAC;IACtC,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B,QAAQ,CAAC,KAAK,CAAC,EAAE,UAAU,CAAC;CAC7B,CAAC;AAWN,qBAAa,kBAAmB,SAAQ,KAAK;IAC3C,YAAY,OAAO,EAAE,MAAM,EAG1B;CACF;AAED,qBAAa,KAAK;IAChB,OAAO,CAAC,QAAQ,CAAC,YAAY,CAAS;IACtC,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAS;IAClC,OAAO,CAAC,QAAQ,CAAC,GAAG,CAAY;IAChC,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAA4B;IACrD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAA4B;IAC5D,OAAO,CAAC,QAAQ,CAAC,SAAS,CAAY;IACtC,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAqB;IAC9C,OAAO,CAAC,QAAQ,CAAC,SAAS,CAAU;IACpC,OAAO,CAAC,QAAQ,CAAC,cAAc,CAAC,CAAiB;IAEjD,YAAY,OAAO,EAAE,YAAY,EAsBhC;IAED,IAAI,SAAS,IAAI,SAAS,MAAM,EAAE,CAEjC;IAEK,GAAG,CACP,SAAS,EAAE,MAAM,EACjB,OAAO,GAAE,eAAoB,GAC5B,OAAO,CAAC,cAAc,CAAC,CAqCzB;YAEa,IAAI;IA+DlB;;;;;;OAMG;YACW,YAAY;YA8BZ,uBAAuB;IA4BrC,OAAO,CAAC,uBAAuB;YAcjB,eAAe;IAmE7B,OAAO,CAAC,IAAI;IAIZ,OAAO,CAAC,OAAO;CAchB"}
|
package/dist/agent.js
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
import { emitEvent, } from "./events.js";
|
|
2
2
|
import { isAbortError } from "./llm.js";
|
|
3
|
+
import { addUsage } from "./providers/usage.js";
|
|
3
4
|
export class AgentProtocolError extends Error {
|
|
4
5
|
constructor(message) {
|
|
5
6
|
super(message);
|
|
@@ -83,11 +84,17 @@ export class Agent {
|
|
|
83
84
|
this.emit({ type: "turn_start", turn });
|
|
84
85
|
this.emit({ type: "message_start", turn });
|
|
85
86
|
const response = await this.requestAssistantMessage(state, turn, signal);
|
|
87
|
+
state.usage = addUsage(state.usage, response.usage);
|
|
86
88
|
const assistant = validateAndCloneAssistantMessage(response.message);
|
|
87
89
|
this.assertUniqueToolCallIds(assistant.toolCalls, state.seenToolCallIds);
|
|
88
90
|
state.messages.push(assistant);
|
|
89
91
|
state.continuation = response.continuation;
|
|
90
|
-
this.emit({
|
|
92
|
+
this.emit({
|
|
93
|
+
type: "message_end",
|
|
94
|
+
turn,
|
|
95
|
+
message: assistant,
|
|
96
|
+
...(response.usage === undefined ? {} : { usage: response.usage }),
|
|
97
|
+
});
|
|
91
98
|
if (assistant.toolCalls.length === 0) {
|
|
92
99
|
this.emit({
|
|
93
100
|
type: "turn_end",
|
|
@@ -100,6 +107,7 @@ export class Agent {
|
|
|
100
107
|
finalMessage: assistant,
|
|
101
108
|
messages: snapshotMessages(state.messages),
|
|
102
109
|
turnCount: state.turnCount,
|
|
110
|
+
...(state.usage === undefined ? {} : { usage: state.usage }),
|
|
103
111
|
};
|
|
104
112
|
}
|
|
105
113
|
const toolResults = [];
|
|
@@ -117,6 +125,7 @@ export class Agent {
|
|
|
117
125
|
status: "max_turns",
|
|
118
126
|
messages: snapshotMessages(state.messages),
|
|
119
127
|
turnCount: state.turnCount,
|
|
128
|
+
...(state.usage === undefined ? {} : { usage: state.usage }),
|
|
120
129
|
};
|
|
121
130
|
}
|
|
122
131
|
/**
|
|
@@ -144,6 +153,12 @@ export class Agent {
|
|
|
144
153
|
if (context.compacted) {
|
|
145
154
|
delete state.continuation;
|
|
146
155
|
}
|
|
156
|
+
// Summarising is a Provider call this run paid for. Only add when there
|
|
157
|
+
// was one: passing undefined would mark the total incomplete on every
|
|
158
|
+
// turn that simply did not need compacting.
|
|
159
|
+
if (context.usage !== undefined) {
|
|
160
|
+
state.usage = addUsage(state.usage, context.usage);
|
|
161
|
+
}
|
|
147
162
|
return context.messages;
|
|
148
163
|
}
|
|
149
164
|
async requestAssistantMessage(state, turn, signal) {
|
|
@@ -243,6 +258,7 @@ export class Agent {
|
|
|
243
258
|
status,
|
|
244
259
|
turnCount: state.turnCount,
|
|
245
260
|
messages: snapshotMessages(state.messages),
|
|
261
|
+
...(state.usage === undefined ? {} : { usage: state.usage }),
|
|
246
262
|
...(error === undefined ? {} : { error }),
|
|
247
263
|
});
|
|
248
264
|
}
|
|
@@ -256,6 +272,7 @@ function abortedResult(state) {
|
|
|
256
272
|
status: "aborted",
|
|
257
273
|
messages: snapshotMessages(state.messages),
|
|
258
274
|
turnCount: state.turnCount,
|
|
275
|
+
...(state.usage === undefined ? {} : { usage: state.usage }),
|
|
259
276
|
};
|
|
260
277
|
}
|
|
261
278
|
function createToolRegistry(tools) {
|
package/dist/agent.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"agent.js","sourceRoot":"","sources":["../src/agent.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,SAAS,GAGV,MAAM,aAAa,CAAC;AACrB,OAAO,EAAE,YAAY,EAAwC,MAAM,UAAU,CAAC;AA6D9E,MAAM,OAAO,kBAAmB,SAAQ,KAAK;IAC3C,YAAY,OAAe;QACzB,KAAK,CAAC,OAAO,CAAC,CAAC;QACf,IAAI,CAAC,IAAI,GAAG,oBAAoB,CAAC;IACnC,CAAC;CACF;AAED,MAAM,OAAO,KAAK;IACC,YAAY,CAAS;IACrB,QAAQ,CAAS;IACjB,GAAG,CAAY;IACf,QAAQ,CAA4B;IACpC,eAAe,CAA4B;IAC3C,SAAS,CAAY;IACrB,OAAO,CAAsB;IAC7B,SAAS,CAAU;IACnB,cAAc,CAAkB;IAEjD,YAAY,OAAqB;QAC/B,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,OAAO,CAAC,QAAQ,CAAC,IAAI,OAAO,CAAC,QAAQ,IAAI,CAAC,EAAE,CAAC;YACrE,MAAM,IAAI,SAAS,CAAC,2CAA2C,CAAC,CAAC;QACnE,CAAC;QAED,IAAI,CAAC,YAAY,GAAG,OAAO,CAAC,YAAY,CAAC;QACzC,IAAI,CAAC,QAAQ,GAAG,OAAO,CAAC,QAAQ,CAAC;QACjC,IAAI,CAAC,GAAG,GAAG,OAAO,CAAC,GAAG,CAAC;QACvB,IAAI,CAAC,SAAS,GAAG,OAAO,CAAC,SAAS,CAAC;QACnC,IAAI,CAAC,QAAQ,GAAG,kBAAkB,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC;QAClD,IAAI,CAAC,eAAe,GAAG,CAAC,GAAG,IAAI,CAAC,QAAQ,CAAC,MAAM,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;YAChE,IAAI,EAAE,IAAI,CAAC,IAAI;YACf,WAAW,EAAE,IAAI,CAAC,WAAW;YAC7B,WAAW,EAAE,eAAe,CAAC,IAAI,CAAC,WAAW,CAAC;SAC/C,CAAC,CAAC,CAAC;QACJ,IAAI,OAAO,CAAC,OAAO,KAAK,SAAS,EAAE,CAAC;YAClC,IAAI,CAAC,OAAO,GAAG,OAAO,CAAC,OAAO,CAAC;QACjC,CAAC;QACD,IAAI,CAAC,SAAS,GAAG,OAAO,CAAC,SAAS,IAAI,IAAI,CAAC;QAC3C,IAAI,OAAO,CAAC,cAAc,KAAK,SAAS,EAAE,CAAC;YACzC,IAAI,CAAC,cAAc,GAAG,OAAO,CAAC,cAAc,CAAC;QAC/C,CAAC;IACH,CAAC;IAED,IAAI,SAAS;QACX,OAAO,CAAC,GAAG,IAAI,CAAC,QAAQ,CAAC,IAAI,EAAE,CAAC,CAAC;IACnC,CAAC;IAED,KAAK,CAAC,GAAG,CACP,SAAiB,EACjB,OAAO,GAAoB,EAAE;QAE7B,IAAI,SAAS,CAAC,IAAI,EAAE,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YAClC,MAAM,IAAI,SAAS,CAAC,+BAA+B,CAAC,CAAC;QACvD,CAAC;QAED,MAAM,KAAK,GAAa;YACtB,QAAQ,EAAE;gBACR,GAAG,eAAe,CAAC,OAAO,CAAC,OAAO,IAAI,EAAE,CAAC;gBACzC,EAAE,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,SAAS,EAAE;aACrC;YACD,eAAe,EAAE,IAAI,GAAG,EAAU;YAClC,SAAS,EAAE,CAAC;SACb,CAAC;QACF,MAAM,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC;QAE9B,IAAI,CAAC,IAAI,CAAC;YACR,IAAI,EAAE,aAAa;YACnB,MAAM,EAAE,SAAS;YACjB,QAAQ,EAAE,IAAI,CAAC,QAAQ;SACxB,CAAC,CAAC;QAEH,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,IAAI,CAAC,KAAK,EAAE,MAAM,CAAC,CAAC;YAC9C,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,MAAM,EAAE,KAAK,CAAC,CAAC;YACnC,OAAO,MAAM,CAAC;QAChB,CAAC;QAAC,OAAO,KAAc,EAAE,CAAC;YACxB,IAAI,YAAY,CAAC,KAAK,CAAC,IAAI,SAAS,CAAC,MAAM,CAAC,EAAE,CAAC;gBAC7C,IAAI,CAAC,OAAO,CAAC,SAAS,EAAE,KAAK,CAAC,CAAC;gBAC/B,OAAO,aAAa,CAAC,KAAK,CAAC,CAAC;YAC9B,CAAC;YACD,IAAI,CAAC,OAAO,CACV,OAAO,EACP,KAAK,EACL,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,eAAe,CACzD,CAAC;YACF,MAAM,KAAK,CAAC;QACd,CAAC;IACH,CAAC;IAEO,KAAK,CAAC,IAAI,CAChB,KAAe,EACf,MAA+B;QAE/B,OAAO,KAAK,CAAC,SAAS,GAAG,IAAI,CAAC,QAAQ,EAAE,CAAC;YACvC,IAAI,SAAS,CAAC,MAAM,CAAC,EAAE,CAAC;gBACtB,OAAO,aAAa,CAAC,KAAK,CAAC,CAAC;YAC9B,CAAC;YAED,KAAK,CAAC,SAAS,IAAI,CAAC,CAAC;YACrB,MAAM,IAAI,GAAG,KAAK,CAAC,SAAS,CAAC;YAC7B,IAAI,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,YAAY,EAAE,IAAI,EAAE,CAAC,CAAC;YACxC,IAAI,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,eAAe,EAAE,IAAI,EAAE,CAAC,CAAC;YAE3C,MAAM,QAAQ,GAAG,MAAM,IAAI,CAAC,uBAAuB,CAAC,KAAK,EAAE,IAAI,EAAE,MAAM,CAAC,CAAC;YACzE,MAAM,SAAS,GAAG,gCAAgC,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC;YACrE,IAAI,CAAC,uBAAuB,CAAC,SAAS,CAAC,SAAS,EAAE,KAAK,CAAC,eAAe,CAAC,CAAC;YACzE,KAAK,CAAC,QAAQ,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;YAC/B,KAAK,CAAC,YAAY,GAAG,QAAQ,CAAC,YAAY,CAAC;YAC3C,IAAI,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,OAAO,EAAE,SAAS,EAAE,CAAC,CAAC;YAE7D,IAAI,SAAS,CAAC,SAAS,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;gBACrC,IAAI,CAAC,IAAI,CAAC;oBACR,IAAI,EAAE,UAAU;oBAChB,IAAI;oBACJ,OAAO,EAAE,SAAS;oBAClB,WAAW,EAAE,EAAE;iBAChB,CAAC,CAAC;gBACH,OAAO;oBACL,MAAM,EAAE,WAAW;oBACnB,YAAY,EAAE,SAAS;oBACvB,QAAQ,EAAE,gBAAgB,CAAC,KAAK,CAAC,QAAQ,CAAC;oBAC1C,SAAS,EAAE,KAAK,CAAC,SAAS;iBAC3B,CAAC;YACJ,CAAC;YAED,MAAM,WAAW,GAAwB,EAAE,CAAC;YAC5C,KAAK,MAAM,QAAQ,IAAI,SAAS,CAAC,SAAS,EAAE,CAAC;gBAC3C,IAAI,SAAS,CAAC,MAAM,CAAC,EAAE,CAAC;oBACtB,OAAO,aAAa,CAAC,KAAK,CAAC,CAAC;gBAC9B,CAAC;gBACD,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,eAAe,CAAC,QAAQ,EAAE,IAAI,EAAE,MAAM,CAAC,CAAC;gBAClE,WAAW,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;gBACzB,KAAK,CAAC,QAAQ,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;YAC9B,CAAC;YACD,IAAI,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,UAAU,EAAE,IAAI,EAAE,OAAO,EAAE,SAAS,EAAE,WAAW,EAAE,CAAC,CAAC;QACzE,CAAC;QAED,OAAO;YACL,MAAM,EAAE,WAAW;YACnB,QAAQ,EAAE,gBAAgB,CAAC,KAAK,CAAC,QAAQ,CAAC;YAC1C,SAAS,EAAE,KAAK,CAAC,SAAS;SAC3B,CAAC;IACJ,CAAC;IAED;;;;;;OAMG;IACK,KAAK,CAAC,YAAY,CACxB,KAAe,EACf,MAA+B;QAE/B,MAAM,QAAQ,GAAG,gBAAgB,CAAC,KAAK,CAAC,QAAQ,CAAC,CAAC;QAClD,IAAI,IAAI,CAAC,cAAc,KAAK,SAAS,EAAE,CAAC;YACtC,OAAO,QAAQ,CAAC;QAClB,CAAC;QAED,MAAM,OAAO,GAAG,MAAM,IAAI,CAAC,cAAc,CAAC,KAAK,CAAC,QAAQ,EAAE;YACxD,GAAG,CAAC,KAAK,CAAC,UAAU,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,KAAK,CAAC,UAAU,EAAE,CAAC;YACzE,GAAG,CAAC,MAAM,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC;SAC5C,CAAC,CAAC;QACH,IAAI,OAAO,CAAC,UAAU,KAAK,SAAS,EAAE,CAAC;YACrC,OAAO,KAAK,CAAC,UAAU,CAAC;QAC1B,CAAC;aAAM,CAAC;YACN,KAAK,CAAC,UAAU,GAAG,OAAO,CAAC,UAAU,CAAC;QACxC,CAAC;QACD,IAAI,OAAO,CAAC,SAAS,EAAE,CAAC;YACtB,OAAO,KAAK,CAAC,YAAY,CAAC;QAC5B,CAAC;QACD,OAAO,OAAO,CAAC,QAAQ,CAAC;IAC1B,CAAC;IAEO,KAAK,CAAC,uBAAuB,CACnC,KAAe,EACf,IAAY,EACZ,MAA+B;QAE/B,MAAM,QAAQ,GAAG,MAAM,IAAI,CAAC,YAAY,CAAC,KAAK,EAAE,MAAM,CAAC,CAAC;QACxD,MAAM,OAAO,GAAG;YACd,YAAY,EAAE,IAAI,CAAC,YAAY;YAC/B,QAAQ;YACR,KAAK,EAAE,IAAI,CAAC,eAAe;YAC3B,GAAG,CAAC,KAAK,CAAC,YAAY,KAAK,SAAS;gBAClC,CAAC,CAAC,EAAE;gBACJ,CAAC,CAAC,EAAE,YAAY,EAAE,KAAK,CAAC,YAAY,EAAE,CAAC;YACzC,GAAG,CAAC,MAAM,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC;SAC5C,CAAC;QAEF,IAAI,IAAI,CAAC,SAAS,IAAI,IAAI,CAAC,GAAG,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;YACpD,OAAO,IAAI,CAAC,GAAG,CAAC,MAAM,CAAC,OAAO,EAAE;gBAC9B,WAAW,EAAE,CAAC,KAAK,EAAE,EAAE;oBACrB,IAAI,KAAK,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;wBACrB,IAAI,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,gBAAgB,EAAE,IAAI,EAAE,KAAK,EAAE,CAAC,CAAC;oBACrD,CAAC;gBACH,CAAC;aACF,CAAC,CAAC;QACL,CAAC;QACD,OAAO,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC;IACpC,CAAC;IAEO,uBAAuB,CAC7B,SAA8B,EAC9B,OAAoB;QAEpB,KAAK,MAAM,QAAQ,IAAI,SAAS,EAAE,CAAC;YACjC,IAAI,OAAO,CAAC,GAAG,CAAC,QAAQ,CAAC,EAAE,CAAC,EAAE,CAAC;gBAC7B,MAAM,IAAI,kBAAkB,CAC1B,+CAA+C,QAAQ,CAAC,EAAE,EAAE,CAC7D,CAAC;YACJ,CAAC;YACD,OAAO,CAAC,GAAG,CAAC,QAAQ,CAAC,EAAE,CAAC,CAAC;QAC3B,CAAC;IACH,CAAC;IAEO,KAAK,CAAC,eAAe,CAC3B,QAAkB,EAClB,IAAY,EACZ,MAA+B;QAE/B,IAAI,CAAC,IAAI,CAAC;YACR,IAAI,EAAE,sBAAsB;YAC5B,IAAI;YACJ,UAAU,EAAE,QAAQ,CAAC,EAAE;YACvB,QAAQ,EAAE,QAAQ,CAAC,IAAI;YACvB,SAAS,EAAE,QAAQ,CAAC,SAAS;SAC9B,CAAC,CAAC;QAEH,MAAM,IAAI,GAAG,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC;QAC9C,IAAI,MAAkB,CAAC;QAEvB,IAAI,IAAI,KAAK,SAAS,EAAE,CAAC;YACvB,MAAM,GAAG;gBACP,OAAO,EAAE,iBAAiB,QAAQ,CAAC,IAAI,EAAE;gBACzC,OAAO,EAAE,IAAI;aACd,CAAC;QACJ,CAAC;aAAM,CAAC;YACN,IAAI,CAAC;gBACH,MAAM,GAAG,kBAAkB,CACzB,MAAM,IAAI,CAAC,OAAO,CAAC,QAAQ,CAAC,SAAS,EAAE;oBACrC,SAAS,EAAE,IAAI,CAAC,SAAS;oBACzB,GAAG,CAAC,MAAM,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC;oBAC3C,QAAQ,EAAE,CAAC,OAAe,EAAE,EAAE;wBAC5B,IAAI,CAAC,IAAI,CAAC;4BACR,IAAI,EAAE,uBAAuB;4BAC7B,IAAI;4BACJ,UAAU,EAAE,QAAQ,CAAC,EAAE;4BACvB,QAAQ,EAAE,QAAQ,CAAC,IAAI;4BACvB,OAAO;yBACR,CAAC,CAAC;oBACL,CAAC;iBACF,CAAC,CACH,CAAC;YACJ,CAAC;YAAC,OAAO,KAAc,EAAE,CAAC;gBACxB,IAAI,YAAY,CAAC,KAAK,CAAC,EAAE,CAAC;oBACxB,MAAM,KAAK,CAAC;gBACd,CAAC;gBACD,MAAM,GAAG;oBACP,OAAO,EAAE,0BAA0B,QAAQ,CAAC,IAAI,EAAE;oBAClD,OAAO,EAAE,IAAI;iBACd,CAAC;YACJ,CAAC;QACH,CAAC;QAED,IAAI,CAAC,IAAI,CAAC;YACR,IAAI,EAAE,oBAAoB;YAC1B,IAAI;YACJ,UAAU,EAAE,QAAQ,CAAC,EAAE;YACvB,QAAQ,EAAE,QAAQ,CAAC,IAAI;YACvB,OAAO,EAAE,MAAM,CAAC,OAAO;YACvB,OAAO,EAAE,MAAM,CAAC,OAAO;SACxB,CAAC,CAAC;QAEH,OAAO;YACL,IAAI,EAAE,MAAM;YACZ,UAAU,EAAE,QAAQ,CAAC,EAAE;YACvB,QAAQ,EAAE,QAAQ,CAAC,IAAI;YACvB,OAAO,EAAE,MAAM,CAAC,OAAO;YACvB,OAAO,EAAE,MAAM,CAAC,OAAO;SACxB,CAAC;IACJ,CAAC;IAEO,IAAI,CAAC,KAAsC;QACjD,SAAS,CAAC,IAAI,CAAC,OAAO,EAAE,KAAK,CAAC,CAAC;IACjC,CAAC;IAEO,OAAO,CACb,MAAsB,EACtB,KAAe,EACf,KAAc;QAEd,IAAI,CAAC,IAAI,CAAC;YACR,IAAI,EAAE,WAAW;YACjB,MAAM;YACN,SAAS,EAAE,KAAK,CAAC,SAAS;YAC1B,QAAQ,EAAE,gBAAgB,CAAC,KAAK,CAAC,QAAQ,CAAC;YAC1C,GAAG,CAAC,KAAK,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC;SAC1C,CAAC,CAAC;IACL,CAAC;CACF;AAED,4EAA4E;AAC5E,SAAS,SAAS,CAAC,MAA+B;IAChD,OAAO,MAAM,KAAK,SAAS,IAAI,MAAM,CAAC,OAAO,CAAC;AAChD,CAAC;AAED,SAAS,aAAa,CAAC,KAAe;IACpC,OAAO;QACL,MAAM,EAAE,SAAS;QACjB,QAAQ,EAAE,gBAAgB,CAAC,KAAK,CAAC,QAAQ,CAAC;QAC1C,SAAS,EAAE,KAAK,CAAC,SAAS;KAC3B,CAAC;AACJ,CAAC;AAED,SAAS,kBAAkB,CAAC,KAAsB;IAChD,MAAM,QAAQ,GAAG,IAAI,GAAG,EAAgB,CAAC;IACzC,KAAK,MAAM,IAAI,IAAI,KAAK,EAAE,CAAC;QACzB,IAAI,IAAI,CAAC,IAAI,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YAC3B,MAAM,IAAI,SAAS,CAAC,+BAA+B,CAAC,CAAC;QACvD,CAAC;QACD,IAAI,QAAQ,CAAC,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;YAC5B,MAAM,IAAI,SAAS,CAAC,wBAAwB,IAAI,CAAC,IAAI,EAAE,CAAC,CAAC;QAC3D,CAAC;QACD,QAAQ,CAAC,GAAG,CAAC,IAAI,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;IAChC,CAAC;IACD,OAAO,QAAQ,CAAC;AAClB,CAAC;AAED,SAAS,gCAAgC,CAAC,KAAc;IACtD,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,KAAK,IAAI,EAAE,CAAC;QAChD,MAAM,IAAI,kBAAkB,CAAC,iDAAiD,CAAC,CAAC;IAClF,CAAC;IAED,MAAM,SAAS,GAAG,KAAkC,CAAC;IACrD,IACE,SAAS,CAAC,IAAI,KAAK,WAAW;QAC9B,OAAO,SAAS,CAAC,OAAO,KAAK,QAAQ;QACrC,CAAC,KAAK,CAAC,OAAO,CAAC,SAAS,CAAC,SAAS,CAAC,EACnC,CAAC;QACD,MAAM,IAAI,kBAAkB,CAAC,iDAAiD,CAAC,CAAC;IAClF,CAAC;IAED,MAAM,SAAS,GAAG,SAAS,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC,QAAQ,EAAE,EAAE;QACrD,IACE,OAAO,QAAQ,KAAK,QAAQ;YAC5B,QAAQ,KAAK,IAAI;YACjB,OAAO,QAAQ,CAAC,EAAE,KAAK,QAAQ;YAC/B,QAAQ,CAAC,EAAE,CAAC,MAAM,KAAK,CAAC;YACxB,OAAO,QAAQ,CAAC,IAAI,KAAK,QAAQ;YACjC,QAAQ,CAAC,IAAI,CAAC,MAAM,KAAK,CAAC,EAC1B,CAAC;YACD,MAAM,IAAI,kBAAkB,CAAC,yCAAyC,CAAC,CAAC;QAC1E,CAAC;QAED,OAAO;YACL,EAAE,EAAE,QAAQ,CAAC,EAAE;YACf,IAAI,EAAE,QAAQ,CAAC,IAAI;YACnB,SAAS,EAAE,eAAe,CAAC,QAAQ,CAAC,SAAS,CAAC;SAC/C,CAAC;IACJ,CAAC,CAAC,CAAC;IAEH,IAAI,SAAS,CAAC,OAAO,CAAC,MAAM,KAAK,CAAC,IAAI,SAAS,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC7D,MAAM,IAAI,kBAAkB,CAC1B,oEAAoE,CACrE,CAAC;IACJ,CAAC;IAED,OAAO;QACL,IAAI,EAAE,WAAW;QACjB,OAAO,EAAE,SAAS,CAAC,OAAO;QAC1B,SAAS;KACV,CAAC;AACJ,CAAC;AAED,SAAS,kBAAkB,CAAC,KAAc;IACxC,IACE,OAAO,KAAK,KAAK,QAAQ;QACzB,KAAK,KAAK,IAAI;QACd,CAAC,CAAC,SAAS,IAAI,KAAK,CAAC;QACrB,CAAC,CAAC,SAAS,IAAI,KAAK,CAAC;QACrB,OAAO,KAAK,CAAC,OAAO,KAAK,QAAQ;QACjC,OAAO,KAAK,CAAC,OAAO,KAAK,SAAS,EAClC,CAAC;QACD,MAAM,IAAI,SAAS,CAAC,kCAAkC,CAAC,CAAC;IAC1D,CAAC;IACD,OAAO,EAAE,OAAO,EAAE,KAAK,CAAC,OAAO,EAAE,OAAO,EAAE,KAAK,CAAC,OAAO,EAAE,CAAC;AAC5D,CAAC;AAED,SAAS,gBAAgB,CAAC,QAA4B;IACpD,OAAO,eAAe,CAAC,QAAQ,CAAC,CAAC;AACnC,CAAC"}
|
|
1
|
+
{"version":3,"file":"agent.js","sourceRoot":"","sources":["../src/agent.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,SAAS,GAGV,MAAM,aAAa,CAAC;AACrB,OAAO,EAAE,YAAY,EAAwC,MAAM,UAAU,CAAC;AAC9E,OAAO,EAAE,QAAQ,EAAmB,MAAM,sBAAsB,CAAC;AAiEjE,MAAM,OAAO,kBAAmB,SAAQ,KAAK;IAC3C,YAAY,OAAe;QACzB,KAAK,CAAC,OAAO,CAAC,CAAC;QACf,IAAI,CAAC,IAAI,GAAG,oBAAoB,CAAC;IACnC,CAAC;CACF;AAED,MAAM,OAAO,KAAK;IACC,YAAY,CAAS;IACrB,QAAQ,CAAS;IACjB,GAAG,CAAY;IACf,QAAQ,CAA4B;IACpC,eAAe,CAA4B;IAC3C,SAAS,CAAY;IACrB,OAAO,CAAsB;IAC7B,SAAS,CAAU;IACnB,cAAc,CAAkB;IAEjD,YAAY,OAAqB;QAC/B,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,OAAO,CAAC,QAAQ,CAAC,IAAI,OAAO,CAAC,QAAQ,IAAI,CAAC,EAAE,CAAC;YACrE,MAAM,IAAI,SAAS,CAAC,2CAA2C,CAAC,CAAC;QACnE,CAAC;QAED,IAAI,CAAC,YAAY,GAAG,OAAO,CAAC,YAAY,CAAC;QACzC,IAAI,CAAC,QAAQ,GAAG,OAAO,CAAC,QAAQ,CAAC;QACjC,IAAI,CAAC,GAAG,GAAG,OAAO,CAAC,GAAG,CAAC;QACvB,IAAI,CAAC,SAAS,GAAG,OAAO,CAAC,SAAS,CAAC;QACnC,IAAI,CAAC,QAAQ,GAAG,kBAAkB,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC;QAClD,IAAI,CAAC,eAAe,GAAG,CAAC,GAAG,IAAI,CAAC,QAAQ,CAAC,MAAM,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;YAChE,IAAI,EAAE,IAAI,CAAC,IAAI;YACf,WAAW,EAAE,IAAI,CAAC,WAAW;YAC7B,WAAW,EAAE,eAAe,CAAC,IAAI,CAAC,WAAW,CAAC;SAC/C,CAAC,CAAC,CAAC;QACJ,IAAI,OAAO,CAAC,OAAO,KAAK,SAAS,EAAE,CAAC;YAClC,IAAI,CAAC,OAAO,GAAG,OAAO,CAAC,OAAO,CAAC;QACjC,CAAC;QACD,IAAI,CAAC,SAAS,GAAG,OAAO,CAAC,SAAS,IAAI,IAAI,CAAC;QAC3C,IAAI,OAAO,CAAC,cAAc,KAAK,SAAS,EAAE,CAAC;YACzC,IAAI,CAAC,cAAc,GAAG,OAAO,CAAC,cAAc,CAAC;QAC/C,CAAC;IACH,CAAC;IAED,IAAI,SAAS;QACX,OAAO,CAAC,GAAG,IAAI,CAAC,QAAQ,CAAC,IAAI,EAAE,CAAC,CAAC;IACnC,CAAC;IAED,KAAK,CAAC,GAAG,CACP,SAAiB,EACjB,OAAO,GAAoB,EAAE;QAE7B,IAAI,SAAS,CAAC,IAAI,EAAE,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YAClC,MAAM,IAAI,SAAS,CAAC,+BAA+B,CAAC,CAAC;QACvD,CAAC;QAED,MAAM,KAAK,GAAa;YACtB,QAAQ,EAAE;gBACR,GAAG,eAAe,CAAC,OAAO,CAAC,OAAO,IAAI,EAAE,CAAC;gBACzC,EAAE,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,SAAS,EAAE;aACrC;YACD,eAAe,EAAE,IAAI,GAAG,EAAU;YAClC,SAAS,EAAE,CAAC;SACb,CAAC;QACF,MAAM,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC;QAE9B,IAAI,CAAC,IAAI,CAAC;YACR,IAAI,EAAE,aAAa;YACnB,MAAM,EAAE,SAAS;YACjB,QAAQ,EAAE,IAAI,CAAC,QAAQ;SACxB,CAAC,CAAC;QAEH,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,IAAI,CAAC,KAAK,EAAE,MAAM,CAAC,CAAC;YAC9C,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,MAAM,EAAE,KAAK,CAAC,CAAC;YACnC,OAAO,MAAM,CAAC;QAChB,CAAC;QAAC,OAAO,KAAc,EAAE,CAAC;YACxB,IAAI,YAAY,CAAC,KAAK,CAAC,IAAI,SAAS,CAAC,MAAM,CAAC,EAAE,CAAC;gBAC7C,IAAI,CAAC,OAAO,CAAC,SAAS,EAAE,KAAK,CAAC,CAAC;gBAC/B,OAAO,aAAa,CAAC,KAAK,CAAC,CAAC;YAC9B,CAAC;YACD,IAAI,CAAC,OAAO,CACV,OAAO,EACP,KAAK,EACL,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,eAAe,CACzD,CAAC;YACF,MAAM,KAAK,CAAC;QACd,CAAC;IACH,CAAC;IAEO,KAAK,CAAC,IAAI,CAChB,KAAe,EACf,MAA+B;QAE/B,OAAO,KAAK,CAAC,SAAS,GAAG,IAAI,CAAC,QAAQ,EAAE,CAAC;YACvC,IAAI,SAAS,CAAC,MAAM,CAAC,EAAE,CAAC;gBACtB,OAAO,aAAa,CAAC,KAAK,CAAC,CAAC;YAC9B,CAAC;YAED,KAAK,CAAC,SAAS,IAAI,CAAC,CAAC;YACrB,MAAM,IAAI,GAAG,KAAK,CAAC,SAAS,CAAC;YAC7B,IAAI,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,YAAY,EAAE,IAAI,EAAE,CAAC,CAAC;YACxC,IAAI,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,eAAe,EAAE,IAAI,EAAE,CAAC,CAAC;YAE3C,MAAM,QAAQ,GAAG,MAAM,IAAI,CAAC,uBAAuB,CAAC,KAAK,EAAE,IAAI,EAAE,MAAM,CAAC,CAAC;YACzE,KAAK,CAAC,KAAK,GAAG,QAAQ,CAAC,KAAK,CAAC,KAAK,EAAE,QAAQ,CAAC,KAAK,CAAC,CAAC;YACpD,MAAM,SAAS,GAAG,gCAAgC,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC;YACrE,IAAI,CAAC,uBAAuB,CAAC,SAAS,CAAC,SAAS,EAAE,KAAK,CAAC,eAAe,CAAC,CAAC;YACzE,KAAK,CAAC,QAAQ,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;YAC/B,KAAK,CAAC,YAAY,GAAG,QAAQ,CAAC,YAAY,CAAC;YAC3C,IAAI,CAAC,IAAI,CAAC;gBACR,IAAI,EAAE,aAAa;gBACnB,IAAI;gBACJ,OAAO,EAAE,SAAS;gBAClB,GAAG,CAAC,QAAQ,CAAC,KAAK,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,QAAQ,CAAC,KAAK,EAAE,CAAC;aACnE,CAAC,CAAC;YAEH,IAAI,SAAS,CAAC,SAAS,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;gBACrC,IAAI,CAAC,IAAI,CAAC;oBACR,IAAI,EAAE,UAAU;oBAChB,IAAI;oBACJ,OAAO,EAAE,SAAS;oBAClB,WAAW,EAAE,EAAE;iBAChB,CAAC,CAAC;gBACH,OAAO;oBACL,MAAM,EAAE,WAAW;oBACnB,YAAY,EAAE,SAAS;oBACvB,QAAQ,EAAE,gBAAgB,CAAC,KAAK,CAAC,QAAQ,CAAC;oBAC1C,SAAS,EAAE,KAAK,CAAC,SAAS;oBAC1B,GAAG,CAAC,KAAK,CAAC,KAAK,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,KAAK,CAAC,KAAK,EAAE,CAAC;iBAC7D,CAAC;YACJ,CAAC;YAED,MAAM,WAAW,GAAwB,EAAE,CAAC;YAC5C,KAAK,MAAM,QAAQ,IAAI,SAAS,CAAC,SAAS,EAAE,CAAC;gBAC3C,IAAI,SAAS,CAAC,MAAM,CAAC,EAAE,CAAC;oBACtB,OAAO,aAAa,CAAC,KAAK,CAAC,CAAC;gBAC9B,CAAC;gBACD,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,eAAe,CAAC,QAAQ,EAAE,IAAI,EAAE,MAAM,CAAC,CAAC;gBAClE,WAAW,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;gBACzB,KAAK,CAAC,QAAQ,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;YAC9B,CAAC;YACD,IAAI,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,UAAU,EAAE,IAAI,EAAE,OAAO,EAAE,SAAS,EAAE,WAAW,EAAE,CAAC,CAAC;QACzE,CAAC;QAED,OAAO;YACL,MAAM,EAAE,WAAW;YACnB,QAAQ,EAAE,gBAAgB,CAAC,KAAK,CAAC,QAAQ,CAAC;YAC1C,SAAS,EAAE,KAAK,CAAC,SAAS;YAC1B,GAAG,CAAC,KAAK,CAAC,KAAK,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,KAAK,CAAC,KAAK,EAAE,CAAC;SAC7D,CAAC;IACJ,CAAC;IAED;;;;;;OAMG;IACK,KAAK,CAAC,YAAY,CACxB,KAAe,EACf,MAA+B;QAE/B,MAAM,QAAQ,GAAG,gBAAgB,CAAC,KAAK,CAAC,QAAQ,CAAC,CAAC;QAClD,IAAI,IAAI,CAAC,cAAc,KAAK,SAAS,EAAE,CAAC;YACtC,OAAO,QAAQ,CAAC;QAClB,CAAC;QAED,MAAM,OAAO,GAAG,MAAM,IAAI,CAAC,cAAc,CAAC,KAAK,CAAC,QAAQ,EAAE;YACxD,GAAG,CAAC,KAAK,CAAC,UAAU,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,KAAK,CAAC,UAAU,EAAE,CAAC;YACzE,GAAG,CAAC,MAAM,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC;SAC5C,CAAC,CAAC;QACH,IAAI,OAAO,CAAC,UAAU,KAAK,SAAS,EAAE,CAAC;YACrC,OAAO,KAAK,CAAC,UAAU,CAAC;QAC1B,CAAC;aAAM,CAAC;YACN,KAAK,CAAC,UAAU,GAAG,OAAO,CAAC,UAAU,CAAC;QACxC,CAAC;QACD,IAAI,OAAO,CAAC,SAAS,EAAE,CAAC;YACtB,OAAO,KAAK,CAAC,YAAY,CAAC;QAC5B,CAAC;QACD,wEAAwE;QACxE,sEAAsE;QACtE,4CAA4C;QAC5C,IAAI,OAAO,CAAC,KAAK,KAAK,SAAS,EAAE,CAAC;YAChC,KAAK,CAAC,KAAK,GAAG,QAAQ,CAAC,KAAK,CAAC,KAAK,EAAE,OAAO,CAAC,KAAK,CAAC,CAAC;QACrD,CAAC;QACD,OAAO,OAAO,CAAC,QAAQ,CAAC;IAC1B,CAAC;IAEO,KAAK,CAAC,uBAAuB,CACnC,KAAe,EACf,IAAY,EACZ,MAA+B;QAE/B,MAAM,QAAQ,GAAG,MAAM,IAAI,CAAC,YAAY,CAAC,KAAK,EAAE,MAAM,CAAC,CAAC;QACxD,MAAM,OAAO,GAAG;YACd,YAAY,EAAE,IAAI,CAAC,YAAY;YAC/B,QAAQ;YACR,KAAK,EAAE,IAAI,CAAC,eAAe;YAC3B,GAAG,CAAC,KAAK,CAAC,YAAY,KAAK,SAAS;gBAClC,CAAC,CAAC,EAAE;gBACJ,CAAC,CAAC,EAAE,YAAY,EAAE,KAAK,CAAC,YAAY,EAAE,CAAC;YACzC,GAAG,CAAC,MAAM,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC;SAC5C,CAAC;QAEF,IAAI,IAAI,CAAC,SAAS,IAAI,IAAI,CAAC,GAAG,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;YACpD,OAAO,IAAI,CAAC,GAAG,CAAC,MAAM,CAAC,OAAO,EAAE;gBAC9B,WAAW,EAAE,CAAC,KAAK,EAAE,EAAE;oBACrB,IAAI,KAAK,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;wBACrB,IAAI,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,gBAAgB,EAAE,IAAI,EAAE,KAAK,EAAE,CAAC,CAAC;oBACrD,CAAC;gBACH,CAAC;aACF,CAAC,CAAC;QACL,CAAC;QACD,OAAO,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC;IACpC,CAAC;IAEO,uBAAuB,CAC7B,SAA8B,EAC9B,OAAoB;QAEpB,KAAK,MAAM,QAAQ,IAAI,SAAS,EAAE,CAAC;YACjC,IAAI,OAAO,CAAC,GAAG,CAAC,QAAQ,CAAC,EAAE,CAAC,EAAE,CAAC;gBAC7B,MAAM,IAAI,kBAAkB,CAC1B,+CAA+C,QAAQ,CAAC,EAAE,EAAE,CAC7D,CAAC;YACJ,CAAC;YACD,OAAO,CAAC,GAAG,CAAC,QAAQ,CAAC,EAAE,CAAC,CAAC;QAC3B,CAAC;IACH,CAAC;IAEO,KAAK,CAAC,eAAe,CAC3B,QAAkB,EAClB,IAAY,EACZ,MAA+B;QAE/B,IAAI,CAAC,IAAI,CAAC;YACR,IAAI,EAAE,sBAAsB;YAC5B,IAAI;YACJ,UAAU,EAAE,QAAQ,CAAC,EAAE;YACvB,QAAQ,EAAE,QAAQ,CAAC,IAAI;YACvB,SAAS,EAAE,QAAQ,CAAC,SAAS;SAC9B,CAAC,CAAC;QAEH,MAAM,IAAI,GAAG,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC;QAC9C,IAAI,MAAkB,CAAC;QAEvB,IAAI,IAAI,KAAK,SAAS,EAAE,CAAC;YACvB,MAAM,GAAG;gBACP,OAAO,EAAE,iBAAiB,QAAQ,CAAC,IAAI,EAAE;gBACzC,OAAO,EAAE,IAAI;aACd,CAAC;QACJ,CAAC;aAAM,CAAC;YACN,IAAI,CAAC;gBACH,MAAM,GAAG,kBAAkB,CACzB,MAAM,IAAI,CAAC,OAAO,CAAC,QAAQ,CAAC,SAAS,EAAE;oBACrC,SAAS,EAAE,IAAI,CAAC,SAAS;oBACzB,GAAG,CAAC,MAAM,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC;oBAC3C,QAAQ,EAAE,CAAC,OAAe,EAAE,EAAE;wBAC5B,IAAI,CAAC,IAAI,CAAC;4BACR,IAAI,EAAE,uBAAuB;4BAC7B,IAAI;4BACJ,UAAU,EAAE,QAAQ,CAAC,EAAE;4BACvB,QAAQ,EAAE,QAAQ,CAAC,IAAI;4BACvB,OAAO;yBACR,CAAC,CAAC;oBACL,CAAC;iBACF,CAAC,CACH,CAAC;YACJ,CAAC;YAAC,OAAO,KAAc,EAAE,CAAC;gBACxB,IAAI,YAAY,CAAC,KAAK,CAAC,EAAE,CAAC;oBACxB,MAAM,KAAK,CAAC;gBACd,CAAC;gBACD,MAAM,GAAG;oBACP,OAAO,EAAE,0BAA0B,QAAQ,CAAC,IAAI,EAAE;oBAClD,OAAO,EAAE,IAAI;iBACd,CAAC;YACJ,CAAC;QACH,CAAC;QAED,IAAI,CAAC,IAAI,CAAC;YACR,IAAI,EAAE,oBAAoB;YAC1B,IAAI;YACJ,UAAU,EAAE,QAAQ,CAAC,EAAE;YACvB,QAAQ,EAAE,QAAQ,CAAC,IAAI;YACvB,OAAO,EAAE,MAAM,CAAC,OAAO;YACvB,OAAO,EAAE,MAAM,CAAC,OAAO;SACxB,CAAC,CAAC;QAEH,OAAO;YACL,IAAI,EAAE,MAAM;YACZ,UAAU,EAAE,QAAQ,CAAC,EAAE;YACvB,QAAQ,EAAE,QAAQ,CAAC,IAAI;YACvB,OAAO,EAAE,MAAM,CAAC,OAAO;YACvB,OAAO,EAAE,MAAM,CAAC,OAAO;SACxB,CAAC;IACJ,CAAC;IAEO,IAAI,CAAC,KAAsC;QACjD,SAAS,CAAC,IAAI,CAAC,OAAO,EAAE,KAAK,CAAC,CAAC;IACjC,CAAC;IAEO,OAAO,CACb,MAAsB,EACtB,KAAe,EACf,KAAc;QAEd,IAAI,CAAC,IAAI,CAAC;YACR,IAAI,EAAE,WAAW;YACjB,MAAM;YACN,SAAS,EAAE,KAAK,CAAC,SAAS;YAC1B,QAAQ,EAAE,gBAAgB,CAAC,KAAK,CAAC,QAAQ,CAAC;YAC1C,GAAG,CAAC,KAAK,CAAC,KAAK,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,KAAK,CAAC,KAAK,EAAE,CAAC;YAC5D,GAAG,CAAC,KAAK,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC;SAC1C,CAAC,CAAC;IACL,CAAC;CACF;AAED,4EAA4E;AAC5E,SAAS,SAAS,CAAC,MAA+B;IAChD,OAAO,MAAM,KAAK,SAAS,IAAI,MAAM,CAAC,OAAO,CAAC;AAChD,CAAC;AAED,SAAS,aAAa,CAAC,KAAe;IACpC,OAAO;QACL,MAAM,EAAE,SAAS;QACjB,QAAQ,EAAE,gBAAgB,CAAC,KAAK,CAAC,QAAQ,CAAC;QAC1C,SAAS,EAAE,KAAK,CAAC,SAAS;QAC1B,GAAG,CAAC,KAAK,CAAC,KAAK,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,KAAK,CAAC,KAAK,EAAE,CAAC;KAC7D,CAAC;AACJ,CAAC;AAED,SAAS,kBAAkB,CAAC,KAAsB;IAChD,MAAM,QAAQ,GAAG,IAAI,GAAG,EAAgB,CAAC;IACzC,KAAK,MAAM,IAAI,IAAI,KAAK,EAAE,CAAC;QACzB,IAAI,IAAI,CAAC,IAAI,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YAC3B,MAAM,IAAI,SAAS,CAAC,+BAA+B,CAAC,CAAC;QACvD,CAAC;QACD,IAAI,QAAQ,CAAC,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;YAC5B,MAAM,IAAI,SAAS,CAAC,wBAAwB,IAAI,CAAC,IAAI,EAAE,CAAC,CAAC;QAC3D,CAAC;QACD,QAAQ,CAAC,GAAG,CAAC,IAAI,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;IAChC,CAAC;IACD,OAAO,QAAQ,CAAC;AAClB,CAAC;AAED,SAAS,gCAAgC,CAAC,KAAc;IACtD,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,KAAK,IAAI,EAAE,CAAC;QAChD,MAAM,IAAI,kBAAkB,CAAC,iDAAiD,CAAC,CAAC;IAClF,CAAC;IAED,MAAM,SAAS,GAAG,KAAkC,CAAC;IACrD,IACE,SAAS,CAAC,IAAI,KAAK,WAAW;QAC9B,OAAO,SAAS,CAAC,OAAO,KAAK,QAAQ;QACrC,CAAC,KAAK,CAAC,OAAO,CAAC,SAAS,CAAC,SAAS,CAAC,EACnC,CAAC;QACD,MAAM,IAAI,kBAAkB,CAAC,iDAAiD,CAAC,CAAC;IAClF,CAAC;IAED,MAAM,SAAS,GAAG,SAAS,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC,QAAQ,EAAE,EAAE;QACrD,IACE,OAAO,QAAQ,KAAK,QAAQ;YAC5B,QAAQ,KAAK,IAAI;YACjB,OAAO,QAAQ,CAAC,EAAE,KAAK,QAAQ;YAC/B,QAAQ,CAAC,EAAE,CAAC,MAAM,KAAK,CAAC;YACxB,OAAO,QAAQ,CAAC,IAAI,KAAK,QAAQ;YACjC,QAAQ,CAAC,IAAI,CAAC,MAAM,KAAK,CAAC,EAC1B,CAAC;YACD,MAAM,IAAI,kBAAkB,CAAC,yCAAyC,CAAC,CAAC;QAC1E,CAAC;QAED,OAAO;YACL,EAAE,EAAE,QAAQ,CAAC,EAAE;YACf,IAAI,EAAE,QAAQ,CAAC,IAAI;YACnB,SAAS,EAAE,eAAe,CAAC,QAAQ,CAAC,SAAS,CAAC;SAC/C,CAAC;IACJ,CAAC,CAAC,CAAC;IAEH,IAAI,SAAS,CAAC,OAAO,CAAC,MAAM,KAAK,CAAC,IAAI,SAAS,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC7D,MAAM,IAAI,kBAAkB,CAC1B,oEAAoE,CACrE,CAAC;IACJ,CAAC;IAED,OAAO;QACL,IAAI,EAAE,WAAW;QACjB,OAAO,EAAE,SAAS,CAAC,OAAO;QAC1B,SAAS;KACV,CAAC;AACJ,CAAC;AAED,SAAS,kBAAkB,CAAC,KAAc;IACxC,IACE,OAAO,KAAK,KAAK,QAAQ;QACzB,KAAK,KAAK,IAAI;QACd,CAAC,CAAC,SAAS,IAAI,KAAK,CAAC;QACrB,CAAC,CAAC,SAAS,IAAI,KAAK,CAAC;QACrB,OAAO,KAAK,CAAC,OAAO,KAAK,QAAQ;QACjC,OAAO,KAAK,CAAC,OAAO,KAAK,SAAS,EAClC,CAAC;QACD,MAAM,IAAI,SAAS,CAAC,kCAAkC,CAAC,CAAC;IAC1D,CAAC;IACD,OAAO,EAAE,OAAO,EAAE,KAAK,CAAC,OAAO,EAAE,OAAO,EAAE,KAAK,CAAC,OAAO,EAAE,CAAC;AAC5D,CAAC;AAED,SAAS,gBAAgB,CAAC,QAA4B;IACpD,OAAO,eAAe,CAAC,QAAQ,CAAC,CAAC;AACnC,CAAC"}
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import type { LLMClient } from "../llm.js";
|
|
1
|
+
import type { LLMClient, Usage } from "../llm.js";
|
|
2
2
|
import type { Message } from "../messages.js";
|
|
3
3
|
export interface CompactionState {
|
|
4
4
|
readonly summary: string;
|
|
@@ -29,7 +29,16 @@ export declare class Compactor {
|
|
|
29
29
|
private readonly llm;
|
|
30
30
|
private readonly fileEffects;
|
|
31
31
|
constructor(llm: LLMClient, options?: CompactorOptions);
|
|
32
|
-
|
|
32
|
+
/**
|
|
33
|
+
* Summarising costs a Provider call of its own. It is returned alongside the
|
|
34
|
+
* state so the run's total includes it: compaction trades a call now for
|
|
35
|
+
* smaller inputs later, and that trade cannot be judged if half of it is
|
|
36
|
+
* invisible.
|
|
37
|
+
*/
|
|
38
|
+
compact(messages: readonly Message[], signal?: AbortSignal): Promise<{
|
|
39
|
+
readonly state: CompactionState;
|
|
40
|
+
readonly usage?: Usage;
|
|
41
|
+
}>;
|
|
33
42
|
private summarise;
|
|
34
43
|
}
|
|
35
44
|
/** Falls back to the raw text when the model does not return usable JSON. */
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"compaction.d.ts","sourceRoot":"","sources":["../../src/context/compaction.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,SAAS,EAAE,MAAM,WAAW,CAAC;
|
|
1
|
+
{"version":3,"file":"compaction.d.ts","sourceRoot":"","sources":["../../src/context/compaction.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,SAAS,EAAE,KAAK,EAAE,MAAM,WAAW,CAAC;AAClD,OAAO,KAAK,EAAE,OAAO,EAAY,MAAM,gBAAgB,CAAC;AAExD,MAAM,WAAW,eAAe;IAC9B,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;IACzB,4EAA4E;IAC5E,QAAQ,CAAC,SAAS,EAAE,SAAS,MAAM,EAAE,CAAC;IACtC,+EAA+E;IAC/E,QAAQ,CAAC,aAAa,EAAE,SAAS,MAAM,EAAE,CAAC;IAC1C,QAAQ,CAAC,SAAS,EAAE,SAAS,MAAM,EAAE,CAAC;IACtC,QAAQ,CAAC,YAAY,EAAE,SAAS,MAAM,EAAE,CAAC;CAC1C;AAED;;;;;;;GAOG;AACH,MAAM,WAAW,aAAa;IAC5B,QAAQ,CAAC,KAAK,EAAE,SAAS,MAAM,EAAE,CAAC;IAClC,QAAQ,CAAC,OAAO,EAAE,SAAS,MAAM,EAAE,CAAC;CACrC;AAED,eAAO,MAAM,oBAAoB,EAAE,aAGlC,CAAC;AAUF,MAAM,WAAW,gBAAgB;IAC/B,QAAQ,CAAC,WAAW,CAAC,EAAE,aAAa,CAAC;CACtC;AAED,qBAAa,SAAS;IAIlB,OAAO,CAAC,QAAQ,CAAC,GAAG;IAHtB,OAAO,CAAC,QAAQ,CAAC,WAAW,CAAgB;IAE5C,YACmB,GAAG,EAAE,SAAS,EAC/B,OAAO,GAAE,gBAAqB,EAG/B;IAED;;;;;OAKG;IACG,OAAO,CACX,QAAQ,EAAE,SAAS,OAAO,EAAE,EAC5B,MAAM,CAAC,EAAE,WAAW,GACnB,OAAO,CAAC;QAAE,QAAQ,CAAC,KAAK,EAAE,eAAe,CAAC;QAAC,QAAQ,CAAC,KAAK,CAAC,EAAE,KAAK,CAAA;KAAE,CAAC,CAOtE;YAEa,SAAS;CAkBxB;AAED,6EAA6E;AAC7E,wBAAgB,YAAY,CAC1B,OAAO,EAAE,MAAM,GACd,IAAI,CAAC,eAAe,EAAE,SAAS,GAAG,WAAW,GAAG,cAAc,CAAC,CAiCjE;AAsED,gFAAgF;AAChF,wBAAgB,qBAAqB,CAAC,KAAK,EAAE,eAAe,GAAG,MAAM,CAiBpE"}
|
|
@@ -16,10 +16,19 @@ export class Compactor {
|
|
|
16
16
|
this.llm = llm;
|
|
17
17
|
this.fileEffects = options.fileEffects ?? DEFAULT_FILE_EFFECTS;
|
|
18
18
|
}
|
|
19
|
+
/**
|
|
20
|
+
* Summarising costs a Provider call of its own. It is returned alongside the
|
|
21
|
+
* state so the run's total includes it: compaction trades a call now for
|
|
22
|
+
* smaller inputs later, and that trade cannot be judged if half of it is
|
|
23
|
+
* invisible.
|
|
24
|
+
*/
|
|
19
25
|
async compact(messages, signal) {
|
|
20
26
|
const files = collectFiles(messages, this.fileEffects);
|
|
21
|
-
const prose = await this.summarise(messages, signal);
|
|
22
|
-
return {
|
|
27
|
+
const { prose, usage } = await this.summarise(messages, signal);
|
|
28
|
+
return {
|
|
29
|
+
state: { ...prose, ...files },
|
|
30
|
+
...(usage === undefined ? {} : { usage }),
|
|
31
|
+
};
|
|
23
32
|
}
|
|
24
33
|
async summarise(messages, signal) {
|
|
25
34
|
const response = await this.llm.complete({
|
|
@@ -28,7 +37,10 @@ export class Compactor {
|
|
|
28
37
|
tools: [],
|
|
29
38
|
...(signal === undefined ? {} : { signal }),
|
|
30
39
|
});
|
|
31
|
-
return
|
|
40
|
+
return {
|
|
41
|
+
prose: parseSummary(response.message.content),
|
|
42
|
+
...(response.usage === undefined ? {} : { usage: response.usage }),
|
|
43
|
+
};
|
|
32
44
|
}
|
|
33
45
|
}
|
|
34
46
|
/** Falls back to the raw text when the model does not return usable JSON. */
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"compaction.js","sourceRoot":"","sources":["../../src/context/compaction.ts"],"names":[],"mappings":"AA0BA,MAAM,CAAC,MAAM,oBAAoB,GAAkB;IACjD,KAAK,EAAE,CAAC,WAAW,CAAC;IACpB,OAAO,EAAE,CAAC,YAAY,EAAE,WAAW,CAAC;CACrC,CAAC;AAEF,MAAM,oBAAoB,GAAG;;;;;;8CAMiB,CAAC;AAM/C,MAAM,OAAO,SAAS;IAID,GAAG;IAHL,WAAW,CAAgB;IAE5C,YACmB,GAAc,EAC/B,OAAO,GAAqB,EAAE;mBADb,GAAG;QAGpB,IAAI,CAAC,WAAW,GAAG,OAAO,CAAC,WAAW,IAAI,oBAAoB,CAAC;IACjE,CAAC;IAED,KAAK,CAAC,OAAO,CACX,QAA4B,EAC5B,MAAoB;QAEpB,MAAM,KAAK,GAAG,YAAY,CAAC,QAAQ,EAAE,IAAI,CAAC,WAAW,CAAC,CAAC;QACvD,MAAM,KAAK,GAAG,MAAM,IAAI,CAAC,SAAS,CAAC,QAAQ,EAAE,MAAM,CAAC,CAAC;
|
|
1
|
+
{"version":3,"file":"compaction.js","sourceRoot":"","sources":["../../src/context/compaction.ts"],"names":[],"mappings":"AA0BA,MAAM,CAAC,MAAM,oBAAoB,GAAkB;IACjD,KAAK,EAAE,CAAC,WAAW,CAAC;IACpB,OAAO,EAAE,CAAC,YAAY,EAAE,WAAW,CAAC;CACrC,CAAC;AAEF,MAAM,oBAAoB,GAAG;;;;;;8CAMiB,CAAC;AAM/C,MAAM,OAAO,SAAS;IAID,GAAG;IAHL,WAAW,CAAgB;IAE5C,YACmB,GAAc,EAC/B,OAAO,GAAqB,EAAE;mBADb,GAAG;QAGpB,IAAI,CAAC,WAAW,GAAG,OAAO,CAAC,WAAW,IAAI,oBAAoB,CAAC;IACjE,CAAC;IAED;;;;;OAKG;IACH,KAAK,CAAC,OAAO,CACX,QAA4B,EAC5B,MAAoB;QAEpB,MAAM,KAAK,GAAG,YAAY,CAAC,QAAQ,EAAE,IAAI,CAAC,WAAW,CAAC,CAAC;QACvD,MAAM,EAAE,KAAK,EAAE,KAAK,EAAE,GAAG,MAAM,IAAI,CAAC,SAAS,CAAC,QAAQ,EAAE,MAAM,CAAC,CAAC;QAChE,OAAO;YACL,KAAK,EAAE,EAAE,GAAG,KAAK,EAAE,GAAG,KAAK,EAAE;YAC7B,GAAG,CAAC,KAAK,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC;SAC1C,CAAC;IACJ,CAAC;IAEO,KAAK,CAAC,SAAS,CACrB,QAA4B,EAC5B,MAAoB;QAKpB,MAAM,QAAQ,GAAG,MAAM,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAC;YACvC,YAAY,EAAE,oBAAoB;YAClC,QAAQ,EAAE,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,gBAAgB,CAAC,QAAQ,CAAC,EAAE,CAAC;YACjE,KAAK,EAAE,EAAE;YACT,GAAG,CAAC,MAAM,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC;SAC5C,CAAC,CAAC;QACH,OAAO;YACL,KAAK,EAAE,YAAY,CAAC,QAAQ,CAAC,OAAO,CAAC,OAAO,CAAC;YAC7C,GAAG,CAAC,QAAQ,CAAC,KAAK,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,QAAQ,CAAC,KAAK,EAAE,CAAC;SACnE,CAAC;IACJ,CAAC;CACF;AAED,6EAA6E;AAC7E,MAAM,UAAU,YAAY,CAC1B,OAAe;IAEf,MAAM,QAAQ,GAAG;QACf,OAAO,EAAE,OAAO,CAAC,IAAI,EAAE;QACvB,SAAS,EAAE,EAAuB;QAClC,YAAY,EAAE,EAAuB;KACtC,CAAC;IAEF,MAAM,KAAK,GAAG,OAAO,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC;IACnC,MAAM,GAAG,GAAG,OAAO,CAAC,WAAW,CAAC,GAAG,CAAC,CAAC;IACrC,IAAI,KAAK,KAAK,CAAC,CAAC,IAAI,GAAG,IAAI,KAAK,EAAE,CAAC;QACjC,OAAO,QAAQ,CAAC;IAClB,CAAC;IAED,IAAI,MAAe,CAAC;IACpB,IAAI,CAAC;QACH,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,KAAK,EAAE,GAAG,GAAG,CAAC,CAAC,CAAC,CAAC;IACrD,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,QAAQ,CAAC;IAClB,CAAC;IACD,IAAI,OAAO,MAAM,KAAK,QAAQ,IAAI,MAAM,KAAK,IAAI,IAAI,KAAK,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,CAAC;QAC3E,OAAO,QAAQ,CAAC;IAClB,CAAC;IAED,MAAM,MAAM,GAAG,MAAiC,CAAC;IACjD,MAAM,OAAO,GACX,OAAO,MAAM,CAAC,OAAO,KAAK,QAAQ,IAAI,MAAM,CAAC,OAAO,CAAC,IAAI,EAAE,CAAC,MAAM,GAAG,CAAC;QACpE,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,IAAI,EAAE;QACvB,CAAC,CAAC,QAAQ,CAAC,OAAO,CAAC;IACvB,OAAO;QACL,OAAO;QACP,SAAS,EAAE,UAAU,CAAC,MAAM,CAAC,SAAS,CAAC;QACvC,YAAY,EAAE,UAAU,CAAC,MAAM,CAAC,YAAY,CAAC;KAC9C,CAAC;AACJ,CAAC;AAED,SAAS,UAAU,CAAC,KAAc;IAChC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,EAAE,CAAC;QAC1B,OAAO,EAAE,CAAC;IACZ,CAAC;IACD,OAAO,KAAK,CAAC,MAAM,CACjB,CAAC,KAAK,EAAmB,EAAE,CAAC,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,CAAC,MAAM,GAAG,CAAC,CAC1E,CAAC;AACJ,CAAC;AAED,SAAS,YAAY,CACnB,QAA4B,EAC5B,OAAsB;IAEtB,MAAM,SAAS,GAAG,IAAI,GAAG,EAAU,CAAC;IACpC,MAAM,aAAa,GAAG,IAAI,GAAG,EAAU,CAAC;IAExC,KAAK,MAAM,OAAO,IAAI,QAAQ,EAAE,CAAC;QAC/B,IAAI,OAAO,CAAC,IAAI,KAAK,WAAW,EAAE,CAAC;YACjC,SAAS;QACX,CAAC;QACD,KAAK,MAAM,IAAI,IAAI,OAAO,CAAC,SAAS,EAAE,CAAC;YACrC,MAAM,QAAQ,GAAG,YAAY,CAAC,IAAI,CAAC,CAAC;YACpC,IAAI,QAAQ,KAAK,SAAS,EAAE,CAAC;gBAC3B,SAAS;YACX,CAAC;YACD,IAAI,OAAO,CAAC,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;gBACxC,aAAa,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC;YAC9B,CAAC;iBAAM,IAAI,OAAO,CAAC,KAAK,CAAC,QAAQ,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;gBAC7C,SAAS,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC;YAC1B,CAAC;QACH,CAAC;IACH,CAAC;IAED,wEAAwE;IACxE,KAAK,MAAM,QAAQ,IAAI,aAAa,EAAE,CAAC;QACrC,SAAS,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;IAC7B,CAAC;IACD,OAAO;QACL,SAAS,EAAE,CAAC,GAAG,SAAS,CAAC,CAAC,IAAI,EAAE;QAChC,aAAa,EAAE,CAAC,GAAG,aAAa,CAAC,CAAC,IAAI,EAAE;KACzC,CAAC;AACJ,CAAC;AAED,SAAS,YAAY,CAAC,IAAc;IAClC,IAAI,OAAO,IAAI,CAAC,SAAS,KAAK,QAAQ,IAAI,IAAI,CAAC,SAAS,KAAK,IAAI,EAAE,CAAC;QAClE,OAAO,SAAS,CAAC;IACnB,CAAC;IACD,MAAM,KAAK,GAAI,IAAI,CAAC,SAAqC,CAAC,IAAI,CAAC;IAC/D,OAAO,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS,CAAC;AAC3E,CAAC;AAED,SAAS,gBAAgB,CAAC,QAA4B;IACpD,OAAO,QAAQ;SACZ,GAAG,CAAC,CAAC,OAAO,EAAE,EAAE;QACf,IAAI,OAAO,CAAC,IAAI,KAAK,WAAW,EAAE,CAAC;YACjC,MAAM,KAAK,GAAG,OAAO,CAAC,SAAS;iBAC5B,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,iBAAiB,IAAI,CAAC,IAAI,IAAI,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,CAAC;iBAC7E,IAAI,CAAC,EAAE,CAAC,CAAC;YACZ,OAAO,cAAc,OAAO,CAAC,OAAO,GAAG,KAAK,EAAE,CAAC;QACjD,CAAC;QACD,IAAI,OAAO,CAAC,IAAI,KAAK,MAAM,EAAE,CAAC;YAC5B,OAAO,iBAAiB,OAAO,CAAC,QAAQ,GAAG,OAAO,CAAC,OAAO,CAAC,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,EAAE,KAAK,OAAO,CAAC,OAAO,EAAE,CAAC;QACrG,CAAC;QACD,OAAO,SAAS,OAAO,CAAC,OAAO,EAAE,CAAC;IACpC,CAAC,CAAC;SACD,IAAI,CAAC,MAAM,CAAC,CAAC;AAClB,CAAC;AAED,gFAAgF;AAChF,MAAM,UAAU,qBAAqB,CAAC,KAAsB;IAC1D,MAAM,QAAQ,GAAG,CAAC,6BAA6B,KAAK,CAAC,OAAO,EAAE,CAAC,CAAC;IAChE,IAAI,KAAK,CAAC,SAAS,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC/B,QAAQ,CAAC,IAAI,CAAC,gBAAgB,KAAK,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACnF,CAAC;IACD,IAAI,KAAK,CAAC,aAAa,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACnC,QAAQ,CAAC,IAAI,CACX,oBAAoB,KAAK,CAAC,aAAa,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAC1E,CAAC;IACJ,CAAC;IACD,IAAI,KAAK,CAAC,SAAS,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC/B,QAAQ,CAAC,IAAI,CAAC,0BAA0B,KAAK,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IAC7F,CAAC;IACD,IAAI,KAAK,CAAC,YAAY,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAClC,QAAQ,CAAC,IAAI,CAAC,mBAAmB,KAAK,CAAC,YAAY,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACzF,CAAC;IACD,OAAO,QAAQ,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;AAC/B,CAAC"}
|
|
@@ -1,4 +1,5 @@
|
|
|
1
1
|
import type { Message } from "../messages.js";
|
|
2
|
+
import type { Usage } from "../llm.js";
|
|
2
3
|
import { Compactor, type CompactionState } from "./compaction.js";
|
|
3
4
|
import { type TokenEstimator } from "./token-estimator.js";
|
|
4
5
|
export declare const DEFAULT_CONTEXT_WINDOW = 128000;
|
|
@@ -43,6 +44,8 @@ export interface BuiltContext {
|
|
|
43
44
|
readonly estimatedTokens: number;
|
|
44
45
|
/** Carry this into the next build to reuse the summary. */
|
|
45
46
|
readonly compaction?: AppliedCompaction;
|
|
47
|
+
/** What summarising cost, so the run's total can include it. */
|
|
48
|
+
readonly usage?: Usage;
|
|
46
49
|
}
|
|
47
50
|
/**
|
|
48
51
|
* Decides what the model sees for one request.
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"context-manager.d.ts","sourceRoot":"","sources":["../../src/context/context-manager.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,gBAAgB,CAAC;AAC9C,OAAO,EACL,SAAS,EAET,KAAK,eAAe,EACrB,MAAM,iBAAiB,CAAC;AACzB,OAAO,EAEL,KAAK,cAAc,EACpB,MAAM,sBAAsB,CAAC;AAE9B,eAAO,MAAM,sBAAsB,SAAU,CAAC;AAC9C,eAAO,MAAM,sBAAsB,QAAS,CAAC;AAC7C,eAAO,MAAM,0BAA0B,QAAS,CAAC;AAEjD,MAAM,WAAW,qBAAqB;IACpC,QAAQ,CAAC,aAAa,CAAC,EAAE,MAAM,CAAC;IAChC,wEAAwE;IACxE,QAAQ,CAAC,aAAa,CAAC,EAAE,MAAM,CAAC;IAChC,2EAA2E;IAC3E,QAAQ,CAAC,gBAAgB,CAAC,EAAE,MAAM,CAAC;IACnC,QAAQ,CAAC,SAAS,CAAC,EAAE,cAAc,CAAC;IACpC,yEAAyE;IACzE,QAAQ,CAAC,SAAS,CAAC,EAAE,SAAS,CAAC;IAC/B,QAAQ,CAAC,YAAY,CAAC,EAAE,CAAC,KAAK,EAAE,eAAe,KAAK,IAAI,CAAC;CAC1D;AAED,MAAM,WAAW,eAAe;IAC9B,QAAQ,CAAC,eAAe,EAAE,MAAM,CAAC;IACjC,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAC;IAC9B,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC;IAC7B,QAAQ,CAAC,KAAK,EAAE,eAAe,CAAC;CACjC;AAED,MAAM,WAAW,iBAAiB;IAChC,QAAQ,CAAC,KAAK,EAAE,eAAe,CAAC;IAChC,qEAAqE;IACrE,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAC;CAC7B;AAED,MAAM,WAAW,YAAY;IAC3B,2EAA2E;IAC3E,QAAQ,CAAC,QAAQ,CAAC,EAAE,iBAAiB,CAAC;IACtC,QAAQ,CAAC,MAAM,CAAC,EAAE,WAAW,CAAC;CAC/B;AAED,MAAM,WAAW,YAAY;IAC3B,QAAQ,CAAC,QAAQ,EAAE,SAAS,OAAO,EAAE,CAAC;IACtC;;;;;OAKG;IACH,QAAQ,CAAC,SAAS,EAAE,OAAO,CAAC;IAC5B,QAAQ,CAAC,eAAe,EAAE,MAAM,CAAC;IACjC,2DAA2D;IAC3D,QAAQ,CAAC,UAAU,CAAC,EAAE,iBAAiB,CAAC;
|
|
1
|
+
{"version":3,"file":"context-manager.d.ts","sourceRoot":"","sources":["../../src/context/context-manager.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,gBAAgB,CAAC;AAC9C,OAAO,KAAK,EAAE,KAAK,EAAE,MAAM,WAAW,CAAC;AACvC,OAAO,EACL,SAAS,EAET,KAAK,eAAe,EACrB,MAAM,iBAAiB,CAAC;AACzB,OAAO,EAEL,KAAK,cAAc,EACpB,MAAM,sBAAsB,CAAC;AAE9B,eAAO,MAAM,sBAAsB,SAAU,CAAC;AAC9C,eAAO,MAAM,sBAAsB,QAAS,CAAC;AAC7C,eAAO,MAAM,0BAA0B,QAAS,CAAC;AAEjD,MAAM,WAAW,qBAAqB;IACpC,QAAQ,CAAC,aAAa,CAAC,EAAE,MAAM,CAAC;IAChC,wEAAwE;IACxE,QAAQ,CAAC,aAAa,CAAC,EAAE,MAAM,CAAC;IAChC,2EAA2E;IAC3E,QAAQ,CAAC,gBAAgB,CAAC,EAAE,MAAM,CAAC;IACnC,QAAQ,CAAC,SAAS,CAAC,EAAE,cAAc,CAAC;IACpC,yEAAyE;IACzE,QAAQ,CAAC,SAAS,CAAC,EAAE,SAAS,CAAC;IAC/B,QAAQ,CAAC,YAAY,CAAC,EAAE,CAAC,KAAK,EAAE,eAAe,KAAK,IAAI,CAAC;CAC1D;AAED,MAAM,WAAW,eAAe;IAC9B,QAAQ,CAAC,eAAe,EAAE,MAAM,CAAC;IACjC,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAC;IAC9B,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC;IAC7B,QAAQ,CAAC,KAAK,EAAE,eAAe,CAAC;CACjC;AAED,MAAM,WAAW,iBAAiB;IAChC,QAAQ,CAAC,KAAK,EAAE,eAAe,CAAC;IAChC,qEAAqE;IACrE,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAC;CAC7B;AAED,MAAM,WAAW,YAAY;IAC3B,2EAA2E;IAC3E,QAAQ,CAAC,QAAQ,CAAC,EAAE,iBAAiB,CAAC;IACtC,QAAQ,CAAC,MAAM,CAAC,EAAE,WAAW,CAAC;CAC/B;AAED,MAAM,WAAW,YAAY;IAC3B,QAAQ,CAAC,QAAQ,EAAE,SAAS,OAAO,EAAE,CAAC;IACtC;;;;;OAKG;IACH,QAAQ,CAAC,SAAS,EAAE,OAAO,CAAC;IAC5B,QAAQ,CAAC,eAAe,EAAE,MAAM,CAAC;IACjC,2DAA2D;IAC3D,QAAQ,CAAC,UAAU,CAAC,EAAE,iBAAiB,CAAC;IACxC,gEAAgE;IAChE,QAAQ,CAAC,KAAK,CAAC,EAAE,KAAK,CAAC;CACxB;AAED;;;;;GAKG;AACH,qBAAa,cAAc;IACzB,OAAO,CAAC,QAAQ,CAAC,aAAa,CAAS;IACvC,OAAO,CAAC,QAAQ,CAAC,aAAa,CAAS;IACvC,OAAO,CAAC,QAAQ,CAAC,gBAAgB,CAAS;IAC1C,OAAO,CAAC,QAAQ,CAAC,SAAS,CAAiB;IAC3C,OAAO,CAAC,QAAQ,CAAC,SAAS,CAAC,CAAY;IACvC,OAAO,CAAC,QAAQ,CAAC,YAAY,CAAC,CAAmC;IAEjE,YAAY,OAAO,GAAE,qBAA0B,EAoB9C;IAED,IAAI,MAAM,IAAI,MAAM,CAEnB;IAEK,KAAK,CACT,QAAQ,EAAE,SAAS,OAAO,EAAE,EAC5B,OAAO,GAAE,YAAiB,GACzB,OAAO,CAAC,YAAY,CAAC,CAwDvB;CACF;AAMD;;;;;;GAMG;AACH,wBAAgB,cAAc,CAC5B,QAAQ,EAAE,SAAS,OAAO,EAAE,EAC5B,SAAS,EAAE,cAAc,EACzB,gBAAgB,EAAE,MAAM,GACvB,MAAM,CA4BR"}
|
|
@@ -67,7 +67,7 @@ export class ContextManager {
|
|
|
67
67
|
}
|
|
68
68
|
const head = effective.slice(0, cut);
|
|
69
69
|
const tail = effective.slice(cut);
|
|
70
|
-
const state = await this.compactor.compact(head, options.signal);
|
|
70
|
+
const { state, usage } = await this.compactor.compact(head, options.signal);
|
|
71
71
|
const compactedMessages = [summaryMessage(state), ...tail];
|
|
72
72
|
const tokensAfter = this.estimator.estimateMessages(compactedMessages);
|
|
73
73
|
// Map the cut back onto the full transcript, allowing for the summary
|
|
@@ -86,6 +86,7 @@ export class ContextManager {
|
|
|
86
86
|
compacted: true,
|
|
87
87
|
estimatedTokens: tokensAfter,
|
|
88
88
|
compaction: { state, splitIndex },
|
|
89
|
+
...(usage === undefined ? {} : { usage }),
|
|
89
90
|
};
|
|
90
91
|
}
|
|
91
92
|
}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"context-manager.js","sourceRoot":"","sources":["../../src/context/context-manager.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"context-manager.js","sourceRoot":"","sources":["../../src/context/context-manager.ts"],"names":[],"mappings":"AAEA,OAAO,EACL,SAAS,EACT,qBAAqB,GAEtB,MAAM,iBAAiB,CAAC;AACzB,OAAO,EACL,uBAAuB,GAExB,MAAM,sBAAsB,CAAC;AAE9B,MAAM,CAAC,MAAM,sBAAsB,GAAG,OAAO,CAAC;AAC9C,MAAM,CAAC,MAAM,sBAAsB,GAAG,MAAM,CAAC;AAC7C,MAAM,CAAC,MAAM,0BAA0B,GAAG,MAAM,CAAC;AAiDjD;;;;;GAKG;AACH,MAAM,OAAO,cAAc;IACR,aAAa,CAAS;IACtB,aAAa,CAAS;IACtB,gBAAgB,CAAS;IACzB,SAAS,CAAiB;IAC1B,SAAS,CAAa;IACtB,YAAY,CAAoC;IAEjE,YAAY,OAAO,GAA0B,EAAE;QAC7C,IAAI,CAAC,aAAa,GAAG,OAAO,CAAC,aAAa,IAAI,sBAAsB,CAAC;QACrE,yEAAyE;QACzE,+DAA+D;QAC/D,IAAI,CAAC,aAAa;YAChB,OAAO,CAAC,aAAa;gBACrB,IAAI,CAAC,GAAG,CAAC,sBAAsB,EAAE,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,aAAa,GAAG,CAAC,CAAC,CAAC,CAAC;QACvE,IAAI,CAAC,gBAAgB;YACnB,OAAO,CAAC,gBAAgB;gBACxB,IAAI,CAAC,GAAG,CAAC,0BAA0B,EAAE,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,aAAa,GAAG,CAAC,CAAC,CAAC,CAAC;QAC3E,IAAI,CAAC,SAAS,GAAG,OAAO,CAAC,SAAS,IAAI,IAAI,uBAAuB,EAAE,CAAC;QACpE,IAAI,OAAO,CAAC,SAAS,KAAK,SAAS,EAAE,CAAC;YACpC,IAAI,CAAC,SAAS,GAAG,OAAO,CAAC,SAAS,CAAC;QACrC,CAAC;QACD,IAAI,OAAO,CAAC,YAAY,KAAK,SAAS,EAAE,CAAC;YACvC,IAAI,CAAC,YAAY,GAAG,OAAO,CAAC,YAAY,CAAC;QAC3C,CAAC;QACD,IAAI,IAAI,CAAC,aAAa,IAAI,IAAI,CAAC,aAAa,EAAE,CAAC;YAC7C,MAAM,IAAI,SAAS,CAAC,mDAAmD,CAAC,CAAC;QAC3E,CAAC;IACH,CAAC;IAED,IAAI,MAAM;QACR,OAAO,IAAI,CAAC,aAAa,GAAG,IAAI,CAAC,aAAa,CAAC;IACjD,CAAC;IAED,KAAK,CAAC,KAAK,CACT,QAA4B,EAC5B,OAAO,GAAiB,EAAE;QAE1B,MAAM,QAAQ,GAAG,OAAO,CAAC,QAAQ,CAAC;QAClC,MAAM,SAAS,GACb,QAAQ,KAAK,SAAS;YACpB,CAAC,CAAC,QAAQ;YACV,CAAC,CAAC,CAAC,cAAc,CAAC,QAAQ,CAAC,KAAK,CAAC,EAAE,GAAG,QAAQ,CAAC,KAAK,CAAC,QAAQ,CAAC,UAAU,CAAC,CAAC,CAAC;QAC/E,MAAM,eAAe,GAAG,IAAI,CAAC,SAAS,CAAC,gBAAgB,CAAC,SAAS,CAAC,CAAC;QAEnE,IAAI,IAAI,CAAC,SAAS,KAAK,SAAS,IAAI,eAAe,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;YACnE,OAAO;gBACL,QAAQ,EAAE,SAAS;gBACnB,SAAS,EAAE,KAAK;gBAChB,eAAe;gBACf,GAAG,CAAC,QAAQ,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,UAAU,EAAE,QAAQ,EAAE,CAAC;aAC5D,CAAC;QACJ,CAAC;QAED,MAAM,GAAG,GAAG,cAAc,CAAC,SAAS,EAAE,IAAI,CAAC,SAAS,EAAE,IAAI,CAAC,gBAAgB,CAAC,CAAC;QAC7E,4EAA4E;QAC5E,oEAAoE;QACpE,IAAI,GAAG,IAAI,CAAC,IAAI,CAAC,QAAQ,KAAK,SAAS,IAAI,GAAG,GAAG,CAAC,CAAC,EAAE,CAAC;YACpD,OAAO;gBACL,QAAQ,EAAE,SAAS;gBACnB,SAAS,EAAE,KAAK;gBAChB,eAAe;gBACf,GAAG,CAAC,QAAQ,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,UAAU,EAAE,QAAQ,EAAE,CAAC;aAC5D,CAAC;QACJ,CAAC;QAED,MAAM,IAAI,GAAG,SAAS,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC;QACrC,MAAM,IAAI,GAAG,SAAS,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;QAClC,MAAM,EAAE,KAAK,EAAE,KAAK,EAAE,GAAG,MAAM,IAAI,CAAC,SAAS,CAAC,OAAO,CAAC,IAAI,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC;QAC5E,MAAM,iBAAiB,GAAG,CAAC,cAAc,CAAC,KAAK,CAAC,EAAE,GAAG,IAAI,CAAC,CAAC;QAC3D,MAAM,WAAW,GAAG,IAAI,CAAC,SAAS,CAAC,gBAAgB,CAAC,iBAAiB,CAAC,CAAC;QAEvE,sEAAsE;QACtE,8DAA8D;QAC9D,MAAM,UAAU,GACd,QAAQ,KAAK,SAAS;YACpB,CAAC,CAAC,GAAG;YACL,CAAC,CAAC,QAAQ,CAAC,UAAU,GAAG,CAAC,GAAG,GAAG,CAAC,CAAC,CAAC;QAEtC,IAAI,CAAC,YAAY,EAAE,CAAC;YAClB,eAAe,EAAE,IAAI,CAAC,MAAM;YAC5B,YAAY,EAAE,eAAe;YAC7B,WAAW;YACX,KAAK;SACN,CAAC,CAAC;QAEH,OAAO;YACL,QAAQ,EAAE,iBAAiB;YAC3B,SAAS,EAAE,IAAI;YACf,eAAe,EAAE,WAAW;YAC5B,UAAU,EAAE,EAAE,KAAK,EAAE,UAAU,EAAE;YACjC,GAAG,CAAC,KAAK,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC;SAC1C,CAAC;IACJ,CAAC;CACF;AAED,SAAS,cAAc,CAAC,KAAsB;IAC5C,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,qBAAqB,CAAC,KAAK,CAAC,EAAE,CAAC;AACjE,CAAC;AAED;;;;;;GAMG;AACH,MAAM,UAAU,cAAc,CAC5B,QAA4B,EAC5B,SAAyB,EACzB,gBAAwB;IAExB,IAAI,MAAM,GAAG,CAAC,CAAC;IACf,IAAI,KAAK,GAAG,QAAQ,CAAC,MAAM,CAAC;IAC5B,OAAO,KAAK,GAAG,CAAC,EAAE,CAAC;QACjB,MAAM,OAAO,GAAG,QAAQ,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC;QACpC,IAAI,OAAO,KAAK,SAAS,EAAE,CAAC;YAC1B,MAAM;QACR,CAAC;QACD,MAAM,IAAI,GAAG,SAAS,CAAC,eAAe,CAAC,OAAO,CAAC,CAAC;QAChD,IAAI,MAAM,GAAG,IAAI,GAAG,gBAAgB,EAAE,CAAC;YACrC,MAAM;QACR,CAAC;QACD,MAAM,IAAI,IAAI,CAAC;QACf,KAAK,IAAI,CAAC,CAAC;IACb,CAAC;IAED,OAAO,KAAK,GAAG,QAAQ,CAAC,MAAM,IAAI,QAAQ,CAAC,KAAK,CAAC,EAAE,IAAI,KAAK,MAAM,EAAE,CAAC;QACnE,KAAK,IAAI,CAAC,CAAC;IACb,CAAC;IAED,4DAA4D;IAC5D,IAAI,KAAK,IAAI,QAAQ,CAAC,MAAM,EAAE,CAAC;QAC7B,KAAK,GAAG,QAAQ,CAAC,MAAM,GAAG,CAAC,CAAC;QAC5B,OAAO,KAAK,GAAG,CAAC,IAAI,QAAQ,CAAC,KAAK,CAAC,EAAE,IAAI,KAAK,MAAM,EAAE,CAAC;YACrD,KAAK,IAAI,CAAC,CAAC;QACb,CAAC;IACH,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC"}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"report.d.ts","sourceRoot":"","sources":["../../src/evals/report.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;
|
|
1
|
+
{"version":3,"file":"report.d.ts","sourceRoot":"","sources":["../../src/evals/report.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AAG9C,MAAM,WAAW,UAAU;IACzB,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;IACzB,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;CAC5B;AA4CD;;;;;GAKG;AACH,wBAAgB,WAAW,CAAC,OAAO,EAAE,SAAS,UAAU,EAAE,GAAG,MAAM,CA0ClE;AAED,wEAAwE;AACxE,wBAAgB,cAAc,CAAC,OAAO,EAAE,SAAS,UAAU,EAAE,GAAG,MAAM,CAgBrE;AAED,wBAAgB,YAAY,CAC1B,OAAO,EAAE,SAAS,UAAU,EAAE,EAC9B,IAAI,EAAE,UAAU,GACf,MAAM,CAkCR"}
|
package/dist/evals/report.js
CHANGED
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
import { formatTokens } from "../providers/usage.js";
|
|
1
2
|
function median(values) {
|
|
2
3
|
if (values.length === 0) {
|
|
3
4
|
return 0;
|
|
@@ -14,6 +15,17 @@ function mean(values) {
|
|
|
14
15
|
}
|
|
15
16
|
return values.reduce((total, value) => total + value, 0) / values.length;
|
|
16
17
|
}
|
|
18
|
+
/** Blank rather than zero when the Provider reported nothing. */
|
|
19
|
+
function medianTokens(result) {
|
|
20
|
+
const totals = result.attempts
|
|
21
|
+
.map((attempt) => attempt.usage?.usage.totalTokens)
|
|
22
|
+
.filter((value) => value !== undefined);
|
|
23
|
+
return totals.length === 0 ? "-" : formatTokens(median(totals));
|
|
24
|
+
}
|
|
25
|
+
function totalTokens(results) {
|
|
26
|
+
return results.reduce((sum, result) => sum +
|
|
27
|
+
result.attempts.reduce((inner, attempt) => inner + (attempt.usage?.usage.totalTokens ?? 0), 0), 0);
|
|
28
|
+
}
|
|
17
29
|
function pad(value, width) {
|
|
18
30
|
return value.length >= width ? value : value + " ".repeat(width - value.length);
|
|
19
31
|
}
|
|
@@ -31,6 +43,7 @@ export function formatTable(results) {
|
|
|
31
43
|
task: result.task,
|
|
32
44
|
pass: `${result.passed}/${result.total}`,
|
|
33
45
|
turns: mean(result.attempts.map((attempt) => attempt.turnCount)).toFixed(1),
|
|
46
|
+
tokens: medianTokens(result),
|
|
34
47
|
tools: [
|
|
35
48
|
...new Set(result.attempts.flatMap((attempt) => attempt.toolsUsed)),
|
|
36
49
|
].join(","),
|
|
@@ -40,19 +53,21 @@ export function formatTable(results) {
|
|
|
40
53
|
task: Math.max(4, ...rows.map((row) => row.task.length)),
|
|
41
54
|
pass: Math.max(4, ...rows.map((row) => row.pass.length)),
|
|
42
55
|
turns: Math.max(5, ...rows.map((row) => row.turns.length)),
|
|
56
|
+
tokens: Math.max(6, ...rows.map((row) => row.tokens.length)),
|
|
43
57
|
tools: Math.max(5, ...rows.map((row) => row.tools.length)),
|
|
44
58
|
};
|
|
45
59
|
const lines = [
|
|
46
|
-
`${pad("task", widths.task)} ${pad("pass", widths.pass)} ${pad("turns", widths.turns)} ${pad("tools", widths.tools)} p50`,
|
|
60
|
+
`${pad("task", widths.task)} ${pad("pass", widths.pass)} ${pad("turns", widths.turns)} ${pad("tokens", widths.tokens)} ${pad("tools", widths.tools)} p50`,
|
|
47
61
|
];
|
|
48
62
|
for (const row of rows) {
|
|
49
|
-
lines.push(`${pad(row.task, widths.task)} ${pad(row.pass, widths.pass)} ${pad(row.turns, widths.turns)} ${pad(row.tools, widths.tools)} ${row.time}`);
|
|
63
|
+
lines.push(`${pad(row.task, widths.task)} ${pad(row.pass, widths.pass)} ${pad(row.turns, widths.turns)} ${pad(row.tokens, widths.tokens)} ${pad(row.tools, widths.tools)} ${row.time}`);
|
|
50
64
|
}
|
|
51
65
|
const passed = results.reduce((total, result) => total + result.passed, 0);
|
|
52
66
|
const total = results.reduce((sum, result) => sum + result.total, 0);
|
|
53
67
|
const percent = total === 0 ? 0 : Math.round((passed / total) * 100);
|
|
68
|
+
const spent = totalTokens(results);
|
|
54
69
|
lines.push("");
|
|
55
|
-
lines.push(`overall ${passed}/${total} (${percent}%)`);
|
|
70
|
+
lines.push(`overall ${passed}/${total} (${percent}%)${spent === 0 ? "" : ` ${formatTokens(spent)} tokens`}`);
|
|
56
71
|
return `${lines.join("\n")}\n`;
|
|
57
72
|
}
|
|
58
73
|
/** Failures, grouped so the same reason across attempts is one line. */
|
|
@@ -83,6 +98,7 @@ export function toJsonReport(results, meta) {
|
|
|
83
98
|
total: result.total,
|
|
84
99
|
passRate: result.total === 0 ? 0 : result.passed / result.total,
|
|
85
100
|
meanTurns: mean(result.attempts.map((attempt) => attempt.turnCount)),
|
|
101
|
+
totalTokens: result.attempts.reduce((sum, attempt) => sum + (attempt.usage?.usage.totalTokens ?? 0), 0),
|
|
86
102
|
medianDurationMs: median(result.attempts.map((attempt) => attempt.durationMs)),
|
|
87
103
|
attempts: result.attempts.map((attempt) => ({
|
|
88
104
|
attempt: attempt.attempt,
|
|
@@ -91,12 +107,14 @@ export function toJsonReport(results, meta) {
|
|
|
91
107
|
turnCount: attempt.turnCount,
|
|
92
108
|
durationMs: attempt.durationMs,
|
|
93
109
|
toolsUsed: attempt.toolsUsed,
|
|
110
|
+
...(attempt.usage === undefined ? {} : { usage: attempt.usage }),
|
|
94
111
|
failures: attempt.failures,
|
|
95
112
|
})),
|
|
96
113
|
})),
|
|
97
114
|
overall: {
|
|
98
115
|
passed: results.reduce((total, result) => total + result.passed, 0),
|
|
99
116
|
total: results.reduce((sum, result) => sum + result.total, 0),
|
|
117
|
+
totalTokens: totalTokens(results),
|
|
100
118
|
},
|
|
101
119
|
};
|
|
102
120
|
}
|
package/dist/evals/report.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"report.js","sourceRoot":"","sources":["../../src/evals/report.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"report.js","sourceRoot":"","sources":["../../src/evals/report.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,YAAY,EAAE,MAAM,uBAAuB,CAAC;AASrD,SAAS,MAAM,CAAC,MAAyB;IACvC,IAAI,MAAM,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QACxB,OAAO,CAAC,CAAC;IACX,CAAC;IACD,MAAM,MAAM,GAAG,CAAC,GAAG,MAAM,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;IACjD,MAAM,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;IAC7C,OAAO,MAAM,CAAC,MAAM,GAAG,CAAC,KAAK,CAAC;QAC5B,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC;QACrE,CAAC,CAAC,CAAC,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC;AAC5B,CAAC;AAED,SAAS,IAAI,CAAC,MAAyB;IACrC,IAAI,MAAM,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QACxB,OAAO,CAAC,CAAC;IACX,CAAC;IACD,OAAO,MAAM,CAAC,MAAM,CAAC,CAAC,KAAK,EAAE,KAAK,EAAE,EAAE,CAAC,KAAK,GAAG,KAAK,EAAE,CAAC,CAAC,GAAG,MAAM,CAAC,MAAM,CAAC;AAC3E,CAAC;AAED,iEAAiE;AACjE,SAAS,YAAY,CAAC,MAAkB;IACtC,MAAM,MAAM,GAAG,MAAM,CAAC,QAAQ;SAC3B,GAAG,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC,OAAO,CAAC,KAAK,EAAE,KAAK,CAAC,WAAW,CAAC;SAClD,MAAM,CAAC,CAAC,KAAK,EAAmB,EAAE,CAAC,KAAK,KAAK,SAAS,CAAC,CAAC;IAC3D,OAAO,MAAM,CAAC,MAAM,KAAK,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,YAAY,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC;AAClE,CAAC;AAED,SAAS,WAAW,CAAC,OAA8B;IACjD,OAAO,OAAO,CAAC,MAAM,CACnB,CAAC,GAAG,EAAE,MAAM,EAAE,EAAE,CACd,GAAG;QACH,MAAM,CAAC,QAAQ,CAAC,MAAM,CACpB,CAAC,KAAK,EAAE,OAAO,EAAE,EAAE,CAAC,KAAK,GAAG,CAAC,OAAO,CAAC,KAAK,EAAE,KAAK,CAAC,WAAW,IAAI,CAAC,CAAC,EACnE,CAAC,CACF,EACH,CAAC,CACF,CAAC;AACJ,CAAC;AAED,SAAS,GAAG,CAAC,KAAa,EAAE,KAAa;IACvC,OAAO,KAAK,CAAC,MAAM,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG,GAAG,CAAC,MAAM,CAAC,KAAK,GAAG,KAAK,CAAC,MAAM,CAAC,CAAC;AAClF,CAAC;AAED;;;;;GAKG;AACH,MAAM,UAAU,WAAW,CAAC,OAA8B;IACxD,IAAI,OAAO,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QACzB,OAAO,iBAAiB,CAAC;IAC3B,CAAC;IAED,MAAM,IAAI,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;QACpC,IAAI,EAAE,MAAM,CAAC,IAAI;QACjB,IAAI,EAAE,GAAG,MAAM,CAAC,MAAM,IAAI,MAAM,CAAC,KAAK,EAAE;QACxC,KAAK,EAAE,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC;QAC3E,MAAM,EAAE,YAAY,CAAC,MAAM,CAAC;QAC5B,KAAK,EAAE;YACL,GAAG,IAAI,GAAG,CAAC,MAAM,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;SACpE,CAAC,IAAI,CAAC,GAAG,CAAC;QACX,IAAI,EAAE,GAAG,CAAC,MAAM,CAAC,MAAM,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC,OAAO,CAAC,UAAU,CAAC,CAAC,GAAG,IAAI,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,GAAG;KAC7F,CAAC,CAAC,CAAC;IAEJ,MAAM,MAAM,GAAG;QACb,IAAI,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,GAAG,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACxD,IAAI,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,GAAG,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACxD,KAAK,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,GAAG,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC;QAC1D,MAAM,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,GAAG,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;QAC5D,KAAK,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,GAAG,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC;KAC3D,CAAC;IAEF,MAAM,KAAK,GAAG;QACZ,GAAG,GAAG,CAAC,MAAM,EAAE,MAAM,CAAC,IAAI,CAAC,KAAK,GAAG,CAAC,MAAM,EAAE,MAAM,CAAC,IAAI,CAAC,KAAK,GAAG,CAAC,OAAO,EAAE,MAAM,CAAC,KAAK,CAAC,KAAK,GAAG,CAAC,QAAQ,EAAE,MAAM,CAAC,MAAM,CAAC,KAAK,GAAG,CAAC,OAAO,EAAE,MAAM,CAAC,KAAK,CAAC,OAAO;KAC/J,CAAC;IACF,KAAK,MAAM,GAAG,IAAI,IAAI,EAAE,CAAC;QACvB,KAAK,CAAC,IAAI,CACR,GAAG,GAAG,CAAC,GAAG,CAAC,IAAI,EAAE,MAAM,CAAC,IAAI,CAAC,KAAK,GAAG,CAAC,GAAG,CAAC,IAAI,EAAE,MAAM,CAAC,IAAI,CAAC,KAAK,GAAG,CAAC,GAAG,CAAC,KAAK,EAAE,MAAM,CAAC,KAAK,CAAC,KAAK,GAAG,CAAC,GAAG,CAAC,MAAM,EAAE,MAAM,CAAC,MAAM,CAAC,KAAK,GAAG,CAAC,GAAG,CAAC,KAAK,EAAE,MAAM,CAAC,KAAK,CAAC,KAAK,GAAG,CAAC,IAAI,EAAE,CACjL,CAAC;IACJ,CAAC;IAED,MAAM,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,KAAK,EAAE,MAAM,EAAE,EAAE,CAAC,KAAK,GAAG,MAAM,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC;IAC3E,MAAM,KAAK,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,MAAM,EAAE,EAAE,CAAC,GAAG,GAAG,MAAM,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC;IACrE,MAAM,OAAO,GAAG,KAAK,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,MAAM,GAAG,KAAK,CAAC,GAAG,GAAG,CAAC,CAAC;IACrE,MAAM,KAAK,GAAG,WAAW,CAAC,OAAO,CAAC,CAAC;IACnC,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACf,KAAK,CAAC,IAAI,CACR,YAAY,MAAM,IAAI,KAAK,KAAK,OAAO,KAAK,KAAK,KAAK,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,KAAK,YAAY,CAAC,KAAK,CAAC,SAAS,EAAE,CACnG,CAAC;IACF,OAAO,GAAG,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC;AACjC,CAAC;AAED,wEAAwE;AACxE,MAAM,UAAU,cAAc,CAAC,OAA8B;IAC3D,MAAM,KAAK,GAAa,EAAE,CAAC;IAC3B,KAAK,MAAM,MAAM,IAAI,OAAO,EAAE,CAAC;QAC7B,MAAM,QAAQ,GAAG,IAAI,GAAG,EAAoB,CAAC;QAC7C,KAAK,MAAM,OAAO,IAAI,MAAM,CAAC,QAAQ,EAAE,CAAC;YACtC,KAAK,MAAM,OAAO,IAAI,OAAO,CAAC,QAAQ,EAAE,CAAC;gBACvC,MAAM,QAAQ,GAAG,QAAQ,CAAC,GAAG,CAAC,OAAO,CAAC,IAAI,EAAE,CAAC;gBAC7C,QAAQ,CAAC,IAAI,CAAC,OAAO,CAAC,OAAO,CAAC,CAAC;gBAC/B,QAAQ,CAAC,GAAG,CAAC,OAAO,EAAE,QAAQ,CAAC,CAAC;YAClC,CAAC;QACH,CAAC;QACD,KAAK,MAAM,CAAC,MAAM,EAAE,QAAQ,CAAC,IAAI,QAAQ,EAAE,CAAC;YAC1C,KAAK,CAAC,IAAI,CAAC,GAAG,MAAM,CAAC,IAAI,YAAY,QAAQ,CAAC,IAAI,CAAC,GAAG,CAAC,KAAK,MAAM,EAAE,CAAC,CAAC;QACxE,CAAC;IACH,CAAC;IACD,OAAO,KAAK,CAAC,MAAM,KAAK,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,GAAG,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC;AAC3D,CAAC;AAED,MAAM,UAAU,YAAY,CAC1B,OAA8B,EAC9B,IAAgB;IAEhB,OAAO;QACL,aAAa,EAAE,CAAC;QAChB,GAAG,IAAI;QACP,KAAK,EAAE,OAAO,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;YAC9B,IAAI,EAAE,MAAM,CAAC,IAAI;YACjB,MAAM,EAAE,MAAM,CAAC,MAAM;YACrB,KAAK,EAAE,MAAM,CAAC,KAAK;YACnB,QAAQ,EAAE,MAAM,CAAC,KAAK,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,MAAM,GAAG,MAAM,CAAC,KAAK;YAC/D,SAAS,EAAE,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;YACpE,WAAW,EAAE,MAAM,CAAC,QAAQ,CAAC,MAAM,CACjC,CAAC,GAAG,EAAE,OAAO,EAAE,EAAE,CAAC,GAAG,GAAG,CAAC,OAAO,CAAC,KAAK,EAAE,KAAK,CAAC,WAAW,IAAI,CAAC,CAAC,EAC/D,CAAC,CACF;YACD,gBAAgB,EAAE,MAAM,CACtB,MAAM,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC,OAAO,CAAC,UAAU,CAAC,CACrD;YACD,QAAQ,EAAE,MAAM,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC,CAAC;gBAC1C,OAAO,EAAE,OAAO,CAAC,OAAO;gBACxB,MAAM,EAAE,OAAO,CAAC,MAAM;gBACtB,MAAM,EAAE,OAAO,CAAC,MAAM;gBACtB,SAAS,EAAE,OAAO,CAAC,SAAS;gBAC5B,UAAU,EAAE,OAAO,CAAC,UAAU;gBAC9B,SAAS,EAAE,OAAO,CAAC,SAAS;gBAC5B,GAAG,CAAC,OAAO,CAAC,KAAK,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,OAAO,CAAC,KAAK,EAAE,CAAC;gBAChE,QAAQ,EAAE,OAAO,CAAC,QAAQ;aAC3B,CAAC,CAAC;SACJ,CAAC,CAAC;QACH,OAAO,EAAE;YACP,MAAM,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC,KAAK,EAAE,MAAM,EAAE,EAAE,CAAC,KAAK,GAAG,MAAM,CAAC,MAAM,EAAE,CAAC,CAAC;YACnE,KAAK,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,MAAM,EAAE,EAAE,CAAC,GAAG,GAAG,MAAM,CAAC,KAAK,EAAE,CAAC,CAAC;YAC7D,WAAW,EAAE,WAAW,CAAC,OAAO,CAAC;SAClC;KACF,CAAC;AACJ,CAAC"}
|