@snylonue/dsh-context-policy 0.1.1 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +14 -10
- package/dist/compaction.d.ts +6 -2
- package/dist/compaction.d.ts.map +1 -1
- package/dist/compaction.js +19 -7
- package/dist/compaction.js.map +1 -1
- package/dist/core.d.ts.map +1 -1
- package/dist/core.js +2 -3
- package/dist/core.js.map +1 -1
- package/dist/projection.d.ts +17 -5
- package/dist/projection.d.ts.map +1 -1
- package/dist/projection.js +23 -25
- package/dist/projection.js.map +1 -1
- package/dist/token-meter.d.ts.map +1 -1
- package/dist/token-meter.js +15 -7
- package/dist/token-meter.js.map +1 -1
- package/dist/tokenizers.d.ts.map +1 -1
- package/dist/tokenizers.js +7 -7
- package/dist/tokenizers.js.map +1 -1
- package/dist/types.d.ts +10 -6
- package/dist/types.d.ts.map +1 -1
- package/package.json +21 -11
- package/src/compaction.ts +25 -8
- package/src/core.ts +2 -3
- package/src/projection.ts +38 -32
- package/src/token-meter.ts +18 -7
- package/src/tokenizers.ts +12 -7
- package/src/types.ts +10 -6
- package/tests/policy.test.ts +307 -8
package/README.md
CHANGED
|
@@ -5,7 +5,7 @@ Percentage-based automatic compaction and model-configurable token accounting fo
|
|
|
5
5
|
## Requirements
|
|
6
6
|
|
|
7
7
|
- Node.js 24 or later.
|
|
8
|
-
- DSH 0.2.0-rc.2.
|
|
8
|
+
- DSH 0.2.0-rc.2 - 0.2.1-rc.1.
|
|
9
9
|
|
|
10
10
|
## Installation
|
|
11
11
|
|
|
@@ -59,6 +59,7 @@ The global entry has the ID percentage-compaction. The following is a profile pa
|
|
|
59
59
|
auto: true
|
|
60
60
|
# summarizationProvider: example-provider # Optional summarization route override.
|
|
61
61
|
# summarizationModel: example-model
|
|
62
|
+
providerPolicies: []
|
|
62
63
|
modelPolicies: []
|
|
63
64
|
headroomTokens: 0 # Accepted for migration; ignored by this plugin.
|
|
64
65
|
```
|
|
@@ -76,6 +77,7 @@ For an isolated preset, place the configuration on its compaction child. Values
|
|
|
76
77
|
| `auto` | true | Enables automatic compaction listeners. |
|
|
77
78
|
| `summarizationProvider` | Unset | Optional provider override for summarization. |
|
|
78
79
|
| `summarizationModel` | Unset | Optional model override for summarization. |
|
|
80
|
+
| `providerPolicies` | [] | Default pressure policy for all models on a provider. |
|
|
79
81
|
| `modelPolicies` | [] | Partial overrides for exact provider/model routes. Both identifiers are required. |
|
|
80
82
|
| `headroomTokens` | Ignored | Accepted for migration from the upstream configuration. |
|
|
81
83
|
|
|
@@ -96,7 +98,7 @@ modelPolicies:
|
|
|
96
98
|
headroomTokens: 0 # Accepted for migration; ignored by this plugin.
|
|
97
99
|
```
|
|
98
100
|
|
|
99
|
-
Use the provider identifier and model name registered in the target DSH installation.
|
|
101
|
+
Use the provider identifier and model name registered in the target DSH installation. Pressure policy fields inherit from service defaults → provider policy → exact model policy. Set `retainTokens` or `retainRatio` to replace the inherited retention form. Summary and overflow handlers continue to use upstream service/model configuration. `auto` is a service-wide option and cannot be overridden per route. The LLM adapter must supply `contextWindow` for pressure compaction.
|
|
100
102
|
|
|
101
103
|
## Token-meter configuration
|
|
102
104
|
|
|
@@ -106,8 +108,9 @@ The global entry has the ID model-token-meter. A profile patch can select a toke
|
|
|
106
108
|
- id: model-token-meter
|
|
107
109
|
config:
|
|
108
110
|
defaultTokenizer: deepseek_v4
|
|
109
|
-
|
|
110
|
-
|
|
111
|
+
providerPolicies:
|
|
112
|
+
- provider: example-provider
|
|
113
|
+
tokenizer: o200k_base
|
|
111
114
|
modelPolicies:
|
|
112
115
|
- provider: example-provider
|
|
113
116
|
model: example-model
|
|
@@ -120,19 +123,19 @@ The global entry has the ID model-token-meter. A profile patch can select a toke
|
|
|
120
123
|
tokenizer: o200k_base
|
|
121
124
|
```
|
|
122
125
|
|
|
123
|
-
The identifiers in this example are placeholders. An exact provider/model policy takes precedence over a model-only policy.
|
|
126
|
+
The identifiers in this example are placeholders. An exact provider/model policy takes precedence over a model-only policy. Each omitted policy field inherits from the provider policy, then from the global default. The matched model policy's explicit fields override the provider policy. With no request header, the global default applies.
|
|
124
127
|
|
|
125
128
|
| Option | Default | Description |
|
|
126
129
|
| --- | --- | --- |
|
|
127
130
|
| `defaultTokenizer` | `deepseek_v4` | Registered tokenizer used when no model or provider tokenizer applies. |
|
|
128
|
-
| `
|
|
131
|
+
| `providerPolicies` | [] | Tokenizer and accounting defaults for all models on a provider. |
|
|
129
132
|
| `modelPolicies` | [] | Route-specific tokenizer and accounting settings. |
|
|
130
133
|
| `tokensPerMessage` | 3 | Per-message framing allowance. |
|
|
131
134
|
| `toolsOverhead` | 4 | Allowance added when tool definitions are present. |
|
|
132
135
|
| `replyPrimingTokens` | 3 | Request-level reply framing allowance. |
|
|
133
136
|
| `includeReasoning` | false | Includes visible reasoning text in the estimated message surface. |
|
|
134
137
|
|
|
135
|
-
The last four options belong to model policy entries. Token allowances must be nonnegative integers. For duplicate routes at the same specificity, the first matching entry is used. Unknown tokenizer names fail when text is counted.
|
|
138
|
+
The last four options belong to provider and model policy entries. Token allowances must be nonnegative integers. For duplicate routes at the same specificity, the first matching entry is used. Unknown tokenizer names fail when text is counted.
|
|
136
139
|
|
|
137
140
|
### Builtin tokenizers
|
|
138
141
|
|
|
@@ -147,8 +150,9 @@ DeepSeek V4 is the global default. Select GPT token counting for all models on a
|
|
|
147
150
|
- id: model-token-meter
|
|
148
151
|
config:
|
|
149
152
|
defaultTokenizer: deepseek_v4
|
|
150
|
-
|
|
151
|
-
|
|
153
|
+
providerPolicies:
|
|
154
|
+
- provider: openai
|
|
155
|
+
tokenizer: o200k_base
|
|
152
156
|
```
|
|
153
157
|
|
|
154
158
|
Use the provider identifiers configured in your DSH profile; `openai` is an example.
|
|
@@ -197,7 +201,7 @@ Compared with DSH 0.2.0-rc.2's official `dsh-token-meter` and `dsh-compaction-ba
|
|
|
197
201
|
|
|
198
202
|
The compaction service extends `BasicCompactionEngine` and replaces automatic pressure range selection. Summarization, durable transactions, manual commands, and overflow listeners use the upstream implementation.
|
|
199
203
|
|
|
200
|
-
The meter uses the public Cordis service lifecycle and projection registration API. Its billing fold is maintained locally because the upstream definition is not exported at runtime.
|
|
204
|
+
The meter uses the public Cordis service lifecycle and projection registration API. Its billing fold is maintained locally because the upstream definition is not exported at runtime. Replay state does not retain fixed-density heuristic prices. Projection state, wire composition, and durable compaction shadow prices all use model-configured token counts. The upstream `heuristicTokens` compatibility field carries the same model count as `tokens`. This does not change independent history estimators owned by other plugins.
|
|
201
205
|
|
|
202
206
|
## Development and release
|
|
203
207
|
|
package/dist/compaction.d.ts
CHANGED
|
@@ -1,11 +1,14 @@
|
|
|
1
1
|
import type { Context } from "@deepseek-ai/cordis";
|
|
2
2
|
import Schema from "@deepseek-ai/schemastery";
|
|
3
3
|
import BasicCompactionEngine from "@deepseek-ai/dsh-compaction-basic";
|
|
4
|
-
import type { BasicCompactionConfig, ResolvedTargetPolicy, ResolvedCompactSpec } from "@deepseek-ai/dsh-compaction-basic";
|
|
4
|
+
import type { BasicCompactionConfig, CompactionPolicyConfig, ResolvedTargetPolicy, ResolvedCompactSpec } from "@deepseek-ai/dsh-compaction-basic";
|
|
5
5
|
import type { CompactionResult } from "@deepseek-ai/dsh-compaction";
|
|
6
6
|
import type { Session } from "@deepseek-ai/dsh-session";
|
|
7
7
|
import type { TokenMeasurement } from "@deepseek-ai/dsh-token-meter";
|
|
8
8
|
export interface PercentageCompactionConfig extends BasicCompactionConfig {
|
|
9
|
+
providerPolicies?: (CompactionPolicyConfig & {
|
|
10
|
+
provider: string;
|
|
11
|
+
})[];
|
|
9
12
|
thresholdRatio: number;
|
|
10
13
|
maxTokens: number;
|
|
11
14
|
modelPolicies: NonNullable<BasicCompactionConfig["modelPolicies"]>;
|
|
@@ -33,7 +36,8 @@ declare module "@deepseek-ai/dsh-session-projection" {
|
|
|
33
36
|
/** Reuse the official summarizer, transactions, commands and overflow listeners. */
|
|
34
37
|
export declare class PercentageCompactionEngine extends BasicCompactionEngine {
|
|
35
38
|
static inject: string[];
|
|
36
|
-
static Config: Schema<
|
|
39
|
+
static Config: Schema<Partial<PercentageCompactionConfig>, PercentageCompactionConfig>;
|
|
40
|
+
private readonly providerPolicies;
|
|
37
41
|
constructor(ctx: Context, config: PercentageCompactionConfig);
|
|
38
42
|
private entryState;
|
|
39
43
|
private policy;
|
package/dist/compaction.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"compaction.d.ts","sourceRoot":"","sources":["../src/compaction.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,qBAAqB,CAAC;AACnD,OAAO,MAAM,MAAM,0BAA0B,CAAC;AAC9C,OAAO,qBAAqB,MAAM,mCAAmC,CAAC;AACtE,OAAO,KAAK,EACX,qBAAqB,EACrB,oBAAoB,EACpB,mBAAmB,EAEnB,MAAM,mCAAmC,CAAC;AAK3C,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,6BAA6B,CAAC;AAIpE,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,0BAA0B,CAAC;AACxD,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,8BAA8B,CAAC;AAErE,MAAM,WAAW,0BAA2B,SAAQ,qBAAqB;IACxE,cAAc,EAAE,MAAM,CAAC;IACvB,SAAS,EAAE,MAAM,CAAC;IAClB,aAAa,EAAE,WAAW,CAAC,qBAAqB,CAAC,eAAe,CAAC,CAAC,CAAC;IACnE,IAAI,EAAE,OAAO,CAAC;CACd;AAED,KAAK,KAAK,GAAG,UAAU,CAAC,qBAAqB,CAAC,iBAAiB,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;AACrE,KAAK,OAAO,GAAG,UAAU,CAAC,qBAAqB,CAAC,iBAAiB,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;
|
|
1
|
+
{"version":3,"file":"compaction.d.ts","sourceRoot":"","sources":["../src/compaction.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,qBAAqB,CAAC;AACnD,OAAO,MAAM,MAAM,0BAA0B,CAAC;AAC9C,OAAO,qBAAqB,MAAM,mCAAmC,CAAC;AACtE,OAAO,KAAK,EACX,qBAAqB,EACrB,sBAAsB,EACtB,oBAAoB,EACpB,mBAAmB,EAEnB,MAAM,mCAAmC,CAAC;AAK3C,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,6BAA6B,CAAC;AAIpE,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,0BAA0B,CAAC;AACxD,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,8BAA8B,CAAC;AAErE,MAAM,WAAW,0BAA2B,SAAQ,qBAAqB;IACxE,gBAAgB,CAAC,EAAE,CAAC,sBAAsB,GAAG;QAAE,QAAQ,EAAE,MAAM,CAAA;KAAE,CAAC,EAAE,CAAC;IACrE,cAAc,EAAE,MAAM,CAAC;IACvB,SAAS,EAAE,MAAM,CAAC;IAClB,aAAa,EAAE,WAAW,CAAC,qBAAqB,CAAC,eAAe,CAAC,CAAC,CAAC;IACnE,IAAI,EAAE,OAAO,CAAC;CACd;AAED,KAAK,KAAK,GAAG,UAAU,CAAC,qBAAqB,CAAC,iBAAiB,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;AACrE,KAAK,OAAO,GAAG,UAAU,CAAC,qBAAqB,CAAC,iBAAiB,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;AAsBvE,+DAA+D;AAC/D,wBAAgB,kBAAkB,CACjC,MAAM,EAAE,oBAAoB,EAC5B,aAAa,EAAE,MAAM,GACnB,mBAAmB,CA0BrB;AAED,8DAA8D;AAC9D,wBAAgB,sBAAsB,CACrC,OAAO,EAAE,OAAO,EAChB,WAAW,EAAE,gBAAgB,EAC7B,YAAY,EAAE,MAAM,EACpB,UAAU,EAAE,OAAO;;;SAyBnB;AAED,UAAU,oBAAoB;IAC7B,MAAM,EAAE,OAAO,CAAC;IAChB,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;IACvB,UAAU,EAAE,OAAO,CAAC;CACpB;AAED,OAAO,QAAQ,qCAAqC,CAAC;IACpD,UAAU,yBAAyB;QAClC,uBAAuB,EAAE,oBAAoB,CAAC;KAC9C;CACD;AA8BD,oFAAoF;AACpF,qBAAa,0BAA2B,SAAQ,qBAAqB;IACpE,OAAgB,MAAM,WAGpB;IACF,OAAgB,MAAM,EAAE,MAAM,CAC7B,OAAO,CAAC,0BAA0B,CAAC,EACnC,0BAA0B,CAC1B,CAeE;IACH,OAAO,CAAC,QAAQ,CAAC,gBAAgB,CAA8D;gBACnF,GAAG,EAAE,OAAO,EAAE,MAAM,EAAE,0BAA0B;IAc5D,OAAO,CAAC,UAAU;IASlB,OAAO,CAAC,MAAM;IA+BC,eAAe,CAC7B,KAAK,EAAE,KAAK,EACZ,OAAO,EAAE,OAAO,EAChB,MAAM,CAAC,EAAE,WAAW,GAClB,OAAO,CAAC,gBAAgB,GAAG,IAAI,CAAC;CAmEnC;AACD,eAAe,0BAA0B,CAAC"}
|
package/dist/compaction.js
CHANGED
|
@@ -14,6 +14,14 @@ const fields = {
|
|
|
14
14
|
compactionRetries: Schema.natural(),
|
|
15
15
|
maxOverflowRetries: Schema.natural(),
|
|
16
16
|
};
|
|
17
|
+
function mergePolicy(base, override) {
|
|
18
|
+
const merged = { ...base, ...Object.fromEntries(Object.entries(override).filter(([, value]) => value !== undefined)) };
|
|
19
|
+
if (override.retainTokens !== undefined && override.retainRatio === undefined)
|
|
20
|
+
delete merged.retainRatio;
|
|
21
|
+
if (override.retainRatio !== undefined && override.retainTokens === undefined)
|
|
22
|
+
delete merged.retainTokens;
|
|
23
|
+
return merged;
|
|
24
|
+
}
|
|
17
25
|
/** Both budgets are percentages of the full context window. */
|
|
18
26
|
export function resolveCompactSpec(policy, contextWindow) {
|
|
19
27
|
if (!Number.isSafeInteger(contextWindow) || contextWindow <= 0)
|
|
@@ -98,22 +106,26 @@ export class PercentageCompactionEngine extends BasicCompactionEngine {
|
|
|
98
106
|
thresholdRatio: Schema.number().min(0).max(1).default(0.8),
|
|
99
107
|
maxTokens: Schema.natural().default(65536),
|
|
100
108
|
auto: Schema.boolean().default(true),
|
|
109
|
+
providerPolicies: Schema.array(Schema.object({ ...fields, provider: Schema.string().required() })).default([]),
|
|
101
110
|
modelPolicies: Schema.array(Schema.object({
|
|
102
111
|
...fields,
|
|
103
112
|
provider: Schema.string().required(),
|
|
104
113
|
model: Schema.string().required(),
|
|
105
114
|
})).default([]),
|
|
106
115
|
});
|
|
116
|
+
providerPolicies;
|
|
107
117
|
constructor(ctx, config) {
|
|
118
|
+
const { providerPolicies = [], ...basic } = config;
|
|
108
119
|
super(ctx, {
|
|
109
|
-
...
|
|
120
|
+
...basic,
|
|
110
121
|
headroomTokens: 0,
|
|
111
|
-
modelPolicies: config.modelPolicies.map((p) =>
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
maxTokens:
|
|
115
|
-
})
|
|
122
|
+
modelPolicies: config.modelPolicies.map((p) => {
|
|
123
|
+
const provider = providerPolicies.find((policy) => policy.provider === p.provider);
|
|
124
|
+
const merged = mergePolicy(provider ?? {}, p);
|
|
125
|
+
return { ...merged, headroomTokens: 0, maxTokens: merged.maxTokens ?? config.maxTokens };
|
|
126
|
+
}),
|
|
116
127
|
});
|
|
128
|
+
this.providerPolicies = structuredClone(providerPolicies);
|
|
117
129
|
ctx.sessionProjections.register(compactionEntry);
|
|
118
130
|
}
|
|
119
131
|
entryState(session) {
|
|
@@ -123,7 +135,7 @@ export class PercentageCompactionEngine extends BasicCompactionEngine {
|
|
|
123
135
|
return state;
|
|
124
136
|
}
|
|
125
137
|
policy(target) {
|
|
126
|
-
const match = this.config.modelPolicies.find((p) => p.provider === target.provider && p.model === target.model);
|
|
138
|
+
const match = this.config.modelPolicies.find((p) => p.provider === target.provider && p.model === target.model) ?? this.providerPolicies.find((p) => p.provider === target.provider);
|
|
127
139
|
let retention;
|
|
128
140
|
if (match?.retainTokens !== undefined)
|
|
129
141
|
retention = { retainTokens: match.retainTokens };
|
package/dist/compaction.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"compaction.js","sourceRoot":"","sources":["../src/compaction.ts"],"names":[],"mappings":"AAEA,OAAO,MAAM,MAAM,0BAA0B,CAAC;AAC9C,OAAO,qBAAqB,MAAM,mCAAmC,CAAC;
|
|
1
|
+
{"version":3,"file":"compaction.js","sourceRoot":"","sources":["../src/compaction.ts"],"names":[],"mappings":"AAEA,OAAO,MAAM,MAAM,0BAA0B,CAAC;AAC9C,OAAO,qBAAqB,MAAM,mCAAmC,CAAC;AAQtE,OAAO,EACN,qBAAqB,EACrB,yBAAyB,GACzB,MAAM,6BAA6B,CAAC;AAErC,OAAO,EAAE,cAAc,EAAE,MAAM,0BAA0B,CAAC;AAE1D,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AAcxB,MAAM,MAAM,GAAG;IACd,cAAc,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;IAC7C,cAAc,EAAE,MAAM,CAAC,OAAO,EAAE;IAChC,WAAW,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;IAC1C,YAAY,EAAE,MAAM,CAAC,OAAO,EAAE;IAC9B,qBAAqB,EAAE,MAAM,CAAC,MAAM,EAAE;IACtC,kBAAkB,EAAE,MAAM,CAAC,MAAM,EAAE;IACnC,SAAS,EAAE,MAAM,CAAC,OAAO,EAAE;IAC3B,iBAAiB,EAAE,MAAM,CAAC,OAAO,EAAE;IACnC,kBAAkB,EAAE,MAAM,CAAC,OAAO,EAAE;CACpC,CAAC;AAEF,SAAS,WAAW,CAAmC,IAA4B,EAAE,QAAW;IAC/F,MAAM,MAAM,GAAG,EAAE,GAAG,IAAI,EAAE,GAAG,MAAM,CAAC,WAAW,CAC9C,MAAM,CAAC,OAAO,CAAC,QAAQ,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,KAAK,CAAC,EAAE,EAAE,CAAC,KAAK,KAAK,SAAS,CAAC,CACnE,EAAO,CAAC;IACT,IAAI,QAAQ,CAAC,YAAY,KAAK,SAAS,IAAI,QAAQ,CAAC,WAAW,KAAK,SAAS;QAAE,OAAO,MAAM,CAAC,WAAW,CAAC;IACzG,IAAI,QAAQ,CAAC,WAAW,KAAK,SAAS,IAAI,QAAQ,CAAC,YAAY,KAAK,SAAS;QAAE,OAAO,MAAM,CAAC,YAAY,CAAC;IAC1G,OAAO,MAAM,CAAC;AACf,CAAC;AAED,+DAA+D;AAC/D,MAAM,UAAU,kBAAkB,CACjC,MAA4B,EAC5B,aAAqB;IAErB,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,aAAa,CAAC,IAAI,aAAa,IAAI,CAAC;QAC7D,MAAM,IAAI,KAAK,CACd,2DAA2D,CAC3D,CAAC;IACH,MAAM,eAAe,GAAG,IAAI,CAAC,KAAK,CAAC,aAAa,GAAG,MAAM,CAAC,cAAc,CAAC,CAAC;IAC1E,MAAM,YAAY,GACjB,MAAM,CAAC,YAAY,KAAK,SAAS;QAChC,CAAC,CAAC,MAAM,CAAC,YAAY;QACrB,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,aAAa,GAAG,MAAM,CAAC,WAAW,CAAC,CAAC;IACnD,IAAI,eAAe,IAAI,CAAC,IAAI,YAAY,IAAI,eAAe;QAC1D,MAAM,IAAI,KAAK,CACd,kEAAkE,CAClE,CAAC;IACH,OAAO,MAAM,CAAC,MAAM,CAAC;QACpB,MAAM,EAAE,MAAM,CAAC,MAAM;QACrB,aAAa;QACb,cAAc,EAAE,MAAM,CAAC,cAAc;QACrC,eAAe;QACf,YAAY;QACZ,qBAAqB,EAAE,MAAM,CAAC,qBAAqB;QACnD,kBAAkB,EAAE,MAAM,CAAC,kBAAkB;QAC7C,SAAS,EAAE,MAAM,CAAC,SAAS;QAC3B,iBAAiB,EAAE,MAAM,CAAC,iBAAiB;QAC3C,kBAAkB,EAAE,MAAM,CAAC,kBAAkB;KAC7C,CAAC,CAAC;AACJ,CAAC;AAED,8DAA8D;AAC9D,MAAM,UAAU,sBAAsB,CACrC,OAAgB,EAChB,WAA6B,EAC7B,YAAoB,EACpB,UAAmB;IAEnB,MAAM,KAAK,GAAG,WAAW,CAAC,KAAK,CAAC;IAChC,IAAI,CAAC,KAAK,CAAC,MAAM;QAAE,OAAO,IAAI,CAAC;IAC/B,MAAM,OAAO,GAAG,OAAO,CAAC,OAAO,CAAC,KAAK,CAAC;IACtC,IACC,OAAO,CAAC,MAAM,KAAK,KAAK,CAAC,MAAM;QAC/B,OAAO,CAAC,IAAI,CAAC,CAAC,GAAG,EAAE,CAAC,EAAE,EAAE,CAAC,GAAG,KAAK,KAAK,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC;QAE9C,MAAM,IAAI,KAAK,CACd,oEAAoE,CACpE,CAAC;IACH,MAAM,KAAK,GAAG,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;IACjC,IAAI,KAAK,GAAG,CAAC,CAAC;IACd,IAAI,IAAI,GAAG,KAAK,CAAC,MAAM,CAAC;IACxB,KAAK,IAAI,CAAC,GAAG,KAAK,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC,IAAI,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC;QAC5C,KAAK,IAAI,KAAK,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC;QACzB,IAAI,GAAG,CAAC,CAAC;QACT,IAAI,KAAK,IAAI,YAAY;YAAE,MAAM;IAClC,CAAC;IACD,OAAO,IAAI,GAAG,KAAK,IAAI,CAAC,yBAAyB,CAAC,OAAO,EAAE,OAAO,CAAC,IAAI,CAAC,CAAC;QACxE,IAAI,EAAE,CAAC;IACR,OAAO,IAAI,IAAI,KAAK;QACnB,CAAC,CAAC,IAAI;QACN,CAAC,CAAC,EAAE,KAAK,EAAE,OAAO,CAAC,KAAK,CAAC,EAAE,GAAG,EAAE,OAAO,CAAC,IAAI,GAAG,CAAC,CAAC,EAAE,CAAC;AACtD,CAAC;AAcD,MAAM,eAAe,GAAoD;IACxE,GAAG,EAAE,yBAAyB;IAC9B,YAAY,EAAE,CAAC;IACf,WAAW,EAAE,CAAC,CAAC,MAAM,CAAC;QACrB,MAAM,EAAE,CAAC,CAAC,OAAO,EAAE;QACnB,OAAO,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,WAAW,EAAE,CAAC,QAAQ,EAAE;QAClD,UAAU,EAAE,CAAC,CAAC,OAAO,EAAE;KACvB,CAAC;IACF,IAAI,EAAE,GAAG,EAAE,CAAC,CAAC,EAAE,MAAM,EAAE,KAAK,EAAE,OAAO,EAAE,IAAI,EAAE,UAAU,EAAE,KAAK,EAAE,CAAC;IACjE,KAAK,EAAE,CAAC,KAAK,EAAE,KAAK,EAAE,EAAE;QACvB,IAAI,KAAK,CAAC,IAAI,KAAK,kBAAkB;YAAE,OAAO,EAAE,GAAG,KAAK,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC;QACzE,IAAI,KAAK,CAAC,IAAI,KAAK,gBAAgB,IAAI,KAAK,CAAC,IAAI,KAAK,kBAAkB;YACvE,OAAO,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,GAAG,KAAK,EAAE,MAAM,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC,KAAK,CAAC;QAC3D,IACC,cAAc,CAAC,KAAK,CAAC;YACrB,CAAC,KAAK,CAAC,OAAO,KAAK,IAAI;gBACtB,CAAC,KAAK,CAAC,SAAS,KAAK,QAAQ;oBAC5B,KAAK,CAAC,SAAS,CAAC,QAAQ,KAAK,KAAK,CAAC,OAAO,CAAC,CAAC;YAE9C,OAAO;gBACN,GAAG,KAAK;gBACR,OAAO,EAAE,KAAK,CAAC,GAAG;gBAClB,UAAU,EAAE,KAAK,CAAC,IAAI,KAAK,gBAAgB;aAC3C,CAAC;QACH,OAAO,KAAK,CAAC;IACd,CAAC;CACD,CAAC;AAEF,oFAAoF;AACpF,MAAM,OAAO,0BAA2B,SAAQ,qBAAqB;IACpE,MAAM,CAAU,MAAM,GAAG;QACxB,GAAG,qBAAqB,CAAC,MAAM;QAC/B,oBAAoB;KACpB,CAAC;IACF,MAAM,CAAU,MAAM,GAGlB,MAAM,CAAC,MAAM,CAAC;QACjB,GAAG,MAAM;QACT,cAAc,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC;QAC1D,SAAS,EAAE,MAAM,CAAC,OAAO,EAAE,CAAC,OAAO,CAAC,KAAK,CAAC;QAC1C,IAAI,EAAE,MAAM,CAAC,OAAO,EAAE,CAAC,OAAO,CAAC,IAAI,CAAC;QACpC,gBAAgB,EAAE,MAAM,CAAC,KAAK,CAC7B,MAAM,CAAC,MAAM,CAAC,EAAE,GAAG,MAAM,EAAE,QAAQ,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE,EAAE,CAAC,CAClE,CAAC,OAAO,CAAC,EAAE,CAAC;QACb,aAAa,EAAE,MAAM,CAAC,KAAK,CAC1B,MAAM,CAAC,MAAM,CAAC;YACb,GAAG,MAAM;YACT,QAAQ,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE;YACpC,KAAK,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE;SACjC,CAAC,CACF,CAAC,OAAO,CAAC,EAAE,CAAC;KACb,CAAC,CAAC;IACc,gBAAgB,CAA8D;IAC/F,YAAY,GAAY,EAAE,MAAkC;QAC3D,MAAM,EAAE,gBAAgB,GAAG,EAAE,EAAE,GAAG,KAAK,EAAE,GAAG,MAAM,CAAC;QACnD,KAAK,CAAC,GAAG,EAAE;YACV,GAAG,KAAK;YACR,cAAc,EAAE,CAAC;YACjB,aAAa,EAAE,MAAM,CAAC,aAAa,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE;gBAC7C,MAAM,QAAQ,GAAG,gBAAgB,CAAC,IAAI,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,MAAM,CAAC,QAAQ,KAAK,CAAC,CAAC,QAAQ,CAAC,CAAC;gBACnF,MAAM,MAAM,GAAG,WAAW,CAAC,QAAQ,IAAI,EAAE,EAAE,CAAC,CAAC,CAAC;gBAC9C,OAAO,EAAE,GAAG,MAAM,EAAE,cAAc,EAAE,CAAC,EAAE,SAAS,EAAE,MAAM,CAAC,SAAS,IAAI,MAAM,CAAC,SAAS,EAAE,CAAC;YAC1F,CAAC,CAAC;SACF,CAAC,CAAC;QACH,IAAI,CAAC,gBAAgB,GAAG,eAAe,CAAC,gBAAgB,CAAC,CAAC;QAC1D,GAAG,CAAC,kBAAkB,CAAC,QAAQ,CAAC,eAAe,CAAC,CAAC;IAClD,CAAC;IACO,UAAU,CAAC,OAAgB;QAClC,MAAM,KAAK,GAAG,IAAI,CAAC,GAAG,CAAC,kBAAkB,CAAC,OAAO,CAChD,OAAO,EACP,yBAAyB,CACzB,CAAC;QACF,IAAI,CAAC,KAAK;YACT,MAAM,IAAI,KAAK,CAAC,mDAAmD,CAAC,CAAC;QACtE,OAAO,KAAK,CAAC;IACd,CAAC;IACO,MAAM,CAAC,MAGd;QACA,MAAM,KAAK,GAAG,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,IAAI,CAC3C,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,QAAQ,KAAK,MAAM,CAAC,QAAQ,IAAI,CAAC,CAAC,KAAK,KAAK,MAAM,CAAC,KAAK,CACjE,IAAI,IAAI,CAAC,gBAAgB,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,QAAQ,KAAK,MAAM,CAAC,QAAQ,CAAC,CAAC;QACvE,IAAI,SAA4B,CAAC;QACjC,IAAI,KAAK,EAAE,YAAY,KAAK,SAAS;YACpC,SAAS,GAAG,EAAE,YAAY,EAAE,KAAK,CAAC,YAAY,EAAE,CAAC;aAC7C,IAAI,KAAK,EAAE,WAAW,KAAK,SAAS;YACxC,SAAS,GAAG,EAAE,WAAW,EAAE,KAAK,CAAC,WAAW,EAAE,CAAC;aAC3C,IAAI,IAAI,CAAC,MAAM,CAAC,YAAY,KAAK,SAAS;YAC9C,SAAS,GAAG,EAAE,YAAY,EAAE,IAAI,CAAC,MAAM,CAAC,YAAY,EAAE,CAAC;;YACnD,SAAS,GAAG,EAAE,WAAW,EAAE,IAAI,CAAC,MAAM,CAAC,WAAW,EAAE,CAAC;QAC1D,OAAO;YACN,cAAc,EAAE,KAAK,EAAE,cAAc,IAAI,IAAI,CAAC,MAAM,CAAC,cAAc;YACnE,cAAc,EAAE,CAAC;YACjB,qBAAqB,EACpB,KAAK,EAAE,qBAAqB,IAAI,IAAI,CAAC,MAAM,CAAC,qBAAqB;YAClE,kBAAkB,EACjB,KAAK,EAAE,kBAAkB,IAAI,IAAI,CAAC,MAAM,CAAC,kBAAkB;YAC5D,SAAS,EAAE,KAAK,EAAE,SAAS,IAAI,IAAI,CAAC,MAAM,CAAC,SAAS;YACpD,iBAAiB,EAChB,KAAK,EAAE,iBAAiB,IAAI,IAAI,CAAC,MAAM,CAAC,iBAAiB;YAC1D,kBAAkB,EACjB,KAAK,EAAE,kBAAkB,IAAI,IAAI,CAAC,MAAM,CAAC,kBAAkB;YAC5D,MAAM;YACN,GAAG,SAAS;SACZ,CAAC;IACH,CAAC;IACQ,KAAK,CAAC,eAAe,CAC7B,KAAY,EACZ,OAAgB,EAChB,MAAoB;QAEpB,MAAM,MAAM,GAAG,KAAK,CAAC,OAAO,CAAC,aAAa,EAAE,CAAC;QAC7C,IAAI,CAAC,MAAM,EAAE,MAAM,CAAC,QAAQ,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,KAAK;YAAE,OAAO,IAAI,CAAC;QAClE,MAAM,MAAM,GAAG;YACd,QAAQ,EAAE,MAAM,CAAC,MAAM,CAAC,QAAQ;YAChC,KAAK,EAAE,MAAM,CAAC,MAAM,CAAC,KAAK;SAC1B,CAAC;QACF,MAAM,KAAK,GAAG,IAAI,CAAC,GAAG,CAAC,UAAU,CAAC;QAClC,IAAI,WAAW,GAAG,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC;QAC/C,MAAM,KAAK,GAAG,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC,kBAAkB,CAAC,CAAC;QAC/C,IAAI,OAAO,KAAK,kBAAkB,EAAE,CAAC;YACpC,KAAK,EAAE,YAAY,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC;YACnC,WAAW,GAAG,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC;YAC3C,MAAM,KAAK,GAAG,sBAAsB,CACnC,KAAK,CAAC,OAAO,EACb,WAAW,EACX,CAAC,EACD,IAAI,CAAC,UAAU,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,UAAU,CACzC,CAAC;YACF,OAAO,KAAK;gBACX,CAAC,CAAC,IAAI,CAAC,aAAa,CAAC,KAAK,CAAC,KAAK,EAAE,KAAK,CAAC,GAAG,EAAE,KAAK,EAAE,MAAM,CAAC;gBAC3D,CAAC,CAAC,IAAI,CAAC;QACT,CAAC;QACD,MAAM,IAAI,GAAG,MAAM,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC,gBAAgB,CAC/C,MAAM,CAAC,QAAQ,EACf,MAAM,CAAC,KAAK,EACZ,MAAM,CACN,CAAC;QACF,IAAI,IAAI,CAAC,UAAU,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,MAAM;YACxC,MAAM,IAAI,qBAAqB,CAC9B,MAAM,EACN,+DAA+D,CAC/D,CAAC;QACH,IAAI,CAAC,IAAI,CAAC,OAAO;YAChB,MAAM,IAAI,KAAK,CACd,0CAA0C;gBACzC,MAAM,CAAC,QAAQ;gBACf,GAAG;gBACH,MAAM,CAAC,KAAK,CACb,CAAC;QACH,MAAM,IAAI,GAAG,kBAAkB,CAC9B,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,EACnB,IAAI,CAAC,OAAO,CAAC,aAAa,CAC1B,CAAC;QACF,IAAI,WAAW,CAAC,WAAW,GAAG,IAAI,CAAC,eAAe;YAAE,OAAO,IAAI,CAAC;QAChE,KAAK,EAAE,YAAY,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC;QACnC,WAAW,GAAG,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC;QAC3C,IAAI,WAAW,CAAC,WAAW,GAAG,IAAI,CAAC,eAAe;YAAE,OAAO,IAAI,CAAC;QAChE,IAAI,MAAM,GAA4B,IAAI,CAAC;QAC3C,KAAK,IAAI,OAAO,GAAG,CAAC,EAAE,OAAO,IAAI,IAAI,CAAC,iBAAiB,EAAE,OAAO,EAAE,EAAE,CAAC;YACpE,MAAM,KAAK,GAAG,sBAAsB,CACnC,KAAK,CAAC,OAAO,EACb,WAAW,EACX,IAAI,CAAC,YAAY,EACjB,IAAI,CAAC,UAAU,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,UAAU,CACzC,CAAC;YACF,IAAI,CAAC,KAAK;gBAAE,OAAO,MAAM,CAAC;YAC1B,MAAM,GAAG,MAAM,IAAI,CAAC,aAAa,CAAC,KAAK,CAAC,KAAK,EAAE,KAAK,CAAC,GAAG,EAAE,KAAK,EAAE,MAAM,CAAC,CAAC;YACzE,WAAW,GAAG,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC;YAC3C,IAAI,WAAW,CAAC,WAAW,GAAG,IAAI,CAAC,eAAe;gBAAE,OAAO,MAAM,CAAC;QACnE,CAAC;QACD,MAAM,IAAI,KAAK,CACd,oDAAoD;YACnD,CAAC,IAAI,CAAC,iBAAiB,GAAG,CAAC,CAAC;YAC5B,WAAW,CACZ,CAAC;IACH,CAAC;;AAEF,eAAe,0BAA0B,CAAC"}
|
package/dist/core.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"core.d.ts","sourceRoot":"","sources":["../src/core.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,WAAW,EAAE,YAAY,EAAE,MAAM,0BAA0B,CAAC;AAC1E,OAAO,KAAK,EAAE,wBAAwB,EAAE,MAAM,8BAA8B,CAAC;AAC7E,OAAO,EAAE,gBAAgB,EAAE,MAAM,0BAA0B,CAAC;AAC5D,OAAO,KAAK,EACX,OAAO,IAAI,UAAU,EAIrB,UAAU,EACV,MAAM,sBAAsB,CAAC;AAC9B,OAAO,KAAK,EACX,WAAW,EACX,UAAU,EACV,WAAW,EACX,UAAU,EACV,MAAM,YAAY,CAAC;AACpB,MAAM,WAAW,aAAa;IAC7B,GAAG,CAAC,EAAE,IAAI,CAAC,UAAU,EAAE,qBAAqB,GAAG,iBAAiB,CAAC,CAAC;CAClE;
|
|
1
|
+
{"version":3,"file":"core.d.ts","sourceRoot":"","sources":["../src/core.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,WAAW,EAAE,YAAY,EAAE,MAAM,0BAA0B,CAAC;AAC1E,OAAO,KAAK,EAAE,wBAAwB,EAAE,MAAM,8BAA8B,CAAC;AAC7E,OAAO,EAAE,gBAAgB,EAAE,MAAM,0BAA0B,CAAC;AAC5D,OAAO,KAAK,EACX,OAAO,IAAI,UAAU,EAIrB,UAAU,EACV,MAAM,sBAAsB,CAAC;AAC9B,OAAO,KAAK,EACX,WAAW,EACX,UAAU,EACV,WAAW,EACX,UAAU,EACV,MAAM,YAAY,CAAC;AACpB,MAAM,WAAW,aAAa;IAC7B,GAAG,CAAC,EAAE,IAAI,CAAC,UAAU,EAAE,qBAAqB,GAAG,iBAAiB,CAAC,CAAC;CAClE;AAmBD,eAAO,MAAM,YAAY,QAAO,WAK9B,CAAC;AAEH,wBAAgB,WAAW,CAAC,KAAK,EAAE,UAAU,UAM5C;AAED,wBAAgB,OAAO,CACtB,KAAK,EAAE,OAAO,CACb,YAAY,EACZ;IAAE,IAAI,EAAE,mBAAmB,GAAG,mBAAmB,CAAA;CAAE,CACnD,GACC,UAAU,GAAG,SAAS,CAIxB;AAED,oEAAoE;AACpE,wBAAgB,IAAI,CAAC,KAAK,EAAE,WAAW,EAAE,KAAK,EAAE,UAAU,GAAG,WAAW,CA0DvE;AA+CD,wBAAgB,YAAY,CAC3B,KAAK,EAAE,SAAS,UAAU,EAAE,EAC5B,MAAM,EAAE,WAAW,GAAG,IAAI,EAC1B,MAAM,EAAE,WAAW,EACnB,QAAQ,EAAE,aAAa;;;;;;;;;;;;;EA2DvB;AAED;;GAEG;AACH,wBAAgB,YAAY,CAC3B,KAAK,EAAE,WAAW,EAClB,MAAM,EAAE,WAAW,EACnB,QAAQ,GAAE,aAAkB,EAC5B,aAAa,CAAC,EAAE,WAAW;;;;;;;;;;;;;;;EAkC3B"}
|
package/dist/core.js
CHANGED
|
@@ -1,6 +1,5 @@
|
|
|
1
1
|
import { SessionLogOffset } from "@deepseek-ai/dsh-session";
|
|
2
2
|
import { canonicalHeader, deriveEventMessage, headerEquals, isSurfaceEvent, } from "@deepseek-ai/dsh-session";
|
|
3
|
-
import { estimateMessage as shadowPrice } from "@deepseek-ai/dsh-token-meter/estimate";
|
|
4
3
|
import { lastAssistantStreamChunk } from "@deepseek-ai/dsh-llm";
|
|
5
4
|
import { deepFreeze } from "@deepseek-ai/dsh-util-values";
|
|
6
5
|
import { countText, resolvePolicy } from "./tokenizers.js";
|
|
@@ -62,7 +61,6 @@ export function fold(state, event) {
|
|
|
62
61
|
const node = {
|
|
63
62
|
seq: event.seq,
|
|
64
63
|
message,
|
|
65
|
-
heuristicTokens: message ? shadowPrice(message) : 0,
|
|
66
64
|
};
|
|
67
65
|
if (event.surfaceOp === "append")
|
|
68
66
|
next.nodes = [...state.nodes, node];
|
|
@@ -154,7 +152,8 @@ export function priceSurface(nodes, header, config, services) {
|
|
|
154
152
|
message.content.some((b) => b.type === "text" || b.type === "tool-call"))
|
|
155
153
|
tokens = body + policy.tokensPerMessage;
|
|
156
154
|
}
|
|
157
|
-
|
|
155
|
+
// Upstream compaction reads this compatibility field for shadowedTokenCount.
|
|
156
|
+
return { seq: node.seq, tokens, heuristicTokens: tokens };
|
|
158
157
|
});
|
|
159
158
|
const surfaceTokens = priced.reduce((sum, node) => sum + node.tokens, 0);
|
|
160
159
|
const toolsTokens = header?.tools?.length
|
package/dist/core.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"core.js","sourceRoot":"","sources":["../src/core.ts"],"names":[],"mappings":"AAEA,OAAO,EAAE,gBAAgB,EAAE,MAAM,0BAA0B,CAAC;AAyB5D,OAAO,EACN,eAAe,EACf,kBAAkB,EAClB,YAAY,EACZ,cAAc,GACd,MAAM,0BAA0B,CAAC;AAClC,OAAO,EAAE,
|
|
1
|
+
{"version":3,"file":"core.js","sourceRoot":"","sources":["../src/core.ts"],"names":[],"mappings":"AAEA,OAAO,EAAE,gBAAgB,EAAE,MAAM,0BAA0B,CAAC;AAyB5D,OAAO,EACN,eAAe,EACf,kBAAkB,EAClB,YAAY,EACZ,cAAc,GACd,MAAM,0BAA0B,CAAC;AAClC,OAAO,EAAE,wBAAwB,EAAE,MAAM,sBAAsB,CAAC;AAChE,OAAO,EAAE,UAAU,EAAE,MAAM,8BAA8B,CAAC;AAC1D,OAAO,EAAE,SAAS,EAAE,aAAa,EAAE,MAAM,iBAAiB,CAAC;AAE3D,MAAM,CAAC,MAAM,YAAY,GAAG,GAAgB,EAAE,CAAC,CAAC;IAC/C,KAAK,EAAE,EAAE;IACT,MAAM,EAAE,IAAI;IACZ,MAAM,EAAE,IAAI;IACZ,WAAW,EAAE,gBAAgB,CAAC,CAAC,CAAC;CAChC,CAAC,CAAC;AAEH,MAAM,UAAU,WAAW,CAAC,KAAiB;IAC5C,OAAO,CACN,KAAK,CAAC,WAAW;QACjB,CAAC,KAAK,CAAC,eAAe,IAAI,CAAC,CAAC;QAC5B,CAAC,KAAK,CAAC,gBAAgB,IAAI,CAAC,CAAC,CAC7B,CAAC;AACH,CAAC;AAED,MAAM,UAAU,OAAO,CACtB,KAGC;IAED,OAAO,KAAK,CAAC,IAAI,KAAK,mBAAmB,IAAI,KAAK,CAAC,IAAI,CAAC,KAAK,KAAK,SAAS;QAC1E,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK;QAClB,CAAC,CAAC,wBAAwB,CAAC,KAAK,CAAC,IAAI,CAAC,MAAM,EAAE,OAAO,CAAC,EAAE,KAAK,CAAC;AAChE,CAAC;AAED,oEAAoE;AACpE,MAAM,UAAU,IAAI,CAAC,KAAkB,EAAE,KAAiB;IACzD,MAAM,IAAI,GAAgB;QACzB,GAAG,KAAK;QACR,WAAW,EAAE,gBAAgB,CAAC,KAAK,CAAC,GAAG,GAAG,CAAC,CAAC;KAC5C,CAAC;IACF,IAAI,KAAK,CAAC,IAAI,KAAK,mBAAmB;QAAE,OAAO,IAAI,CAAC;IACpD,IAAI,KAAK,CAAC,IAAI,KAAK,gBAAgB;QAClC,IAAI,CAAC,MAAM,GAAG,eAAe,CAAC,KAAK,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;IAClD,4EAA4E;IAC5E,IAAI,KAAK,CAAC,IAAI,KAAK,mBAAmB,EAAE,CAAC;QACxC,MAAM,KAAK,GAAG,OAAO,CAAC,KAAK,CAAC,CAAC;QAC7B,IAAI,KAAK,IAAI,WAAW,CAAC,KAAK,CAAC,GAAG,CAAC,IAAI,IAAI,CAAC,MAAM;YACjD,IAAI,CAAC,MAAM,GAAG;gBACb,MAAM,EAAE,IAAI,CAAC,MAAM;gBACnB,KAAK,EAAE,KAAK,CAAC,KAAK;gBAClB,KAAK;aACL,CAAC;IACJ,CAAC;IACD,IAAI,KAAK,CAAC,IAAI,KAAK,eAAe,EAAE,CAAC;QACpC,MAAM,OAAO,GAAG,IAAI,GAAG,CACtB,KAAK,CAAC,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,GAAG,EAAE,IAAI,GAAG,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,CAC/D,CAAC;QACF,IAAI,CAAC,KAAK,GAAG,KAAK,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE;YACrC,MAAM,OAAO,GAAG,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;YACtC,IAAI,CAAC,OAAO,IAAI,CAAC,IAAI,CAAC,OAAO;gBAAE,OAAO,IAAI,CAAC;YAC3C,IAAI,KAAK,GAAG,CAAC,CAAC;YACd,MAAM,OAAO,GAAG,IAAI,CAAC,OAAO,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAClD,KAAK,CAAC,IAAI,KAAK,OAAO;gBACrB,CAAC,CAAC,KAAK;gBACP,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC,KAAK,EAAE,CAAC;oBACrB,CAAC,CAAC,EAAE,GAAG,KAAK,EAAE,SAAS,EAAE,IAAa,EAAE;oBACxC,CAAC,CAAC,KAAK,CACT,CAAC;YACF,OAAO,EAAE,GAAG,IAAI,EAAE,OAAO,EAAE,EAAE,GAAG,IAAI,CAAC,OAAO,EAAE,OAAO,EAAE,EAAE,CAAC;QAC3D,CAAC,CAAC,CAAC;IACJ,CAAC;IACD,IAAI,KAAK,CAAC,IAAI,KAAK,eAAe;QAAE,OAAO,IAAI,CAAC;IAChD,IAAI,cAAc,CAAC,KAAK,CAAC,EAAE,CAAC;QAC3B,MAAM,OAAO,GAAG,kBAAkB,CAAC,KAAK,CAAC,CAAC;QAC1C,MAAM,IAAI,GAAG;YACZ,GAAG,EAAE,KAAK,CAAC,GAAG;YACd,OAAO;SACP,CAAC;QACF,IAAI,KAAK,CAAC,SAAS,KAAK,QAAQ;YAAE,IAAI,CAAC,KAAK,GAAG,CAAC,GAAG,KAAK,CAAC,KAAK,EAAE,IAAI,CAAC,CAAC;aACjE,CAAC;YACL,MAAM,EAAE,QAAQ,EAAE,MAAM,EAAE,GAAG,KAAK,CAAC,SAAS,CAAC;YAC7C,MAAM,KAAK,GAAG,KAAK,CAAC,KAAK,CAAC,SAAS,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,GAAG,KAAK,QAAQ,CAAC,CAAC;YAC/D,MAAM,GAAG,GAAG,KAAK,CAAC,KAAK,CAAC,SAAS,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,GAAG,KAAK,MAAM,CAAC,CAAC;YAC3D,IAAI,KAAK,GAAG,CAAC,IAAI,GAAG,GAAG,KAAK;gBAC3B,MAAM,IAAI,KAAK,CAAC,sCAAsC,KAAK,CAAC,GAAG,EAAE,CAAC,CAAC;YACpE,IAAI,CAAC,KAAK,GAAG;gBACZ,GAAG,KAAK,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,EAAE,KAAK,CAAC;gBAC9B,IAAI;gBACJ,GAAG,KAAK,CAAC,KAAK,CAAC,KAAK,CAAC,GAAG,GAAG,CAAC,CAAC;aAC7B,CAAC;QACH,CAAC;IACF,CAAC;IACD,OAAO,IAAI,CAAC;AACb,CAAC;AAED,SAAS,aAAa,CACrB,MAA0B,EAC1B,MAAwC,EACxC,WAAwB;IAExB,MAAM,KAAK,GAAG,CAAC,IAAY,EAAE,EAAE,CAAC,SAAS,CAAC,IAAI,EAAE,MAAM,CAAC,SAAS,CAAC,CAAC;IAClE,IAAI,MAAM,GAAG,CAAC,CAAC;IACf,KAAK,MAAM,KAAK,IAAI,MAAM,EAAE,CAAC;QAC5B,QAAQ,KAAK,CAAC,IAAI,EAAE,CAAC;YACpB,KAAK,MAAM;gBACV,MAAM,IAAI,KAAK,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;gBAC5B,MAAM;YACP,KAAK,WAAW;gBACf,IAAI,MAAM,CAAC,gBAAgB;oBAAE,MAAM,IAAI,KAAK,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;gBACzD,MAAM;YACP,KAAK,WAAW;gBACf,MAAM,IAAI,KAAK,CAAC,KAAK,CAAC,IAAI,CAAC,GAAG,KAAK,CAAC,KAAK,CAAC,SAAS,CAAC,CAAC;gBACrD,IAAI,KAAK,CAAC,EAAE;oBAAE,MAAM,IAAI,KAAK,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;gBACxC,MAAM;YACP,KAAK,OAAO,CAAC,CAAC,CAAC;gBACd,IAAI,WAAW,CAAC,YAAY,EAAE,CAAC;oBAC9B,MAAM,KAAK,GAAG,WAAW,CAAC,WAAW,CAAC,WAAW,CAAC,WAAW,EAAE,CAAC,CAAC;oBACjE,MAAM,IAAI,KAAK,CAAC,YAAY,GAAG,KAAK,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;gBAClD,CAAC;qBAAM,CAAC;oBACP,kFAAkF;oBAClF,WAAW,CAAC,cAAc,GAAG,IAAI,CAAC;oBAClC,MAAM,IAAI,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,KAAK,CAAC,CAAC,CAAC;gBACxC,CAAC;gBACD,MAAM;YACP,CAAC;YACD,KAAK,MAAM;gBACV,IAAI,WAAW,CAAC,QAAQ;oBACvB,MAAM,IAAI,KAAK,CAAC,WAAW,CAAC,QAAQ,CAAC,KAAK,CAAC,UAAU,CAAC,CAAC,CAAC;qBACpD,CAAC;oBACL,WAAW,CAAC,aAAa,GAAG,IAAI,CAAC;oBACjC,MAAM,IAAI,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,KAAK,CAAC,CAAC,CAAC;gBACxC,CAAC;gBACD,MAAM;YACP;gBACC,MAAM,IAAI,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,KAAK,CAAC,CAAC,CAAC;QACzC,CAAC;IACF,CAAC;IACD,OAAO,MAAM,CAAC;AACf,CAAC;AAED,MAAM,UAAU,YAAY,CAC3B,KAA4B,EAC5B,MAA0B,EAC1B,MAAmB,EACnB,QAAuB;IAEvB,MAAM,MAAM,GAAG,aAAa,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IAC7C,MAAM,KAAK,GAAG,MAAM,EAAE,MAAM,CAAC;IAC7B,MAAM,GAAG,GAAG,QAAQ,CAAC,GAAG,CAAC;IACzB,MAAM,WAAW,GAAgB;QAChC,WAAW,EAAE,EAAE;QACf,WAAW,EAAE,CAAC;QACd,YAAY,EACX,KAAK,IAAI,QAAQ,CAAC,GAAG,EAAE,mBAAmB,CAAC,KAAK,CAAC,QAAQ,EAAE,KAAK,CAAC,KAAK,CAAC;QACxE,QAAQ,EAAE,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,GAAG,CAAC,eAAe,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,SAAS;QAC7D,cAAc,EAAE,KAAK;QACrB,aAAa,EAAE,KAAK;KACpB,CAAC;IACF,MAAM,MAAM,GAAG,KAAK,CAAC,OAAO,CAC3B,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,OAAO,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,KAAK,OAAO,CAAC,IAAI,EAAE,CACjE,CAAC;IACF,WAAW,CAAC,WAAW;QACtB,WAAW,CAAC,YAAY,IAAI,MAAM,CAAC,MAAM;YACxC,CAAC,CAAC,WAAW,CAAC,YAAY,CAAC,WAAW,CAAC,MAAM,CAAC;YAC9C,CAAC,CAAC,EAAE,CAAC;IACP,IACC,WAAW,CAAC,YAAY;QACxB,CAAC,WAAW,CAAC,WAAW,CAAC,MAAM,KAAK,MAAM,CAAC,MAAM;YAChD,WAAW,CAAC,WAAW,CAAC,IAAI,CAC3B,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,MAAM,CAAC,aAAa,CAAC,CAAC,CAAC,YAAY,CAAC,IAAI,CAAC,CAAC,YAAY,GAAG,CAAC,CAClE,CAAC;QAEH,MAAM,IAAI,KAAK,CAAC,6BAA6B,CAAC,CAAC;IAChD,MAAM,MAAM,GAAG,KAAK,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE;QACjC,MAAM,OAAO,GAAG,IAAI,CAAC,OAAO,CAAC;QAC7B,IAAI,MAAM,GAAG,CAAC,CAAC;QACf,IAAI,OAAO,EAAE,CAAC;YACb,MAAM,IAAI,GAAG,aAAa,CAAC,OAAO,CAAC,OAAO,EAAE,MAAM,EAAE,WAAW,CAAC,CAAC;YACjE,IACC,IAAI,GAAG,CAAC;gBACR,OAAO,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,KAAK,MAAM,IAAI,CAAC,CAAC,IAAI,KAAK,WAAW,CAAC;gBAExE,MAAM,GAAG,IAAI,GAAG,MAAM,CAAC,gBAAgB,CAAC;QAC1C,CAAC;QACD,6EAA6E;QAC7E,OAAO,EAAE,GAAG,EAAE,IAAI,CAAC,GAAG,EAAE,MAAM,EAAE,eAAe,EAAE,MAAM,EAAE,CAAC;IAC3D,CAAC,CAAC,CAAC;IACH,MAAM,aAAa,GAAG,MAAM,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,IAAI,EAAE,EAAE,CAAC,GAAG,GAAG,IAAI,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC;IACzE,MAAM,WAAW,GAAG,MAAM,EAAE,KAAK,EAAE,MAAM;QACxC,CAAC,CAAC,SAAS,CAAC,IAAI,CAAC,SAAS,CAAC,MAAM,CAAC,KAAK,CAAC,EAAE,MAAM,CAAC,SAAS,CAAC;YAC1D,MAAM,CAAC,aAAa;QACrB,CAAC,CAAC,CAAC,CAAC;IACL,MAAM,aAAa,GAAG,MAAM,IAAI,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,MAAM,CAAC,kBAAkB,CAAC,CAAC,CAAC,CAAC,CAAC;IAC7E,OAAO;QACN,KAAK,EAAE,MAAM;QACb,aAAa;QACb,WAAW;QACX,aAAa;QACb,aAAa,EAAE,aAAa,GAAG,WAAW,GAAG,aAAa;QAC1D,MAAM;QACN,cAAc,EAAE,WAAW,CAAC,cAAc;QAC1C,aAAa,EAAE,WAAW,CAAC,aAAa;KACxC,CAAC;AACH,CAAC;AAED;;GAEG;AACH,MAAM,UAAU,YAAY,CAC3B,KAAkB,EAClB,MAAmB,EACnB,WAA0B,EAAE,EAC5B,aAA2B;IAE3B,MAAM,MAAM,GACX,aAAa,KAAK,SAAS,CAAC,CAAC,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,eAAe,CAAC,aAAa,CAAC,CAAC;IAC7E,MAAM,OAAO,GAAG,YAAY,CAAC,KAAK,CAAC,KAAK,EAAE,MAAM,EAAE,MAAM,EAAE,QAAQ,CAAC,CAAC;IACpE,MAAM,MAAM,GAAG,KAAK,CAAC,MAAM,CAAC;IAC5B,MAAM,QAAQ,GAAG,MAAM,IAAI,MAAM,IAAI,YAAY,CAAC,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IACzE,IAAI,QAAkC,CAAC;IACvC,IAAI,kBAAkB,GAAG,CAAC,CAAC;IAC3B,IAAI,QAAQ,EAAE,CAAC;QACd,MAAM,QAAQ,GAAG,YAAY,CAAC,MAAM,CAAC,KAAK,EAAE,MAAM,EAAE,MAAM,EAAE,QAAQ,CAAC,CAAC;QACtE,QAAQ,GAAG;YACV,IAAI,EAAE,OAAO;YACb,MAAM,EAAE,WAAW,CAAC,MAAM,CAAC,KAAK,CAAC;YACjC,KAAK,EAAE,MAAM,CAAC,KAAK;SACnB,CAAC;QACF,kBAAkB,GAAG,OAAO,CAAC,aAAa,GAAG,QAAQ,CAAC,aAAa,CAAC;IACrE,CAAC;;QACA,QAAQ;YACP,CAAC,MAAM,IAAI,KAAK,CAAC,KAAK,CAAC,MAAM,KAAK,CAAC;gBAClC,CAAC,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,MAAM,EAAE,CAAC,EAAE;gBAC7B,CAAC,CAAC,EAAE,IAAI,EAAE,WAAW,EAAE,MAAM,EAAE,OAAO,CAAC,aAAa,EAAE,CAAC;IAC1D,OAAO,UAAU,CAAC;QACjB,WAAW,EAAE,KAAK,CAAC,WAAW;QAC9B,QAAQ;QACR,kBAAkB;QAClB,WAAW,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,QAAQ,CAAC,MAAM,GAAG,kBAAkB,CAAC;QAC9D,aAAa,EAAE,OAAO,CAAC,aAAa;QACpC,WAAW,EAAE,OAAO,CAAC,WAAW;QAChC,KAAK,EAAE,OAAO,CAAC,KAAK;QACpB,SAAS,EAAE,OAAO,CAAC,MAAM,CAAC,SAAS;QACnC,cAAc,EAAE,OAAO,CAAC,cAAc;QACtC,aAAa,EAAE,OAAO,CAAC,aAAa;KACpC,CAAC,CAAC;AACJ,CAAC"}
|
package/dist/projection.d.ts
CHANGED
|
@@ -8,15 +8,19 @@ import type { MeterServices } from "./core.js";
|
|
|
8
8
|
import type { MeterConfig, ReplayState, ReplayNode } from "./types.js";
|
|
9
9
|
export declare const replayProjection: ProjectionDefinition<"contextPolicyMeter">;
|
|
10
10
|
interface DisplayState extends ReplayState {
|
|
11
|
+
systems: {
|
|
12
|
+
seq: SessionSeq;
|
|
13
|
+
tokens: number;
|
|
14
|
+
}[];
|
|
11
15
|
nodes: (ReplayNode & {
|
|
12
16
|
system: boolean;
|
|
17
|
+
heuristicTokens: number;
|
|
13
18
|
})[];
|
|
14
19
|
surfaceTokens: number;
|
|
15
20
|
breakdown: ContextBreakdownProjection;
|
|
16
21
|
contextWindow?: number;
|
|
17
22
|
}
|
|
18
23
|
declare function initialDisplayState(): DisplayState;
|
|
19
|
-
declare function foldDisplay(state: DisplayState, event: SessionEvent): DisplayState;
|
|
20
24
|
export declare function createDisplayProjections(config: MeterConfig, services: MeterServices): {
|
|
21
25
|
pressure: {
|
|
22
26
|
key: "contextPressure";
|
|
@@ -41,11 +45,15 @@ export declare function createDisplayProjections(config: MeterConfig, services:
|
|
|
41
45
|
usage: import("@deepseek-ai/dsh-llm").TokenUsage;
|
|
42
46
|
}>>;
|
|
43
47
|
logRevision: z.ZodPipe<z.ZodNumber, z.ZodTransform<SessionLogOffset, number>>;
|
|
48
|
+
systems: z.ZodArray<z.ZodObject<{
|
|
49
|
+
seq: z.ZodPipe<z.ZodNumber, z.ZodTransform<SessionSeq, number>>;
|
|
50
|
+
tokens: z.ZodNumber;
|
|
51
|
+
}, z.core.$strip>>;
|
|
44
52
|
nodes: z.ZodArray<z.ZodObject<{
|
|
45
53
|
seq: z.ZodPipe<z.ZodNumber, z.ZodTransform<SessionSeq, number>>;
|
|
46
54
|
message: z.ZodNullable<z.ZodCustom<Message, Message>>;
|
|
47
|
-
heuristicTokens: z.ZodNumber;
|
|
48
55
|
system: z.ZodBoolean;
|
|
56
|
+
heuristicTokens: z.ZodNumber;
|
|
49
57
|
}, z.core.$strip>>;
|
|
50
58
|
surfaceTokens: z.ZodNumber;
|
|
51
59
|
breakdown: z.ZodObject<{
|
|
@@ -56,7 +64,7 @@ export declare function createDisplayProjections(config: MeterConfig, services:
|
|
|
56
64
|
contextWindow: z.ZodOptional<z.ZodNumber>;
|
|
57
65
|
}, z.core.$strip>;
|
|
58
66
|
init: typeof initialDisplayState;
|
|
59
|
-
apply:
|
|
67
|
+
apply: (state: DisplayState, event: SessionEvent) => DisplayState;
|
|
60
68
|
};
|
|
61
69
|
breakdown: {
|
|
62
70
|
key: "contextBreakdown";
|
|
@@ -81,11 +89,15 @@ export declare function createDisplayProjections(config: MeterConfig, services:
|
|
|
81
89
|
usage: import("@deepseek-ai/dsh-llm").TokenUsage;
|
|
82
90
|
}>>;
|
|
83
91
|
logRevision: z.ZodPipe<z.ZodNumber, z.ZodTransform<SessionLogOffset, number>>;
|
|
92
|
+
systems: z.ZodArray<z.ZodObject<{
|
|
93
|
+
seq: z.ZodPipe<z.ZodNumber, z.ZodTransform<SessionSeq, number>>;
|
|
94
|
+
tokens: z.ZodNumber;
|
|
95
|
+
}, z.core.$strip>>;
|
|
84
96
|
nodes: z.ZodArray<z.ZodObject<{
|
|
85
97
|
seq: z.ZodPipe<z.ZodNumber, z.ZodTransform<SessionSeq, number>>;
|
|
86
98
|
message: z.ZodNullable<z.ZodCustom<Message, Message>>;
|
|
87
|
-
heuristicTokens: z.ZodNumber;
|
|
88
99
|
system: z.ZodBoolean;
|
|
100
|
+
heuristicTokens: z.ZodNumber;
|
|
89
101
|
}, z.core.$strip>>;
|
|
90
102
|
surfaceTokens: z.ZodNumber;
|
|
91
103
|
breakdown: z.ZodObject<{
|
|
@@ -96,7 +108,7 @@ export declare function createDisplayProjections(config: MeterConfig, services:
|
|
|
96
108
|
contextWindow: z.ZodOptional<z.ZodNumber>;
|
|
97
109
|
}, z.core.$strip>;
|
|
98
110
|
init: typeof initialDisplayState;
|
|
99
|
-
apply:
|
|
111
|
+
apply: (state: DisplayState, event: SessionEvent) => DisplayState;
|
|
100
112
|
};
|
|
101
113
|
};
|
|
102
114
|
export {};
|
package/dist/projection.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"projection.d.ts","sourceRoot":"","sources":["../src/projection.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,gBAAgB,EAAE,UAAU,EAAE,MAAM,0BAA0B,CAAC;AACxE,OAAO,KAAK,EAAE,WAAW,EAAE,YAAY,EAAE,MAAM,0BAA0B,CAAC;AAC1E,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,sBAAsB,CAAC;AACpD,OAAO,KAAK,EAAE,oBAAoB,EAAE,MAAM,qCAAqC,CAAC;AAChF,OAAO,KAAK,EACX,yBAAyB,EACzB,0BAA0B,EAC1B,MAAM,8BAA8B,CAAC;
|
|
1
|
+
{"version":3,"file":"projection.d.ts","sourceRoot":"","sources":["../src/projection.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,gBAAgB,EAAE,UAAU,EAAE,MAAM,0BAA0B,CAAC;AACxE,OAAO,KAAK,EAAE,WAAW,EAAE,YAAY,EAAE,MAAM,0BAA0B,CAAC;AAC1E,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,sBAAsB,CAAC;AACpD,OAAO,KAAK,EAAE,oBAAoB,EAAE,MAAM,qCAAqC,CAAC;AAChF,OAAO,KAAK,EACX,yBAAyB,EACzB,0BAA0B,EAC1B,MAAM,8BAA8B,CAAC;AACtC,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AAExB,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,WAAW,CAAC;AAC/C,OAAO,KAAK,EAAE,WAAW,EAAE,WAAW,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC;AAavE,eAAO,MAAM,gBAAgB,EAAE,oBAAoB,CAAC,oBAAoB,CAMvE,CAAC;AAGF,UAAU,YAAa,SAAQ,WAAW;IACzC,OAAO,EAAE;QAAE,GAAG,EAAE,UAAU,CAAC;QAAC,MAAM,EAAE,MAAM,CAAA;KAAE,EAAE,CAAC;IAC/C,KAAK,EAAE,CAAC,UAAU,GAAG;QAAE,MAAM,EAAE,OAAO,CAAC;QAAC,eAAe,EAAE,MAAM,CAAA;KAAE,CAAC,EAAE,CAAC;IACrE,aAAa,EAAE,MAAM,CAAC;IACtB,SAAS,EAAE,0BAA0B,CAAC;IACtC,aAAa,CAAC,EAAE,MAAM,CAAC;CACvB;AAeD,iBAAS,mBAAmB,IAAI,YAAY,CAQ3C;AAmDD,wBAAgB,wBAAwB,CACvC,MAAM,EAAE,WAAW,EACnB,QAAQ,EAAE,aAAa;;;;;;;;;0BAmDP,YAAY;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;uBAZZ,YAAY,SAAS,YAAY;;;;;;;;;;0BAoBjC,YAAY;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;uBApBZ,YAAY,SAAS,YAAY;;EAwBjD"}
|
package/dist/projection.js
CHANGED
|
@@ -1,12 +1,10 @@
|
|
|
1
1
|
import { SessionLogOffset, SessionSeq } from "@deepseek-ai/dsh-session";
|
|
2
|
-
import { estimateToolsTokens } from "@deepseek-ai/dsh-token-meter/estimate";
|
|
3
2
|
import { z } from "zod";
|
|
4
3
|
import { initialState, fold, measureState } from "./core.js";
|
|
5
4
|
const count = z.number().int().nonnegative();
|
|
6
5
|
const nodeSchema = z.object({
|
|
7
6
|
seq: count.transform(SessionSeq),
|
|
8
7
|
message: z.custom().nullable(),
|
|
9
|
-
heuristicTokens: count,
|
|
10
8
|
});
|
|
11
9
|
const replaySchema = z.object({
|
|
12
10
|
nodes: z.array(nodeSchema),
|
|
@@ -16,7 +14,7 @@ const replaySchema = z.object({
|
|
|
16
14
|
});
|
|
17
15
|
export const replayProjection = {
|
|
18
16
|
key: "contextPolicyMeter",
|
|
19
|
-
stateVersion:
|
|
17
|
+
stateVersion: 2,
|
|
20
18
|
stateSchema: replaySchema,
|
|
21
19
|
init: initialState,
|
|
22
20
|
apply: fold,
|
|
@@ -27,7 +25,8 @@ const breakdownSchema = z.object({
|
|
|
27
25
|
messageTokens: count,
|
|
28
26
|
});
|
|
29
27
|
const displaySchema = replaySchema.extend({
|
|
30
|
-
|
|
28
|
+
systems: z.array(z.object({ seq: count.transform(SessionSeq), tokens: count })),
|
|
29
|
+
nodes: z.array(nodeSchema.extend({ system: z.boolean(), heuristicTokens: count })),
|
|
31
30
|
surfaceTokens: count,
|
|
32
31
|
breakdown: breakdownSchema,
|
|
33
32
|
contextWindow: z.number().int().positive().optional(),
|
|
@@ -35,12 +34,13 @@ const displaySchema = replaySchema.extend({
|
|
|
35
34
|
function initialDisplayState() {
|
|
36
35
|
return {
|
|
37
36
|
...initialState(),
|
|
37
|
+
systems: [],
|
|
38
38
|
nodes: [],
|
|
39
39
|
surfaceTokens: 0,
|
|
40
40
|
breakdown: { systemTokens: 0, toolsTokens: 0, messageTokens: 0 },
|
|
41
41
|
};
|
|
42
42
|
}
|
|
43
|
-
function foldDisplay(state, event) {
|
|
43
|
+
function foldDisplay(state, event, config, services) {
|
|
44
44
|
const next = fold(state, event);
|
|
45
45
|
const { contextWindow, ...withoutWindow } = {
|
|
46
46
|
...next,
|
|
@@ -52,28 +52,27 @@ function foldDisplay(state, event) {
|
|
|
52
52
|
next.anchor === state.anchor &&
|
|
53
53
|
window === state.contextWindow)
|
|
54
54
|
return state;
|
|
55
|
-
const
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
: nodes.
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
55
|
+
const measurement = measureState(next, config, services);
|
|
56
|
+
// Empty system events have no derived Message but still occupy a surface position.
|
|
57
|
+
const systemSeqs = new Set(state.systems.map((node) => node.seq));
|
|
58
|
+
const nodes = next.nodes.map((node, index) => ({
|
|
59
|
+
...node,
|
|
60
|
+
system: systemSeqs.has(node.seq) ||
|
|
61
|
+
(event.type === "system/message" && node.seq === event.seq),
|
|
62
|
+
// Native consumers use this field; it carries the model count, not a second estimator.
|
|
63
|
+
heuristicTokens: measurement.nodes[index].tokens,
|
|
64
|
+
}));
|
|
65
|
+
const surfaceTokens = measurement.surfaceTokens;
|
|
66
|
+
const systems = nodes.flatMap((node) => node.system ? [{ seq: node.seq, tokens: node.heuristicTokens }] : []);
|
|
67
|
+
const systemTokens = systems.findLast((node) => node.tokens > 0)?.tokens ?? 0;
|
|
68
68
|
return {
|
|
69
69
|
...withoutWindow,
|
|
70
|
+
systems,
|
|
70
71
|
nodes,
|
|
71
72
|
surfaceTokens,
|
|
72
73
|
breakdown: {
|
|
73
74
|
systemTokens,
|
|
74
|
-
toolsTokens:
|
|
75
|
-
? state.breakdown.toolsTokens
|
|
76
|
-
: estimateToolsTokens(next.header ?? undefined),
|
|
75
|
+
toolsTokens: measurement.toolsTokens,
|
|
77
76
|
messageTokens: surfaceTokens - systemTokens,
|
|
78
77
|
},
|
|
79
78
|
...(window === undefined ? {} : { contextWindow: window }),
|
|
@@ -86,8 +85,7 @@ export function createDisplayProjections(config, services) {
|
|
|
86
85
|
if (cached)
|
|
87
86
|
return cached;
|
|
88
87
|
const measurement = measureState(state, config, services);
|
|
89
|
-
const
|
|
90
|
-
const systemTokens = measurement.nodes.find((node) => node.seq === system?.seq)?.tokens ?? 0;
|
|
88
|
+
const systemTokens = measurement.nodes.findLast((node, index) => state.nodes[index].system && node.tokens > 0)?.tokens ?? 0;
|
|
91
89
|
const pressure = {
|
|
92
90
|
projectedTokens: measurement.totalTokens,
|
|
93
91
|
...(measurement.baseline.kind === "usage"
|
|
@@ -107,10 +105,10 @@ export function createDisplayProjections(config, services) {
|
|
|
107
105
|
return value;
|
|
108
106
|
};
|
|
109
107
|
const replay = {
|
|
110
|
-
stateVersion:
|
|
108
|
+
stateVersion: 9,
|
|
111
109
|
stateSchema: displaySchema,
|
|
112
110
|
init: initialDisplayState,
|
|
113
|
-
apply: foldDisplay,
|
|
111
|
+
apply: (state, event) => foldDisplay(state, event, config, services),
|
|
114
112
|
};
|
|
115
113
|
const pressure = {
|
|
116
114
|
...replay,
|
package/dist/projection.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"projection.js","sourceRoot":"","sources":["../src/projection.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,gBAAgB,EAAE,UAAU,EAAE,MAAM,0BAA0B,CAAC;AAQxE,OAAO,EAAE,
|
|
1
|
+
{"version":3,"file":"projection.js","sourceRoot":"","sources":["../src/projection.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,gBAAgB,EAAE,UAAU,EAAE,MAAM,0BAA0B,CAAC;AAQxE,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AACxB,OAAO,EAAE,YAAY,EAAE,IAAI,EAAE,YAAY,EAAE,MAAM,WAAW,CAAC;AAI7D,MAAM,KAAK,GAAG,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,WAAW,EAAE,CAAC;AAC7C,MAAM,UAAU,GAAG,CAAC,CAAC,MAAM,CAAC;IAC3B,GAAG,EAAE,KAAK,CAAC,SAAS,CAAC,UAAU,CAAC;IAChC,OAAO,EAAE,CAAC,CAAC,MAAM,EAAW,CAAC,QAAQ,EAAE;CACvC,CAAC,CAAC;AACH,MAAM,YAAY,GAAG,CAAC,CAAC,MAAM,CAAC;IAC7B,KAAK,EAAE,CAAC,CAAC,KAAK,CAAC,UAAU,CAAC;IAC1B,MAAM,EAAE,CAAC,CAAC,MAAM,EAAe,CAAC,QAAQ,EAAE;IAC1C,MAAM,EAAE,CAAC,CAAC,MAAM,EAAsC,CAAC,QAAQ,EAAE;IACjE,WAAW,EAAE,KAAK,CAAC,SAAS,CAAC,gBAAgB,CAAC;CAC9C,CAAC,CAAC;AACH,MAAM,CAAC,MAAM,gBAAgB,GAA+C;IAC3E,GAAG,EAAE,oBAAoB;IACzB,YAAY,EAAE,CAAC;IACf,WAAW,EAAE,YAAY;IACzB,IAAI,EAAE,YAAY;IAClB,KAAK,EAAE,IAAI;CACX,CAAC;AAUF,MAAM,eAAe,GAAG,CAAC,CAAC,MAAM,CAAC;IAChC,YAAY,EAAE,KAAK;IACnB,WAAW,EAAE,KAAK;IAClB,aAAa,EAAE,KAAK;CACpB,CAAC,CAAC;AACH,MAAM,aAAa,GAAG,YAAY,CAAC,MAAM,CAAC;IACzC,OAAO,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,MAAM,CAAC,EAAE,GAAG,EAAE,KAAK,CAAC,SAAS,CAAC,UAAU,CAAC,EAAE,MAAM,EAAE,KAAK,EAAE,CAAC,CAAC;IAC/E,KAAK,EAAE,CAAC,CAAC,KAAK,CACb,UAAU,CAAC,MAAM,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC,OAAO,EAAE,EAAE,eAAe,EAAE,KAAK,EAAE,CAAC,CAClE;IACD,aAAa,EAAE,KAAK;IACpB,SAAS,EAAE,eAAe;IAC1B,aAAa,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,EAAE;CACrD,CAAC,CAAC;AACH,SAAS,mBAAmB;IAC3B,OAAO;QACN,GAAG,YAAY,EAAE;QACjB,OAAO,EAAE,EAAE;QACX,KAAK,EAAE,EAAE;QACT,aAAa,EAAE,CAAC;QAChB,SAAS,EAAE,EAAE,YAAY,EAAE,CAAC,EAAE,WAAW,EAAE,CAAC,EAAE,aAAa,EAAE,CAAC,EAAE;KAChE,CAAC;AACH,CAAC;AACD,SAAS,WAAW,CACnB,KAAmB,EACnB,KAAmB,EACnB,MAAmB,EACnB,QAAuB;IAEvB,MAAM,IAAI,GAAG,IAAI,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC;IAChC,MAAM,EAAE,aAAa,EAAE,GAAG,aAAa,EAAE,GAAG;QAC3C,GAAG,IAAI;QACP,aAAa,EAAE,KAAK,CAAC,aAAa;KAClC,CAAC;IACF,MAAM,MAAM,GACX,KAAK,CAAC,IAAI,KAAK,iBAAiB,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,aAAa,CAAC,CAAC,CAAC,aAAa,CAAC;IAC7E,IACC,IAAI,CAAC,KAAK,KAAK,KAAK,CAAC,KAAK;QAC1B,IAAI,CAAC,MAAM,KAAK,KAAK,CAAC,MAAM;QAC5B,IAAI,CAAC,MAAM,KAAK,KAAK,CAAC,MAAM;QAC5B,MAAM,KAAK,KAAK,CAAC,aAAa;QAE9B,OAAO,KAAK,CAAC;IACd,MAAM,WAAW,GAAG,YAAY,CAAC,IAAI,EAAE,MAAM,EAAE,QAAQ,CAAC,CAAC;IACzD,mFAAmF;IACnF,MAAM,UAAU,GAAG,IAAI,GAAG,CAAC,KAAK,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC;IAClE,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,KAAK,EAAE,EAAE,CAAC,CAAC;QAC9C,GAAG,IAAI;QACP,MAAM,EACL,UAAU,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC;YACxB,CAAC,KAAK,CAAC,IAAI,KAAK,gBAAgB,IAAI,IAAI,CAAC,GAAG,KAAK,KAAK,CAAC,GAAG,CAAC;QAC5D,uFAAuF;QACvF,eAAe,EAAE,WAAW,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,MAAM;KAChD,CAAC,CAAC,CAAC;IACJ,MAAM,aAAa,GAAG,WAAW,CAAC,aAAa,CAAC;IAChD,MAAM,OAAO,GAAG,KAAK,CAAC,OAAO,CAAC,CAAC,IAAI,EAAE,EAAE,CACtC,IAAI,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,EAAE,GAAG,EAAE,IAAI,CAAC,GAAG,EAAE,MAAM,EAAE,IAAI,CAAC,eAAe,EAAE,CAAC,CAAC,CAAC,CAAC,EAAE,CACpE,CAAC;IACF,MAAM,YAAY,GAAG,OAAO,CAAC,QAAQ,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,EAAE,MAAM,IAAI,CAAC,CAAC;IAC9E,OAAO;QACN,GAAG,aAAa;QAChB,OAAO;QACP,KAAK;QACL,aAAa;QACb,SAAS,EAAE;YACV,YAAY;YACZ,WAAW,EAAE,WAAW,CAAC,WAAW;YACpC,aAAa,EAAE,aAAa,GAAG,YAAY;SAC3C;QACD,GAAG,CAAC,MAAM,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,aAAa,EAAE,MAAM,EAAE,CAAC;KAC1D,CAAC;AACH,CAAC;AAED,MAAM,UAAU,wBAAwB,CACvC,MAAmB,EACnB,QAAuB;IAEvB,MAAM,KAAK,GAAG,IAAI,OAAO,EAMtB,CAAC;IACJ,MAAM,IAAI,GAAG,CAAC,KAAmB,EAAE,EAAE;QACpC,MAAM,MAAM,GAAG,KAAK,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC;QAChC,IAAI,MAAM;YAAE,OAAO,MAAM,CAAC;QAC1B,MAAM,WAAW,GAAG,YAAY,CAAC,KAAK,EAAE,MAAM,EAAE,QAAQ,CAAC,CAAC;QAC1D,MAAM,YAAY,GACjB,WAAW,CAAC,KAAK,CAAC,QAAQ,CACzB,CAAC,IAAI,EAAE,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,MAAM,IAAI,IAAI,CAAC,MAAM,GAAG,CAAC,CAC7D,EAAE,MAAM,IAAI,CAAC,CAAC;QAChB,MAAM,QAAQ,GAA8B;YAC3C,eAAe,EAAE,WAAW,CAAC,WAAW;YACxC,GAAG,CAAC,WAAW,CAAC,QAAQ,CAAC,IAAI,KAAK,OAAO;gBACxC,CAAC,CAAC,EAAE,cAAc,EAAE,WAAW,CAAC,QAAQ,CAAC,MAAM,EAAE;gBACjD,CAAC,CAAC,EAAE,CAAC;YACN,GAAG,CAAC,KAAK,CAAC,aAAa,KAAK,SAAS;gBACpC,CAAC,CAAC,EAAE;gBACJ,CAAC,CAAC,EAAE,aAAa,EAAE,KAAK,CAAC,aAAa,EAAE,CAAC;SAC1C,CAAC;QACF,MAAM,SAAS,GAAG;YACjB,YAAY;YACZ,WAAW,EAAE,WAAW,CAAC,WAAW;YACpC,aAAa,EAAE,WAAW,CAAC,aAAa,GAAG,YAAY;SACvD,CAAC;QACF,MAAM,KAAK,GAAG,EAAE,QAAQ,EAAE,SAAS,EAAE,CAAC;QACtC,KAAK,CAAC,GAAG,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC;QACxB,OAAO,KAAK,CAAC;IACd,CAAC,CAAC;IACF,MAAM,MAAM,GAAG;QACd,YAAY,EAAE,CAAC;QACf,WAAW,EAAE,aAAa;QAC1B,IAAI,EAAE,mBAAmB;QACzB,KAAK,EAAE,CAAC,KAAmB,EAAE,KAAmB,EAAE,EAAE,CACnD,WAAW,CAAC,KAAK,EAAE,KAAK,EAAE,MAAM,EAAE,QAAQ,CAAC;KAC5C,CAAC;IACF,MAAM,QAAQ,GAAG;QAChB,GAAG,MAAM;QACT,GAAG,EAAE,iBAAiB;QACtB,IAAI,EAAE;YACL,UAAU,EAAE,CAAC,CAAC,MAAM,CAAC;gBACpB,cAAc,EAAE,KAAK,CAAC,QAAQ,EAAE;gBAChC,eAAe,EAAE,KAAK;gBACtB,aAAa,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,QAAQ,EAAE,CAAC,QAAQ,EAAE;aACrD,CAAC;YACF,IAAI,EAAE,CAAC,KAAmB,EAAE,EAAE,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,QAAQ;SACnD;KAC+D,CAAC;IAClE,MAAM,SAAS,GAAG;QACjB,GAAG,MAAM;QACT,GAAG,EAAE,kBAAkB;QACvB,IAAI,EAAE;YACL,UAAU,EAAE,eAAe;YAC3B,IAAI,EAAE,CAAC,KAAmB,EAAE,EAAE,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,SAAS;SACpD;KACgE,CAAC;IACnE,OAAO,EAAE,QAAQ,EAAE,SAAS,EAAE,CAAC;AAChC,CAAC"}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"token-meter.d.ts","sourceRoot":"","sources":["../src/token-meter.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAE,MAAM,qBAAqB,CAAC;AAC9C,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,qBAAqB,CAAC;AACnD,OAAO,KAAK,EAAE,WAAW,EAAE,OAAO,EAAE,MAAM,0BAA0B,CAAC;AACrE,OAAO,EAAE,UAAU,EAAE,MAAM,0BAA0B,CAAC;AACtD,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,sBAAsB,CAAC;AACpD,OAAO,KAAK,EAAE,WAAW,EAAE,gBAAgB,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AAC7E,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,WAAW,CAAC;AAC/C,OAAO,qCAAqC,CAAC;AAC7C,OAAO,QAAQ,MAAM,0BAA0B,CAAC;AAKhD,OAAO,QAAQ,qCAAqC,CAAC;IACpD,UAAU,yBAAyB;QAClC,kBAAkB,EAAE,WAAW,CAAC;KAChC;CACD;
|
|
1
|
+
{"version":3,"file":"token-meter.d.ts","sourceRoot":"","sources":["../src/token-meter.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAE,MAAM,qBAAqB,CAAC;AAC9C,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,qBAAqB,CAAC;AACnD,OAAO,KAAK,EAAE,WAAW,EAAE,OAAO,EAAE,MAAM,0BAA0B,CAAC;AACrE,OAAO,EAAE,UAAU,EAAE,MAAM,0BAA0B,CAAC;AACtD,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,sBAAsB,CAAC;AACpD,OAAO,KAAK,EAAE,WAAW,EAAE,gBAAgB,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AAC7E,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,WAAW,CAAC;AAC/C,OAAO,qCAAqC,CAAC;AAC7C,OAAO,QAAQ,MAAM,0BAA0B,CAAC;AAKhD,OAAO,QAAQ,qCAAqC,CAAC;IACpD,UAAU,yBAAyB;QAClC,kBAAkB,EAAE,WAAW,CAAC;KAChC;CACD;AAUD,eAAO,MAAM,MAAM,EAAE,QAAQ,CAAC,gBAAgB,EAAE,WAAW,CAmBzD,CAAC;AACH,eAAO,MAAM,MAAM,UAAyB,CAAC;AAE7C,qBAAa,eAAgB,SAAQ,OAAO;gBAC/B,GAAG,EAAE,OAAO,EAAE,MAAM,EAAE,WAAW;IAc7C,MAAM,EAAE,WAAW,CAAC;IACpB,QAAQ,EAAE,aAAa,CAAC;IACxB,OAAO,CAAC,OAAO,EAAE,OAAO,EAAE,aAAa,CAAC,EAAE,WAAW;;;;;;;;;;;;;;;;IAWrD,eAAe,CAAC,OAAO,EAAE,OAAO;CAQhC;AAED,wBAAgB,KAAK,CAAC,GAAG,EAAE,OAAO,EAAE,MAAM,EAAE,WAAW,QAEtD;;;;;;AAED,wBAAyC"}
|
package/dist/token-meter.js
CHANGED
|
@@ -5,18 +5,26 @@ import settings from "@deepseek-ai/schemastery";
|
|
|
5
5
|
import { measureState, priceSurface } from "./core.js";
|
|
6
6
|
import { createDisplayProjections, replayProjection } from "./projection.js";
|
|
7
7
|
import { usageProjection } from "./usage.js";
|
|
8
|
+
const policyFields = {
|
|
9
|
+
tokenizer: settings.string(),
|
|
10
|
+
tokensPerMessage: settings.number().min(0).step(1),
|
|
11
|
+
toolsOverhead: settings.number().min(0).step(1),
|
|
12
|
+
replyPrimingTokens: settings.number().min(0).step(1),
|
|
13
|
+
includeReasoning: settings.boolean(),
|
|
14
|
+
};
|
|
8
15
|
export const Config = settings.object({
|
|
9
16
|
defaultTokenizer: settings.string().default("deepseek_v4"),
|
|
10
|
-
|
|
17
|
+
providerPolicies: settings
|
|
18
|
+
.array(settings.object({
|
|
19
|
+
...policyFields,
|
|
20
|
+
provider: settings.string().required(),
|
|
21
|
+
}))
|
|
22
|
+
.default([]),
|
|
11
23
|
modelPolicies: settings
|
|
12
24
|
.array(settings.object({
|
|
25
|
+
...policyFields,
|
|
13
26
|
provider: settings.string(),
|
|
14
27
|
model: settings.string().required(),
|
|
15
|
-
tokenizer: settings.string(),
|
|
16
|
-
tokensPerMessage: settings.number().min(0).step(1),
|
|
17
|
-
toolsOverhead: settings.number().min(0).step(1),
|
|
18
|
-
replyPrimingTokens: settings.number().min(0).step(1),
|
|
19
|
-
includeReasoning: settings.boolean(),
|
|
20
28
|
}))
|
|
21
29
|
.default([]),
|
|
22
30
|
});
|
|
@@ -45,7 +53,7 @@ export class ModelTokenMeter extends Service {
|
|
|
45
53
|
return measureState(state, this.config, this.services, requestHeader);
|
|
46
54
|
}
|
|
47
55
|
estimateMessage(message) {
|
|
48
|
-
return priceSurface([{ seq: SessionSeq(0), message
|
|
56
|
+
return priceSurface([{ seq: SessionSeq(0), message }], null, this.config, this.services).surfaceTokens;
|
|
49
57
|
}
|
|
50
58
|
}
|
|
51
59
|
export function apply(ctx, config) {
|
package/dist/token-meter.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"token-meter.js","sourceRoot":"","sources":["../src/token-meter.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAE,MAAM,qBAAqB,CAAC;AAG9C,OAAO,EAAE,UAAU,EAAE,MAAM,0BAA0B,CAAC;AAItD,OAAO,qCAAqC,CAAC;AAC7C,OAAO,QAAQ,MAAM,0BAA0B,CAAC;AAChD,OAAO,EAAE,YAAY,EAAE,YAAY,EAAE,MAAM,WAAW,CAAC;AACvD,OAAO,EAAE,wBAAwB,EAAE,gBAAgB,EAAE,MAAM,iBAAiB,CAAC;AAC7E,OAAO,EAAE,eAAe,EAAE,MAAM,YAAY,CAAC;AAQ7C,MAAM,
|
|
1
|
+
{"version":3,"file":"token-meter.js","sourceRoot":"","sources":["../src/token-meter.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAE,MAAM,qBAAqB,CAAC;AAG9C,OAAO,EAAE,UAAU,EAAE,MAAM,0BAA0B,CAAC;AAItD,OAAO,qCAAqC,CAAC;AAC7C,OAAO,QAAQ,MAAM,0BAA0B,CAAC;AAChD,OAAO,EAAE,YAAY,EAAE,YAAY,EAAE,MAAM,WAAW,CAAC;AACvD,OAAO,EAAE,wBAAwB,EAAE,gBAAgB,EAAE,MAAM,iBAAiB,CAAC;AAC7E,OAAO,EAAE,eAAe,EAAE,MAAM,YAAY,CAAC;AAQ7C,MAAM,YAAY,GAAG;IACpB,SAAS,EAAE,QAAQ,CAAC,MAAM,EAAE;IAC5B,gBAAgB,EAAE,QAAQ,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC;IAClD,aAAa,EAAE,QAAQ,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC;IAC/C,kBAAkB,EAAE,QAAQ,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC;IACpD,gBAAgB,EAAE,QAAQ,CAAC,OAAO,EAAE;CACpC,CAAC;AAEF,MAAM,CAAC,MAAM,MAAM,GAA4C,QAAQ,CAAC,MAAM,CAAC;IAC9E,gBAAgB,EAAE,QAAQ,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,aAAa,CAAC;IAC1D,gBAAgB,EAAE,QAAQ;SACxB,KAAK,CACL,QAAQ,CAAC,MAAM,CAAC;QACf,GAAG,YAAY;QACf,QAAQ,EAAE,QAAQ,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE;KACtC,CAAC,CACF;SACA,OAAO,CAAC,EAAE,CAAC;IACb,aAAa,EAAE,QAAQ;SACrB,KAAK,CACL,QAAQ,CAAC,MAAM,CAAC;QACf,GAAG,YAAY;QACf,QAAQ,EAAE,QAAQ,CAAC,MAAM,EAAE;QAC3B,KAAK,EAAE,QAAQ,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE;KACnC,CAAC,CACF;SACA,OAAO,CAAC,EAAE,CAAC;CACb,CAAC,CAAC;AACH,MAAM,CAAC,MAAM,MAAM,GAAG,CAAC,oBAAoB,CAAC,CAAC;AAE7C,MAAM,OAAO,eAAgB,SAAQ,OAAO;IAC3C,YAAY,GAAY,EAAE,MAAmB;QAC5C,KAAK,CAAC,GAAG,EAAE,YAAY,CAAC,CAAC;QACzB,IAAI,CAAC,MAAM,GAAG,eAAe,CAAC,MAAM,CAAC,CAAC;QACtC,IAAI,CAAC,QAAQ,GAAG;YACf,IAAI,GAAG;gBACN,OAAO,GAAG,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC;YACvB,CAAC;SACD,CAAC;QACF,MAAM,OAAO,GAAG,wBAAwB,CAAC,IAAI,CAAC,MAAM,EAAE,IAAI,CAAC,QAAQ,CAAC,CAAC;QACrE,GAAG,CAAC,kBAAkB,CAAC,QAAQ,CAAC,eAAe,CAAC,CAAC;QACjD,GAAG,CAAC,kBAAkB,CAAC,QAAQ,CAAC,OAAO,CAAC,QAAQ,CAAC,CAAC;QAClD,GAAG,CAAC,kBAAkB,CAAC,QAAQ,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;QACnD,GAAG,CAAC,kBAAkB,CAAC,QAAQ,CAAC,gBAAgB,CAAC,CAAC;IACnD,CAAC;IACD,MAAM,CAAc;IACpB,QAAQ,CAAgB;IACxB,OAAO,CAAC,OAAgB,EAAE,aAA2B;QACpD,MAAM,KAAK,GAAG,IAAI,CAAC,GAAG,CAAC,kBAAkB,CAAC,OAAO,CAChD,OAAO,EACP,oBAAoB,CACpB,CAAC;QACF,IAAI,CAAC,KAAK;YACT,MAAM,IAAI,KAAK,CACd,8DAA8D,CAC9D,CAAC;QACH,OAAO,YAAY,CAAC,KAAK,EAAE,IAAI,CAAC,MAAM,EAAE,IAAI,CAAC,QAAQ,EAAE,aAAa,CAAC,CAAC;IACvE,CAAC;IACD,eAAe,CAAC,OAAgB;QAC/B,OAAO,YAAY,CAClB,CAAC,EAAE,GAAG,EAAE,UAAU,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,CAAC,EACjC,IAAI,EACJ,IAAI,CAAC,MAAM,EACX,IAAI,CAAC,QAAQ,CACb,CAAC,aAAa,CAAC;IACjB,CAAC;CACD;AAED,MAAM,UAAU,KAAK,CAAC,GAAY,EAAE,MAAmB;IACtD,IAAI,eAAe,CAAC,GAAG,EAAE,MAAM,CAAC,CAAC;AAClC,CAAC;AAED,eAAe,EAAE,MAAM,EAAE,MAAM,EAAE,KAAK,EAAE,CAAC"}
|
package/dist/tokenizers.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"tokenizers.d.ts","sourceRoot":"","sources":["../src/tokenizers.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,0BAA0B,CAAC;AAC5D,OAAO,KAAK,EAAE,WAAW,EAAE,cAAc,EAAE,MAAM,YAAY,CAAC;AAc9D,wBAAgB,iBAAiB,CAChC,IAAI,EAAE,MAAM,EACZ,KAAK,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,MAAM,cAW/B;AAED,wBAAgB,SAAS,CAAC,IAAI,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,GAAG,MAAM,CAyBjE;AAED,wBAAgB,aAAa,CAC5B,MAAM,EAAE,WAAW,EACnB,MAAM,EAAE,WAAW,GAAG,IAAI,GACxB,QAAQ,CAAC,cAAc,CAAC,
|
|
1
|
+
{"version":3,"file":"tokenizers.d.ts","sourceRoot":"","sources":["../src/tokenizers.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,0BAA0B,CAAC;AAC5D,OAAO,KAAK,EAAE,WAAW,EAAE,cAAc,EAAE,MAAM,YAAY,CAAC;AAc9D,wBAAgB,iBAAiB,CAChC,IAAI,EAAE,MAAM,EACZ,KAAK,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,MAAM,cAW/B;AAED,wBAAgB,SAAS,CAAC,IAAI,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,GAAG,MAAM,CAyBjE;AAED,wBAAgB,aAAa,CAC5B,MAAM,EAAE,WAAW,EACnB,MAAM,EAAE,WAAW,GAAG,IAAI,GACxB,QAAQ,CAAC,cAAc,CAAC,CAsB1B"}
|