dsh-plugin-profiler 0.0.0-stage → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +105 -2
- package/cordis.patch.yml +3 -0
- package/package.json +43 -4
- package/src/index.js +270 -0
- package/src/report.js +268 -0
package/README.md
CHANGED
|
@@ -1,3 +1,106 @@
|
|
|
1
|
-
#
|
|
1
|
+
# dsh-plugin-profiler
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
**Your DSH did not get slower because of the model. It got slower because of what you installed.**
|
|
4
|
+
|
|
5
|
+
`/diet` prints what your current plugin set costs while the session runs: how many tokens your tool definitions eat on every request, how much wall time sits outside the tool body on the hot path, how many listeners each plugin puts on that path, and which three plugins to look at first.
|
|
6
|
+
|
|
7
|
+
This is not another benchmark. `dsh-benchup` and `dsh-eval` answer "is DSH good?" with black-box totals over a task. This answers "what is my install costing me right now?" using data from the live event bus and the live tool registry, with no changes to any other plugin.
|
|
8
|
+
|
|
9
|
+
| | Answers | What it needs from you |
|
|
10
|
+
| --- | --- | --- |
|
|
11
|
+
| `dsh-benchup` | Is this model or config good overall? | a benchmark task to run |
|
|
12
|
+
| `dsh-eval` | Does it choose the right tool? | a task suite |
|
|
13
|
+
| this plugin | What is my installed plugin set costing me right now? | the session you are already in |
|
|
14
|
+
|
|
15
|
+
## What it reports
|
|
16
|
+
|
|
17
|
+
| Section | Source | Meaning |
|
|
18
|
+
| --- | --- | --- |
|
|
19
|
+
| Context tax | `ctx.tools.schemas()` | Estimated tokens for every tool definition sent on each request, plus the heaviest tools |
|
|
20
|
+
| Hot path | prepended `agent/pre-step`, `agent/request`, `tools/pre-execute` listeners | Wall time of the remaining chain, builtins included, provider and tool body excluded |
|
|
21
|
+
| Tool calls | `tools/pre-execute` → `tools/result` | Wall time per tool call, keyed by `callId` |
|
|
22
|
+
| Listener fan-out | live `ctx.events._hooks` snapshot | Which plugin registered how many listeners, and how many sit on hot events |
|
|
23
|
+
| Event volume | `internal/dispatch` | How often each event actually fires |
|
|
24
|
+
|
|
25
|
+
## Sample output
|
|
26
|
+
|
|
27
|
+
From a synthetic two-plugin run against the real rc.7 runtime (not fabricated numbers):
|
|
28
|
+
|
|
29
|
+
```text
|
|
30
|
+
DSH plugin tax · 0s observed
|
|
31
|
+
context 4 tools ≈ 210 tokens per request
|
|
32
|
+
hot path 1 call, 37.3ms of host work (provider and tool body excluded)
|
|
33
|
+
listeners 2 plugins · 4 on hot events
|
|
34
|
+
cut first plugin-b (3 hot listeners), plugin-a (1 hot listener)
|
|
35
|
+
fan-out is a suspicion, not a measurement: rc.7 cannot time another plugin's callback
|
|
36
|
+
|
|
37
|
+
Hot chains (wall time of every listener after this one, builtins included)
|
|
38
|
+
event calls total avg max
|
|
39
|
+
agent/pre-step 1 37.3ms 37.3ms 37.3ms
|
|
40
|
+
before each step
|
|
41
|
+
|
|
42
|
+
Context tax, per tool definition
|
|
43
|
+
4 tools ≈ 210 tokens; heaviest: bash ≈ 68, read_file ≈ 54, apply_patch ≈ 45
|
|
44
|
+
Owner is not exposed by rc.7, so this is per tool, not per plugin.
|
|
45
|
+
|
|
46
|
+
Listeners per plugin (snapshot)
|
|
47
|
+
plugin-b agent/pre-step ×1, llm/stream ×1, tools/result ×1
|
|
48
|
+
plugin-a agent/pre-step ×1
|
|
49
|
+
|
|
50
|
+
Event volume (top dispatches, all modes)
|
|
51
|
+
agent/pre-step ×1 · tools/result ×1
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
## What this cannot measure
|
|
55
|
+
|
|
56
|
+
Read this before installing. It is the reason the card says "fan-out is a suspicion":
|
|
57
|
+
|
|
58
|
+
- **No per-plugin milliseconds.** rc.7 gives `internal/listener` only a veto over registration (`vendor/cordis/src/events.ts`, `on()` returns a non-null value as the disposer instead of registering), and it has no post-dispatch event. A plugin therefore cannot wrap another plugin's callback. Any tool claiming "plugin A's pre-execute took N ms" on rc.7 is not measuring what it says.
|
|
59
|
+
- **No tool owner.** `tools.register()` stores the definition but not the registering fiber, so the context tax is priced per tool, not per plugin.
|
|
60
|
+
- **Two scopes are intentionally excluded from the tax line.** `llm/stream` is timed only up to interception, not stream consumption, and the provider call dominates it. Tool body time is measured separately under "Tool calls".
|
|
61
|
+
|
|
62
|
+
If a host seam lands later (a post-registration or post-dispatch hook, plus an owner field on tool records), this plugin can add real per-plugin attribution without changing its report format.
|
|
63
|
+
|
|
64
|
+
## Robustness
|
|
65
|
+
|
|
66
|
+
Written so it can never be the reason your host breaks:
|
|
67
|
+
|
|
68
|
+
- Every listener body is contained: a bookkeeping failure is dropped, never thrown into the chain the host is running.
|
|
69
|
+
- A dispatch with no `next` callback (a plain emit of a waterfall event name) passes through untouched instead of being called as a function.
|
|
70
|
+
- No `commands` service, or a `/diet` name already taken by another copy, costs the command only — counters keep running, `/profiler` is still attempted, and the rest of the session is unaffected.
|
|
71
|
+
- `tools.schemas()` throwing, circular parameter objects, or a proxy that refuses enumeration degrade that one section and leave the rest of the card intact.
|
|
72
|
+
- Counters are bounded (200 in-flight calls, 200 distinct events), so a very long session cannot grow them without limit.
|
|
73
|
+
- Tables are padded by display width, so Chinese, Japanese or emoji plugin names line up like ASCII ones.
|
|
74
|
+
|
|
75
|
+
## Usage
|
|
76
|
+
|
|
77
|
+
Add the package to your DSH plugin list like any other plugin, then:
|
|
78
|
+
|
|
79
|
+
- `/diet` — print the tax card and the detail sections
|
|
80
|
+
- `/diet reset` — clear counters and keep the live listener inventory
|
|
81
|
+
- `/profiler` — alias of `/diet`
|
|
82
|
+
|
|
83
|
+
No config keys. No network calls. No file writes. If the `commands` service is absent, counters still collect silently and nothing is shown.
|
|
84
|
+
|
|
85
|
+
## Limitations
|
|
86
|
+
|
|
87
|
+
- Chain timing includes builtins and any other listener in the remaining chain; it is the cost of the path, not one plugin.
|
|
88
|
+
- The listener snapshot is live: plugins loaded after `/diet` are visible on the next call, plugins already unloaded are gone.
|
|
89
|
+
- Token estimates use ~4 characters per token; treat them as relative weights, not billing.
|
|
90
|
+
- `commands.register` is called once at load; a hot-reloaded commands service is not re-bound.
|
|
91
|
+
|
|
92
|
+
## Compatibility
|
|
93
|
+
|
|
94
|
+
DeepSeek Harness `0.1.0-rc.7` (`@deepseek-ai/cordis` 4.x event bus). Node `^22.19.0 || >=24.0.0`.
|
|
95
|
+
|
|
96
|
+
## Tests
|
|
97
|
+
|
|
98
|
+
```bash
|
|
99
|
+
npm test
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
Ten tests run against the real vendored cordis runtime: silent degradation, chain timing with two plugins, tool call timing, tool-definition pricing, reset, missing commands service, duplicate command name, dispatch without `next`, unserializable tool parameters, and a throwing tools service.
|
|
103
|
+
|
|
104
|
+
## License
|
|
105
|
+
|
|
106
|
+
MIT
|
package/cordis.patch.yml
ADDED
package/package.json
CHANGED
|
@@ -1,6 +1,45 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dsh-plugin-profiler",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"
|
|
5
|
-
"
|
|
6
|
-
|
|
3
|
+
"version": "0.1.1",
|
|
4
|
+
"description": "Plugin load audit for DSH: hot event chains, tool execution cost, tool-definition context cost, and per-plugin listener inventory",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"main": "src/index.js",
|
|
7
|
+
"exports": {
|
|
8
|
+
".": "./src/index.js"
|
|
9
|
+
},
|
|
10
|
+
"files": [
|
|
11
|
+
"src",
|
|
12
|
+
"README.md",
|
|
13
|
+
"cordis.patch.yml"
|
|
14
|
+
],
|
|
15
|
+
"dsh": {
|
|
16
|
+
"bundle": {
|
|
17
|
+
"patch": "./cordis.patch.yml"
|
|
18
|
+
}
|
|
19
|
+
},
|
|
20
|
+
"keywords": [
|
|
21
|
+
"dsh",
|
|
22
|
+
"deepseek-harness",
|
|
23
|
+
"plugin",
|
|
24
|
+
"profiler",
|
|
25
|
+
"performance",
|
|
26
|
+
"context",
|
|
27
|
+
"diagnostics"
|
|
28
|
+
],
|
|
29
|
+
"engines": {
|
|
30
|
+
"node": "^22.19.0 || >=24.0.0"
|
|
31
|
+
},
|
|
32
|
+
"scripts": {
|
|
33
|
+
"test": "node --test tests/profiler.test.js"
|
|
34
|
+
},
|
|
35
|
+
"repository": {
|
|
36
|
+
"type": "git",
|
|
37
|
+
"url": "git+https://github.com/liuhange789/dsh-plugin-profiler.git"
|
|
38
|
+
},
|
|
39
|
+
"homepage": "https://github.com/liuhange789/dsh-plugin-profiler#readme",
|
|
40
|
+
"bugs": {
|
|
41
|
+
"url": "https://github.com/liuhange789/dsh-plugin-profiler/issues"
|
|
42
|
+
},
|
|
43
|
+
"author": "liuhange789",
|
|
44
|
+
"license": "MIT"
|
|
45
|
+
}
|
package/src/index.js
ADDED
|
@@ -0,0 +1,270 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* dsh-plugin-profiler — a plugin load audit for DeepSeek Harness.
|
|
3
|
+
*
|
|
4
|
+
* It reads what rc.7 exposes without patching another plugin:
|
|
5
|
+
* - dispatch volume per event (`internal/dispatch`)
|
|
6
|
+
* - wall time of the remaining chain (prepended waterfall listener)
|
|
7
|
+
* - tool wall time per tool (`tools/pre-execute` → `tools/result`)
|
|
8
|
+
* - context cost of visible tools (`ctx.tools.schemas()`)
|
|
9
|
+
* - listener inventory per plugin (read-only snapshot of `ctx.events._hooks`)
|
|
10
|
+
*
|
|
11
|
+
* It does NOT split one chain into per-plugin milliseconds. rc.7 hands
|
|
12
|
+
* listeners to `internal/listener` only to allow or refuse registration, and
|
|
13
|
+
* has no post-dispatch hook, so no plugin can time another plugin's callback.
|
|
14
|
+
* See README "What this cannot measure".
|
|
15
|
+
*
|
|
16
|
+
* Robustness contract: this plugin never throws into the host. Every listener
|
|
17
|
+
* body, every command handler and every optional-service call is contained; if
|
|
18
|
+
* anything it reads is missing or hostile it drops that section and keeps the
|
|
19
|
+
* rest of the session running.
|
|
20
|
+
*/
|
|
21
|
+
|
|
22
|
+
import {
|
|
23
|
+
collectListeners,
|
|
24
|
+
collectTools,
|
|
25
|
+
renderReport,
|
|
26
|
+
} from './report.js'
|
|
27
|
+
|
|
28
|
+
export const name = 'plugin-profiler'
|
|
29
|
+
|
|
30
|
+
/** Waterfall events whose remaining chain is timed around `next()`. */
|
|
31
|
+
const TIMED_CHAINS = [
|
|
32
|
+
['agent/pre-step', 'before each step'],
|
|
33
|
+
['agent/request', 'request assembly'],
|
|
34
|
+
['llm/stream', 'stream interception only, not consumption'],
|
|
35
|
+
]
|
|
36
|
+
|
|
37
|
+
/** Bounds that keep counters small even after a very long session. */
|
|
38
|
+
const MAX_INFLIGHT = 200
|
|
39
|
+
const MAX_EVENTS = 200
|
|
40
|
+
|
|
41
|
+
const clock = typeof performance !== 'undefined' && typeof performance.now === 'function'
|
|
42
|
+
? () => performance.now()
|
|
43
|
+
: () => Date.now()
|
|
44
|
+
|
|
45
|
+
function newState() {
|
|
46
|
+
return {
|
|
47
|
+
since: Date.now(),
|
|
48
|
+
chains: new Map(),
|
|
49
|
+
dispatches: new Map(),
|
|
50
|
+
tools: new Map(),
|
|
51
|
+
inflight: new Map(),
|
|
52
|
+
}
|
|
53
|
+
}
|
|
54
|
+
|
|
55
|
+
/** Read an optional service without ever letting a proxy getter break the load. */
|
|
56
|
+
function optionalService(ctx, key) {
|
|
57
|
+
try {
|
|
58
|
+
const value = ctx.get(key, false)
|
|
59
|
+
return value && typeof value === 'object' ? value : undefined
|
|
60
|
+
} catch {
|
|
61
|
+
return undefined
|
|
62
|
+
}
|
|
63
|
+
}
|
|
64
|
+
|
|
65
|
+
/** Run bookkeeping that must not be able to fail a host dispatch. */
|
|
66
|
+
function safely(work) {
|
|
67
|
+
try {
|
|
68
|
+
work()
|
|
69
|
+
} catch {
|
|
70
|
+
// A counter can never justify breaking the caller's chain.
|
|
71
|
+
}
|
|
72
|
+
}
|
|
73
|
+
|
|
74
|
+
function recordChain(state, event, note, ms) {
|
|
75
|
+
let row = state.chains.get(event)
|
|
76
|
+
if (!row) {
|
|
77
|
+
row = { event, note, calls: 0, totalMs: 0, maxMs: 0 }
|
|
78
|
+
state.chains.set(event, row)
|
|
79
|
+
}
|
|
80
|
+
row.calls += 1
|
|
81
|
+
row.totalMs += ms
|
|
82
|
+
row.maxMs = Math.max(row.maxMs, ms)
|
|
83
|
+
}
|
|
84
|
+
|
|
85
|
+
function recordTool(state, tool, ms) {
|
|
86
|
+
let row = state.tools.get(tool)
|
|
87
|
+
if (!row) {
|
|
88
|
+
row = { tool, calls: 0, totalMs: 0, maxMs: 0 }
|
|
89
|
+
state.tools.set(tool, row)
|
|
90
|
+
}
|
|
91
|
+
row.calls += 1
|
|
92
|
+
row.totalMs += ms
|
|
93
|
+
row.maxMs = Math.max(row.maxMs, ms)
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
function describeThrown(error) {
|
|
97
|
+
try {
|
|
98
|
+
const text = error instanceof Error ? error.message : String(error)
|
|
99
|
+
return text.length > 160 ? `${text.slice(0, 159)}…` : text
|
|
100
|
+
} catch {
|
|
101
|
+
return 'unknown error'
|
|
102
|
+
}
|
|
103
|
+
}
|
|
104
|
+
|
|
105
|
+
function warn(ctx, message) {
|
|
106
|
+
try {
|
|
107
|
+
ctx.logger?.warn?.(`[plugin-profiler] ${message}`)
|
|
108
|
+
} catch {
|
|
109
|
+
// Logging is never load-bearing.
|
|
110
|
+
}
|
|
111
|
+
}
|
|
112
|
+
|
|
113
|
+
/** Attach a registered command's disposer to this plugin's fiber, so unload unregisters it. */
|
|
114
|
+
function keepCommandAlive(ctx, dispose) {
|
|
115
|
+
if (typeof dispose !== 'function') return
|
|
116
|
+
if (typeof ctx.effect !== 'function') return
|
|
117
|
+
try {
|
|
118
|
+
ctx.effect(() => dispose, 'plugin-profiler command')
|
|
119
|
+
} catch {
|
|
120
|
+
// If the fiber is already leaving, drop the registration immediately.
|
|
121
|
+
try {
|
|
122
|
+
dispose()
|
|
123
|
+
} catch {
|
|
124
|
+
// Nothing left to release.
|
|
125
|
+
}
|
|
126
|
+
}
|
|
127
|
+
}
|
|
128
|
+
|
|
129
|
+
function registerCommand(ctx, commands, definition) {
|
|
130
|
+
try {
|
|
131
|
+
const dispose = commands.register(definition)
|
|
132
|
+
keepCommandAlive(ctx, dispose)
|
|
133
|
+
return true
|
|
134
|
+
} catch (error) {
|
|
135
|
+
// A duplicate name (second copy of this plugin, or a foreign /diet) must
|
|
136
|
+
// cost the command, never the plugin.
|
|
137
|
+
warn(ctx, `could not register /${definition.name}: ${describeThrown(error)}`)
|
|
138
|
+
return false
|
|
139
|
+
}
|
|
140
|
+
}
|
|
141
|
+
|
|
142
|
+
/** Resolve the tool payload's display name without trusting property access. */
|
|
143
|
+
function toolNameOf(exec) {
|
|
144
|
+
try {
|
|
145
|
+
return exec?.name == null ? '?' : String(exec.name)
|
|
146
|
+
} catch {
|
|
147
|
+
return '?'
|
|
148
|
+
}
|
|
149
|
+
}
|
|
150
|
+
|
|
151
|
+
function callIdOf(exec) {
|
|
152
|
+
try {
|
|
153
|
+
return exec?.callId == null ? undefined : String(exec.callId)
|
|
154
|
+
} catch {
|
|
155
|
+
return undefined
|
|
156
|
+
}
|
|
157
|
+
}
|
|
158
|
+
|
|
159
|
+
export function apply(ctx) {
|
|
160
|
+
const state = newState()
|
|
161
|
+
|
|
162
|
+
// Every non-internal dispatch, whatever its mode (emit/parallel/serial/bail/waterfall).
|
|
163
|
+
ctx.on('internal/dispatch', (mode, event) => {
|
|
164
|
+
safely(() => {
|
|
165
|
+
if (typeof event !== 'string' || event.startsWith('internal/')) return
|
|
166
|
+
const seen = state.dispatches.get(event)
|
|
167
|
+
if (seen === undefined && state.dispatches.size >= MAX_EVENTS) return
|
|
168
|
+
state.dispatches.set(event, (seen ?? 0) + 1)
|
|
169
|
+
})
|
|
170
|
+
}, { global: true })
|
|
171
|
+
|
|
172
|
+
// Outermost listener: time everything the rest of the chain does, `next()` included.
|
|
173
|
+
// Rest arguments keep this signature-agnostic across the three events.
|
|
174
|
+
for (const [event, note] of TIMED_CHAINS) {
|
|
175
|
+
ctx.on(event, async (...args) => {
|
|
176
|
+
const next = args[args.length - 1]
|
|
177
|
+
// A non-waterfall dispatch (a plain emit of the same name) has no `next`;
|
|
178
|
+
// pass through instead of trying to call something that is not a function.
|
|
179
|
+
if (typeof next !== 'function') return next
|
|
180
|
+
const start = clock()
|
|
181
|
+
try {
|
|
182
|
+
return await next()
|
|
183
|
+
} finally {
|
|
184
|
+
safely(() => recordChain(state, event, note, clock() - start))
|
|
185
|
+
}
|
|
186
|
+
}, { prepend: true, global: true })
|
|
187
|
+
}
|
|
188
|
+
|
|
189
|
+
// Pre-tool gates: time the gate chain and remember when the call started.
|
|
190
|
+
ctx.on('tools/pre-execute', async (...args) => {
|
|
191
|
+
const next = args[args.length - 1]
|
|
192
|
+
if (typeof next !== 'function') return next
|
|
193
|
+
const start = clock()
|
|
194
|
+
safely(() => {
|
|
195
|
+
const exec = args[0]
|
|
196
|
+
const callId = callIdOf(exec)
|
|
197
|
+
if (callId === undefined) return
|
|
198
|
+
if (state.inflight.size >= MAX_INFLIGHT) state.inflight.clear()
|
|
199
|
+
state.inflight.set(callId, { name: toolNameOf(exec), start })
|
|
200
|
+
})
|
|
201
|
+
try {
|
|
202
|
+
return await next()
|
|
203
|
+
} finally {
|
|
204
|
+
safely(() => recordChain(state, 'tools/pre-execute', 'pre-tool gates', clock() - start))
|
|
205
|
+
}
|
|
206
|
+
}, { prepend: true, global: true })
|
|
207
|
+
|
|
208
|
+
ctx.on('tools/result', (exec) => {
|
|
209
|
+
safely(() => {
|
|
210
|
+
const callId = callIdOf(exec)
|
|
211
|
+
if (callId === undefined) return
|
|
212
|
+
const started = state.inflight.get(callId)
|
|
213
|
+
if (!started) return
|
|
214
|
+
state.inflight.delete(callId)
|
|
215
|
+
recordTool(state, started.name, clock() - started.start)
|
|
216
|
+
})
|
|
217
|
+
}, { global: true })
|
|
218
|
+
|
|
219
|
+
const commands = optionalService(ctx, 'commands')
|
|
220
|
+
if (!commands || typeof commands.register !== 'function') return
|
|
221
|
+
|
|
222
|
+
const handler = (invocation) => {
|
|
223
|
+
try {
|
|
224
|
+
let arg = ''
|
|
225
|
+
try {
|
|
226
|
+
arg = String(invocation?.rawInput ?? '').trim().toLowerCase()
|
|
227
|
+
} catch {
|
|
228
|
+
arg = ''
|
|
229
|
+
}
|
|
230
|
+
if (arg === 'reset') {
|
|
231
|
+
reset(state)
|
|
232
|
+
return { kind: 'success', text: 'Plugin tax counters reset.' }
|
|
233
|
+
}
|
|
234
|
+
if (arg !== '') return { kind: 'error', text: 'Usage: /diet [reset]' }
|
|
235
|
+
return {
|
|
236
|
+
kind: 'success',
|
|
237
|
+
text: renderReport(state, {
|
|
238
|
+
listeners: collectListeners(ctx, name),
|
|
239
|
+
schemas: collectTools(ctx),
|
|
240
|
+
selfName: name,
|
|
241
|
+
}, Date.now()),
|
|
242
|
+
}
|
|
243
|
+
} catch (error) {
|
|
244
|
+
return {
|
|
245
|
+
kind: 'error',
|
|
246
|
+
text: `Plugin tax report failed: ${describeThrown(error)}. Counters are still collecting; try /diet again.`,
|
|
247
|
+
}
|
|
248
|
+
}
|
|
249
|
+
}
|
|
250
|
+
|
|
251
|
+
// `/diet` is the memorable surface; `/profiler` is kept as an alias.
|
|
252
|
+
registerCommand(ctx, commands, {
|
|
253
|
+
name: 'diet',
|
|
254
|
+
description: 'Plugin tax: context cost, hot path, and uninstall candidates',
|
|
255
|
+
handler,
|
|
256
|
+
})
|
|
257
|
+
registerCommand(ctx, commands, {
|
|
258
|
+
name: 'profiler',
|
|
259
|
+
description: 'Alias of /diet',
|
|
260
|
+
handler,
|
|
261
|
+
})
|
|
262
|
+
}
|
|
263
|
+
|
|
264
|
+
function reset(state) {
|
|
265
|
+
state.since = Date.now()
|
|
266
|
+
state.chains.clear()
|
|
267
|
+
state.dispatches.clear()
|
|
268
|
+
state.tools.clear()
|
|
269
|
+
state.inflight.clear()
|
|
270
|
+
}
|
package/src/report.js
ADDED
|
@@ -0,0 +1,268 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Pure formatting and snapshot collection for dsh-plugin-profiler.
|
|
3
|
+
*
|
|
4
|
+
* Everything here is read-only. The listener inventory walks the public
|
|
5
|
+
* `ctx.events._hooks` records (each record carries the registering context),
|
|
6
|
+
* and the tool context cost comes from `ctx.tools.schemas()`.
|
|
7
|
+
*/
|
|
8
|
+
|
|
9
|
+
const TOP_ROWS = 8
|
|
10
|
+
const NAME_WIDTH = 22
|
|
11
|
+
const CHARS_PER_TOKEN = 4
|
|
12
|
+
|
|
13
|
+
/** Events that every turn or every tool call must run through. */
|
|
14
|
+
const HOT_EVENTS = new Set([
|
|
15
|
+
'agent/pre-step',
|
|
16
|
+
'agent/request',
|
|
17
|
+
'llm/stream',
|
|
18
|
+
'tools/pre-execute',
|
|
19
|
+
'tools/execute',
|
|
20
|
+
'tools/post-execute',
|
|
21
|
+
'tools/result',
|
|
22
|
+
])
|
|
23
|
+
|
|
24
|
+
/** Coerce anything to text without trusting a hostile `toString`. */
|
|
25
|
+
function safeText(value) {
|
|
26
|
+
if (value == null) return ''
|
|
27
|
+
try {
|
|
28
|
+
return String(value)
|
|
29
|
+
} catch {
|
|
30
|
+
return ''
|
|
31
|
+
}
|
|
32
|
+
}
|
|
33
|
+
|
|
34
|
+
/** Estimate prompt tokens for one tool schema (~4 characters per token). */
|
|
35
|
+
export function estimateTokens(schema) {
|
|
36
|
+
try {
|
|
37
|
+
const text = JSON.stringify({
|
|
38
|
+
name: schema?.name,
|
|
39
|
+
description: schema?.description,
|
|
40
|
+
parameters: schema?.parameters,
|
|
41
|
+
})
|
|
42
|
+
if (typeof text === 'string') return Math.ceil(text.length / CHARS_PER_TOKEN)
|
|
43
|
+
} catch {
|
|
44
|
+
// Circular, BigInt or otherwise unserializable parameters: price what we can read.
|
|
45
|
+
}
|
|
46
|
+
const fallback = `${safeText(schema?.name)}${safeText(schema?.description)}`
|
|
47
|
+
return Math.ceil(fallback.length / CHARS_PER_TOKEN)
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
/** List one `_hooks` table as `[event, hooks]` pairs, tolerating a Map or a hostile object. */
|
|
51
|
+
function hookEntries(hooks) {
|
|
52
|
+
try {
|
|
53
|
+
if (hooks instanceof Map) return [...hooks.entries()]
|
|
54
|
+
if (hooks && typeof hooks === 'object') return Object.entries(hooks)
|
|
55
|
+
} catch {
|
|
56
|
+
// A proxy that refuses enumeration simply yields no inventory.
|
|
57
|
+
}
|
|
58
|
+
return []
|
|
59
|
+
}
|
|
60
|
+
|
|
61
|
+
/** Group registered listeners by owning plugin, skipping internal events. */
|
|
62
|
+
export function collectListeners(ctx, selfName) {
|
|
63
|
+
const byPlugin = new Map()
|
|
64
|
+
let hooks
|
|
65
|
+
try {
|
|
66
|
+
hooks = ctx?.events?._hooks
|
|
67
|
+
} catch {
|
|
68
|
+
return byPlugin
|
|
69
|
+
}
|
|
70
|
+
for (const [event, list] of hookEntries(hooks)) {
|
|
71
|
+
if (typeof event !== 'string' || event.startsWith('internal/')) continue
|
|
72
|
+
if (!Array.isArray(list)) continue
|
|
73
|
+
for (const hook of list) {
|
|
74
|
+
let owner
|
|
75
|
+
try {
|
|
76
|
+
owner = hook?.ctx?.fiber?.name
|
|
77
|
+
} catch {
|
|
78
|
+
continue
|
|
79
|
+
}
|
|
80
|
+
if (typeof owner !== 'string' || owner.length === 0 || owner === selfName) continue
|
|
81
|
+
let events = byPlugin.get(owner)
|
|
82
|
+
if (!events) {
|
|
83
|
+
events = new Map()
|
|
84
|
+
byPlugin.set(owner, events)
|
|
85
|
+
}
|
|
86
|
+
events.set(event, (events.get(event) ?? 0) + 1)
|
|
87
|
+
}
|
|
88
|
+
}
|
|
89
|
+
return byPlugin
|
|
90
|
+
}
|
|
91
|
+
|
|
92
|
+
/** Visible tool schemas, or an empty list when the tools service is absent. */
|
|
93
|
+
export function collectTools(ctx) {
|
|
94
|
+
try {
|
|
95
|
+
const tools = ctx.get('tools', false)
|
|
96
|
+
if (!tools || typeof tools.schemas !== 'function') return []
|
|
97
|
+
const schemas = tools.schemas()
|
|
98
|
+
return Array.isArray(schemas) ? schemas : []
|
|
99
|
+
} catch {
|
|
100
|
+
return []
|
|
101
|
+
}
|
|
102
|
+
}
|
|
103
|
+
|
|
104
|
+
/** Human-readable duration with a unit that never drops significant digits. */
|
|
105
|
+
export function formatMs(ms) {
|
|
106
|
+
if (!Number.isFinite(ms) || ms < 0) return '-'
|
|
107
|
+
if (ms < 1) return `${ms.toFixed(2)}ms`
|
|
108
|
+
if (ms < 1000) return `${ms.toFixed(1)}ms`
|
|
109
|
+
return `${(ms / 1000).toFixed(2)}s`
|
|
110
|
+
}
|
|
111
|
+
|
|
112
|
+
/** Double-width code points (CJK, Hangul, fullwidth forms, emoji) so tables stay aligned. */
|
|
113
|
+
const WIDE = /[\u1100-\u115F\u2E80-\u303E\u3041-\u33FF\u3400-\u4DBF\u4E00-\u9FFF\uA000-\uA4CF\uAC00-\uD7A3\uF900-\uFAFF\uFE10-\uFE19\uFE30-\uFE6F\uFF00-\uFF60\uFFE0-\uFFE6]|[\u{1F000}-\u{1FAFF}]/u
|
|
114
|
+
|
|
115
|
+
function charWidth(ch) {
|
|
116
|
+
return WIDE.test(ch) ? 2 : 1
|
|
117
|
+
}
|
|
118
|
+
|
|
119
|
+
function displayWidth(text) {
|
|
120
|
+
let width = 0
|
|
121
|
+
for (const ch of text) width += charWidth(ch)
|
|
122
|
+
return width
|
|
123
|
+
}
|
|
124
|
+
|
|
125
|
+
function pad(text, width) {
|
|
126
|
+
const value = safeText(text)
|
|
127
|
+
const current = displayWidth(value)
|
|
128
|
+
return current >= width ? value : value + ' '.repeat(width - current)
|
|
129
|
+
}
|
|
130
|
+
|
|
131
|
+
function clip(text, width) {
|
|
132
|
+
const value = safeText(text)
|
|
133
|
+
if (displayWidth(value) <= width) return value
|
|
134
|
+
let out = ''
|
|
135
|
+
let used = 0
|
|
136
|
+
for (const ch of value) {
|
|
137
|
+
const step = charWidth(ch)
|
|
138
|
+
if (used + step > width - 1) break
|
|
139
|
+
out += ch
|
|
140
|
+
used += step
|
|
141
|
+
}
|
|
142
|
+
return `${out}…`
|
|
143
|
+
}
|
|
144
|
+
|
|
145
|
+
function elapsed(since, now) {
|
|
146
|
+
const total = Math.max(0, Math.floor((now - since) / 1000))
|
|
147
|
+
const minutes = Math.floor(total / 60)
|
|
148
|
+
const seconds = total % 60
|
|
149
|
+
return minutes > 0 ? `${minutes}m ${seconds}s` : `${seconds}s`
|
|
150
|
+
}
|
|
151
|
+
|
|
152
|
+
/** Listener fan-out on hot events, per plugin. Higher fan-out = likelier cost. */
|
|
153
|
+
export function hotFanout(listeners) {
|
|
154
|
+
const rows = []
|
|
155
|
+
for (const [plugin, events] of listeners) {
|
|
156
|
+
let hot = 0
|
|
157
|
+
for (const [event, count] of events) {
|
|
158
|
+
if (HOT_EVENTS.has(event)) hot += count
|
|
159
|
+
}
|
|
160
|
+
if (hot > 0) rows.push({ plugin, hot })
|
|
161
|
+
}
|
|
162
|
+
rows.sort((a, b) => b.hot - a.hot || a.plugin.localeCompare(b.plugin))
|
|
163
|
+
return rows
|
|
164
|
+
}
|
|
165
|
+
|
|
166
|
+
/**
|
|
167
|
+
* Render the audit.
|
|
168
|
+
* @param state - counters maintained by the plugin.
|
|
169
|
+
* @param snapshot - `{ listeners, schemas, selfName }` gathered at call time.
|
|
170
|
+
* @param now - current epoch milliseconds.
|
|
171
|
+
*/
|
|
172
|
+
export function renderReport(state, snapshot, now) {
|
|
173
|
+
const listeners = snapshot.listeners
|
|
174
|
+
const schemas = snapshot.schemas
|
|
175
|
+
const chains = [...state.chains.values()]
|
|
176
|
+
const tools = [...state.tools.values()]
|
|
177
|
+
const volume = [...state.dispatches.entries()]
|
|
178
|
+
|
|
179
|
+
const measured = chains.length > 0 || tools.length > 0 || schemas.length > 0 || listeners.size > 0
|
|
180
|
+
if (!measured) {
|
|
181
|
+
return 'Nothing measured yet — run a turn, then call /diet again.'
|
|
182
|
+
}
|
|
183
|
+
|
|
184
|
+
const lines = []
|
|
185
|
+
const fanout = hotFanout(listeners)
|
|
186
|
+
const hotListeners = fanout.reduce((sum, row) => sum + row.hot, 0)
|
|
187
|
+
const contextTokens = schemas.reduce((sum, schema) => sum + estimateTokens(schema), 0)
|
|
188
|
+
const hotChains = chains.filter(row => HOST_EVENTS_FOR_TAX.has(row.event))
|
|
189
|
+
const hotCalls = hotChains.reduce((sum, row) => sum + row.calls, 0)
|
|
190
|
+
const hotMs = hotChains.reduce((sum, row) => sum + row.totalMs, 0)
|
|
191
|
+
|
|
192
|
+
// The shareable card: three numbers and a cut list, short enough to paste.
|
|
193
|
+
lines.push(`DSH plugin tax · ${elapsed(state.since, now)} observed`)
|
|
194
|
+
lines.push(` context ${schemas.length} tool${schemas.length === 1 ? '' : 's'} ≈ ${contextTokens.toLocaleString('en-US')} tokens per request`)
|
|
195
|
+
lines.push(` hot path ${hotCalls} call${hotCalls === 1 ? '' : 's'}, ${formatMs(hotMs)} of host work (provider and tool body excluded)`)
|
|
196
|
+
lines.push(` listeners ${listeners.size} plugin${listeners.size === 1 ? '' : 's'} · ${hotListeners} on hot events`)
|
|
197
|
+
if (fanout.length > 0) {
|
|
198
|
+
const cuts = fanout.slice(0, 3).map(row => `${row.plugin} (${row.hot} hot listener${row.hot === 1 ? '' : 's'})`)
|
|
199
|
+
lines.push(` cut first ${cuts.join(', ')}`)
|
|
200
|
+
lines.push(' fan-out is a suspicion, not a measurement: rc.7 cannot time another plugin\'s callback')
|
|
201
|
+
}
|
|
202
|
+
|
|
203
|
+
if (chains.length > 0) {
|
|
204
|
+
lines.push('')
|
|
205
|
+
lines.push('Hot chains (wall time of every listener after this one, builtins included)')
|
|
206
|
+
lines.push(`${pad('event', NAME_WIDTH)}${pad('calls', 7)}${pad('total', 10)}${pad('avg', 10)}max`)
|
|
207
|
+
chains.sort((a, b) => b.totalMs - a.totalMs)
|
|
208
|
+
for (const row of chains.slice(0, TOP_ROWS)) {
|
|
209
|
+
const avg = row.calls > 0 ? row.totalMs / row.calls : 0
|
|
210
|
+
lines.push(`${pad(clip(row.event, NAME_WIDTH - 2), NAME_WIDTH)}${pad(String(row.calls), 7)}${pad(formatMs(row.totalMs), 10)}${pad(formatMs(avg), 10)}${formatMs(row.maxMs)}`)
|
|
211
|
+
if (row.note) lines.push(` ${row.note}`)
|
|
212
|
+
}
|
|
213
|
+
if (state.inflight.size > 0) {
|
|
214
|
+
lines.push(` (${state.inflight.size} tool call${state.inflight.size === 1 ? '' : 's'} still in flight, not counted yet)`)
|
|
215
|
+
}
|
|
216
|
+
}
|
|
217
|
+
|
|
218
|
+
if (tools.length > 0) {
|
|
219
|
+
lines.push('')
|
|
220
|
+
lines.push('Tool calls (pre-execute → tools/result)')
|
|
221
|
+
lines.push(`${pad('tool', NAME_WIDTH)}${pad('calls', 7)}${pad('total', 10)}max`)
|
|
222
|
+
tools.sort((a, b) => b.totalMs - a.totalMs)
|
|
223
|
+
for (const row of tools.slice(0, TOP_ROWS)) {
|
|
224
|
+
lines.push(`${pad(clip(row.tool, NAME_WIDTH - 2), NAME_WIDTH)}${pad(String(row.calls), 7)}${pad(formatMs(row.totalMs), 10)}${formatMs(row.maxMs)}`)
|
|
225
|
+
}
|
|
226
|
+
}
|
|
227
|
+
|
|
228
|
+
if (schemas.length > 0) {
|
|
229
|
+
const priced = schemas.map(schema => ({ name: safeText(schema?.name) || '?', tokens: estimateTokens(schema) }))
|
|
230
|
+
priced.sort((a, b) => b.tokens - a.tokens)
|
|
231
|
+
const total = priced.reduce((sum, row) => sum + row.tokens, 0)
|
|
232
|
+
lines.push('')
|
|
233
|
+
lines.push('Context tax, per tool definition')
|
|
234
|
+
lines.push(`${schemas.length} tool${schemas.length === 1 ? '' : 's'} ≈ ${total.toLocaleString('en-US')} tokens; heaviest: ${priced.slice(0, 3).map(row => `${row.name} ≈ ${row.tokens}`).join(', ')}`)
|
|
235
|
+
lines.push(' Owner is not exposed by rc.7, so this is per tool, not per plugin.')
|
|
236
|
+
}
|
|
237
|
+
|
|
238
|
+
if (listeners.size > 0) {
|
|
239
|
+
lines.push('')
|
|
240
|
+
lines.push('Listeners per plugin (snapshot)')
|
|
241
|
+
const rows = [...listeners.entries()].map(([plugin, events]) => ({
|
|
242
|
+
plugin,
|
|
243
|
+
total: [...events.values()].reduce((sum, count) => sum + count, 0),
|
|
244
|
+
text: [...events.entries()].sort((a, b) => b[1] - a[1]).map(([event, count]) => `${event} ×${count}`).join(', '),
|
|
245
|
+
}))
|
|
246
|
+
rows.sort((a, b) => b.total - a.total)
|
|
247
|
+
for (const row of rows.slice(0, TOP_ROWS)) {
|
|
248
|
+
lines.push(`${pad(clip(row.plugin, NAME_WIDTH - 2), NAME_WIDTH)}${clip(row.text, 96)}`)
|
|
249
|
+
}
|
|
250
|
+
if (rows.length > TOP_ROWS) lines.push(` … ${rows.length - TOP_ROWS} more plugin(s)`)
|
|
251
|
+
}
|
|
252
|
+
|
|
253
|
+
if (volume.length > 0) {
|
|
254
|
+
lines.push('')
|
|
255
|
+
lines.push('Event volume (top dispatches, all modes)')
|
|
256
|
+
volume.sort((a, b) => b[1] - a[1])
|
|
257
|
+
lines.push(volume.slice(0, 5).map(([event, count]) => `${event} ×${count}`).join(' · '))
|
|
258
|
+
}
|
|
259
|
+
|
|
260
|
+
return lines.join('\n')
|
|
261
|
+
}
|
|
262
|
+
|
|
263
|
+
/** Chains that do not include provider or tool-body time, used for the tax line. */
|
|
264
|
+
const HOST_EVENTS_FOR_TAX = new Set([
|
|
265
|
+
'agent/pre-step',
|
|
266
|
+
'agent/request',
|
|
267
|
+
'tools/pre-execute',
|
|
268
|
+
])
|