pi-better-web-search 0.1.5 → 0.1.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,13 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.1.6
4
+
5
+ - Keep oversized results as valid JSON, sharing the character budget across
6
+ search results and fetched pages rather than cutting off the serialized tail.
7
+ Report affected paths and mark shortened strings; retain unchanged provider
8
+ data when it fits, and compact whitespace before dropping data.
9
+ - Fix premature truncation immediately below the configured output limit.
10
+
3
11
  ## 0.1.5
4
12
 
5
13
  - Rank models inside a provider family by capability tier first (cheapest tier
package/README.md CHANGED
@@ -8,7 +8,7 @@ provider supports it, automatically falls back across authenticated providers,
8
8
  and returns provider data as-is.
9
9
 
10
10
  [![CI](https://github.com/zhexusun10/pi-better-web-search/actions/workflows/ci.yml/badge.svg)](https://github.com/zhexusun10/pi-better-web-search/actions/workflows/ci.yml)
11
- [![npm](https://img.shields.io/badge/npm-v0.1.5-blue)](https://www.npmjs.com/package/pi-better-web-search)
11
+ [![npm](https://img.shields.io/npm/v/pi-better-web-search)](https://www.npmjs.com/package/pi-better-web-search)
12
12
  [![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
13
13
  [![Pi 1.3](https://img.shields.io/badge/Pi-1.3-blue.svg)](https://pi.dev)
14
14
  [![Node](https://img.shields.io/badge/node-%3E%3D22.19-339933.svg)](https://nodejs.org)
@@ -58,8 +58,8 @@ Requirements: **pi 1.3+** and **Node.js 22.19+**.
58
58
  provider supports URL analysis. `urls` can be used alone or with `query`.
59
59
  - **Raw results.** Provider data is returned in a JSON wrapper without
60
60
  deduplication, URL normalization, junk filtering, re-ranking, summarization,
61
- citation-marker insertion, or appended Sources sections. Only a final
62
- character guard truncates oversized output.
61
+ citation-marker insertion, or appended Sources sections. A final character
62
+ guard applies structured truncation only to oversized output.
63
63
  - **Correct semantics.** The tool description is exactly
64
64
  `Search the web or analyze specified URLs.` The schema makes clear that
65
65
  `urls` does not require `query`.
@@ -250,18 +250,36 @@ this:
250
250
  - Citation markers are not inserted, and no "Sources" or "Additional results"
251
251
  sections are appended.
252
252
  - Redirects are not resolved and sources are not derived.
253
- - Unknown provider fields are preserved.
254
-
255
- The only transformation is a final output-size guard. The model-facing JSON is
256
- truncated to **64,000 characters** by default and marked with `[Truncated]`;
257
- `details.truncated` is set to `true`. Change the limit with:
253
+ - Unknown provider fields are preserved when the result fits the size guard.
254
+
255
+ A final output-size guard limits successful model-facing results to **64,000
256
+ Unicode code points** by default. Normal-sized results retain their original
257
+ fields and formatting. If indentation alone exceeds the limit, compact JSON is
258
+ returned without dropping data or setting the truncation flag.
259
+
260
+ When even compact JSON is too large, the guard allocates space across sibling
261
+ fields and array entries, redistributing unused space. It tries to preserve
262
+ identifying fields such as URLs, titles and errors, while shortening long
263
+ strings or omitting fields and trailing array entries when needed. The result
264
+ remains valid JSON. Shortened strings end in `…[truncated]` when space permits,
265
+ and the model-facing `_truncation` object reports `truncated`, `originalChars`
266
+ (the original indented JSON size), and `affectedPaths`. `moreAffectedPaths`
267
+ indicates that the path list is incomplete. Array paths can indicate omitted
268
+ entries; with very small budgets, not every URL or field can be preserved.
269
+ `details.truncated` is also set to `true` when data is actually removed.
270
+
271
+ Change the limit **before starting pi**:
258
272
 
259
273
  ```bash
260
274
  export PI_BETTER_WEB_SEARCH_MAX_CHARS=100000
261
275
  ```
262
276
 
263
- Truncation is Unicode code-point safe. The guard is intentionally applied after
264
- the raw JSON is assembled so normal-sized provider payloads are not altered.
277
+ In PowerShell, use `$env:PI_BETTER_WEB_SEARCH_MAX_CHARS = "100000"`. The limit
278
+ is in code points, **not tokens**, and does not adapt to the current model's
279
+ context window. Extremely small limits below 19 characters cannot fit a
280
+ model-visible truncation flag; the result is still valid JSON and
281
+ `details.truncated` remains true. There is no continuation/pagination cache:
282
+ retry individual URLs or raise the limit if more content is needed.
265
283
 
266
284
  ## Exposure rules
267
285
 
@@ -46,7 +46,7 @@ pi install ./path/to/pi-better-web-search
46
46
  `urls`。`urls` 可以单独使用,也可以和 `query` 一起使用。
47
47
  - **原始结果。** provider 数据放在 JSON 外层中返回,不去重、不规范化 URL、
48
48
  不过滤垃圾 URL、不重排、不总结、不插入 citation 标记、不追加 Sources。
49
- 只做最终字符数限制和截断。
49
+ 只有超出最终字符数限制时才进行结构化截断。
50
50
  - **正确语义。** 工具描述是 `Search the web or analyze specified URLs.`;
51
51
  schema 明确说明 `urls` 不依赖 `query`。
52
52
  - **不碰系统提示词。** 不添加 prompt snippet、guideline、loadout hook、
@@ -215,17 +215,32 @@ Provider 数据放在 JSON 外层中返回,不做清理。示例:
215
215
  - 结果不重排、不总结。
216
216
  - 不插入 citation 标记,不追加 "Sources" 或 "Additional results"。
217
217
  - 不解析 redirect,不派生 sources。
218
- - provider 的未知字段保留。
218
+ - 未超出大小限制时,provider 的未知字段也会保留。
219
219
 
220
- 唯一的转换是最终输出大小保护。模型可见的 JSON 默认截断到
221
- **64,000 字符**,末尾标记 `[Truncated]`,`details.truncated` 为 `true`。
222
- 可通过环境变量调整:
220
+ 最终大小保护默认限制成功结果不超过 **64,000 个 Unicode code point**。
221
+ 未超限时保留原始字段及格式;如果只是缩进导致超限,会先压缩 JSON,
222
+ 不丢失数据,也不标记为截断。
223
+
224
+ 压缩后仍超限时,按字段和数组条目分配预算,未用完的额度会重新分配;
225
+ 尽量保留 URL、标题和错误等标识信息。必要时会缩短字符串或省略字段、
226
+ 数组末尾的条目,但返回值始终是合法 JSON。空间允许时,缩短的字符串以
227
+ `…[truncated]` 结尾;模型可见的 `_truncation` 对象包含 `truncated`、
228
+ `originalChars`(原始缩进 JSON 的字符数)和 `affectedPaths`。
229
+ `moreAffectedPaths` 表示路径列表不完整,数组路径也可能表示部分条目被省略。
230
+ 预算极小时无法保证保留所有 URL 或字段。确实丢失数据时,
231
+ `details.truncated` 也会设为 `true`。
232
+
233
+ 可在**启动 pi 之前**通过环境变量调整:
223
234
 
224
235
  ```bash
225
236
  export PI_BETTER_WEB_SEARCH_MAX_CHARS=100000
226
237
  ```
227
238
 
228
- 截断按 Unicode code point 处理,不会拆开 surrogate pair。
239
+ PowerShell 使用 `$env:PI_BETTER_WEB_SEARCH_MAX_CHARS = "100000"`。
240
+ 上限是 code point,**不是 token**,不会根据当前模型的上下文窗口自动调整。
241
+ 极小上限(少于 19 字符)无法在模型可见内容中容纳截断标记,此时仍返回
242
+ 合法 JSON,并在 `details.truncated` 中报告。当前没有续读或分页缓存;
243
+ 需要更多内容时可单独重新抓取 URL 或提高上限。
229
244
 
230
245
  ## 暴露规则
231
246
 
@@ -38,7 +38,7 @@ tool execute
38
38
  ├─ run search candidates until one succeeds
39
39
  ├─ run fetch candidates per URL until each URL succeeds or candidates end
40
40
  ├─ write search/fetch preferences + fingerprint on success
41
- └─ format raw JSON, truncate only if the character guard is exceeded
41
+ └─ format raw JSON; compact or structurally truncate if the guard is exceeded
42
42
  ```
43
43
 
44
44
  ## Exposure
@@ -160,9 +160,16 @@ The result is a JSON wrapper:
160
160
 
161
161
  Fields inside `search` and `fetched[].data` are provider data. The extension does
162
162
  not deduplicate, normalize, filter, re-rank, summarize, insert citation markers,
163
- or append source sections. The only transformation is the final character
164
- guard: the serialized JSON is truncated at 64,000 characters by default
165
- (`PI_BETTER_WEB_SEARCH_MAX_CHARS`) and marked with `[Truncated]`.
163
+ or append source sections. The final guard keeps successful model-facing output
164
+ within 64,000 Unicode code points by default (`PI_BETTER_WEB_SEARCH_MAX_CHARS`).
165
+ It preserves ordinary formatted JSON, or uses compact JSON if that alone fits.
166
+ Otherwise it budgets sibling fields and array entries independently, preferring
167
+ identifiers when space is tight. Strings can be shortened, and fields or trailing
168
+ array entries omitted; the output remains valid JSON. `_truncation` records the
169
+ original indented JSON length and affected paths, while `details.truncated` marks
170
+ actual data loss. For very small limits below 19 characters, only the details
171
+ flag fits. This is a character guard, not a token or model-context limit; it does
172
+ not provide pagination or retain the complete provider payload for later use.
166
173
 
167
174
  ## Provider adapter contract
168
175
 
@@ -30,7 +30,7 @@ local `pi-web-search` reference checkout, `.env` files, and `DESIGN.md`.
30
30
  npm run build
31
31
  ```
32
32
 
33
- This produces `dist/pi-better-web-search-0.1.4.tgz`. The tarball is not
33
+ This produces `dist/pi-better-web-search-0.1.6.tgz`. The tarball is not
34
34
  committed.
35
35
 
36
36
  ## Publish
@@ -42,7 +42,7 @@ npm publish --access public
42
42
  Alternatively, publish the verified tarball directly:
43
43
 
44
44
  ```bash
45
- npm publish dist/pi-better-web-search-0.1.4.tgz --access public
45
+ npm publish dist/pi-better-web-search-0.1.6.tgz --access public
46
46
  ```
47
47
 
48
48
  `prepublishOnly` reruns the checks, tests, and package verification. npm may
@@ -51,10 +51,10 @@ require browser or OTP authentication.
51
51
  ## Tag and release
52
52
 
53
53
  ```bash
54
- git tag -a v0.1.4 -m "v0.1.4"
55
- git push origin v0.1.4
56
- gh release create v0.1.4 dist/pi-better-web-search-0.1.4.tgz \
57
- --title "v0.1.4" \
54
+ git tag -a v0.1.6 -m "v0.1.6"
55
+ git push origin v0.1.6
56
+ gh release create v0.1.6 dist/pi-better-web-search-0.1.6.tgz \
57
+ --title "v0.1.6" \
58
58
  --notes-file /path/to/release-notes.md
59
59
  ```
60
60
 
package/docs/testing.md CHANGED
@@ -12,12 +12,12 @@ npm run verify:package
12
12
  ```
13
13
 
14
14
  - `npm run check` runs `tsc --noEmit`.
15
- - `npm test` runs the 116 unit and integration tests with Node's test runner.
15
+ - `npm test` runs the unit and integration tests with Node's test runner.
16
16
  - `npm run verify:package` runs `npm pack --dry-run --json` and checks that the
17
17
  tarball contains runtime sources, docs, assets, and metadata, while excluding
18
18
  tests, scripts, CI files, the local reference checkout, and `DESIGN.md`.
19
19
  - `npm run build` runs the checks, tests, package verification, and packs
20
- `dist/pi-better-web-search-0.1.0.tgz`.
20
+ `dist/pi-better-web-search-0.1.6.tgz`.
21
21
 
22
22
  Tests import the TypeScript sources directly. The suite mocks `globalThis.fetch`
23
23
  and uses temporary cache directories. Coverage includes:
@@ -30,7 +30,8 @@ and uses temporary cache directories. Coverage includes:
30
30
  - search and fetch fallback, including per-URL fallback;
31
31
  - provider request shapes and SSE parsing;
32
32
  - HTTP/stream errors and abort propagation;
33
- - character-guard truncation;
33
+ - structured character-guard truncation, valid JSON, Unicode and escaping,
34
+ boundary lengths, and fair allocation across search results and URLs;
34
35
  - one-tool registration and no system-prompt hooks.
35
36
 
36
37
  ## Live DeepSeek test
package/lib/format.ts CHANGED
@@ -1,9 +1,10 @@
1
1
  import type { AgentToolResult } from "@earendil-works/pi-coding-agent";
2
+ import { formatJsonWithinLimit } from "./truncate-json.ts";
2
3
 
3
- const configuredOutputLimit = Number(process.env.PI_BETTER_WEB_SEARCH_MAX_CHARS);
4
+ const configuredOutputLimit = Math.floor(Number(process.env.PI_BETTER_WEB_SEARCH_MAX_CHARS));
4
5
  export const MAX_OUTPUT_CHARS =
5
- Number.isFinite(configuredOutputLimit) && configuredOutputLimit > 0
6
- ? Math.floor(configuredOutputLimit)
6
+ Number.isFinite(configuredOutputLimit) && configuredOutputLimit >= 1
7
+ ? configuredOutputLimit
7
8
  : 64_000;
8
9
 
9
10
  export interface WebSearchOutput {
@@ -41,8 +42,6 @@ export interface WebSearchDetails {
41
42
  error?: string;
42
43
  }
43
44
 
44
- const TRUNCATION_MARKER = "\n\n[Truncated]";
45
-
46
45
  /**
47
46
  * Truncate by Unicode code point so a surrogate pair is never split. The
48
47
  * returned `text` is at most `maxChars` code points.
@@ -61,11 +60,9 @@ export function formatSuccess(
61
60
  output: WebSearchOutput,
62
61
  details: WebSearchDetails,
63
62
  ): AgentToolResult<WebSearchDetails> {
64
- const json = JSON.stringify(output, null, 2);
65
- const marker = TRUNCATION_MARKER.slice(0, Math.max(0, MAX_OUTPUT_CHARS));
66
- const { text, truncated } = truncateText(json, Math.max(0, MAX_OUTPUT_CHARS - marker.length));
63
+ const { text, truncated } = formatJsonWithinLimit(output, MAX_OUTPUT_CHARS);
67
64
  return {
68
- content: [{ type: "text", text: truncated ? text + marker : text }],
65
+ content: [{ type: "text", text }],
69
66
  details: truncated ? { ...details, truncated: true } : details,
70
67
  };
71
68
  }
@@ -0,0 +1,299 @@
1
+ type JsonValue = null | boolean | number | string | JsonValue[] | { [key: string]: JsonValue };
2
+
3
+ interface ChangeTracker {
4
+ paths: string[];
5
+ seen: Set<string>;
6
+ more: boolean;
7
+ }
8
+
9
+ interface TruncationInfo {
10
+ truncated: true;
11
+ originalChars: number;
12
+ affectedPaths: string[];
13
+ moreAffectedPaths: boolean;
14
+ }
15
+
16
+ interface Slot {
17
+ index: number;
18
+ key?: string;
19
+ value: JsonValue;
20
+ minimum: number;
21
+ full: number;
22
+ allocation: number;
23
+ }
24
+
25
+ const MAX_RECORDED_PATHS = 64;
26
+ const MAX_PATH_LENGTH = 160;
27
+ const MAX_PATH_BUDGET = 2_048;
28
+ const STRING_MARKER = "…[truncated]";
29
+ const IDENTITY_KEYS = ["url", "uri", "title", "name", "id", "error", "status", "query", "answer", "content", "text"];
30
+ const IMPORTANT_KEYS = [...IDENTITY_KEYS, "search", "fetched", "results", "data"];
31
+ const KEY_PRIORITY = new Map(IMPORTANT_KEYS.map((key, index) => [key, index]));
32
+
33
+ /** Count Unicode code points, rather than UTF-16 code units. */
34
+ function charCount(text: string): number {
35
+ let count = 0;
36
+ for (const _character of text) count += 1;
37
+ return count;
38
+ }
39
+
40
+ function recordChange(tracker: ChangeTracker, path: string): void {
41
+ if (tracker.seen.has(path)) return;
42
+ if (tracker.paths.length >= MAX_RECORDED_PATHS) {
43
+ tracker.more = true;
44
+ return;
45
+ }
46
+ tracker.seen.add(path);
47
+ tracker.paths.push(path);
48
+ }
49
+
50
+ function propertyPath(parent: string, key: string): string {
51
+ // A very long or unusual provider field name is represented by its parent,
52
+ // rather than allowing an unbounded field name to consume the path budget.
53
+ if (parent.length + key.length > MAX_PATH_LENGTH) return parent;
54
+ if (/^[A-Za-z_$][\w$]*$/.test(key)) return `${parent}.${key}`;
55
+ return `${parent}[${JSON.stringify(key)}]`;
56
+ }
57
+
58
+ function indexPath(parent: string, index: number): string {
59
+ const path = `${parent}[${index}]`;
60
+ return path.length <= MAX_PATH_LENGTH ? path : parent;
61
+ }
62
+
63
+ /** Encoded length of one code point inside a JSON string (without quotes). */
64
+ function encodedCharacterLength(character: string): number {
65
+ if (character.length === 2) return 1; // a surrogate pair is one code point
66
+ const code = character.charCodeAt(0);
67
+ if (character === '"' || character === "\\") return 2;
68
+ if (code === 8 || code === 9 || code === 10 || code === 12 || code === 13) return 2;
69
+ if (code < 32 || (code >= 0xd800 && code <= 0xdfff)) return 6;
70
+ return 1;
71
+ }
72
+
73
+ function fitString(value: string, budget: number): string {
74
+ // A JSON string needs two quotes. Include a marker whenever it fits; paths in
75
+ // _truncation also identify shortened strings when the budget is very small.
76
+ const marker = budget >= charCount(STRING_MARKER) + 2 ? STRING_MARKER : budget >= 3 ? "…" : "";
77
+ let remaining = budget - 2 - charCount(marker);
78
+ const prefix: string[] = [];
79
+ for (const character of value) {
80
+ const size = encodedCharacterLength(character);
81
+ if (size > remaining) break;
82
+ prefix.push(character);
83
+ remaining -= size;
84
+ }
85
+ return prefix.join("") + marker;
86
+ }
87
+
88
+ /** Distribute the available space fairly, giving unused space to larger values. */
89
+ function allocate(slots: Slot[], available: number): void {
90
+ let remaining = available;
91
+ for (const slot of slots) {
92
+ slot.allocation = slot.minimum;
93
+ remaining -= slot.minimum;
94
+ }
95
+
96
+ const pending = slots
97
+ .filter((slot) => slot.full > slot.minimum)
98
+ .sort((a, b) => (a.full - a.minimum) - (b.full - b.minimum) || a.index - b.index);
99
+
100
+ for (let i = 0; i < pending.length && remaining > 0; i += 1) {
101
+ const slot = pending[i];
102
+ const count = pending.length - i;
103
+ const share = Math.floor(remaining / count);
104
+ const demand = slot.full - slot.minimum;
105
+ if (demand <= share) {
106
+ slot.allocation += demand;
107
+ remaining -= demand;
108
+ continue;
109
+ }
110
+
111
+ const remainder = remaining % count;
112
+ for (let j = i; j < pending.length; j += 1) {
113
+ pending[j].allocation += share + (j - i < remainder ? 1 : 0);
114
+ }
115
+ break;
116
+ }
117
+ }
118
+
119
+ function createFitter(tracker: ChangeTracker) {
120
+ const sizes = new WeakMap<object, number>();
121
+
122
+ const sizeOf = (value: JsonValue): number => {
123
+ if (value !== null && typeof value === "object") {
124
+ const cached = sizes.get(value);
125
+ if (cached !== undefined) return cached;
126
+ const size = charCount(JSON.stringify(value));
127
+ sizes.set(value, size);
128
+ return size;
129
+ }
130
+ return charCount(JSON.stringify(value));
131
+ };
132
+
133
+ const minimumSize = (value: JsonValue): number => {
134
+ if (typeof value === "string" || Array.isArray(value)) return 2; // "" or []
135
+ if (value !== null && typeof value === "object") {
136
+ // An array of hundreds of empty objects is not useful. Reserve enough
137
+ // for one identifying or text field when the item has one; if the
138
+ // budget cannot accommodate all entries, report an omitted suffix.
139
+ for (const key of IDENTITY_KEYS) {
140
+ if (!Object.hasOwn(value, key)) continue;
141
+ const child = value[key];
142
+ if (child !== null && typeof child === "object") continue;
143
+ const childMinimum = typeof child === "string" ? Math.min(sizeOf(child), 128) : sizeOf(child);
144
+ return 2 + charCount(JSON.stringify(key)) + 1 + childMinimum;
145
+ }
146
+ return 2;
147
+ }
148
+ return sizeOf(value);
149
+ };
150
+
151
+ const fit = (value: JsonValue, budget: number, path: string): JsonValue => {
152
+ if (sizeOf(value) <= budget) return value;
153
+
154
+ if (typeof value === "string") {
155
+ recordChange(tracker, path);
156
+ return fitString(value, budget);
157
+ }
158
+
159
+ if (Array.isArray(value)) {
160
+ const slots: Slot[] = [];
161
+ let fixed = 2; // brackets and commas
162
+ let minimums = 0;
163
+ for (let index = 0; index < value.length; index += 1) {
164
+ const child = value[index];
165
+ const minimum = minimumSize(child);
166
+ const punctuation = slots.length > 0 ? 1 : 0;
167
+ if (fixed + punctuation + minimums + minimum > budget) {
168
+ // Preserve array order and report that one or more entries were omitted.
169
+ recordChange(tracker, path);
170
+ break;
171
+ }
172
+ fixed += punctuation;
173
+ minimums += minimum;
174
+ slots.push({ index, value: child, minimum, full: sizeOf(child), allocation: minimum });
175
+ }
176
+ allocate(slots, budget - fixed);
177
+ return slots.map((slot) => fit(slot.value, slot.allocation, indexPath(path, slot.index)));
178
+ }
179
+
180
+ if (value !== null && typeof value === "object") {
181
+ const entries = Object.entries(value);
182
+ // Prefer identifiers and the search/fetch containers when even the
183
+ // structural overhead does not fit. Restore the original field order in
184
+ // the result; no provider data is re-ranked.
185
+ const candidates = entries.map(([key, child], index) => ({ key, child, index }));
186
+ candidates.sort((a, b) =>
187
+ (KEY_PRIORITY.get(a.key) ?? IMPORTANT_KEYS.length) - (KEY_PRIORITY.get(b.key) ?? IMPORTANT_KEYS.length)
188
+ || a.index - b.index,
189
+ );
190
+ const slots: Slot[] = [];
191
+ let fixed = 2; // braces, keys, colons and commas
192
+ let minimums = 0;
193
+ for (const { key, child, index } of candidates) {
194
+ const minimum = minimumSize(child);
195
+ const punctuation = (slots.length > 0 ? 1 : 0) + charCount(JSON.stringify(key)) + 1;
196
+ if (fixed + punctuation + minimums + minimum > budget) {
197
+ recordChange(tracker, propertyPath(path, key));
198
+ continue;
199
+ }
200
+ fixed += punctuation;
201
+ minimums += minimum;
202
+ slots.push({ index, key, value: child, minimum, full: sizeOf(child), allocation: minimum });
203
+ }
204
+ slots.sort((a, b) => a.index - b.index);
205
+ allocate(slots, budget - fixed);
206
+ const result: { [key: string]: JsonValue } = Object.create(null);
207
+ for (const slot of slots) {
208
+ const key = slot.key!;
209
+ result[key] = fit(slot.value, slot.allocation, propertyPath(path, key));
210
+ }
211
+ return result;
212
+ }
213
+
214
+ // Parents reserve the complete serialized size for non-string scalars.
215
+ throw new Error("Insufficient JSON budget for scalar value");
216
+ };
217
+
218
+ return fit;
219
+ }
220
+
221
+ function minimalTruncatedJson(limit: number): string {
222
+ for (const text of ['{"_truncation":{"truncated":true}}', '{"_truncated":true}', "{}", "0"]) {
223
+ if (text.length <= limit) return text;
224
+ }
225
+ throw new RangeError("The JSON output limit must be at least one character");
226
+ }
227
+
228
+ /**
229
+ * Preserve ordinary output exactly. If compact JSON still exceeds the limit,
230
+ * fit its fields and array entries independently and report affected paths.
231
+ * The returned text is valid JSON and never exceeds `maxChars` code points.
232
+ */
233
+ export function formatJsonWithinLimit(
234
+ output: object,
235
+ maxChars: number,
236
+ ): { text: string; truncated: boolean } {
237
+ if (!Number.isFinite(maxChars) || maxChars < 1) {
238
+ throw new RangeError("The JSON output limit must be at least one character");
239
+ }
240
+ const limit = Math.floor(maxChars);
241
+ const pretty = JSON.stringify(output, null, 2);
242
+ const originalChars = charCount(pretty);
243
+ if (originalChars <= limit) return { text: pretty, truncated: false };
244
+
245
+ // Work from the already serialized value, so JSON's handling of undefined,
246
+ // dates and toJSON is identical even if an input object is not plain data.
247
+ const parsed = JSON.parse(pretty) as { [key: string]: JsonValue };
248
+ const compact = JSON.stringify(parsed);
249
+ if (charCount(compact) <= limit) return { text: compact, truncated: false };
250
+
251
+ const info: TruncationInfo = {
252
+ truncated: true,
253
+ originalChars,
254
+ affectedPaths: [],
255
+ moreAffectedPaths: false,
256
+ };
257
+ // Account for the metadata entry and a comma joining it to the root object.
258
+ const overhead = charCount(JSON.stringify({ _truncation: info })) - 2 + 1;
259
+ if (limit - overhead < 2) return { text: minimalTruncatedJson(limit), truncated: true };
260
+
261
+ const build = (reservedPaths: number) => {
262
+ const tracker: ChangeTracker = { paths: [], seen: new Set(), more: false };
263
+ const fit = createFitter(tracker);
264
+ const fitted = fit(parsed, limit - overhead - reservedPaths, "$") as { [key: string]: JsonValue };
265
+ return { fitted, tracker };
266
+ };
267
+
268
+ let result = build(0);
269
+ // Reserve just enough room for the first pass's affected paths, capped so
270
+ // reporting never consumes a disproportionate share of the result budget.
271
+ const desiredPathSpace = Math.min(
272
+ MAX_PATH_BUDGET,
273
+ Math.max(64, Math.floor(limit * 0.05)),
274
+ charCount(JSON.stringify(result.tracker.paths)) - 2,
275
+ );
276
+ if (desiredPathSpace > 0 && limit - overhead - desiredPathSpace >= 2) {
277
+ result = build(desiredPathSpace);
278
+ }
279
+
280
+ const combined: { [key: string]: unknown } = Object.create(null);
281
+ combined._truncation = info;
282
+ for (const [key, value] of Object.entries(result.fitted)) {
283
+ if (key !== "_truncation") combined[key] = value;
284
+ }
285
+
286
+ let remaining = limit - charCount(JSON.stringify(combined));
287
+ for (const path of result.tracker.paths) {
288
+ const cost = charCount(JSON.stringify(path)) + (info.affectedPaths.length > 0 ? 1 : 0);
289
+ if (cost > remaining) {
290
+ info.moreAffectedPaths = true;
291
+ break;
292
+ }
293
+ info.affectedPaths.push(path);
294
+ remaining -= cost;
295
+ }
296
+ if (result.tracker.more) info.moreAffectedPaths = true;
297
+
298
+ return { text: JSON.stringify(combined), truncated: true };
299
+ }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-better-web-search",
3
- "version": "0.1.5",
3
+ "version": "0.1.6",
4
4
  "description": "A better provider-native web_search tool for pi: raw results, automatic provider fallback, URL analysis, and native DeepSeek Search/Fetch.",
5
5
  "type": "module",
6
6
  "license": "MIT",