pi-better-web-search 0.1.5 → 0.1.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,24 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.1.7
4
+
5
+ - Keep recognized URL/URI/link and title/name/ID fields intact when truncating;
6
+ omit entries that cannot fit rather than returning partial links. Preserve
7
+ provider order and drop incomplete URL tokens at text boundaries.
8
+ - Raise the default model-facing output limit from 64,000 to 96,000 Unicode
9
+ code points; the environment override remains available.
10
+ - Replace model-facing truncation metadata and per-string markers with a single
11
+ `[truncated]` line at the end of oversized results. Keep the existing fair
12
+ allocation across search results and fetched pages.
13
+
14
+ ## 0.1.6
15
+
16
+ - Keep oversized results as valid JSON, sharing the character budget across
17
+ search results and fetched pages rather than cutting off the serialized tail.
18
+ Report affected paths and mark shortened strings; retain unchanged provider
19
+ data when it fits, and compact whitespace before dropping data.
20
+ - Fix premature truncation immediately below the configured output limit.
21
+
3
22
  ## 0.1.5
4
23
 
5
24
  - Rank models inside a provider family by capability tier first (cheapest tier
package/README.md CHANGED
@@ -8,7 +8,7 @@ provider supports it, automatically falls back across authenticated providers,
8
8
  and returns provider data as-is.
9
9
 
10
10
  [![CI](https://github.com/zhexusun10/pi-better-web-search/actions/workflows/ci.yml/badge.svg)](https://github.com/zhexusun10/pi-better-web-search/actions/workflows/ci.yml)
11
- [![npm](https://img.shields.io/badge/npm-v0.1.5-blue)](https://www.npmjs.com/package/pi-better-web-search)
11
+ [![npm](https://img.shields.io/npm/v/pi-better-web-search)](https://www.npmjs.com/package/pi-better-web-search)
12
12
  [![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
13
13
  [![Pi 1.3](https://img.shields.io/badge/Pi-1.3-blue.svg)](https://pi.dev)
14
14
  [![Node](https://img.shields.io/badge/node-%3E%3D22.19-339933.svg)](https://nodejs.org)
@@ -58,8 +58,8 @@ Requirements: **pi 1.3+** and **Node.js 22.19+**.
58
58
  provider supports URL analysis. `urls` can be used alone or with `query`.
59
59
  - **Raw results.** Provider data is returned in a JSON wrapper without
60
60
  deduplication, URL normalization, junk filtering, re-ranking, summarization,
61
- citation-marker insertion, or appended Sources sections. Only a final
62
- character guard truncates oversized output.
61
+ citation-marker insertion, or appended Sources sections. A final character
62
+ guard applies structured truncation only to oversized output.
63
63
  - **Correct semantics.** The tool description is exactly
64
64
  `Search the web or analyze specified URLs.` The schema makes clear that
65
65
  `urls` does not require `query`.
@@ -250,18 +250,42 @@ this:
250
250
  - Citation markers are not inserted, and no "Sources" or "Additional results"
251
251
  sections are appended.
252
252
  - Redirects are not resolved and sources are not derived.
253
- - Unknown provider fields are preserved.
254
-
255
- The only transformation is a final output-size guard. The model-facing JSON is
256
- truncated to **64,000 characters** by default and marked with `[Truncated]`;
257
- `details.truncated` is set to `true`. Change the limit with:
253
+ - Unknown provider fields are preserved when the result fits the size guard.
254
+
255
+ A final output-size guard limits successful model-facing results to **96,000
256
+ Unicode code points** by default. Normal-sized results retain their original
257
+ fields and formatting. If indentation alone exceeds the limit, compact JSON is
258
+ returned without dropping data or setting the truncation flag.
259
+
260
+ When even compact JSON is too large, the guard allocates space across sibling
261
+ fields and array entries, redistributing unused space. Recognized link and
262
+ identity fields (`url`, `uri`, `href`, `link`, `title`, `name`, `id`, and common
263
+ aliases) stay complete; if they cannot fit, the associated entry or branch is
264
+ omitted instead. Standalone URLs and entries in link/citation arrays also stay
265
+ whole. When a text cut would split a recognizable embedded URL token (such as
266
+ `https://`, `mailto:`, or `www.`), the incomplete token is omitted. Relative
267
+ links or bare domains in unrecognized fields cannot always be identified. Other long strings may be shortened, and fields or trailing array
268
+ entries omitted; results are not re-ranked or summarized.
269
+
270
+ The JSON portion remains valid, followed by a single `[truncated]` line. No
271
+ truncation metadata or per-string markers are added to the model-facing JSON.
272
+ Because of the trailing line, the complete text is not itself valid JSON.
273
+ `details.truncated` is also set to `true` when data is actually removed.
274
+
275
+ Change the limit **before starting pi**:
258
276
 
259
277
  ```bash
260
278
  export PI_BETTER_WEB_SEARCH_MAX_CHARS=100000
261
279
  ```
262
280
 
263
- Truncation is Unicode code-point safe. The guard is intentionally applied after
264
- the raw JSON is assembled so normal-sized provider payloads are not altered.
281
+ In PowerShell, use `$env:PI_BETTER_WEB_SEARCH_MAX_CHARS = "100000"`. The limit
282
+ is in code points, **not tokens**, and does not adapt to the current model's
283
+ context window. The marker counts toward the limit. If truncation is needed and
284
+ the limit is 11–13 characters, only `[truncated]` is returned; below 11
285
+ characters the marker does not fit and only a minimal JSON value is returned.
286
+ `details.truncated` remains true in both cases. There is no continuation or
287
+ pagination cache: retry individual URLs or raise the limit if more content is
288
+ needed.
265
289
 
266
290
  ## Exposure rules
267
291
 
@@ -366,7 +390,7 @@ npm run build
366
390
  ```
367
391
 
368
392
  `npm run build` runs the checks and tests, verifies the npm file list, and packs
369
- `dist/pi-better-web-search-0.1.0.tgz`. Nothing is published automatically.
393
+ `dist/pi-better-web-search-0.1.7.tgz`. Nothing is published automatically.
370
394
 
371
395
  The test suite uses plain `.mjs` files that import the `.ts` sources directly.
372
396
  It mocks `globalThis.fetch`, uses a temporary cache file, and exercises
@@ -46,7 +46,7 @@ pi install ./path/to/pi-better-web-search
46
46
  `urls`。`urls` 可以单独使用,也可以和 `query` 一起使用。
47
47
  - **原始结果。** provider 数据放在 JSON 外层中返回,不去重、不规范化 URL、
48
48
  不过滤垃圾 URL、不重排、不总结、不插入 citation 标记、不追加 Sources。
49
- 只做最终字符数限制和截断。
49
+ 只有超出最终字符数限制时才进行结构化截断。
50
50
  - **正确语义。** 工具描述是 `Search the web or analyze specified URLs.`;
51
51
  schema 明确说明 `urls` 不依赖 `query`。
52
52
  - **不碰系统提示词。** 不添加 prompt snippet、guideline、loadout hook、
@@ -215,17 +215,38 @@ Provider 数据放在 JSON 外层中返回,不做清理。示例:
215
215
  - 结果不重排、不总结。
216
216
  - 不插入 citation 标记,不追加 "Sources" 或 "Additional results"。
217
217
  - 不解析 redirect,不派生 sources。
218
- - provider 的未知字段保留。
218
+ - 未超出大小限制时,provider 的未知字段也会保留。
219
219
 
220
- 唯一的转换是最终输出大小保护。模型可见的 JSON 默认截断到
221
- **64,000 字符**,末尾标记 `[Truncated]`,`details.truncated` 为 `true`。
222
- 可通过环境变量调整:
220
+ 最终大小保护默认限制成功结果不超过 **96,000 个 Unicode code point**。
221
+ 未超限时保留原始字段及格式;如果只是缩进导致超限,会先压缩 JSON,
222
+ 不丢失数据,也不标记为截断。
223
+
224
+ 压缩后仍超限时,按字段和数组条目分配预算,未用完的额度会重新分配。
225
+ 已识别的链接及标识字段(包括 `url`、`uri`、`href`、`link`、`title`、
226
+ `name`、`id` 和常见别名)保持完整;放不下时省略对应条目或分支,而不是
227
+ 返回半截值。独立 URL 和链接、引用数组中的条目也保持完整;正文截断点
228
+ 若落在可识别的 URL(如 `https://`、`mailto:` 或 `www.`)中间,会省略该
229
+ 未完整显示的 URL。未知字段中的相对链接或裸域名无法保证都能识别。
230
+ 其他长字符串可能被缩短,字段或数组末尾的条目可能被省略;
231
+ 不对结果重排或总结。
232
+
233
+ JSON 部分仍然合法,末尾仅追加一行 `[truncated]`。不会向模型可见的 JSON
234
+ 中添加截断元数据,也不会逐个标记缩短的字符串。由于末尾追加了这一行,
235
+ 整个文本不再是合法 JSON。确实丢失数据时,`details.truncated` 也会设为
236
+ `true`。
237
+
238
+ 可在**启动 pi 之前**通过环境变量调整:
223
239
 
224
240
  ```bash
225
241
  export PI_BETTER_WEB_SEARCH_MAX_CHARS=100000
226
242
  ```
227
243
 
228
- 截断按 Unicode code point 处理,不会拆开 surrogate pair。
244
+ PowerShell 使用 `$env:PI_BETTER_WEB_SEARCH_MAX_CHARS = "100000"`。
245
+ 上限是 code point,**不是 token**,不会根据当前模型的上下文窗口自动调整。
246
+ 标记也计入上限。需要截断且上限为 11–13 字符时,只返回 `[truncated]`;
247
+ 少于 11 字符时放不下标记,只返回一个最小 JSON 值。这两种情况仍会通过
248
+ `details.truncated` 报告截断。当前没有续读或分页缓存;需要更多内容时可
249
+ 单独重新抓取 URL 或提高上限。
229
250
 
230
251
  ## 暴露规则
231
252
 
@@ -308,7 +329,7 @@ npm run build
308
329
  ```
309
330
 
310
331
  `npm run build` 会运行检查、测试、包内容验证,并生成
311
- `dist/pi-better-web-search-0.1.0.tgz`。不会自动发布。
332
+ `dist/pi-better-web-search-0.1.7.tgz`。不会自动发布。
312
333
 
313
334
  测试是 `.mjs` 文件,直接导入 `.ts` 源码,mock `globalThis.fetch`,使用临时缓存
314
335
  目录。详见 [测试](testing.md) 和 [发布](publishing.md)。
@@ -38,7 +38,7 @@ tool execute
38
38
  ├─ run search candidates until one succeeds
39
39
  ├─ run fetch candidates per URL until each URL succeeds or candidates end
40
40
  ├─ write search/fetch preferences + fingerprint on success
41
- └─ format raw JSON, truncate only if the character guard is exceeded
41
+ └─ format raw JSON; compact or structurally truncate if the guard is exceeded
42
42
  ```
43
43
 
44
44
  ## Exposure
@@ -160,9 +160,22 @@ The result is a JSON wrapper:
160
160
 
161
161
  Fields inside `search` and `fetched[].data` are provider data. The extension does
162
162
  not deduplicate, normalize, filter, re-rank, summarize, insert citation markers,
163
- or append source sections. The only transformation is the final character
164
- guard: the serialized JSON is truncated at 64,000 characters by default
165
- (`PI_BETTER_WEB_SEARCH_MAX_CHARS`) and marked with `[Truncated]`.
163
+ or append source sections. The final guard keeps successful model-facing output
164
+ within 96,000 Unicode code points by default (`PI_BETTER_WEB_SEARCH_MAX_CHARS`).
165
+ It preserves ordinary formatted JSON, or uses compact JSON if that alone fits.
166
+ Otherwise it budgets sibling fields and array entries independently. Recognized
167
+ link and identity fields stay whole; entries or branches that cannot fit them
168
+ are omitted instead, without re-ranking. Standalone URLs and link/citation array
169
+ entries remain intact, and a recognizable URL crossing a text cut is omitted
170
+ rather than partially emitted. Unknown relative links or bare domains may not be
171
+ identifiable. Other strings can be shortened, and fields or trailing array entries
172
+ omitted. The JSON portion
173
+ stays valid, followed by one `[truncated]` line; the complete text is therefore
174
+ not valid JSON. No truncation metadata is injected into the model-facing JSON,
175
+ while `details.truncated` marks actual data loss. When truncation is needed with a limit of 11–13 characters,
176
+ only the marker is returned; below 11, a minimal JSON value is returned instead.
177
+ This is a character guard, not a token or model-context limit; it does not
178
+ provide pagination or retain the complete provider payload for later use.
166
179
 
167
180
  ## Provider adapter contract
168
181
 
@@ -30,7 +30,7 @@ local `pi-web-search` reference checkout, `.env` files, and `DESIGN.md`.
30
30
  npm run build
31
31
  ```
32
32
 
33
- This produces `dist/pi-better-web-search-0.1.4.tgz`. The tarball is not
33
+ This produces `dist/pi-better-web-search-0.1.7.tgz`. The tarball is not
34
34
  committed.
35
35
 
36
36
  ## Publish
@@ -42,7 +42,7 @@ npm publish --access public
42
42
  Alternatively, publish the verified tarball directly:
43
43
 
44
44
  ```bash
45
- npm publish dist/pi-better-web-search-0.1.4.tgz --access public
45
+ npm publish dist/pi-better-web-search-0.1.7.tgz --access public
46
46
  ```
47
47
 
48
48
  `prepublishOnly` reruns the checks, tests, and package verification. npm may
@@ -51,10 +51,10 @@ require browser or OTP authentication.
51
51
  ## Tag and release
52
52
 
53
53
  ```bash
54
- git tag -a v0.1.4 -m "v0.1.4"
55
- git push origin v0.1.4
56
- gh release create v0.1.4 dist/pi-better-web-search-0.1.4.tgz \
57
- --title "v0.1.4" \
54
+ git tag -a v0.1.7 -m "v0.1.7"
55
+ git push origin v0.1.7
56
+ gh release create v0.1.7 dist/pi-better-web-search-0.1.7.tgz \
57
+ --title "v0.1.7" \
58
58
  --notes-file /path/to/release-notes.md
59
59
  ```
60
60
 
package/docs/testing.md CHANGED
@@ -12,12 +12,12 @@ npm run verify:package
12
12
  ```
13
13
 
14
14
  - `npm run check` runs `tsc --noEmit`.
15
- - `npm test` runs the 116 unit and integration tests with Node's test runner.
15
+ - `npm test` runs the unit and integration tests with Node's test runner.
16
16
  - `npm run verify:package` runs `npm pack --dry-run --json` and checks that the
17
17
  tarball contains runtime sources, docs, assets, and metadata, while excluding
18
18
  tests, scripts, CI files, the local reference checkout, and `DESIGN.md`.
19
19
  - `npm run build` runs the checks, tests, package verification, and packs
20
- `dist/pi-better-web-search-0.1.0.tgz`.
20
+ `dist/pi-better-web-search-0.1.7.tgz`.
21
21
 
22
22
  Tests import the TypeScript sources directly. The suite mocks `globalThis.fetch`
23
23
  and uses temporary cache directories. Coverage includes:
@@ -30,7 +30,9 @@ and uses temporary cache directories. Coverage includes:
30
30
  - search and fetch fallback, including per-URL fallback;
31
31
  - provider request shapes and SSE parsing;
32
32
  - HTTP/stream errors and abort propagation;
33
- - character-guard truncation;
33
+ - structured character-guard truncation, a single trailing marker, valid JSON
34
+ before the marker, Unicode and escaping, boundary lengths, fair allocation,
35
+ and intact links and identifiers across search results and fetched pages;
34
36
  - one-tool registration and no system-prompt hooks.
35
37
 
36
38
  ## Live DeepSeek test
package/lib/format.ts CHANGED
@@ -1,10 +1,11 @@
1
1
  import type { AgentToolResult } from "@earendil-works/pi-coding-agent";
2
+ import { formatJsonWithinLimit } from "./truncate-json.ts";
2
3
 
3
- const configuredOutputLimit = Number(process.env.PI_BETTER_WEB_SEARCH_MAX_CHARS);
4
+ const configuredOutputLimit = Math.floor(Number(process.env.PI_BETTER_WEB_SEARCH_MAX_CHARS));
4
5
  export const MAX_OUTPUT_CHARS =
5
- Number.isFinite(configuredOutputLimit) && configuredOutputLimit > 0
6
- ? Math.floor(configuredOutputLimit)
7
- : 64_000;
6
+ Number.isFinite(configuredOutputLimit) && configuredOutputLimit >= 1
7
+ ? configuredOutputLimit
8
+ : 96_000;
8
9
 
9
10
  export interface WebSearchOutput {
10
11
  query?: string;
@@ -41,8 +42,6 @@ export interface WebSearchDetails {
41
42
  error?: string;
42
43
  }
43
44
 
44
- const TRUNCATION_MARKER = "\n\n[Truncated]";
45
-
46
45
  /**
47
46
  * Truncate by Unicode code point so a surrogate pair is never split. The
48
47
  * returned `text` is at most `maxChars` code points.
@@ -61,11 +60,9 @@ export function formatSuccess(
61
60
  output: WebSearchOutput,
62
61
  details: WebSearchDetails,
63
62
  ): AgentToolResult<WebSearchDetails> {
64
- const json = JSON.stringify(output, null, 2);
65
- const marker = TRUNCATION_MARKER.slice(0, Math.max(0, MAX_OUTPUT_CHARS));
66
- const { text, truncated } = truncateText(json, Math.max(0, MAX_OUTPUT_CHARS - marker.length));
63
+ const { text, truncated } = formatJsonWithinLimit(output, MAX_OUTPUT_CHARS);
67
64
  return {
68
- content: [{ type: "text", text: truncated ? text + marker : text }],
65
+ content: [{ type: "text", text }],
69
66
  details: truncated ? { ...details, truncated: true } : details,
70
67
  };
71
68
  }
@@ -0,0 +1,277 @@
1
+ type JsonValue = null | boolean | number | string | JsonValue[] | { [key: string]: JsonValue };
2
+
3
+ interface Slot {
4
+ index: number;
5
+ key?: string;
6
+ value: JsonValue;
7
+ minimum: number;
8
+ full: number;
9
+ allocation: number;
10
+ }
11
+
12
+ const TRUNCATION_MARKER = "[truncated]";
13
+ const IDENTITY_KEYS = ["url", "uri", "href", "link", "title", "name", "id", "error", "status", "query", "answer", "content", "text"];
14
+ const IMPORTANT_KEYS = [...IDENTITY_KEYS, "search", "fetched", "results", "data"];
15
+ const KEY_PRIORITY = new Map(IMPORTANT_KEYS.map((key, index) => [key, index]));
16
+
17
+ /** Link and identity fields are kept intact or their containing entry is omitted. */
18
+ function isProtectedKey(key: string): boolean {
19
+ return /^(?:url|uri|href|link|permalink|website|title|name|id)$/i.test(key)
20
+ || /[_-](?:url|uri|href|link|permalink|website|title|name|id)$/i.test(key)
21
+ || /(?:URL|URI|HREF|LINK|TITLE|NAME|Url|Uri|Href|Link|Title|Name|Id|ID)$/.test(key);
22
+ }
23
+
24
+ function isLinkCollectionKey(key: string): boolean {
25
+ return /^(?:urls?|uris?|hrefs?|links?|permalinks?|websites?|titles?|names?|ids?|sources?|citations?)$/i.test(key)
26
+ || /[_-](?:urls|uris|hrefs|links|titles|names|ids)$/i.test(key)
27
+ || /(?:URLs|URIs|Urls|Uris|Hrefs|Links|Titles|Names|Ids|IDs)$/.test(key);
28
+ }
29
+
30
+ function isStandaloneUrl(value: string): boolean {
31
+ return /^(?:[a-z][a-z0-9+.-]{0,31}:\/\/|mailto:|urn:|doi:|tel:|www\.)\S+$/i.test(value);
32
+ }
33
+
34
+ /** Count Unicode code points, rather than UTF-16 code units. */
35
+ function charCount(text: string): number {
36
+ let count = 0;
37
+ for (const _character of text) count += 1;
38
+ return count;
39
+ }
40
+
41
+ /** Encoded length of one code point inside a JSON string (without quotes). */
42
+ function encodedCharacterLength(character: string): number {
43
+ if (character.length === 2) return 1; // a surrogate pair is one code point
44
+ const code = character.charCodeAt(0);
45
+ if (character === '"' || character === "\\") return 2;
46
+ if (code === 8 || code === 9 || code === 10 || code === 12 || code === 13) return 2;
47
+ if (code < 32 || (code >= 0xd800 && code <= 0xdfff)) return 6;
48
+ return 1;
49
+ }
50
+
51
+ function fitString(value: string, budget: number): string {
52
+ // Reserve the quotes; the single truncation marker is added after the JSON.
53
+ let remaining = budget - 2;
54
+ const prefix: string[] = [];
55
+ for (const character of value) {
56
+ const size = encodedCharacterLength(character);
57
+ if (size > remaining) break;
58
+ prefix.push(character);
59
+ remaining -= size;
60
+ }
61
+ const visible = prefix.join("");
62
+ if (visible.length === value.length) return visible;
63
+
64
+ // A cut inside a URL embedded in prose must not leave a plausible but broken
65
+ // link. Drop the incomplete URL token instead of changing its address.
66
+ const urlStart = /\b(?:[a-z][a-z0-9+.-]{0,31}:\/\/|mailto:|urn:|doi:|tel:|www\.)/gi;
67
+ for (let match = urlStart.exec(value); match && match.index < visible.length; match = urlStart.exec(value)) {
68
+ let end = match.index + match[0].length;
69
+ while (end < value.length && !/\s/.test(value[end])) end += 1;
70
+ if (end > visible.length) return visible.slice(0, match.index);
71
+ urlStart.lastIndex = end;
72
+ }
73
+ return visible;
74
+ }
75
+
76
+ /** Distribute the available space fairly, giving unused space to larger values. */
77
+ function allocate(slots: Slot[], available: number): void {
78
+ let remaining = available;
79
+ for (const slot of slots) {
80
+ slot.allocation = slot.minimum;
81
+ remaining -= slot.minimum;
82
+ }
83
+
84
+ const pending = slots
85
+ .filter((slot) => slot.full > slot.minimum)
86
+ .sort((a, b) => (a.full - a.minimum) - (b.full - b.minimum) || a.index - b.index);
87
+
88
+ for (let i = 0; i < pending.length && remaining > 0; i += 1) {
89
+ const slot = pending[i];
90
+ const count = pending.length - i;
91
+ const share = Math.floor(remaining / count);
92
+ const demand = slot.full - slot.minimum;
93
+ if (demand <= share) {
94
+ slot.allocation += demand;
95
+ remaining -= demand;
96
+ continue;
97
+ }
98
+
99
+ const remainder = remaining % count;
100
+ for (let j = i; j < pending.length; j += 1) {
101
+ pending[j].allocation += share + (j - i < remainder ? 1 : 0);
102
+ }
103
+ break;
104
+ }
105
+ }
106
+
107
+ function createFitter() {
108
+ const sizes = new WeakMap<object, number>();
109
+ const requiredSizes = new WeakMap<object, number>();
110
+
111
+ const sizeOf = (value: JsonValue): number => {
112
+ if (value !== null && typeof value === "object") {
113
+ const cached = sizes.get(value);
114
+ if (cached !== undefined) return cached;
115
+ const size = charCount(JSON.stringify(value));
116
+ sizes.set(value, size);
117
+ return size;
118
+ }
119
+ return charCount(JSON.stringify(value));
120
+ };
121
+
122
+ function requiredPropertySize(key: string, value: JsonValue): number | undefined {
123
+ if (isProtectedKey(key) && !Array.isArray(value)) return sizeOf(value);
124
+ if (value !== null && typeof value === "object") {
125
+ if (Array.isArray(value)) return undefined;
126
+ const nested = requiredSize(value);
127
+ return nested > 0 ? nested : undefined;
128
+ }
129
+ if (typeof value === "string" && isStandaloneUrl(value)) return sizeOf(value);
130
+ return undefined;
131
+ }
132
+
133
+ // Reserve every direct or nested identity field before allocating space to
134
+ // optional content. Arrays can still omit a suffix of complete entries.
135
+ function requiredSize(value: { [key: string]: JsonValue }): number {
136
+ const cached = requiredSizes.get(value);
137
+ if (cached !== undefined) return cached;
138
+ let size = 2;
139
+ let count = 0;
140
+ for (const [key, child] of Object.entries(value)) {
141
+ const required = requiredPropertySize(key, child);
142
+ if (required === undefined) continue;
143
+ size += (count > 0 ? 1 : 0) + charCount(JSON.stringify(key)) + 1 + required;
144
+ count += 1;
145
+ }
146
+ const result = count > 0 ? size : 0;
147
+ requiredSizes.set(value, result);
148
+ return result;
149
+ }
150
+
151
+ const minimumSize = (value: JsonValue, key = "", linkCollection = false): number => {
152
+ if (Array.isArray(value)) return 2; // []
153
+ if (isProtectedKey(key)) return sizeOf(value);
154
+ if (typeof value === "string") {
155
+ return linkCollection || isStandaloneUrl(value) ? sizeOf(value) : 2;
156
+ }
157
+ if (value !== null && typeof value === "object") {
158
+ const required = requiredSize(value);
159
+ if (required > 0) return required;
160
+ // Avoid arrays of empty objects when a useful text field is available.
161
+ for (const identity of IDENTITY_KEYS) {
162
+ if (!Object.hasOwn(value, identity)) continue;
163
+ const child = value[identity];
164
+ if (child !== null && typeof child === "object") continue;
165
+ const childMinimum = typeof child === "string" ? Math.min(sizeOf(child), 128) : sizeOf(child);
166
+ return 2 + charCount(JSON.stringify(identity)) + 1 + childMinimum;
167
+ }
168
+ return 2;
169
+ }
170
+ return sizeOf(value);
171
+ };
172
+
173
+ const fit = (value: JsonValue, budget: number, linkCollection = false, root = false): JsonValue => {
174
+ if (sizeOf(value) <= budget) return value;
175
+
176
+ if (typeof value === "string") return fitString(value, budget);
177
+
178
+ if (Array.isArray(value)) {
179
+ const slots: Slot[] = [];
180
+ let fixed = 2; // brackets and commas
181
+ let minimums = 0;
182
+ for (let index = 0; index < value.length; index += 1) {
183
+ const child = value[index];
184
+ const minimum = minimumSize(child, "", linkCollection);
185
+ const punctuation = slots.length > 0 ? 1 : 0;
186
+ if (fixed + punctuation + minimums + minimum > budget) {
187
+ // Preserve array order when some entries must be omitted.
188
+ break;
189
+ }
190
+ fixed += punctuation;
191
+ minimums += minimum;
192
+ slots.push({ index, value: child, minimum, full: sizeOf(child), allocation: minimum });
193
+ }
194
+ allocate(slots, budget - fixed);
195
+ return slots.map((slot) => fit(slot.value, slot.allocation, linkCollection));
196
+ }
197
+
198
+ if (value !== null && typeof value === "object") {
199
+ // An object cannot be shown with only a fragment of a required identity.
200
+ if (!root && requiredSize(value) > budget) return Object.create(null) as JsonValue;
201
+ const entries = Object.entries(value);
202
+ // Reserve required identifiers first, but restore the original field
203
+ // order in the result; no provider data or array entry is re-ranked.
204
+ const candidates = entries.map(([key, child], index) => ({
205
+ key, child, index, required: requiredPropertySize(key, child) !== undefined,
206
+ }));
207
+ candidates.sort((a, b) =>
208
+ Number(b.required) - Number(a.required)
209
+ || (KEY_PRIORITY.get(a.key) ?? IMPORTANT_KEYS.length) - (KEY_PRIORITY.get(b.key) ?? IMPORTANT_KEYS.length)
210
+ || a.index - b.index,
211
+ );
212
+ const slots: Slot[] = [];
213
+ let fixed = 2; // braces, keys, colons and commas
214
+ let minimums = 0;
215
+ for (const { key, child, index } of candidates) {
216
+ const minimum = minimumSize(child, key);
217
+ const punctuation = (slots.length > 0 ? 1 : 0) + charCount(JSON.stringify(key)) + 1;
218
+ if (fixed + punctuation + minimums + minimum > budget) continue;
219
+ fixed += punctuation;
220
+ minimums += minimum;
221
+ slots.push({ index, key, value: child, minimum, full: sizeOf(child), allocation: minimum });
222
+ }
223
+ slots.sort((a, b) => a.index - b.index);
224
+ allocate(slots, budget - fixed);
225
+ const result: { [key: string]: JsonValue } = Object.create(null);
226
+ for (const slot of slots) {
227
+ const key = slot.key!;
228
+ result[key] = fit(slot.value, slot.allocation, isLinkCollectionKey(key));
229
+ }
230
+ return result;
231
+ }
232
+
233
+ // Parents reserve the complete serialized size for non-string scalars.
234
+ throw new Error("Insufficient JSON budget for scalar value");
235
+ };
236
+
237
+ return fit;
238
+ }
239
+
240
+ /**
241
+ * Preserve ordinary output exactly. If compact JSON still exceeds the limit,
242
+ * fit its fields and array entries independently and append one marker line.
243
+ * The JSON portion stays valid and the entire text fits within `maxChars` code
244
+ * points. Tiny limits may only fit the marker or a minimal JSON value.
245
+ */
246
+ export function formatJsonWithinLimit(
247
+ output: object,
248
+ maxChars: number,
249
+ ): { text: string; truncated: boolean } {
250
+ if (!Number.isFinite(maxChars) || maxChars < 1) {
251
+ throw new RangeError("The JSON output limit must be at least one character");
252
+ }
253
+ const limit = Math.floor(maxChars);
254
+ const pretty = JSON.stringify(output, null, 2);
255
+ if (charCount(pretty) <= limit) return { text: pretty, truncated: false };
256
+
257
+ // Work from the already serialized value, so JSON's handling of undefined,
258
+ // dates and toJSON is identical even if an input object is not plain data.
259
+ const parsed = JSON.parse(pretty) as JsonValue;
260
+ const compact = JSON.stringify(parsed);
261
+ if (charCount(compact) <= limit) return { text: compact, truncated: false };
262
+
263
+ // Below 14 characters there is not enough room for both an empty JSON object
264
+ // and the marker on a new line. Prefer a complete marker whenever it fits.
265
+ if (limit < TRUNCATION_MARKER.length + 3) {
266
+ const text = limit >= TRUNCATION_MARKER.length ? TRUNCATION_MARKER : limit >= 2 ? "{}" : "0";
267
+ return { text, truncated: true };
268
+ }
269
+
270
+ const budget = limit - TRUNCATION_MARKER.length - 1; // newline + marker
271
+ const fit = createFitter();
272
+ // A toJSON method can return a scalar that cannot be shortened to fit.
273
+ const fitted = parsed === null || typeof parsed === "boolean" || typeof parsed === "number"
274
+ ? {}
275
+ : fit(parsed, budget, false, true);
276
+ return { text: `${JSON.stringify(fitted)}\n${TRUNCATION_MARKER}`, truncated: true };
277
+ }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-better-web-search",
3
- "version": "0.1.5",
3
+ "version": "0.1.7",
4
4
  "description": "A better provider-native web_search tool for pi: raw results, automatic provider fallback, URL analysis, and native DeepSeek Search/Fetch.",
5
5
  "type": "module",
6
6
  "license": "MIT",