@astralyn/pi 0.86.1 → 0.87.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (125) hide show
  1. package/CHANGELOG.md +45 -0
  2. package/dist/bundle/chunks/{anthropic-messages-R2LYJUDD.js → anthropic-messages-XWFTZP2N.js} +1 -1
  3. package/dist/bundle/chunks/{chunk-GTBMYBFG.js → chunk-772GXEFD.js} +50 -44
  4. package/dist/bundle/chunks/chunk-CDMUKIXY.js +2 -0
  5. package/dist/bundle/chunks/github-copilot.js +1 -1
  6. package/dist/bundle/chunks/{openai-completions-6SMWQWQN.js → openai-completions-SHS3CAEL.js} +2 -2
  7. package/dist/bundle/chunks/{virtual-modules-OHUZ2T7X.js → virtual-modules-LQO2MG3T.js} +1 -1
  8. package/dist/bundle/cli.js +1 -1
  9. package/dist/bundle/index.js +1 -1
  10. package/dist/bundle/rpc-entry.js +1 -1
  11. package/dist/cli/args.d.ts.map +1 -1
  12. package/dist/cli/args.js +14 -4
  13. package/dist/cli/args.js.map +1 -1
  14. package/dist/cli/file-processor.d.ts +1 -1
  15. package/dist/cli/file-processor.d.ts.map +1 -1
  16. package/dist/cli/file-processor.js.map +1 -1
  17. package/dist/core/agent-session-runtime.js +1 -1
  18. package/dist/core/agent-session-runtime.js.map +1 -1
  19. package/dist/core/agent-session.d.ts +47 -4
  20. package/dist/core/agent-session.d.ts.map +1 -1
  21. package/dist/core/agent-session.js +476 -139
  22. package/dist/core/agent-session.js.map +1 -1
  23. package/dist/core/background/service.d.ts +9 -0
  24. package/dist/core/background/service.d.ts.map +1 -1
  25. package/dist/core/background/service.js +13 -1
  26. package/dist/core/background/service.js.map +1 -1
  27. package/dist/core/background/session.d.ts +5 -0
  28. package/dist/core/background/session.d.ts.map +1 -1
  29. package/dist/core/background/session.js +2 -1
  30. package/dist/core/background/session.js.map +1 -1
  31. package/dist/core/cache-warmer.d.ts +1 -0
  32. package/dist/core/cache-warmer.d.ts.map +1 -1
  33. package/dist/core/cache-warmer.js +15 -1
  34. package/dist/core/cache-warmer.js.map +1 -1
  35. package/dist/core/compaction/compaction.d.ts +3 -1
  36. package/dist/core/compaction/compaction.d.ts.map +1 -1
  37. package/dist/core/compaction/compaction.js +155 -57
  38. package/dist/core/compaction/compaction.js.map +1 -1
  39. package/dist/core/crash-log.d.ts +5 -0
  40. package/dist/core/crash-log.d.ts.map +1 -1
  41. package/dist/core/crash-log.js +68 -0
  42. package/dist/core/crash-log.js.map +1 -1
  43. package/dist/core/export-html/template.js +6 -1
  44. package/dist/core/extensions/index.d.ts +1 -1
  45. package/dist/core/extensions/index.d.ts.map +1 -1
  46. package/dist/core/extensions/index.js.map +1 -1
  47. package/dist/core/extensions/runner.d.ts +16 -3
  48. package/dist/core/extensions/runner.d.ts.map +1 -1
  49. package/dist/core/extensions/runner.js +110 -5
  50. package/dist/core/extensions/runner.js.map +1 -1
  51. package/dist/core/extensions/types.d.ts +77 -6
  52. package/dist/core/extensions/types.d.ts.map +1 -1
  53. package/dist/core/extensions/types.js.map +1 -1
  54. package/dist/core/index.d.ts +1 -1
  55. package/dist/core/index.d.ts.map +1 -1
  56. package/dist/core/index.js.map +1 -1
  57. package/dist/core/model-config.d.ts +52 -0
  58. package/dist/core/model-config.d.ts.map +1 -1
  59. package/dist/core/model-config.js +16 -0
  60. package/dist/core/model-config.js.map +1 -1
  61. package/dist/core/model-resolver.js +1 -1
  62. package/dist/core/model-resolver.js.map +1 -1
  63. package/dist/core/prompt-templates.d.ts +6 -1
  64. package/dist/core/prompt-templates.d.ts.map +1 -1
  65. package/dist/core/prompt-templates.js +61 -35
  66. package/dist/core/prompt-templates.js.map +1 -1
  67. package/dist/core/provider-composer.d.ts +1 -0
  68. package/dist/core/provider-composer.d.ts.map +1 -1
  69. package/dist/core/provider-composer.js +19 -0
  70. package/dist/core/provider-composer.js.map +1 -1
  71. package/dist/core/resource-loader.d.ts.map +1 -1
  72. package/dist/core/resource-loader.js +6 -2
  73. package/dist/core/resource-loader.js.map +1 -1
  74. package/dist/core/sdk.d.ts.map +1 -1
  75. package/dist/core/sdk.js +3 -4
  76. package/dist/core/sdk.js.map +1 -1
  77. package/dist/core/session-manager.d.ts +36 -9
  78. package/dist/core/session-manager.d.ts.map +1 -1
  79. package/dist/core/session-manager.js +97 -7
  80. package/dist/core/session-manager.js.map +1 -1
  81. package/dist/core/tools/read.d.ts +4 -1
  82. package/dist/core/tools/read.d.ts.map +1 -1
  83. package/dist/core/tools/read.js +5 -1
  84. package/dist/core/tools/read.js.map +1 -1
  85. package/dist/extensions/btw/agent.d.ts.map +1 -1
  86. package/dist/extensions/btw/agent.js +8 -5
  87. package/dist/extensions/btw/agent.js.map +1 -1
  88. package/dist/index.d.ts +2 -2
  89. package/dist/index.d.ts.map +1 -1
  90. package/dist/index.js +1 -1
  91. package/dist/index.js.map +1 -1
  92. package/dist/main.d.ts.map +1 -1
  93. package/dist/main.js +4 -3
  94. package/dist/main.js.map +1 -1
  95. package/dist/modes/interactive/bug-report.d.ts.map +1 -1
  96. package/dist/modes/interactive/bug-report.js +4 -0
  97. package/dist/modes/interactive/bug-report.js.map +1 -1
  98. package/dist/modes/interactive/components/tree-selector.d.ts.map +1 -1
  99. package/dist/modes/interactive/components/tree-selector.js +7 -0
  100. package/dist/modes/interactive/components/tree-selector.js.map +1 -1
  101. package/dist/modes/interactive/interactive-mode.d.ts +3 -0
  102. package/dist/modes/interactive/interactive-mode.d.ts.map +1 -1
  103. package/dist/modes/interactive/interactive-mode.js +64 -2
  104. package/dist/modes/interactive/interactive-mode.js.map +1 -1
  105. package/dist/utils/mime.js +1 -1
  106. package/dist/utils/mime.js.map +1 -1
  107. package/dist/utils/tool-result-images.d.ts +3 -1
  108. package/dist/utils/tool-result-images.d.ts.map +1 -1
  109. package/dist/utils/tool-result-images.js +4 -1
  110. package/dist/utils/tool-result-images.js.map +1 -1
  111. package/docs/bundled/extensions/background.md +3 -3
  112. package/docs/bundled/extensions/subagent.md +1 -1
  113. package/docs/compaction.md +24 -5
  114. package/docs/extensions.md +72 -8
  115. package/docs/models.md +30 -1
  116. package/docs/sdk.md +15 -4
  117. package/docs/session-format.md +25 -4
  118. package/docs/usage.md +3 -0
  119. package/examples/README.md +11 -1
  120. package/examples/plugins/pi-example-plugin/src/session.ts +3 -2
  121. package/examples/rpc-client.ts +35 -0
  122. package/examples/rpc-extension-ui.ts +25 -5
  123. package/npm-shrinkwrap.json +43 -43
  124. package/package.json +8 -8
  125. package/dist/bundle/chunks/chunk-DAEM5JIZ.js +0 -2
@@ -8,7 +8,7 @@ export function detectSupportedImageMimeType(buffer) {
8
8
  if (startsWith(buffer, PNG_SIGNATURE)) {
9
9
  return isPng(buffer) && !isAnimatedPng(buffer) ? "image/png" : null;
10
10
  }
11
- if (startsWithAscii(buffer, 0, "GIF")) {
11
+ if (startsWithAscii(buffer, 0, "GIF87a") || startsWithAscii(buffer, 0, "GIF89a")) {
12
12
  return "image/gif";
13
13
  }
14
14
  if (startsWithAscii(buffer, 0, "RIFF") && startsWithAscii(buffer, 8, "WEBP")) {
@@ -1 +1 @@
1
- {"version":3,"file":"mime.js","sourceRoot":"","sources":["../../src/utils/mime.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,IAAI,EAAE,MAAM,kBAAkB,CAAC;AAExC,MAAM,sBAAsB,GAAG,IAAI,CAAC;AACpC,MAAM,aAAa,GAAG,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,CAAC,CAAC;AAEvE,MAAM,UAAU,4BAA4B,CAAC,MAAkB;IAC9D,IAAI,UAAU,CAAC,MAAM,EAAE,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,CAAC,CAAC,EAAE,CAAC;QAC5C,OAAO,MAAM,CAAC,CAAC,CAAC,KAAK,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,YAAY,CAAC;IACjD,CAAC;IACD,IAAI,UAAU,CAAC,MAAM,EAAE,aAAa,CAAC,EAAE,CAAC;QACvC,OAAO,KAAK,CAAC,MAAM,CAAC,IAAI,CAAC,aAAa,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,WAAW,CAAC,CAAC,CAAC,IAAI,CAAC;IACrE,CAAC;IACD,IAAI,eAAe,CAAC,MAAM,EAAE,CAAC,EAAE,KAAK,CAAC,EAAE,CAAC;QACvC,OAAO,WAAW,CAAC;IACpB,CAAC;IACD,IAAI,eAAe,CAAC,MAAM,EAAE,CAAC,EAAE,MAAM,CAAC,IAAI,eAAe,CAAC,MAAM,EAAE,CAAC,EAAE,MAAM,CAAC,EAAE,CAAC;QAC9E,OAAO,YAAY,CAAC;IACrB,CAAC;IACD,IAAI,eAAe,CAAC,MAAM,EAAE,CAAC,EAAE,IAAI,CAAC,IAAI,KAAK,CAAC,MAAM,CAAC,EAAE,CAAC;QACvD,OAAO,WAAW,CAAC;IACpB,CAAC;IACD,OAAO,IAAI,CAAC;AACb,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,oCAAoC,CAAC,QAAgB;IAC1E,MAAM,UAAU,GAAG,MAAM,IAAI,CAAC,QAAQ,EAAE,GAAG,CAAC,CAAC;IAC7C,IAAI,CAAC;QACJ,MAAM,MAAM,GAAG,MAAM,CAAC,KAAK,CAAC,sBAAsB,CAAC,CAAC;QACpD,MAAM,EAAE,SAAS,EAAE,GAAG,MAAM,UAAU,CAAC,IAAI,CAAC,MAAM,EAAE,CAAC,EAAE,sBAAsB,EAAE,CAAC,CAAC,CAAC;QAClF,OAAO,4BAA4B,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,EAAE,SAAS,CAAC,CAAC,CAAC;IACpE,CAAC;YAAS,CAAC;QACV,MAAM,UAAU,CAAC,KAAK,EAAE,CAAC;IAC1B,CAAC;AACF,CAAC;AAED,SAAS,KAAK,CAAC,MAAkB;IAChC,OAAO,CACN,MAAM,CAAC,MAAM,IAAI,EAAE,IAAI,YAAY,CAAC,MAAM,EAAE,aAAa,CAAC,MAAM,CAAC,KAAK,EAAE,IAAI,eAAe,CAAC,MAAM,EAAE,EAAE,EAAE,MAAM,CAAC,CAC/G,CAAC;AACH,CAAC;AAED,SAAS,aAAa,CAAC,MAAkB;IACxC,IAAI,MAAM,GAAG,aAAa,CAAC,MAAM,CAAC;IAClC,OAAO,MAAM,GAAG,CAAC,IAAI,MAAM,CAAC,MAAM,EAAE,CAAC;QACpC,MAAM,WAAW,GAAG,YAAY,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;QACjD,MAAM,eAAe,GAAG,MAAM,GAAG,CAAC,CAAC;QACnC,IAAI,eAAe,CAAC,MAAM,EAAE,eAAe,EAAE,MAAM,CAAC;YAAE,OAAO,IAAI,CAAC;QAClE,IAAI,eAAe,CAAC,MAAM,EAAE,eAAe,EAAE,MAAM,CAAC;YAAE,OAAO,KAAK,CAAC;QAEnE,MAAM,UAAU,GAAG,MAAM,GAAG,CAAC,GAAG,WAAW,GAAG,CAAC,CAAC;QAChD,IAAI,UAAU,IAAI,MAAM,IAAI,UAAU,GAAG,MAAM,CAAC,MAAM;YAAE,OAAO,KAAK,CAAC;QACrE,MAAM,GAAG,UAAU,CAAC;IACrB,CAAC;IACD,OAAO,KAAK,CAAC;AACd,CAAC;AAED,SAAS,KAAK,CAAC,MAAkB;IAChC,IAAI,MAAM,CAAC,MAAM,GAAG,EAAE;QAAE,OAAO,KAAK,CAAC;IAErC,MAAM,gBAAgB,GAAG,YAAY,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC;IACjD,MAAM,eAAe,GAAG,YAAY,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;IACjD,MAAM,aAAa,GAAG,YAAY,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;IAC/C,IAAI,gBAAgB,KAAK,CAAC,IAAI,gBAAgB,GAAG,EAAE;QAAE,OAAO,KAAK,CAAC;IAClE,IAAI,eAAe,GAAG,EAAE,GAAG,aAAa;QAAE,OAAO,KAAK,CAAC;IACvD,IAAI,gBAAgB,KAAK,CAAC,IAAI,eAAe,IAAI,gBAAgB;QAAE,OAAO,KAAK,CAAC;IAEhF,IAAI,WAAmB,CAAC;IACxB,IAAI,YAAoB,CAAC;IACzB,IAAI,aAAa,KAAK,EAAE,EAAE,CAAC;QAC1B,WAAW,GAAG,YAAY,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;QACvC,YAAY,GAAG,YAAY,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;IACzC,CAAC;SAAM,IAAI,aAAa,IAAI,EAAE,IAAI,aAAa,IAAI,GAAG,EAAE,CAAC;QACxD,IAAI,MAAM,CAAC,MAAM,GAAG,EAAE;YAAE,OAAO,KAAK,CAAC;QACrC,WAAW,GAAG,YAAY,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;QACvC,YAAY,GAAG,YAAY,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;IACzC,CAAC;SAAM,CAAC;QACP,OAAO,KAAK,CAAC;IACd,CAAC;IAED,OAAO,WAAW,KAAK,CAAC,IAAI,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,EAAE,EAAE,EAAE,EAAE,EAAE,EAAE,CAAC,CAAC,QAAQ,CAAC,YAAY,CAAC,CAAC;AAC1E,CAAC;AAED,SAAS,YAAY,CAAC,MAAkB,EAAE,MAAc;IACvD,OAAO,CAAC,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC;AACjE,CAAC;AAED,SAAS,YAAY,CAAC,MAAkB,EAAE,MAAc;IACvD,OAAO,CACN,CAAC,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,GAAG,SAAS;QACjC,CAAC,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,IAAI,EAAE,CAAC;QACjC,CAAC,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,CAAC;QAChC,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,CACzB,CAAC;AACH,CAAC;AAED,SAAS,YAAY,CAAC,MAAkB,EAAE,MAAc;IACvD,OAAO,CACN,CAAC,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;QACrB,CAAC,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,CAAC;QAChC,CAAC,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,IAAI,EAAE,CAAC;QACjC,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,GAAG,SAAS,CACrC,CAAC;AACH,CAAC;AAED,SAAS,UAAU,CAAC,MAAkB,EAAE,KAAe;IACtD,IAAI,MAAM,CAAC,MAAM,GAAG,KAAK,CAAC,MAAM;QAAE,OAAO,KAAK,CAAC;IAC/C,OAAO,KAAK,CAAC,KAAK,CAAC,CAAC,IAAI,EAAE,KAAK,EAAE,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,KAAK,IAAI,CAAC,CAAC;AAC7D,CAAC;AAED,SAAS,eAAe,CAAC,MAAkB,EAAE,MAAc,EAAE,IAAY;IACxE,IAAI,MAAM,CAAC,MAAM,GAAG,MAAM,GAAG,IAAI,CAAC,MAAM;QAAE,OAAO,KAAK,CAAC;IACvD,KAAK,IAAI,KAAK,GAAG,CAAC,EAAE,KAAK,GAAG,IAAI,CAAC,MAAM,EAAE,KAAK,EAAE,EAAE,CAAC;QAClD,IAAI,MAAM,CAAC,MAAM,GAAG,KAAK,CAAC,KAAK,IAAI,CAAC,UAAU,CAAC,KAAK,CAAC;YAAE,OAAO,KAAK,CAAC;IACrE,CAAC;IACD,OAAO,IAAI,CAAC;AACb,CAAC","sourcesContent":["import { open } from \"node:fs/promises\";\n\nconst IMAGE_TYPE_SNIFF_BYTES = 4100;\nconst PNG_SIGNATURE = [0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a];\n\nexport function detectSupportedImageMimeType(buffer: Uint8Array): string | null {\n\tif (startsWith(buffer, [0xff, 0xd8, 0xff])) {\n\t\treturn buffer[3] === 0xf7 ? null : \"image/jpeg\";\n\t}\n\tif (startsWith(buffer, PNG_SIGNATURE)) {\n\t\treturn isPng(buffer) && !isAnimatedPng(buffer) ? \"image/png\" : null;\n\t}\n\tif (startsWithAscii(buffer, 0, \"GIF\")) {\n\t\treturn \"image/gif\";\n\t}\n\tif (startsWithAscii(buffer, 0, \"RIFF\") && startsWithAscii(buffer, 8, \"WEBP\")) {\n\t\treturn \"image/webp\";\n\t}\n\tif (startsWithAscii(buffer, 0, \"BM\") && isBmp(buffer)) {\n\t\treturn \"image/bmp\";\n\t}\n\treturn null;\n}\n\nexport async function detectSupportedImageMimeTypeFromFile(filePath: string): Promise<string | null> {\n\tconst fileHandle = await open(filePath, \"r\");\n\ttry {\n\t\tconst buffer = Buffer.alloc(IMAGE_TYPE_SNIFF_BYTES);\n\t\tconst { bytesRead } = await fileHandle.read(buffer, 0, IMAGE_TYPE_SNIFF_BYTES, 0);\n\t\treturn detectSupportedImageMimeType(buffer.subarray(0, bytesRead));\n\t} finally {\n\t\tawait fileHandle.close();\n\t}\n}\n\nfunction isPng(buffer: Uint8Array): boolean {\n\treturn (\n\t\tbuffer.length >= 16 && readUint32BE(buffer, PNG_SIGNATURE.length) === 13 && startsWithAscii(buffer, 12, \"IHDR\")\n\t);\n}\n\nfunction isAnimatedPng(buffer: Uint8Array): boolean {\n\tlet offset = PNG_SIGNATURE.length;\n\twhile (offset + 8 <= buffer.length) {\n\t\tconst chunkLength = readUint32BE(buffer, offset);\n\t\tconst chunkTypeOffset = offset + 4;\n\t\tif (startsWithAscii(buffer, chunkTypeOffset, \"acTL\")) return true;\n\t\tif (startsWithAscii(buffer, chunkTypeOffset, \"IDAT\")) return false;\n\n\t\tconst nextOffset = offset + 8 + chunkLength + 4;\n\t\tif (nextOffset <= offset || nextOffset > buffer.length) return false;\n\t\toffset = nextOffset;\n\t}\n\treturn false;\n}\n\nfunction isBmp(buffer: Uint8Array): boolean {\n\tif (buffer.length < 26) return false;\n\n\tconst declaredFileSize = readUint32LE(buffer, 2);\n\tconst pixelDataOffset = readUint32LE(buffer, 10);\n\tconst dibHeaderSize = readUint32LE(buffer, 14);\n\tif (declaredFileSize !== 0 && declaredFileSize < 26) return false;\n\tif (pixelDataOffset < 14 + dibHeaderSize) return false;\n\tif (declaredFileSize !== 0 && pixelDataOffset >= declaredFileSize) return false;\n\n\tlet colorPlanes: number;\n\tlet bitsPerPixel: number;\n\tif (dibHeaderSize === 12) {\n\t\tcolorPlanes = readUint16LE(buffer, 22);\n\t\tbitsPerPixel = readUint16LE(buffer, 24);\n\t} else if (dibHeaderSize >= 40 && dibHeaderSize <= 124) {\n\t\tif (buffer.length < 30) return false;\n\t\tcolorPlanes = readUint16LE(buffer, 26);\n\t\tbitsPerPixel = readUint16LE(buffer, 28);\n\t} else {\n\t\treturn false;\n\t}\n\n\treturn colorPlanes === 1 && [1, 4, 8, 16, 24, 32].includes(bitsPerPixel);\n}\n\nfunction readUint16LE(buffer: Uint8Array, offset: number): number {\n\treturn (buffer[offset] ?? 0) + ((buffer[offset + 1] ?? 0) << 8);\n}\n\nfunction readUint32BE(buffer: Uint8Array, offset: number): number {\n\treturn (\n\t\t(buffer[offset] ?? 0) * 0x1000000 +\n\t\t((buffer[offset + 1] ?? 0) << 16) +\n\t\t((buffer[offset + 2] ?? 0) << 8) +\n\t\t(buffer[offset + 3] ?? 0)\n\t);\n}\n\nfunction readUint32LE(buffer: Uint8Array, offset: number): number {\n\treturn (\n\t\t(buffer[offset] ?? 0) +\n\t\t((buffer[offset + 1] ?? 0) << 8) +\n\t\t((buffer[offset + 2] ?? 0) << 16) +\n\t\t(buffer[offset + 3] ?? 0) * 0x1000000\n\t);\n}\n\nfunction startsWith(buffer: Uint8Array, bytes: number[]): boolean {\n\tif (buffer.length < bytes.length) return false;\n\treturn bytes.every((byte, index) => buffer[index] === byte);\n}\n\nfunction startsWithAscii(buffer: Uint8Array, offset: number, text: string): boolean {\n\tif (buffer.length < offset + text.length) return false;\n\tfor (let index = 0; index < text.length; index++) {\n\t\tif (buffer[offset + index] !== text.charCodeAt(index)) return false;\n\t}\n\treturn true;\n}\n"]}
1
+ {"version":3,"file":"mime.js","sourceRoot":"","sources":["../../src/utils/mime.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,IAAI,EAAE,MAAM,kBAAkB,CAAC;AAExC,MAAM,sBAAsB,GAAG,IAAI,CAAC;AACpC,MAAM,aAAa,GAAG,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,CAAC,CAAC;AAEvE,MAAM,UAAU,4BAA4B,CAAC,MAAkB;IAC9D,IAAI,UAAU,CAAC,MAAM,EAAE,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,CAAC,CAAC,EAAE,CAAC;QAC5C,OAAO,MAAM,CAAC,CAAC,CAAC,KAAK,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,YAAY,CAAC;IACjD,CAAC;IACD,IAAI,UAAU,CAAC,MAAM,EAAE,aAAa,CAAC,EAAE,CAAC;QACvC,OAAO,KAAK,CAAC,MAAM,CAAC,IAAI,CAAC,aAAa,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,WAAW,CAAC,CAAC,CAAC,IAAI,CAAC;IACrE,CAAC;IACD,IAAI,eAAe,CAAC,MAAM,EAAE,CAAC,EAAE,QAAQ,CAAC,IAAI,eAAe,CAAC,MAAM,EAAE,CAAC,EAAE,QAAQ,CAAC,EAAE,CAAC;QAClF,OAAO,WAAW,CAAC;IACpB,CAAC;IACD,IAAI,eAAe,CAAC,MAAM,EAAE,CAAC,EAAE,MAAM,CAAC,IAAI,eAAe,CAAC,MAAM,EAAE,CAAC,EAAE,MAAM,CAAC,EAAE,CAAC;QAC9E,OAAO,YAAY,CAAC;IACrB,CAAC;IACD,IAAI,eAAe,CAAC,MAAM,EAAE,CAAC,EAAE,IAAI,CAAC,IAAI,KAAK,CAAC,MAAM,CAAC,EAAE,CAAC;QACvD,OAAO,WAAW,CAAC;IACpB,CAAC;IACD,OAAO,IAAI,CAAC;AACb,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,oCAAoC,CAAC,QAAgB;IAC1E,MAAM,UAAU,GAAG,MAAM,IAAI,CAAC,QAAQ,EAAE,GAAG,CAAC,CAAC;IAC7C,IAAI,CAAC;QACJ,MAAM,MAAM,GAAG,MAAM,CAAC,KAAK,CAAC,sBAAsB,CAAC,CAAC;QACpD,MAAM,EAAE,SAAS,EAAE,GAAG,MAAM,UAAU,CAAC,IAAI,CAAC,MAAM,EAAE,CAAC,EAAE,sBAAsB,EAAE,CAAC,CAAC,CAAC;QAClF,OAAO,4BAA4B,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,EAAE,SAAS,CAAC,CAAC,CAAC;IACpE,CAAC;YAAS,CAAC;QACV,MAAM,UAAU,CAAC,KAAK,EAAE,CAAC;IAC1B,CAAC;AACF,CAAC;AAED,SAAS,KAAK,CAAC,MAAkB;IAChC,OAAO,CACN,MAAM,CAAC,MAAM,IAAI,EAAE,IAAI,YAAY,CAAC,MAAM,EAAE,aAAa,CAAC,MAAM,CAAC,KAAK,EAAE,IAAI,eAAe,CAAC,MAAM,EAAE,EAAE,EAAE,MAAM,CAAC,CAC/G,CAAC;AACH,CAAC;AAED,SAAS,aAAa,CAAC,MAAkB;IACxC,IAAI,MAAM,GAAG,aAAa,CAAC,MAAM,CAAC;IAClC,OAAO,MAAM,GAAG,CAAC,IAAI,MAAM,CAAC,MAAM,EAAE,CAAC;QACpC,MAAM,WAAW,GAAG,YAAY,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;QACjD,MAAM,eAAe,GAAG,MAAM,GAAG,CAAC,CAAC;QACnC,IAAI,eAAe,CAAC,MAAM,EAAE,eAAe,EAAE,MAAM,CAAC;YAAE,OAAO,IAAI,CAAC;QAClE,IAAI,eAAe,CAAC,MAAM,EAAE,eAAe,EAAE,MAAM,CAAC;YAAE,OAAO,KAAK,CAAC;QAEnE,MAAM,UAAU,GAAG,MAAM,GAAG,CAAC,GAAG,WAAW,GAAG,CAAC,CAAC;QAChD,IAAI,UAAU,IAAI,MAAM,IAAI,UAAU,GAAG,MAAM,CAAC,MAAM;YAAE,OAAO,KAAK,CAAC;QACrE,MAAM,GAAG,UAAU,CAAC;IACrB,CAAC;IACD,OAAO,KAAK,CAAC;AACd,CAAC;AAED,SAAS,KAAK,CAAC,MAAkB;IAChC,IAAI,MAAM,CAAC,MAAM,GAAG,EAAE;QAAE,OAAO,KAAK,CAAC;IAErC,MAAM,gBAAgB,GAAG,YAAY,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC;IACjD,MAAM,eAAe,GAAG,YAAY,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;IACjD,MAAM,aAAa,GAAG,YAAY,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;IAC/C,IAAI,gBAAgB,KAAK,CAAC,IAAI,gBAAgB,GAAG,EAAE;QAAE,OAAO,KAAK,CAAC;IAClE,IAAI,eAAe,GAAG,EAAE,GAAG,aAAa;QAAE,OAAO,KAAK,CAAC;IACvD,IAAI,gBAAgB,KAAK,CAAC,IAAI,eAAe,IAAI,gBAAgB;QAAE,OAAO,KAAK,CAAC;IAEhF,IAAI,WAAmB,CAAC;IACxB,IAAI,YAAoB,CAAC;IACzB,IAAI,aAAa,KAAK,EAAE,EAAE,CAAC;QAC1B,WAAW,GAAG,YAAY,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;QACvC,YAAY,GAAG,YAAY,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;IACzC,CAAC;SAAM,IAAI,aAAa,IAAI,EAAE,IAAI,aAAa,IAAI,GAAG,EAAE,CAAC;QACxD,IAAI,MAAM,CAAC,MAAM,GAAG,EAAE;YAAE,OAAO,KAAK,CAAC;QACrC,WAAW,GAAG,YAAY,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;QACvC,YAAY,GAAG,YAAY,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;IACzC,CAAC;SAAM,CAAC;QACP,OAAO,KAAK,CAAC;IACd,CAAC;IAED,OAAO,WAAW,KAAK,CAAC,IAAI,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,EAAE,EAAE,EAAE,EAAE,EAAE,EAAE,CAAC,CAAC,QAAQ,CAAC,YAAY,CAAC,CAAC;AAC1E,CAAC;AAED,SAAS,YAAY,CAAC,MAAkB,EAAE,MAAc;IACvD,OAAO,CAAC,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC;AACjE,CAAC;AAED,SAAS,YAAY,CAAC,MAAkB,EAAE,MAAc;IACvD,OAAO,CACN,CAAC,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,GAAG,SAAS;QACjC,CAAC,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,IAAI,EAAE,CAAC;QACjC,CAAC,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,CAAC;QAChC,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,CACzB,CAAC;AACH,CAAC;AAED,SAAS,YAAY,CAAC,MAAkB,EAAE,MAAc;IACvD,OAAO,CACN,CAAC,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;QACrB,CAAC,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,CAAC;QAChC,CAAC,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,IAAI,EAAE,CAAC;QACjC,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,GAAG,SAAS,CACrC,CAAC;AACH,CAAC;AAED,SAAS,UAAU,CAAC,MAAkB,EAAE,KAAe;IACtD,IAAI,MAAM,CAAC,MAAM,GAAG,KAAK,CAAC,MAAM;QAAE,OAAO,KAAK,CAAC;IAC/C,OAAO,KAAK,CAAC,KAAK,CAAC,CAAC,IAAI,EAAE,KAAK,EAAE,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,KAAK,IAAI,CAAC,CAAC;AAC7D,CAAC;AAED,SAAS,eAAe,CAAC,MAAkB,EAAE,MAAc,EAAE,IAAY;IACxE,IAAI,MAAM,CAAC,MAAM,GAAG,MAAM,GAAG,IAAI,CAAC,MAAM;QAAE,OAAO,KAAK,CAAC;IACvD,KAAK,IAAI,KAAK,GAAG,CAAC,EAAE,KAAK,GAAG,IAAI,CAAC,MAAM,EAAE,KAAK,EAAE,EAAE,CAAC;QAClD,IAAI,MAAM,CAAC,MAAM,GAAG,KAAK,CAAC,KAAK,IAAI,CAAC,UAAU,CAAC,KAAK,CAAC;YAAE,OAAO,KAAK,CAAC;IACrE,CAAC;IACD,OAAO,IAAI,CAAC;AACb,CAAC","sourcesContent":["import { open } from \"node:fs/promises\";\n\nconst IMAGE_TYPE_SNIFF_BYTES = 4100;\nconst PNG_SIGNATURE = [0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a];\n\nexport function detectSupportedImageMimeType(buffer: Uint8Array): string | null {\n\tif (startsWith(buffer, [0xff, 0xd8, 0xff])) {\n\t\treturn buffer[3] === 0xf7 ? null : \"image/jpeg\";\n\t}\n\tif (startsWith(buffer, PNG_SIGNATURE)) {\n\t\treturn isPng(buffer) && !isAnimatedPng(buffer) ? \"image/png\" : null;\n\t}\n\tif (startsWithAscii(buffer, 0, \"GIF87a\") || startsWithAscii(buffer, 0, \"GIF89a\")) {\n\t\treturn \"image/gif\";\n\t}\n\tif (startsWithAscii(buffer, 0, \"RIFF\") && startsWithAscii(buffer, 8, \"WEBP\")) {\n\t\treturn \"image/webp\";\n\t}\n\tif (startsWithAscii(buffer, 0, \"BM\") && isBmp(buffer)) {\n\t\treturn \"image/bmp\";\n\t}\n\treturn null;\n}\n\nexport async function detectSupportedImageMimeTypeFromFile(filePath: string): Promise<string | null> {\n\tconst fileHandle = await open(filePath, \"r\");\n\ttry {\n\t\tconst buffer = Buffer.alloc(IMAGE_TYPE_SNIFF_BYTES);\n\t\tconst { bytesRead } = await fileHandle.read(buffer, 0, IMAGE_TYPE_SNIFF_BYTES, 0);\n\t\treturn detectSupportedImageMimeType(buffer.subarray(0, bytesRead));\n\t} finally {\n\t\tawait fileHandle.close();\n\t}\n}\n\nfunction isPng(buffer: Uint8Array): boolean {\n\treturn (\n\t\tbuffer.length >= 16 && readUint32BE(buffer, PNG_SIGNATURE.length) === 13 && startsWithAscii(buffer, 12, \"IHDR\")\n\t);\n}\n\nfunction isAnimatedPng(buffer: Uint8Array): boolean {\n\tlet offset = PNG_SIGNATURE.length;\n\twhile (offset + 8 <= buffer.length) {\n\t\tconst chunkLength = readUint32BE(buffer, offset);\n\t\tconst chunkTypeOffset = offset + 4;\n\t\tif (startsWithAscii(buffer, chunkTypeOffset, \"acTL\")) return true;\n\t\tif (startsWithAscii(buffer, chunkTypeOffset, \"IDAT\")) return false;\n\n\t\tconst nextOffset = offset + 8 + chunkLength + 4;\n\t\tif (nextOffset <= offset || nextOffset > buffer.length) return false;\n\t\toffset = nextOffset;\n\t}\n\treturn false;\n}\n\nfunction isBmp(buffer: Uint8Array): boolean {\n\tif (buffer.length < 26) return false;\n\n\tconst declaredFileSize = readUint32LE(buffer, 2);\n\tconst pixelDataOffset = readUint32LE(buffer, 10);\n\tconst dibHeaderSize = readUint32LE(buffer, 14);\n\tif (declaredFileSize !== 0 && declaredFileSize < 26) return false;\n\tif (pixelDataOffset < 14 + dibHeaderSize) return false;\n\tif (declaredFileSize !== 0 && pixelDataOffset >= declaredFileSize) return false;\n\n\tlet colorPlanes: number;\n\tlet bitsPerPixel: number;\n\tif (dibHeaderSize === 12) {\n\t\tcolorPlanes = readUint16LE(buffer, 22);\n\t\tbitsPerPixel = readUint16LE(buffer, 24);\n\t} else if (dibHeaderSize >= 40 && dibHeaderSize <= 124) {\n\t\tif (buffer.length < 30) return false;\n\t\tcolorPlanes = readUint16LE(buffer, 26);\n\t\tbitsPerPixel = readUint16LE(buffer, 28);\n\t} else {\n\t\treturn false;\n\t}\n\n\treturn colorPlanes === 1 && [1, 4, 8, 16, 24, 32].includes(bitsPerPixel);\n}\n\nfunction readUint16LE(buffer: Uint8Array, offset: number): number {\n\treturn (buffer[offset] ?? 0) + ((buffer[offset + 1] ?? 0) << 8);\n}\n\nfunction readUint32BE(buffer: Uint8Array, offset: number): number {\n\treturn (\n\t\t(buffer[offset] ?? 0) * 0x1000000 +\n\t\t((buffer[offset + 1] ?? 0) << 16) +\n\t\t((buffer[offset + 2] ?? 0) << 8) +\n\t\t(buffer[offset + 3] ?? 0)\n\t);\n}\n\nfunction readUint32LE(buffer: Uint8Array, offset: number): number {\n\treturn (\n\t\t(buffer[offset] ?? 0) +\n\t\t((buffer[offset + 1] ?? 0) << 8) +\n\t\t((buffer[offset + 2] ?? 0) << 16) +\n\t\t(buffer[offset + 3] ?? 0) * 0x1000000\n\t);\n}\n\nfunction startsWith(buffer: Uint8Array, bytes: number[]): boolean {\n\tif (buffer.length < bytes.length) return false;\n\treturn bytes.every((byte, index) => buffer[index] === byte);\n}\n\nfunction startsWithAscii(buffer: Uint8Array, offset: number, text: string): boolean {\n\tif (buffer.length < offset + text.length) return false;\n\tfor (let index = 0; index < text.length; index++) {\n\t\tif (buffer[offset + index] !== text.charCodeAt(index)) return false;\n\t}\n\treturn true;\n}\n"]}
@@ -1,8 +1,10 @@
1
- import type { ImageContent, TextContent } from "@earendil-works/pi-ai";
1
+ import type { ImageContent, ModelImageResizeOptions, TextContent } from "@earendil-works/pi-ai";
2
2
  export type ToolResultContent = TextContent | ImageContent;
3
3
  export interface NormalizeToolResultImagesOptions {
4
4
  /** Whether oversized images are resized to inline provider limits. Default: true */
5
5
  autoResizeImages?: boolean;
6
+ /** Model-specific resize profile. Uses the conservative built-in defaults when omitted. */
7
+ resizeOptions?: ModelImageResizeOptions;
6
8
  }
7
9
  /**
8
10
  * Normalize image blocks returned by tool results.
@@ -1 +1 @@
1
- {"version":3,"file":"tool-result-images.d.ts","sourceRoot":"","sources":["../../src/utils/tool-result-images.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,YAAY,EAAE,WAAW,EAAE,MAAM,uBAAuB,CAAC;AAGvE,MAAM,MAAM,iBAAiB,GAAG,WAAW,GAAG,YAAY,CAAC;AAE3D,MAAM,WAAW,gCAAgC;IAChD,oFAAoF;IACpF,gBAAgB,CAAC,EAAE,OAAO,CAAC;CAC3B;AAED;;;;;;;;;;GAUG;AACH,wBAAsB,yBAAyB,CAC9C,OAAO,EAAE,iBAAiB,EAAE,EAC5B,OAAO,CAAC,EAAE,gCAAgC,GACxC,OAAO,CAAC,iBAAiB,EAAE,CAAC,CAqC9B"}
1
+ {"version":3,"file":"tool-result-images.d.ts","sourceRoot":"","sources":["../../src/utils/tool-result-images.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,YAAY,EAAE,uBAAuB,EAAE,WAAW,EAAE,MAAM,uBAAuB,CAAC;AAGhG,MAAM,MAAM,iBAAiB,GAAG,WAAW,GAAG,YAAY,CAAC;AAE3D,MAAM,WAAW,gCAAgC;IAChD,oFAAoF;IACpF,gBAAgB,CAAC,EAAE,OAAO,CAAC;IAC3B,2FAA2F;IAC3F,aAAa,CAAC,EAAE,uBAAuB,CAAC;CACxC;AAED;;;;;;;;;;GAUG;AACH,wBAAsB,yBAAyB,CAC9C,OAAO,EAAE,iBAAiB,EAAE,EAC5B,OAAO,CAAC,EAAE,gCAAgC,GACxC,OAAO,CAAC,iBAAiB,EAAE,CAAC,CAwC9B"}
@@ -22,7 +22,10 @@ export async function normalizeToolResultImages(content, options) {
22
22
  normalized.push(block);
23
23
  continue;
24
24
  }
25
- const processed = await processImage(Buffer.from(block.data, "base64"), block.mimeType, { autoResizeImages });
25
+ const processed = await processImage(Buffer.from(block.data, "base64"), block.mimeType, {
26
+ autoResizeImages,
27
+ resizeOptions: options?.resizeOptions,
28
+ });
26
29
  if (!processed.ok) {
27
30
  // Unlike `read`, keep the original block. The tool already produced this image and the
28
31
  // failure may just be an unavailable image backend, so passing it through preserves the
@@ -1 +1 @@
1
- {"version":3,"file":"tool-result-images.js","sourceRoot":"","sources":["../../src/utils/tool-result-images.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,YAAY,EAAE,MAAM,oBAAoB,CAAC;AASlD;;;;;;;;;;GAUG;AACH,MAAM,CAAC,KAAK,UAAU,yBAAyB,CAC9C,OAA4B,EAC5B,OAA0C;IAE1C,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,IAAI,KAAK,OAAO,CAAC,EAAE,CAAC;QACtD,OAAO,OAAO,CAAC;IAChB,CAAC;IAED,MAAM,gBAAgB,GAAG,OAAO,EAAE,gBAAgB,IAAI,IAAI,CAAC;IAC3D,MAAM,UAAU,GAAwB,EAAE,CAAC;IAC3C,IAAI,OAAO,GAAG,KAAK,CAAC;IAEpB,KAAK,MAAM,KAAK,IAAI,OAAO,EAAE,CAAC;QAC7B,IAAI,KAAK,CAAC,IAAI,KAAK,OAAO,EAAE,CAAC;YAC5B,UAAU,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;YACvB,SAAS;QACV,CAAC;QAED,MAAM,SAAS,GAAG,MAAM,YAAY,CAAC,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,EAAE,QAAQ,CAAC,EAAE,KAAK,CAAC,QAAQ,EAAE,EAAE,gBAAgB,EAAE,CAAC,CAAC;QAC9G,IAAI,CAAC,SAAS,CAAC,EAAE,EAAE,CAAC;YACnB,uFAAuF;YACvF,wFAAwF;YACxF,uEAAuE;YACvE,UAAU,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;YACvB,SAAS;QACV,CAAC;QAED,IAAI,SAAS,CAAC,IAAI,KAAK,KAAK,CAAC,IAAI,IAAI,SAAS,CAAC,QAAQ,KAAK,KAAK,CAAC,QAAQ,IAAI,SAAS,CAAC,KAAK,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YAC5G,UAAU,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;YACvB,SAAS;QACV,CAAC;QAED,UAAU,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,OAAO,EAAE,IAAI,EAAE,SAAS,CAAC,IAAI,EAAE,QAAQ,EAAE,SAAS,CAAC,QAAQ,EAAE,CAAC,CAAC;QACvF,IAAI,SAAS,CAAC,KAAK,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YAChC,UAAU,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,SAAS,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;QACrE,CAAC;QACD,OAAO,GAAG,IAAI,CAAC;IAChB,CAAC;IAED,OAAO,OAAO,CAAC,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,OAAO,CAAC;AACvC,CAAC","sourcesContent":["import type { ImageContent, TextContent } from \"@earendil-works/pi-ai\";\nimport { processImage } from \"./image-process.ts\";\n\nexport type ToolResultContent = TextContent | ImageContent;\n\nexport interface NormalizeToolResultImagesOptions {\n\t/** Whether oversized images are resized to inline provider limits. Default: true */\n\tautoResizeImages?: boolean;\n}\n\n/**\n * Normalize image blocks returned by tool results.\n *\n * The `read` tool and `@file` CLI attachments run their images through `processImage`, but tools\n * that produce images themselves (extensions, MCP bridges, screenshot tools) hand back arbitrary\n * base64 payloads that go straight into session history and every subsequent provider request.\n * Oversized images make the provider reject the whole conversation, not just the offending turn,\n * so normalize them once as they enter history.\n *\n * Returns the original array when nothing changed so callers can skip rewriting the result.\n */\nexport async function normalizeToolResultImages(\n\tcontent: ToolResultContent[],\n\toptions?: NormalizeToolResultImagesOptions,\n): Promise<ToolResultContent[]> {\n\tif (!content.some((block) => block.type === \"image\")) {\n\t\treturn content;\n\t}\n\n\tconst autoResizeImages = options?.autoResizeImages ?? true;\n\tconst normalized: ToolResultContent[] = [];\n\tlet changed = false;\n\n\tfor (const block of content) {\n\t\tif (block.type !== \"image\") {\n\t\t\tnormalized.push(block);\n\t\t\tcontinue;\n\t\t}\n\n\t\tconst processed = await processImage(Buffer.from(block.data, \"base64\"), block.mimeType, { autoResizeImages });\n\t\tif (!processed.ok) {\n\t\t\t// Unlike `read`, keep the original block. The tool already produced this image and the\n\t\t\t// failure may just be an unavailable image backend, so passing it through preserves the\n\t\t\t// behavior tools have today instead of silently deleting their output.\n\t\t\tnormalized.push(block);\n\t\t\tcontinue;\n\t\t}\n\n\t\tif (processed.data === block.data && processed.mimeType === block.mimeType && processed.hints.length === 0) {\n\t\t\tnormalized.push(block);\n\t\t\tcontinue;\n\t\t}\n\n\t\tnormalized.push({ type: \"image\", data: processed.data, mimeType: processed.mimeType });\n\t\tif (processed.hints.length > 0) {\n\t\t\tnormalized.push({ type: \"text\", text: processed.hints.join(\"\\n\") });\n\t\t}\n\t\tchanged = true;\n\t}\n\n\treturn changed ? normalized : content;\n}\n"]}
1
+ {"version":3,"file":"tool-result-images.js","sourceRoot":"","sources":["../../src/utils/tool-result-images.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,YAAY,EAAE,MAAM,oBAAoB,CAAC;AAWlD;;;;;;;;;;GAUG;AACH,MAAM,CAAC,KAAK,UAAU,yBAAyB,CAC9C,OAA4B,EAC5B,OAA0C;IAE1C,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,IAAI,KAAK,OAAO,CAAC,EAAE,CAAC;QACtD,OAAO,OAAO,CAAC;IAChB,CAAC;IAED,MAAM,gBAAgB,GAAG,OAAO,EAAE,gBAAgB,IAAI,IAAI,CAAC;IAC3D,MAAM,UAAU,GAAwB,EAAE,CAAC;IAC3C,IAAI,OAAO,GAAG,KAAK,CAAC;IAEpB,KAAK,MAAM,KAAK,IAAI,OAAO,EAAE,CAAC;QAC7B,IAAI,KAAK,CAAC,IAAI,KAAK,OAAO,EAAE,CAAC;YAC5B,UAAU,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;YACvB,SAAS;QACV,CAAC;QAED,MAAM,SAAS,GAAG,MAAM,YAAY,CAAC,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,EAAE,QAAQ,CAAC,EAAE,KAAK,CAAC,QAAQ,EAAE;YACvF,gBAAgB;YAChB,aAAa,EAAE,OAAO,EAAE,aAAa;SACrC,CAAC,CAAC;QACH,IAAI,CAAC,SAAS,CAAC,EAAE,EAAE,CAAC;YACnB,uFAAuF;YACvF,wFAAwF;YACxF,uEAAuE;YACvE,UAAU,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;YACvB,SAAS;QACV,CAAC;QAED,IAAI,SAAS,CAAC,IAAI,KAAK,KAAK,CAAC,IAAI,IAAI,SAAS,CAAC,QAAQ,KAAK,KAAK,CAAC,QAAQ,IAAI,SAAS,CAAC,KAAK,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YAC5G,UAAU,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;YACvB,SAAS;QACV,CAAC;QAED,UAAU,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,OAAO,EAAE,IAAI,EAAE,SAAS,CAAC,IAAI,EAAE,QAAQ,EAAE,SAAS,CAAC,QAAQ,EAAE,CAAC,CAAC;QACvF,IAAI,SAAS,CAAC,KAAK,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YAChC,UAAU,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,SAAS,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;QACrE,CAAC;QACD,OAAO,GAAG,IAAI,CAAC;IAChB,CAAC;IAED,OAAO,OAAO,CAAC,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,OAAO,CAAC;AACvC,CAAC","sourcesContent":["import type { ImageContent, ModelImageResizeOptions, TextContent } from \"@earendil-works/pi-ai\";\nimport { processImage } from \"./image-process.ts\";\n\nexport type ToolResultContent = TextContent | ImageContent;\n\nexport interface NormalizeToolResultImagesOptions {\n\t/** Whether oversized images are resized to inline provider limits. Default: true */\n\tautoResizeImages?: boolean;\n\t/** Model-specific resize profile. Uses the conservative built-in defaults when omitted. */\n\tresizeOptions?: ModelImageResizeOptions;\n}\n\n/**\n * Normalize image blocks returned by tool results.\n *\n * The `read` tool and `@file` CLI attachments run their images through `processImage`, but tools\n * that produce images themselves (extensions, MCP bridges, screenshot tools) hand back arbitrary\n * base64 payloads that go straight into session history and every subsequent provider request.\n * Oversized images make the provider reject the whole conversation, not just the offending turn,\n * so normalize them once as they enter history.\n *\n * Returns the original array when nothing changed so callers can skip rewriting the result.\n */\nexport async function normalizeToolResultImages(\n\tcontent: ToolResultContent[],\n\toptions?: NormalizeToolResultImagesOptions,\n): Promise<ToolResultContent[]> {\n\tif (!content.some((block) => block.type === \"image\")) {\n\t\treturn content;\n\t}\n\n\tconst autoResizeImages = options?.autoResizeImages ?? true;\n\tconst normalized: ToolResultContent[] = [];\n\tlet changed = false;\n\n\tfor (const block of content) {\n\t\tif (block.type !== \"image\") {\n\t\t\tnormalized.push(block);\n\t\t\tcontinue;\n\t\t}\n\n\t\tconst processed = await processImage(Buffer.from(block.data, \"base64\"), block.mimeType, {\n\t\t\tautoResizeImages,\n\t\t\tresizeOptions: options?.resizeOptions,\n\t\t});\n\t\tif (!processed.ok) {\n\t\t\t// Unlike `read`, keep the original block. The tool already produced this image and the\n\t\t\t// failure may just be an unavailable image backend, so passing it through preserves the\n\t\t\t// behavior tools have today instead of silently deleting their output.\n\t\t\tnormalized.push(block);\n\t\t\tcontinue;\n\t\t}\n\n\t\tif (processed.data === block.data && processed.mimeType === block.mimeType && processed.hints.length === 0) {\n\t\t\tnormalized.push(block);\n\t\t\tcontinue;\n\t\t}\n\n\t\tnormalized.push({ type: \"image\", data: processed.data, mimeType: processed.mimeType });\n\t\tif (processed.hints.length > 0) {\n\t\t\tnormalized.push({ type: \"text\", text: processed.hints.join(\"\\n\") });\n\t\t}\n\t\tchanged = true;\n\t}\n\n\treturn changed ? normalized : content;\n}\n"]}
@@ -107,7 +107,7 @@ Only the current structured completion format is specialized. Older text-only co
107
107
 
108
108
  ## Lifetime
109
109
 
110
- Background execution belongs to the current session runtime, not a daemon. Parent-turn cancellation still cancels foreground-owned work; after detach it does not cancel background work. Shutdown, `/reload`, `/new`, `/resume`, and `/fork` close admission, stop delivery, cancel work and perform bounded cleanup. `/tree` cancels executions whose launch anchor is absent from the destination branch and suppresses their completion delivery there; navigating back to a branch revives its undelivered completions, and ordinary conversation progress along the same branch does not cancel them. Active processes and workers are not reattached across process restart or copied into a fork.
110
+ Background execution belongs to the current session runtime, not a daemon. Parent-turn cancellation still cancels foreground-owned work; after detach it does not cancel background work. Shutdown, `/reload`, `/new`, `/resume`, and `/fork` close admission, stop delivery, cancel work and perform bounded cleanup. `/tree` cancels executions whose launch anchor is absent from the destination branch and suppresses their completion delivery there; navigating back to a branch revives its undelivered completions, and ordinary conversation progress along the same branch does not cancel them. A completion already handed to the prompt queue when its run was aborted is not recalled — the next run delivers it, like any other queued message. Active processes and workers are not reattached across process restart or copied into a fork.
111
111
 
112
112
  The panel is an observer, not a cleanup engine. Admission, bounded history/output retention, completion delivery and headless exit policy are enforced by the core service and hosting mode.
113
113
 
@@ -121,9 +121,9 @@ Terminal snapshots persist as version-2 `background-task-result` custom entries,
121
121
 
122
122
  Delivered history outside the selected branch can be released and restored on return, so it does not fill the new branch's history budget. Undelivered completions stay protected while their branch is hidden. Pending delivery, pins and active reads use a separate allowance within the total runtime retention cap; restoration respects the same cap.
123
123
 
124
- Completion delivery waits for a safe idle boundary, respects queued user work, and sends one bounded completion at a time. A Subagent group produces one summary, not a separate wake-up per worker. A terminal `bg wait` coordinates with automatic delivery only after its tool result is persisted; aborting during output reading does not lose the pending completion. Direct SDK waits remain observational until explicitly acknowledged. Progress and repeated reads do not inject messages or add usage. `agent_settled` still describes the main agent, not the end of all background executions.
124
+ Completion delivery follows the same queueing path as interactive input. While the agent is running, a completion is steered into that run, so it lands right after the current tool batch — exactly like a message the user types mid-run, and behind any already queued steering. An idle session starts a completion turn instead. User preflight (input hooks, model and authentication checks, compaction) still holds delivery, so a user message is always the first thing a new run sees. One bounded completion is in flight at a time. A Subagent group produces one summary, not a separate wake-up per worker. A terminal `bg wait` coordinates with automatic delivery only after its tool result is persisted: while a `bg wait`, `bg read` or a watched `/bg` row holds a task pinned, that task is not announced, and aborting during output reading does not lose the pending completion. Direct SDK waits remain observational until explicitly acknowledged. Progress and repeated reads do not inject messages or add usage. `agent_settled` still describes the main agent, not the end of all background executions.
125
125
 
126
- Failed completion delivery leaves the terminal result available for inspection rather than silently discarding it, and warns through the extension error channel. The next user prompt retries delivery automatically; SDK hosts can also call `session.retryBackgroundNotifications()` explicitly after resolving the failure. There is no infinite timer retry loop.
126
+ Failed completion delivery leaves the terminal result available for inspection rather than silently discarding it, and warns through the extension error channel. The next user prompt retries delivery automatically; SDK hosts can also call `session.retryBackgroundNotifications()` explicitly after resolving the failure. There is no infinite timer retry loop. A completion that a run emitted but never persisted, and a completion dropped by clearing the queue, fail the same way; one still sitting in the queue when its run ends stays claimed, because the next run delivers that same message.
127
127
 
128
128
  Queued extension `nextTurn` context accompanies completion turns and stays queued until each message is persisted. A failed or partially persisted turn therefore retries only the context that has not been saved.
129
129
 
@@ -85,7 +85,7 @@ Only `explorer` and `general` keys are valid. A malformed, unsupported, unreadab
85
85
  - A retryable failure that produced no turns or tool use may be retried at task level up to two more times. Runs with partial work are never restarted.
86
86
  - Retry backoff does not hold a concurrency slot; another queued worker can run while the failed task waits.
87
87
  - Cancellation of the owning execution interrupts queued and retrying workers immediately.
88
- - Background completion is delivered as one bounded group summary, with reports in input order, at a safe idle boundary after queued user work. Its saved card has independent profile/status/report/error fields and explicit report-truncation flags; report text is never reparsed as worker metadata. Every worker gets a bounded report share. `bg wait` can deliver the terminal result instead; repeated `bg read` calls do not trigger notifications or billing.
88
+ - Background completion is delivered as one bounded group summary, with reports in input order, through the host's normal prompt queueing: steered into a running turn right after its current tool batch, or as its own turn when the session is idle. Its saved card has independent profile/status/report/error fields and explicit report-truncation flags; report text is never reparsed as worker metadata. Every worker gets a bounded report share. `bg wait` can deliver the terminal result instead; repeated `bg read` calls do not trigger notifications or billing.
89
89
  - Managed worker usage settles through an independent persisted ledger, not through panel visits or result reads. Accrued usage from retries, failures, cancellations, and provider-supplied worker compaction is retained; missing provider usage is not fabricated. Worker tokens do not count as parent context occupancy.
90
90
 
91
91
  Child sessions share the parent's canonical model/authentication runtime, so extension-registered providers and current credentials do not need to be mirrored into a second runtime.
@@ -17,7 +17,7 @@ Pi has two summarization mechanisms:
17
17
  | Compaction | Context exceeds threshold, or `/compact` | Summarize old messages to free up context |
18
18
  | Branch summarization | `/tree` navigation | Preserve context when switching branches |
19
19
 
20
- Both use the same structured summary format and track file operations cumulatively. Compaction and branch-summary requests use fresh routing session IDs and, where supported by the provider, disable prompt-cache writes because these one-off prompts are unlikely to be reused.
20
+ Both use the same structured summary format and track file operations cumulatively. Summarization requests disable prompt-cache writes because these one-off prompts are unlikely to be reused.
21
21
 
22
22
  ## Compaction
23
23
 
@@ -33,7 +33,9 @@ contextTokens > contextWindow − reserveTokens
33
33
 
34
34
  > **Distribution note:** upstream Pi's default is a fixed 16384-token reserve, which on large windows delays compaction until ~98% usage. This distribution derives the reserve from `triggerPercent` instead. Setting `compaction.reserveTokens` explicitly — globally or per model — still wins and restores upstream's fixed-reserve behavior for that model.
35
35
 
36
- During a multi-turn agent run, Pi checks this threshold after tools finish and their results are appended, before starting the next assistant response. If the threshold is crossed, Pi compacts inside the same agent run and resumes with the summary and retained messages. It skips this between-turn check when the completed tool batch terminates the run and no queued message requires another response. Pi also checks the threshold before a new user prompt and after a low-level agent run ends.
36
+ During a multi-turn agent run, Pi checks this threshold against the canonical session projection after tools finish and their results are appended, before starting the next assistant response. If the threshold is crossed, Pi compacts during `prepareNextTurn`, then performs the existing catch-up steering poll before `turn_start`, and resumes with the summary and retained messages. It skips this between-turn check when the completed tool batch terminates the run and no queued message requires another response. Pi also checks the threshold before a new user prompt and performs final-attempt overflow recovery after a low-level agent run ends.
37
+
38
+ A provider context-overflow error or an early final `stopReason: "length"` can select one compact-and-retry recovery attempt. Length responses with tool calls retain their synthetic failed tool results and follow the ordinary tool/queue scheduler rather than forcing the run to end.
37
39
 
38
40
  This between-turn trigger is separate from **split-turn compaction** below: the trigger determines *when* compaction runs, while split-turn support determines *where* a very long turn can be cut.
39
41
 
@@ -41,8 +43,8 @@ You can also trigger manually with `/compact [instructions]`, where optional ins
41
43
 
42
44
  ### How It Works
43
45
 
44
- 1. **Find cut point**: Walk backwards from newest message until the effective retention target is reached: `keepRecentTokens` (default 20k), capped at half the trigger line when the reserve comes from `triggerPercent`
45
- 2. **Extract messages**: Collect messages from the previous kept boundary (or session start) up to the cut point
46
+ 1. **Find cut point**: Walk backwards through the finalized session projection until the effective retention target is reached: `keepRecentTokens` (default 20k), capped at half the trigger line when the reserve comes from `triggerPercent`
47
+ 2. **Extract messages**: Collect projected messages from the previous kept boundary (or session start) up to the cut point
46
48
  3. **Generate summary**: Call LLM to summarize with structured format, passing the previous summary as iterative context when present
47
49
  4. **Append entry**: Save `CompactionEntry` with summary and `firstKeptEntryId`
48
50
  5. **Rebuilds context**: Session rebuilds the context for the next request, using summary + messages from `firstKeptEntryId` onwards
@@ -83,7 +85,22 @@ What the LLM sees:
83
85
  prompt from cmp messages from firstKeptEntryId
84
86
  ```
85
87
 
86
- On repeated compactions, the summarized span starts at the previous compaction's kept boundary (`firstKeptEntryId`), not at the compaction entry itself, falling back to the entry after the previous compaction if that kept entry cannot be found in the path. This preserves messages that survived the earlier compaction by including them in the next summarization pass as well. Pi also recalculates `tokensBefore` from the rebuilt session context before writing the new `CompactionEntry`, so the token count reflects the actual pre-compaction context being replaced.
88
+ On repeated compactions, the summarized span starts at the previous compaction's kept boundary (`firstKeptEntryId`), not at the compaction entry itself, falling back to the entry after the previous compaction if that kept entry cannot be found in the path. A retain-none compaction records its own ID as `firstKeptEntryId`; repeated compaction starts after that entry. This preserves messages that survived the earlier compaction by including them in the next summarization pass as well. Pi also recalculates `tokensBefore` from the rebuilt, context-edited session projection before writing the new `CompactionEntry`, so the token count reflects the actual pre-compaction context being replaced. Omitted raw entries remain stored but do not affect cut selection, summaries, checkpoints, or token estimates.
89
+
90
+ ### Overflow and Length Recovery Ordering
91
+
92
+ Recovery preserves the existing lifecycle and queue order. The completed attempt remains visible to `turn_end` and `agent_end`; post-run recovery then repairs persisted model context before a fresh retry:
93
+
94
+ ```text
95
+ persist final assistant response
96
+ → extension/public turn_end
97
+ → extension/public agent_end
98
+ → append context_edit omissions for the selected attempt
99
+ → for overflow/length: run session_before_compact and append compaction on success
100
+ → start the retry as a fresh run
101
+ ```
102
+
103
+ If recovery compaction fails or is cancelled, Pi keeps the omission edits, appends no compaction, and schedules no internal retry. Existing queued work remains governed by ordinary steering and follow-up rules. `agent_before_settle` sees the repaired projection after recovery processing. Raw transcript history, exports, billing totals, and history-search extensions can still inspect the omitted attempt.
87
104
 
88
105
  ### Split Turns
89
106
 
@@ -125,6 +142,8 @@ Never cut at tool results (they must stay with their tool call).
125
142
 
126
143
  Because tool results are not valid cut points, a batch of trailing tool results can exceed `keepRecentTokens` on its own, leaving no cut point at or after the entry that crosses the budget. The cut then lands on the last valid cut point — the least the message structure allows Pi to keep. This matters most for sessions with very few cut points, such as a subagent worker, whose whole session is a single turn: one prompt followed by assistant and tool result entries only.
127
144
 
145
+ Preparation advances the kept boundary into a context-invisible suffix only when that suffix contains an omitted assistant attempt and no unomitted context-producing entries. Recovery `context_edit` omissions satisfy this rule; intrinsically context-invisible metadata may coexist with them. Metadata alone and newly appended custom messages do not move the cut. A replacement edit affecting the candidate input or summarized prefix also blocks advancement because the omitted assistant answered the pre-edit input; replacements of suffix entries that are ultimately omitted remain safe. This allows an over-budget recovered input to be summarized while retaining the edits that keep the abandoned attempt omitted, without making bookkeeping change whether new model input is preserved verbatim.
146
+
128
147
  ### CompactionEntry Structure
129
148
 
130
149
  The public `CompactionEntry` type is documented in [Session Format](session-format.md):
@@ -296,7 +296,8 @@ user sends prompt ────────────────────
296
296
  │ ┌─── turn (repeats while LLM calls tools) ───┐ │
297
297
  │ │ │ │
298
298
  │ ├─► turn_start │ │
299
- │ ├─► context (can modify messages) │ │
299
+ │ ├─► context (can modify conversation messages) │
300
+ │ ├─► context_with_system (can modify the full transcript)
300
301
  │ ├─► before_provider_headers (can mutate headers) |
301
302
  │ ├─► before_provider_request (can inspect or replace payload)
302
303
  │ ├─► after_provider_response (status + headers, before stream consume)
@@ -308,10 +309,14 @@ user sends prompt ────────────────────
308
309
  │ │ ├─► tool_result (can modify) │ │
309
310
  │ │ └─► tool_execution_end │ │
310
311
  │ │ │ │
311
- │ └─► turn_end │ │
312
+ │ └─► turn_end (can append entries and continue) │ │
313
+ │ └─► threshold compaction before a naturally required next turn
312
314
  │ │
313
315
  ├─► agent_end │
314
- └─► agent_settled (no retry/compaction/follow-up left) │
316
+ ├─► retry backoff or final-attempt recovery (when selected)
317
+ │ └─► fresh agent_start on successful recovery │
318
+ ├─► agent_before_settle (can append entries and continue)│
319
+ └─► agent_settled (final, notification only) │
315
320
  │
316
321
  user sends another prompt ◄────────────────────────────────┘
317
322
 
@@ -579,9 +584,9 @@ The `systemPromptOptions` field gives extensions access to the same structured d
579
584
 
580
585
  Inside `before_agent_start`, `event.systemPrompt` and `ctx.getSystemPrompt()` both reflect the chained system prompt as of the current handler. Later `before_agent_start` handlers can still modify it again.
581
586
 
582
- #### agent_start / agent_end / agent_settled
587
+ #### agent_start / agent_end / agent_before_settle / agent_settled
583
588
 
584
- `agent_start` fires when a low-level agent run begins. `agent_end` fires when that run ends, but Pi may still auto-retry, auto-compact and retry, or continue with queued follow-up messages. Use `agent_settled` for status integrations that need to know Pi will not continue running automatically.
589
+ `agent_start` fires when a low-level agent run begins. `agent_end` fires when that run ends, but Pi may still auto-retry, auto-compact and retry, or continue with queued follow-up messages. `agent_before_settle` is the final actionable boundary: it can append session entries and request one continuation. `agent_settled` is final and notification-only; use it for status integrations that need to know Pi will not continue running automatically.
585
590
 
586
591
  ```typescript
587
592
  pi.on("agent_start", async (_event, ctx) => {});
@@ -590,11 +595,28 @@ pi.on("agent_end", async (event, ctx) => {
590
595
  // event.messages - messages from this low-level run
591
596
  });
592
597
 
598
+ let addedReviewReminder = false;
599
+ pi.on("agent_before_settle", async (event, ctx) => {
600
+ if (addedReviewReminder) return;
601
+ addedReviewReminder = true;
602
+ return {
603
+ entries: [...event.entries, {
604
+ type: "custom_message",
605
+ customType: "review-reminder",
606
+ content: "Review the final diff before replying.",
607
+ display: false,
608
+ }],
609
+ continue: true,
610
+ };
611
+ });
612
+
593
613
  pi.on("agent_settled", async (_event, ctx) => {
594
- // ctx.isIdle() is true here unless another extension started a new run.
614
+ // ctx.isIdle() is true; runs requested here start after all settled handlers finish.
595
615
  });
596
616
  ```
597
617
 
618
+ If the run is aborted while `agent_before_settle` handlers are running, valid returned entries are still committed, but requested continuation is suppressed. Work requested from `agent_settled` is deferred until every settled handler completes, so notification dispatch is non-reentrant. In this distribution, Background completion delivery also stays paused until those deferred runs have started, so a deferred prompt never races a completion turn for the same idle gap.
619
+
598
620
  #### ui_prompt_start / ui_prompt_end
599
621
 
600
622
  Notification-only lifecycle events for blocking user-facing extension UI prompts. They fire around `ctx.ui.select()`, `ctx.ui.confirm()`, `ctx.ui.input()`, `ctx.ui.editor()`, and `ctx.ui.custom()` so host/status integrations can report "waiting for user" instead of just "running".
@@ -622,12 +644,35 @@ pi.on("turn_start", async (event, ctx) => {
622
644
  // event.turnIndex, event.timestamp
623
645
  });
624
646
 
647
+ let replacedResponse = false;
625
648
  pi.on("turn_end", async (event, ctx) => {
626
649
  // event.turnIndex, event.message, event.toolResults
650
+ // event.entries contains the structural entries proposed so far.
651
+ if (replacedResponse || event.outcome !== "completed" || event.toolResults.length > 0) return;
652
+ replacedResponse = true;
653
+ return {
654
+ entries: [
655
+ ...event.entries,
656
+ { type: "context_edit", targetId: event.messageEntryId, replacement: null },
657
+ {
658
+ type: "custom_message",
659
+ customType: "replacement-instruction",
660
+ content: "Answer again using the persisted user request.",
661
+ display: false,
662
+ },
663
+ ],
664
+ continue: true,
665
+ };
627
666
  });
628
667
  ```
629
668
 
630
- Between-tool auto-compaction runs after the completed tool turn's `turn_end`. If it cannot produce a safe next context, Pi stops before queue polling or another provider request. The upstream stateful Agent represents that package-local stop as an additional assistant error/aborted message lifecycle and `turn_end`, followed by `agent_end`; SDK, JSON, and RPC consumers can use the preceding `compaction_end` event for the detailed compaction failure or retained-context warning.
669
+ `turn_end` runs after the assistant and tool-result messages have been persisted and before the low-level `turn_end` event. Retry backoff and final-attempt recovery still happen after `agent_end`, preserving their existing lifecycle and queue ordering; `agent_before_settle` sees the repaired projection after that work completes. Boundary handlers run in extension load and registration order. Each handler sees prior proposals in `event.entries` and sees `event.context` rebuilt from them. Returning `entries` or `continue` replaces only that field; omitted fields preserve the current proposal. Allowed draft entry types are `custom`, `custom_message`, `context_edit`, and `compaction`. The complete proposal is validated before it is appended in list order after all handlers finish; a handler error is reported and later handlers still run. Validation prevents partially applied semantic errors, but persistence is not transactional.
670
+
671
+ `continue: true` ensures one next provider request for that boundary invocation. If tool results, steering, or a follow-up already cause that request, they satisfy the decision and no additional request is made; otherwise Pi makes one context-only request. Error and aborted responses remain hard exits. `continue: false` never suppresses natural work. Guard continuation conditions: an unconditional `continue: true` is evaluated again after the next response and can create an endless loop. A `custom_message` draft contributes a user-role model message but is extension-authored: it does not run human input hooks, slash commands, skills, or prompt templates.
672
+
673
+ Host integrations that construct `TurnEndEvent` values must provide `messageEntryId`, `toolResultEntryIds`, `outcome`, `entries`, `continue`, and `context`. `ExtensionEvent` exhaustive switches must also handle `agent_before_settle`. `ExtensionRunner.emit()` excludes actionable turn boundaries; dispatch `turn_end` and `agent_before_settle` through `emitBoundary(baseEvent, buildContext)` so handlers receive chained previews. Other dedicated runner methods still return results for events such as `session_before_*`.
674
+
675
+ Between-tool auto-compaction runs after the completed tool turn's `turn_end`, against the canonical session projection. If it cannot produce a safe next context, Pi stops before queue polling or another provider request. The upstream stateful Agent represents that package-local stop as an additional assistant error/aborted message lifecycle and `turn_end`, followed by `agent_end`; SDK, JSON, and RPC consumers can use the preceding `compaction_end` event for the detailed compaction failure or retained-context warning.
631
676
 
632
677
  #### message_start / message_update / message_end
633
678
 
@@ -695,12 +740,31 @@ Fired before each LLM call. Modify messages non-destructively. See [Session Form
695
740
 
696
741
  ```typescript
697
742
  pi.on("context", async (event, ctx) => {
698
- // event.messages - deep copy, safe to modify
743
+ // event.messages - deep copy without system messages, safe to modify
699
744
  const filtered = event.messages.filter(m => !shouldPrune(m));
700
745
  return { messages: filtered };
701
746
  });
702
747
  ```
703
748
 
749
+ `event.messages` holds the conversation without system messages. The prompt and tool declarations belong to Pi and are not part of this hook: when the handler returns a changed list, Pi replays the current prompt sections and tool declarations into one leading system message ahead of the returned messages. Filtering, windowing, or slicing from a compaction summary therefore cannot drop the prompt or the tools. An unchanged list keeps mid-conversation system messages in place, so models that accept them retain their cached prefix. System messages a handler adds are kept after Pi's head. To change the prompt or the tool set durably, use [`before_agent_start`](#before_agent_start) or `pi.setActiveTools()`; to edit system messages for one request, use [`context_with_system`](#context_with_system).
750
+
751
+ #### context_with_system
752
+
753
+ Fired before each LLM call, after every `context` handler has run and Pi has restored the prompt and tool state. `event.messages` is the full transcript, including the leading system message and any mid-conversation prompt or tool patches (see [Session Format](session-format.md#sessionmessageentry)). The returned messages are sent as they are: this hook owns the prompt and tool declarations for the request.
754
+
755
+ ```typescript
756
+ import { getCurrentSystemMessage } from "@earendil-works/pi-ai";
757
+
758
+ pi.on("context_with_system", async (event, ctx) => {
759
+ const cut = findCutIndex(event.messages);
760
+ // Fold the dropped prefix so its prompt and tool state survives as the new head.
761
+ const head = getCurrentSystemMessage(event.messages.slice(0, cut));
762
+ return { messages: head ? [head, ...event.messages.slice(cut)] : event.messages.slice(cut) };
763
+ });
764
+ ```
765
+
766
+ Rules: keep a system message at index 0 (providers read the prompt and initial tool declarations there; Pi reports an error if a handler drops it). Removing a system message removes the tool declarations and section patches it carries. Check your output with `getCurrentSystemPrompt()` and `getCurrentTools()` from `@earendil-works/pi-ai`. Handlers run in extension load order; a `systemPrompt` forced from `before_agent_start` is still projected onto the request afterwards. `ctx.getContextSnapshot()` reuses the request prefix these hooks produced, so BTW sees the same transcript the provider received.
767
+
704
768
  #### before_provider_headers
705
769
 
706
770
  Fired after the outgoing HTTP headers are assembled. Use it to add, override, or remove request headers.
package/docs/models.md CHANGED
@@ -14,6 +14,7 @@ Pi reads this file at startup and on a model refresh, including when you open `/
14
14
  - [Value Resolution](#value-resolution)
15
15
  - [Custom Headers](#custom-headers)
16
16
  - [Model Configuration](#model-configuration)
17
+ - [Image Input Limits](#image-input-limits)
17
18
  - [Thinking Level Map](#thinking-level-map)
18
19
  - [Prompt Cache Lifetimes](#prompt-cache-lifetimes)
19
20
  - [Overriding Built-in Providers](#overriding-built-in-providers)
@@ -243,6 +244,7 @@ A `models` entry supplies a complete custom model definition with useful default
243
244
  | `reasoning` | No | `false` | Whether the model supports Pi thinking levels. |
244
245
  | `thinkingLevelMap` | No | omitted | Maps Pi thinking levels to provider values and declares unsupported levels; see below. |
245
246
  | `input` | No | `["text"]` | Supported input kinds: `text` and `image`. |
247
+ | `inputLimits` | No | omitted | Request limits and image preprocessing for this model; see [Image Input Limits](#image-input-limits). |
246
248
  | `contextWindow` | No | `128000` | Context window in tokens. |
247
249
  | `maxTokens` | No | `16384` | Maximum generated tokens. |
248
250
  | `samplingParams` | No | omitted | Free-form sampling parameters merged verbatim into every request body; see [Sampling Parameters](#sampling-parameters). |
@@ -275,6 +277,33 @@ When `cost` is supplied on a `models` entry, it must contain all four base rates
275
277
  }
276
278
  ```
277
279
 
280
+ ### Image Input Limits
281
+
282
+ Use `inputLimits.images.resize` to configure how new images are encoded before they enter conversation history:
283
+
284
+ ```json
285
+ {
286
+ "id": "vision-model",
287
+ "input": ["text", "image"],
288
+ "inputLimits": {
289
+ "images": {
290
+ "resize": {
291
+ "maxWidth": 1568,
292
+ "maxHeight": 1568,
293
+ "maxBytes": 524288,
294
+ "jpegQuality": 75
295
+ }
296
+ }
297
+ }
298
+ }
299
+ ```
300
+
301
+ `maxBytes` is the maximum base64-encoded payload size. Omitted resize fields use Pi's conservative defaults: 2000×2000, 4.5 MiB encoded, and JPEG quality 80. Built-in vision models carry that profile explicitly so unknown gateways never receive larger images than before.
302
+
303
+ Pi applies the selected model's resize profile to `@file` attachments, the `read` tool, and images returned by tools. Images are encoded once before they enter history, after `before_agent_start` handlers have selected the request model; changing models later does not rewrite historical images or invalidate the cached conversation prefix. The `images.autoResize` setting can disable resizing globally.
304
+
305
+ The catalog can also record `inputLimits.maxRequestBytes`, `images.maxPerMessage`, and `images.maxPerRequest`. These fields describe hard provider limits; Pi does not yet rewrite or reject conversation history based on them.
306
+
278
307
  ### Prompt Cache Lifetimes
279
308
 
280
309
  `promptCache` states how long the provider keeps a prompt cache entry alive for each retention tier Pi can request (`short` is the default tier; `long` is used when `PI_CACHE_RETENTION=long`). Values are seconds and are estimates: providers publish ranges, so pick the conservative end.
@@ -405,7 +434,7 @@ The built-in models remain. A new custom `id` is added, while a matching ID is r
405
434
  }
406
435
  ```
407
436
 
408
- An override supports `name`, `reasoning`, `thinkingLevelMap`, `input`, `cost`, `promptCache` (merged per tier), `contextWindow`, `maxTokens`, `samplingParams` (merged per key), `headers`, and `compat`. Its `cost` base rates are individually optional and retain omitted values; a supplied `tiers` array replaces the existing array. Thinking maps merge by level. `compat` merges by field, and its `openRouterRouting`, `vercelGatewayRouting`, and `chatTemplateKwargs` objects merge by key.
437
+ An override supports `name`, `reasoning`, `thinkingLevelMap`, `input`, `inputLimits`, `cost`, `promptCache` (merged per tier), `contextWindow`, `maxTokens`, `samplingParams` (merged per key), `headers`, and `compat`. Its `cost` base rates are individually optional and retain omitted values; a supplied `tiers` array replaces the existing array. Thinking maps merge by level. `inputLimits` deep-merges down to the `images.resize` profile. `compat` merges by field, and its `openRouterRouting`, `vercelGatewayRouting`, and `chatTemplateKwargs` objects merge by key.
409
438
 
410
439
  Use a `promptCache` override to enable cache warming through a proxy whose backing cache you know, for example OpenRouter routed to Anthropic:
411
440
 
package/docs/sdk.md CHANGED
@@ -90,6 +90,8 @@ interface AgentSession {
90
90
 
91
91
  // State access
92
92
  agent: Agent;
93
+ sessionManager: SessionManager;
94
+ refreshContext(): void;
93
95
  model: Model | undefined;
94
96
  thinkingLevel: ThinkingLevel;
95
97
  messages: AgentMessage[];
@@ -278,7 +280,7 @@ An embedding can opt in with `await session.bindExtensions({ backgroundEnabled:
278
280
  The host must choose:
279
281
 
280
282
  - **Lifetime and cancellation:** a settled main-agent turn is not the end of background work. `session.abort()` cancels foreground-owned work, not detached work. Use `session.background.kill(id)` for a whole execution, or `await session.background.shutdown(graceMs?)` to close admission, cancel all owned work and drain within a bounded grace period (default 2 seconds, maximum 60 seconds). A non-cooperating executor can outlive that grace period.
281
- - **Result-driven turns and cost:** automatic completions can trigger new model turns at safe idle boundaries, after queued user work. Use the nestable `session.pauseBackgroundNotifications()` to defer delivery; its returned release function is idempotent. Pausing does not stop execution or its costs. Budget worker activity and completion-triggered turns explicitly.
283
+ - **Result-driven turns and cost:** automatic completions join the host's prompt queue, so a running turn receives them as steering and an idle session starts a completion turn. Use the nestable `session.pauseBackgroundNotifications()` to defer delivery; its returned release function is idempotent. Pausing does not stop execution or its costs. Budget worker activity and completion-triggered turns explicitly.
282
284
  - **Observation and delivery:** `session.background` exposes the [public supervision methods](extensions.md#ctxbackground). `read()` and `wait()` observe; a terminal snapshot alone does not acknowledge model delivery. The bundled `bg wait` result carries `details.backgroundTaskId`, which the host acknowledges only after persisting the tool result. A direct SDK waiter that consumes the result outside model history can explicitly call `session.background.markDelivered(task.id)` after successful consumption. A timeout or cancelled wait does not stop the execution. Failed notification delivery leaves the result available for inspection; after fixing the host failure, call `session.retryBackgroundNotifications()` explicitly. Pi does not retry a failed delivery forever on a timer.
283
285
  - **Shutdown:** `await runtime.dispose()` handles background shutdown and extension lifecycle when using the runtime API. For a directly owned session, drain Background, abort the main agent, then dispose; synchronous `session.dispose()` alone is not an awaited drain.
284
286
 
@@ -329,7 +331,7 @@ The `Agent` class (from `@earendil-works/pi-agent-core`) handles the core LLM in
329
331
  // Access current state
330
332
  const state = session.agent.state;
331
333
 
332
- // state.messages: AgentMessage[] - conversation history
334
+ // state.messages: AgentMessage[] - refreshed inspection cache of the model-visible conversation
333
335
  // state.model: Model - current model
334
336
  // state.thinkingLevel: ThinkingLevel - current thinking level
335
337
  // state.systemPrompt: string - read-only, replayed from the transcript's system messages
@@ -337,8 +339,8 @@ const state = session.agent.state;
337
339
  // state.streamingMessage?: AgentMessage - current partial assistant message
338
340
  // state.errorMessage?: string - latest assistant error
339
341
 
340
- // Replace messages (useful for branching or restoration)
341
- session.agent.state.messages = messages; // copies the top-level array
342
+ // Model-visible messages are projected from session.sessionManager.
343
+ // agent.state.messages is a refreshed inspection cache; do not assign it for restoration.
342
344
 
343
345
  // Replace tools
344
346
  session.agent.state.tools = tools; // copies the top-level array
@@ -347,6 +349,15 @@ session.agent.state.tools = tools; // copies the top-level array
347
349
  await session.agent.waitForIdle();
348
350
  ```
349
351
 
352
+ Provider requests use `session.sessionManager` as the canonical finalized context. Assigning `session.agent.state.messages` does not replace persisted context and is overwritten at the next request boundary. Restore externally stored history when constructing the session instead:
353
+
354
+ ```typescript
355
+ const restoredManager = SessionManager.inMemory(process.cwd(), { id: sessionId }, entries);
356
+ const { session } = await createAgentSession({ sessionManager: restoredManager });
357
+ ```
358
+
359
+ For an existing session, use `session.navigateTree(entryId)` to move its active branch. Use `session.sessionManager.appendMessage(...)` plus `session.refreshContext()` only when intentionally appending externally managed entries. `ctx.getContextSnapshot()` and the bundled BTW extension read the same projection.
360
+
350
361
  ### Events
351
362
 
352
363
  Subscribe to events to receive streaming output and lifecycle notifications.
@@ -33,11 +33,14 @@ The published packages provide the session and message types used here:
33
33
  ```typescript
34
34
  import {
35
35
  type CompactionEntry,
36
+ type ContextEditEntry,
36
37
  type NewSessionOptions,
38
+ type ProjectedSessionEntry,
37
39
  type SessionContext,
38
40
  type SessionEntry,
39
41
  type SessionHeader,
40
42
  type SessionInfo,
43
+ type SessionProjection,
41
44
  type SessionTreeNode,
42
45
  SessionManager,
43
46
  } from "@astralyn/pi";
@@ -287,7 +290,7 @@ Created when context is compacted. Stores a summary of earlier messages and a co
287
290
  {"type":"compaction","id":"f6g7h8i9","parentId":"e5f6g7h8","timestamp":"2024-12-03T14:10:00.000Z","summary":"User discussed X, Y, Z...","firstKeptEntryId":"c3d4e5f6","tokensBefore":50000,"systemMessage":{"role":"system","content":"You are a coding assistant.","toolsAdded":[],"timestamp":1733235000000}}
288
291
  ```
289
292
 
290
- `firstKeptEntryId` is required. It identifies the first entry retained from before the compaction entry. When rebuilding context, Pi replaces older summarized entries with the compaction summary and keeps the range beginning at this entry.
293
+ `firstKeptEntryId` is required. It identifies the first entry retained from before the compaction entry. When rebuilding context, Pi replaces older summarized entries with the compaction summary and keeps the range beginning at this entry. A retain-none compaction stores its own ID in this field, so no preceding entries are retained.
291
294
 
292
295
  Optional fields:
293
296
  - `systemMessage`: The replayed prompt sections and tool declarations at the compaction boundary; it becomes the leading system message of the compacted context, and system messages among the kept entries are dropped in its favor. It is absent on older session entries.
@@ -295,6 +298,16 @@ Optional fields:
295
298
  - `details`: Implementation-specific data (e.g., `{ readFiles: string[], modifiedFiles: string[] }` for default, or custom data for extensions)
296
299
  - `fromHook`: `true` if generated by an extension, `false`/`undefined` if pi-generated (legacy field name)
297
300
 
301
+ ### ContextEditEntry
302
+
303
+ Append-only edit of one earlier context-producing entry. It changes only future model context; the target entry and its metadata remain unchanged in raw history, UI, exports, and session accounting.
304
+
305
+ ```json
306
+ {"type":"context_edit","id":"g6h7i8j9","parentId":"f6g7h8i9","timestamp":"2024-12-03T14:11:00.000Z","targetId":"c3d4e5f6","replacement":null}
307
+ ```
308
+
309
+ Targets may be user, assistant, tool-result, or custom-message entries. `replacement: null` omits the target from model context. A non-null `replacement` replaces only the target message content. String replacements for assistant and tool-result entries are normalized to one text block because those roles require content arrays. If several edits target the same entry, the latest edit on the active branch wins. Edits are branch-relative: navigating to a point before the edit reveals the target's original contribution again. Pi writes these entries itself when auto-retry or overflow recovery abandons an assistant attempt, and extensions can append them from the `turn_end` and `agent_before_settle` boundaries; `/tree` and HTML export list them as bookkeeping entries.
310
+
298
311
  ### BranchSummaryEntry
299
312
 
300
313
  Created when switching branches via `/tree` with an LLM generated summary of the left branch up to the common ancestor. Captures context from the abandoned path.
@@ -436,7 +449,9 @@ Entries normally form one tree, but navigation APIs can create multiple roots:
436
449
 
437
450
  Earlier summarized entries are omitted. Non-message entries in the selected ranges remain in the returned list for rendering and state handling. If a malformed session's `firstKeptEntryId` is not found before the compaction entry, step 2 contributes no entries.
438
451
 
439
- `buildSessionContext()` returns a `SessionContext`, not provider-ready `Message[]` directly:
452
+ `buildSessionProjection()` applies the latest `context_edit` for each selected target and returns the model-visible messages together with their source entries (`ProjectedSessionEntry`). Omitted targets produce no message; replacements retain the source entry's role and metadata while changing only content. The raw selected entries are not modified. This projection is what `AgentSession` sends to the provider; `agent.state.messages` is only a refreshed copy of it.
453
+
454
+ `buildSessionContext()` builds on that projection and returns a `SessionContext`, not provider-ready `Message[]` directly:
440
455
 
441
456
  1. It determines `thinkingLevel` and `model` from the entire active path, including entries omitted by compaction. A `model_change` sets the model, and an assistant message later on the path sets it from that message's `provider` and `model`.
442
457
  2. It projects the selected entries to `AgentMessage[]` in the order above:
@@ -444,6 +459,7 @@ Earlier summarized entries are omitted. Non-message entries in the selected rang
444
459
  - `compaction` → its complete system checkpoint followed by a `CompactionSummaryMessage`. Pre-compaction system messages are folded into that checkpoint rather than replayed from the retained range.
445
460
  - `branch_summary` → `BranchSummaryMessage`.
446
461
  - `custom_message` → `CustomMessage`.
462
+ - `context_edit` → no message of its own; it changes the target's contribution.
447
463
  - `usage`, `custom`, model/thinking changes, labels, and session-info entries → no message.
448
464
 
449
465
  Use `convertToLlm()` from `@astralyn/pi` when provider-ready `Message[]` are required. It passes base user, assistant, and tool-result messages through; it converts Pi-specific context roles to user messages and omits `bashExecution` messages marked `excludeFromContext`.
@@ -480,6 +496,9 @@ for (const line of lines) {
480
496
  case "custom_message":
481
497
  console.log(`[${entry.id}] Extension message (${entry.customType}): ${entry.content}`);
482
498
  break;
499
+ case "context_edit":
500
+ console.log(`[${entry.id}] Context ${entry.replacement === null ? "omit" : "replace"}: ${entry.targetId}`);
501
+ break;
483
502
  case "label":
484
503
  console.log(`[${entry.id}] Label "${entry.label}" on ${entry.targetId}`);
485
504
  break;
@@ -526,11 +545,12 @@ All of these return the appended entry ID:
526
545
  - `appendMessage(message: Message | CustomMessage | BashExecutionMessage): string`
527
546
  - `appendThinkingLevelChange(thinkingLevel: string): string`
528
547
  - `appendModelChange(provider: string, modelId: string): string`
529
- - `appendCompaction<T = unknown>(summary: string, firstKeptEntryId: string, tokensBefore: number, details?: T, fromHook?: boolean, usage?: Usage): string`
548
+ - `appendCompaction<T = unknown>(summary: string, firstKeptEntryId: string | null, tokensBefore: number, details?: T, fromHook?: boolean, usage?: Usage): string` — `null` creates a retain-none compaction that records its own ID as the kept boundary.
530
549
  - `appendCustomEntry(customType: string, data?: unknown): string`
531
550
  - `appendSessionInfo(name: string): string`
532
551
  - `appendCustomMessageEntry<T = unknown>(customType: string, content: string | (TextContent | ImageContent)[], display: boolean, details?: T): string`
533
552
  - `appendLabelChange(targetId: string, label: string | undefined): string`
553
+ - `appendContextEdit(targetId: string, replacement: { content: ContextEditableContent } | null): string` — the target must be a user, assistant, tool-result, or custom-message entry on the active branch.
534
554
 
535
555
  ### Instance Methods — Tree Navigation
536
556
 
@@ -546,7 +566,8 @@ All of these return the appended entry ID:
546
566
  ### Instance Methods — Context and Info
547
567
 
548
568
  - `buildContextEntries(): SessionEntry[]` — get the compaction-aware active branch.
549
- - `buildSessionContext(): SessionContext` — get `AgentMessage[]`, thinking level, and model; use `convertToLlm()` for provider messages.
569
+ - `buildSessionProjection(): SessionProjection` — get the context-edited model-visible messages with their source entries, plus thinking level and model.
570
+ - `buildSessionContext(): SessionContext` — get `AgentMessage[]`, thinking level, and model from that projection; use `convertToLlm()` for provider messages.
550
571
  - `getEntries(): SessionEntry[]` and `getHeader(): SessionHeader | null`
551
572
  - `getSessionName(): string | undefined`
552
573
  - `getCwd(): string`, `getSessionDir(): string`, `getSessionId(): string`, and `getSessionFile(): string | undefined`
package/docs/usage.md CHANGED
@@ -168,10 +168,13 @@ See [Pi Packages](packages.md) for package sources and security notes.
168
168
  |------|-------------|
169
169
  | default | Interactive mode |
170
170
  | `-p`, `--print` | Print response and exit |
171
+ | `--mode text` | Text output; still opens the terminal UI when stdin and stdout are terminals |
171
172
  | `--mode json` | Output all events as JSON lines; see [JSON mode](json.md) |
172
173
  | `--mode rpc` | RPC mode over stdin/stdout; see [RPC mode](rpc.md) |
173
174
  | `--export <in> [out]` | Export a session to HTML |
174
175
 
176
+ `--mode` accepts only `text`, `json`, or `rpc`. A missing or invalid value is reported as an error and pi exits with a nonzero status instead of falling back to another mode.
177
+
175
178
  In print mode, pi also reads piped stdin and merges it into the initial prompt:
176
179
 
177
180
  ```bash