bunnyquery 1.3.5 → 1.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +76 -7
- package/bunnyquery.js +137 -44
- package/dist/engine.cjs +135 -40
- package/dist/engine.cjs.map +1 -1
- package/dist/engine.d.mts +70 -1
- package/dist/engine.d.ts +70 -1
- package/dist/engine.mjs +130 -41
- package/dist/engine.mjs.map +1 -1
- package/package.json +1 -1
- package/src/engine/attachment_parsers.ts +112 -0
- package/src/engine/config.ts +11 -0
- package/src/engine/index.ts +12 -0
- package/src/engine/office.ts +26 -4
- package/src/engine/prompts/indexing_user_message.ts +20 -0
- package/src/engine/requests.ts +17 -5
- package/src/engine/session.ts +9 -0
package/README.md
CHANGED
|
@@ -24,6 +24,9 @@ you can build your own chat UI on top of it. See
|
|
|
24
24
|
first load and on scroll-up.
|
|
25
25
|
- **Attachments** — drag-and-drop files and folders, per-file upload status
|
|
26
26
|
(uploading / failed / indexed), and overflow collapsing for large batches.
|
|
27
|
+
- **Attachment parser plugins** — register a client-side parser so the widget
|
|
28
|
+
extracts text in the browser from formats the model can't otherwise read, and
|
|
29
|
+
indexes it directly. See [Attachment parser plugins](#attachment-parser-plugins).
|
|
27
30
|
- **Settings panel** — in-place inside the chat: light/dark theme, account details,
|
|
28
31
|
newsletter subscription, clear history, and remove account.
|
|
29
32
|
- **Theming** — light and dark modes via CSS custom properties; the choice is
|
|
@@ -110,17 +113,19 @@ Mounts the widget. Returns the `BunnyQuery` object.
|
|
|
110
113
|
| `dev` | `boolean` | `false` | Use the development MCP host and `skapi.app` db-CDN host instead of production. |
|
|
111
114
|
| `mcpBaseUrl` | `string` | `null` | Override the MCP OAuth server base URL entirely (advanced). |
|
|
112
115
|
| `hostDomain` | `string` | `null` | db-CDN host for temporary file URLs. Defaults to `skapi.app` (dev) / `skapi.com` (prod). |
|
|
116
|
+
| `attachmentParsers` | `array` | `null` | Client-side attachment parsers. See [Attachment parser plugins](#attachment-parser-plugins). |
|
|
113
117
|
|
|
114
118
|
### Methods
|
|
115
119
|
|
|
116
120
|
The `BunnyQuery` global also exposes:
|
|
117
121
|
|
|
118
|
-
| Method
|
|
119
|
-
|
|
|
120
|
-
| `setTheme(theme)`
|
|
121
|
-
| `toggleTheme()`
|
|
122
|
-
| `logout()`
|
|
123
|
-
| `
|
|
122
|
+
| Method | Description |
|
|
123
|
+
| ---------------------------------- | ----------------------------------------------------------------------------------- |
|
|
124
|
+
| `setTheme(theme)` | Apply `"light"` or `"dark"` and persist it. |
|
|
125
|
+
| `toggleTheme()` | Switch between light and dark. |
|
|
126
|
+
| `logout()` | Sign the current user out and return to the login view. |
|
|
127
|
+
| `registerAttachmentParser(parser)` | Register a client-side attachment parser. May be called before or after `init()`. See [Attachment parser plugins](#attachment-parser-plugins). |
|
|
128
|
+
| `version` | The widget's package version string. Also logged to the console on `init()`. |
|
|
124
129
|
|
|
125
130
|
```js
|
|
126
131
|
BunnyQuery.setTheme("dark");
|
|
@@ -129,7 +134,71 @@ BunnyQuery.logout();
|
|
|
129
134
|
```
|
|
130
135
|
|
|
131
136
|
> `init()` is idempotent — calling it twice logs a warning and returns the existing
|
|
132
|
-
> instance rather than re-mounting.
|
|
137
|
+
> instance rather than re-mounting. On a successful mount it logs its version, e.g.
|
|
138
|
+
> `[bunnyquery] v1.3.5`.
|
|
139
|
+
|
|
140
|
+
## Attachment parser plugins
|
|
141
|
+
|
|
142
|
+
By default the chat agent reads most uploads via the model's `web_fetch` tool
|
|
143
|
+
(text, CSV, PDF, …), and Office/OpenDocument/EPUB files are extracted on the
|
|
144
|
+
server. For any format that can be read by **neither** (e.g. a proprietary
|
|
145
|
+
binary format), register a **parser plugin**: it runs in the browser, turns the
|
|
146
|
+
uploaded file into text (or an HTML string), and the widget sends that content
|
|
147
|
+
**inline** for indexing — no `web_fetch`, no server extraction for that file.
|
|
148
|
+
|
|
149
|
+
BunnyQuery ships only the **mechanism**. You bring the parsing library (so the
|
|
150
|
+
widget stays lean and you choose which formats and which library).
|
|
151
|
+
|
|
152
|
+
A parser is a plain object:
|
|
153
|
+
|
|
154
|
+
```ts
|
|
155
|
+
interface AttachmentParser {
|
|
156
|
+
name?: string; // label, used in logs
|
|
157
|
+
match: (file: { name: string; mime?: string }) => boolean; // handle this file?
|
|
158
|
+
parse: (file: File) => string | null | undefined | Promise<string | null | undefined>; // text or HTML; falsy = skip
|
|
159
|
+
}
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
The first parser whose `match` returns `true` wins. A parser that throws or
|
|
163
|
+
returns nothing is ignored — the file falls back to its normal path. Output is
|
|
164
|
+
capped (~200k chars) before it is inlined.
|
|
165
|
+
|
|
166
|
+
### Example
|
|
167
|
+
|
|
168
|
+
Load whatever parsing library reads your format, then register a parser that
|
|
169
|
+
turns a `File` into text:
|
|
170
|
+
|
|
171
|
+
```html
|
|
172
|
+
<!-- bring your own parsing library, e.g. from a CDN -->
|
|
173
|
+
<script src="https://cdn.example.com/my-format-parser.js"></script>
|
|
174
|
+
<script>
|
|
175
|
+
BunnyQuery.registerAttachmentParser({
|
|
176
|
+
name: "my-format",
|
|
177
|
+
match: (file) => /\.myext$/i.test(file.name),
|
|
178
|
+
parse: async (file) => {
|
|
179
|
+
const bytes = new Uint8Array(await file.arrayBuffer());
|
|
180
|
+
return window.myFormatParser.toText(bytes); // return plain text OR an HTML string
|
|
181
|
+
},
|
|
182
|
+
});
|
|
183
|
+
|
|
184
|
+
BunnyQuery.init(skapi, "chatbox", { theme: "light" });
|
|
185
|
+
</script>
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
Equivalent one-shot form via init options:
|
|
189
|
+
|
|
190
|
+
```js
|
|
191
|
+
BunnyQuery.init(skapi, "chatbox", {
|
|
192
|
+
attachmentParsers: [ myParser ],
|
|
193
|
+
});
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
Bundler consumers can import the same registry from the engine:
|
|
197
|
+
|
|
198
|
+
```js
|
|
199
|
+
import { registerAttachmentParser } from "bunnyquery/engine";
|
|
200
|
+
registerAttachmentParser(myParser);
|
|
201
|
+
```
|
|
133
202
|
|
|
134
203
|
## Theming
|
|
135
204
|
|
package/bunnyquery.js
CHANGED
|
@@ -1,10 +1,52 @@
|
|
|
1
1
|
(function () {
|
|
2
2
|
'use strict';
|
|
3
3
|
|
|
4
|
+
// src/engine/attachment_parsers.ts
|
|
5
|
+
var MAX_PARSED_CONTENT_CHARS = 2e5;
|
|
6
|
+
var _parsers = [];
|
|
7
|
+
function registerAttachmentParser(parser) {
|
|
8
|
+
if (parser && typeof parser.match === "function" && typeof parser.parse === "function" && _parsers.indexOf(parser) === -1) {
|
|
9
|
+
_parsers.push(parser);
|
|
10
|
+
}
|
|
11
|
+
}
|
|
12
|
+
function findAttachmentParser(name, mime) {
|
|
13
|
+
for (let i = 0; i < _parsers.length; i++) {
|
|
14
|
+
try {
|
|
15
|
+
if (_parsers[i].match({ name, mime })) return _parsers[i];
|
|
16
|
+
} catch {
|
|
17
|
+
}
|
|
18
|
+
}
|
|
19
|
+
return void 0;
|
|
20
|
+
}
|
|
21
|
+
async function parseAttachmentContent(file, name, mime) {
|
|
22
|
+
const parser = findAttachmentParser(name, mime);
|
|
23
|
+
if (!parser) return null;
|
|
24
|
+
let raw;
|
|
25
|
+
try {
|
|
26
|
+
raw = await parser.parse(file);
|
|
27
|
+
} catch (err) {
|
|
28
|
+
console.error(
|
|
29
|
+
`[chat-engine] attachment parser ${parser.name || "(unnamed)"} failed for ${name}:`,
|
|
30
|
+
err
|
|
31
|
+
);
|
|
32
|
+
return null;
|
|
33
|
+
}
|
|
34
|
+
let text = (raw == null ? "" : String(raw)).trim();
|
|
35
|
+
if (!text) return null;
|
|
36
|
+
if (text.length > MAX_PARSED_CONTENT_CHARS) {
|
|
37
|
+
text = text.slice(0, MAX_PARSED_CONTENT_CHARS) + `
|
|
38
|
+
...[truncated for length; original ${text.length} characters]`;
|
|
39
|
+
}
|
|
40
|
+
return text;
|
|
41
|
+
}
|
|
42
|
+
|
|
4
43
|
// src/engine/config.ts
|
|
5
44
|
var _config = null;
|
|
6
45
|
function configureChatEngine(config) {
|
|
7
46
|
_config = config;
|
|
47
|
+
if (config.attachmentParsers) {
|
|
48
|
+
for (const parser of config.attachmentParsers) registerAttachmentParser(parser);
|
|
49
|
+
}
|
|
8
50
|
}
|
|
9
51
|
function chatEngineConfig() {
|
|
10
52
|
if (!_config) {
|
|
@@ -31,13 +73,41 @@
|
|
|
31
73
|
"pptx",
|
|
32
74
|
"pptm",
|
|
33
75
|
"hwp",
|
|
34
|
-
"hwpx"
|
|
76
|
+
"hwpx",
|
|
77
|
+
"ods",
|
|
78
|
+
"odt",
|
|
79
|
+
"odp",
|
|
80
|
+
"epub"
|
|
81
|
+
]);
|
|
82
|
+
var WEB_FETCHABLE_TEXT_EXTENSIONS = /* @__PURE__ */ new Set([
|
|
83
|
+
"csv",
|
|
84
|
+
"tsv",
|
|
85
|
+
"tab",
|
|
86
|
+
"txt",
|
|
87
|
+
"text",
|
|
88
|
+
"md",
|
|
89
|
+
"markdown",
|
|
90
|
+
"json",
|
|
91
|
+
"ndjson",
|
|
92
|
+
"jsonl",
|
|
93
|
+
"xml",
|
|
94
|
+
"yaml",
|
|
95
|
+
"yml",
|
|
96
|
+
"log",
|
|
97
|
+
// RTF is a TEXT format (not a binary zip), so web_fetch can read it and the
|
|
98
|
+
// model decodes its control words. Pin it here so a `.rtf` reported as
|
|
99
|
+
// `application/msword` isn't misrouted to server-side extraction (which has no
|
|
100
|
+
// .rtf extractor → "unsupported format" note).
|
|
101
|
+
"rtf",
|
|
102
|
+
"htm",
|
|
103
|
+
"html"
|
|
35
104
|
]);
|
|
36
105
|
function isOfficeFile(name, mime) {
|
|
37
106
|
const ext = (name || "").split(".").pop()?.toLowerCase() || "";
|
|
38
107
|
if (OFFICE_FILE_EXTENSIONS.has(ext)) return true;
|
|
108
|
+
if (WEB_FETCHABLE_TEXT_EXTENSIONS.has(ext)) return false;
|
|
39
109
|
const m = (mime || "").toLowerCase();
|
|
40
|
-
return m.includes("officedocument") || m.includes("hwp") || m === "application/msword" || m === "application/vnd.ms-excel" || m === "application/vnd.ms-powerpoint";
|
|
110
|
+
return m.includes("officedocument") || m.includes("opendocument") || m.includes("hwp") || m.includes("epub") || m === "application/msword" || m === "application/vnd.ms-excel" || m === "application/vnd.ms-powerpoint";
|
|
41
111
|
}
|
|
42
112
|
var _extractPlaceholderSeq = 0;
|
|
43
113
|
function makeExtractPlaceholder(seed) {
|
|
@@ -158,6 +228,14 @@ File metadata:
|
|
|
158
228
|
` + (attachment.mime ? `- mime type: ${attachment.mime}
|
|
159
229
|
` : "") + (typeof attachment.size === "number" ? `- size (bytes): ${attachment.size}
|
|
160
230
|
` : "");
|
|
231
|
+
if (options?.inlineContent) {
|
|
232
|
+
return head + `
|
|
233
|
+
The file's content was parsed by the client and is provided inline below. Read it directly \u2014 do NOT fetch any URL for this file. Use the storage path above (not this content) for the "src::" unique_id.
|
|
234
|
+
|
|
235
|
+
----- BEGIN FILE CONTENT -----
|
|
236
|
+
${options.inlineContent}
|
|
237
|
+
----- END FILE CONTENT -----`;
|
|
238
|
+
}
|
|
161
239
|
if (options?.inlineContentPlaceholder) {
|
|
162
240
|
return head + `
|
|
163
241
|
The file's text content was extracted on the server and is provided inline below. Read it directly \u2014 do NOT fetch any URL for this file. Use the storage path above (not this content) for the "src::" unique_id.
|
|
@@ -653,14 +731,14 @@ ${options.inlineContentPlaceholder}
|
|
|
653
731
|
});
|
|
654
732
|
}
|
|
655
733
|
async function notifyAgentSaveAttachment(info) {
|
|
656
|
-
const { platform, service, owner, attachment } = info;
|
|
657
|
-
const office = isOfficeFile(attachment.name, attachment.mime);
|
|
734
|
+
const { platform, service, owner, attachment, parsedContent } = info;
|
|
735
|
+
const office = !parsedContent && isOfficeFile(attachment.name, attachment.mime);
|
|
658
736
|
const placeholder = office ? makeExtractPlaceholder(attachment.storagePath) : void 0;
|
|
659
737
|
const extractContent = office && placeholder ? [{ path: attachment.storagePath, placeholder, name: attachment.name, mime: attachment.mime }] : void 0;
|
|
660
738
|
const skapiExtract = extractContent && extractContent.length ? { _skapi_extract: extractContent } : {};
|
|
661
739
|
const userMessage = buildIndexingUserMessage(
|
|
662
740
|
attachment,
|
|
663
|
-
placeholder ? { inlineContentPlaceholder: placeholder } : void 0
|
|
741
|
+
parsedContent ? { inlineContent: parsedContent } : placeholder ? { inlineContentPlaceholder: placeholder } : void 0
|
|
664
742
|
);
|
|
665
743
|
const systemPrompt = buildIndexingSystemPrompt({
|
|
666
744
|
service,
|
|
@@ -1837,44 +1915,49 @@ ${options.inlineContentPlaceholder}
|
|
|
1837
1915
|
att.storagePath = member.storagePath;
|
|
1838
1916
|
}
|
|
1839
1917
|
var mime = member.file.type || self.host.getMimeType(member.file.name);
|
|
1840
|
-
return
|
|
1841
|
-
|
|
1842
|
-
|
|
1843
|
-
|
|
1844
|
-
|
|
1845
|
-
|
|
1846
|
-
|
|
1847
|
-
|
|
1848
|
-
|
|
1849
|
-
|
|
1850
|
-
|
|
1851
|
-
|
|
1852
|
-
|
|
1853
|
-
url
|
|
1854
|
-
}
|
|
1855
|
-
}).then(function(ack) {
|
|
1856
|
-
if (ack && typeof ack.id === "string") {
|
|
1857
|
-
self.bgTaskQueue.push({
|
|
1858
|
-
serviceId: id.serviceId,
|
|
1859
|
-
platform: id.platform,
|
|
1860
|
-
id: ack.id,
|
|
1861
|
-
filename: member.file.name,
|
|
1918
|
+
return Promise.resolve(
|
|
1919
|
+
parseAttachmentContent(member.file, member.file.name, mime || void 0)
|
|
1920
|
+
).then(function(parsedContent) {
|
|
1921
|
+
return notifyAgentSaveAttachment({
|
|
1922
|
+
platform: id.platform,
|
|
1923
|
+
model: id.model,
|
|
1924
|
+
service: id.serviceId,
|
|
1925
|
+
owner: id.owner,
|
|
1926
|
+
userId: id.userId || id.serviceId,
|
|
1927
|
+
serviceName: id.serviceName,
|
|
1928
|
+
serviceDescription: id.serviceDescription,
|
|
1929
|
+
attachment: {
|
|
1930
|
+
name: member.file.name,
|
|
1862
1931
|
storagePath: member.storagePath,
|
|
1863
|
-
isReindex: hadExists,
|
|
1864
1932
|
mime: mime || void 0,
|
|
1865
1933
|
size: member.file.size,
|
|
1866
|
-
|
|
1867
|
-
|
|
1868
|
-
|
|
1869
|
-
|
|
1870
|
-
|
|
1871
|
-
|
|
1872
|
-
|
|
1873
|
-
|
|
1874
|
-
|
|
1875
|
-
|
|
1876
|
-
|
|
1877
|
-
|
|
1934
|
+
url
|
|
1935
|
+
},
|
|
1936
|
+
parsedContent: parsedContent || void 0
|
|
1937
|
+
}).then(function(ack) {
|
|
1938
|
+
if (ack && typeof ack.id === "string") {
|
|
1939
|
+
self.bgTaskQueue.push({
|
|
1940
|
+
serviceId: id.serviceId,
|
|
1941
|
+
platform: id.platform,
|
|
1942
|
+
id: ack.id,
|
|
1943
|
+
filename: member.file.name,
|
|
1944
|
+
storagePath: member.storagePath,
|
|
1945
|
+
isReindex: hadExists,
|
|
1946
|
+
mime: mime || void 0,
|
|
1947
|
+
size: member.file.size,
|
|
1948
|
+
status: ack.status === "running" ? "running" : "pending",
|
|
1949
|
+
poll: ack.poll
|
|
1950
|
+
});
|
|
1951
|
+
self.drainBgTaskQueue();
|
|
1952
|
+
}
|
|
1953
|
+
}, function(e) {
|
|
1954
|
+
console.error("[chat-engine] indexing request failed", e);
|
|
1955
|
+
anyIndexFailed = true;
|
|
1956
|
+
if (!att.errorCode && !att.errorDetail) {
|
|
1957
|
+
att.errorCode = e && (e.code || e.body && e.body.code) || "";
|
|
1958
|
+
att.errorDetail = e && (e.message || e.body && e.body.message) || (typeof e === "string" ? e : "");
|
|
1959
|
+
}
|
|
1960
|
+
});
|
|
1878
1961
|
});
|
|
1879
1962
|
});
|
|
1880
1963
|
});
|
|
@@ -1958,6 +2041,7 @@ ${options.inlineContentPlaceholder}
|
|
|
1958
2041
|
(function() {
|
|
1959
2042
|
var MCP_PROD = "https://mcp.broadwayinc.computer";
|
|
1960
2043
|
var MCP_DEV = "https://mcp-dev.broadwayinc.computer";
|
|
2044
|
+
var BQ_VERSION = "1.4.0" ;
|
|
1961
2045
|
var ATTACHMENT_URL_EXPIRES_SECONDS = 600;
|
|
1962
2046
|
var GOOGLE_AUTH_URL = "https://accounts.google.com/o/oauth2/v2/auth";
|
|
1963
2047
|
var GOOGLE_TOKEN_URL = "https://oauth2.googleapis.com/token";
|
|
@@ -2179,7 +2263,7 @@ ${options.inlineContentPlaceholder}
|
|
|
2179
2263
|
if (typeof S.skapi.getConnectionInfo === "function") return S.skapi.getConnectionInfo();
|
|
2180
2264
|
return S.skapi.connection || null;
|
|
2181
2265
|
}).then(function(conn) {
|
|
2182
|
-
console.log("[bunnyquery] loadServiceInfo", conn);
|
|
2266
|
+
if (S.opts && S.opts.dev) console.log("[bunnyquery] loadServiceInfo", conn);
|
|
2183
2267
|
if (conn) {
|
|
2184
2268
|
S.serviceId = conn.service || S.serviceId;
|
|
2185
2269
|
S.owner = conn.owner || S.owner;
|
|
@@ -5001,8 +5085,10 @@ ${options.inlineContentPlaceholder}
|
|
|
5001
5085
|
googleClientSecretName: "ggl",
|
|
5002
5086
|
signupConfirmationUrl: null,
|
|
5003
5087
|
// defaults to current host page
|
|
5004
|
-
hostDomain: null
|
|
5088
|
+
hostDomain: null,
|
|
5005
5089
|
// db-CDN host; null → skapi.app (dev) / skapi.com (prod)
|
|
5090
|
+
attachmentParsers: null
|
|
5091
|
+
// client-side attachment parsers, e.g. [createHwpParser()]
|
|
5006
5092
|
}, opts || {});
|
|
5007
5093
|
S.mountEl = mountEl;
|
|
5008
5094
|
clear(mountEl);
|
|
@@ -5010,6 +5096,7 @@ ${options.inlineContentPlaceholder}
|
|
|
5010
5096
|
mountEl.appendChild(S.root);
|
|
5011
5097
|
applyTheme(loadTheme());
|
|
5012
5098
|
S.booted = true;
|
|
5099
|
+
console.log("[bunnyquery] v" + BQ_VERSION);
|
|
5013
5100
|
configureChatEngine({
|
|
5014
5101
|
clientSecretRequest: function(o) {
|
|
5015
5102
|
return S.skapi.clientSecretRequest(o);
|
|
@@ -5018,7 +5105,9 @@ ${options.inlineContentPlaceholder}
|
|
|
5018
5105
|
return S.skapi.clientSecretRequestHistory(p, f);
|
|
5019
5106
|
},
|
|
5020
5107
|
mcpBaseUrl: mcpBaseUrl(),
|
|
5021
|
-
poll: 0
|
|
5108
|
+
poll: 0,
|
|
5109
|
+
// Client-side attachment parsers (e.g. an .hwp parser) passed via init opts.
|
|
5110
|
+
attachmentParsers: S.opts.attachmentParsers || void 0
|
|
5022
5111
|
});
|
|
5023
5112
|
if (!S._resizeBound && typeof window !== "undefined" && window.addEventListener) {
|
|
5024
5113
|
S._resizeBound = true;
|
|
@@ -5031,12 +5120,16 @@ ${options.inlineContentPlaceholder}
|
|
|
5031
5120
|
}
|
|
5032
5121
|
var PUBLIC = {
|
|
5033
5122
|
init,
|
|
5123
|
+
// Register a client-side attachment parser (e.g. createHwpParser()) so the
|
|
5124
|
+
// widget parses matching uploads in-browser and sends the text for indexing.
|
|
5125
|
+
// Can be called before or after init(); also settable via init opts.attachmentParsers.
|
|
5126
|
+
registerAttachmentParser,
|
|
5034
5127
|
setTheme: function(t) {
|
|
5035
5128
|
applyTheme(t);
|
|
5036
5129
|
},
|
|
5037
5130
|
toggleTheme,
|
|
5038
5131
|
logout,
|
|
5039
|
-
version:
|
|
5132
|
+
version: BQ_VERSION,
|
|
5040
5133
|
_state: S
|
|
5041
5134
|
// exposed for later-phase modules / debugging
|
|
5042
5135
|
};
|
package/dist/engine.cjs
CHANGED
|
@@ -1,9 +1,57 @@
|
|
|
1
1
|
'use strict';
|
|
2
2
|
|
|
3
|
+
// src/engine/attachment_parsers.ts
|
|
4
|
+
var MAX_PARSED_CONTENT_CHARS = 2e5;
|
|
5
|
+
var _parsers = [];
|
|
6
|
+
function registerAttachmentParser(parser) {
|
|
7
|
+
if (parser && typeof parser.match === "function" && typeof parser.parse === "function" && _parsers.indexOf(parser) === -1) {
|
|
8
|
+
_parsers.push(parser);
|
|
9
|
+
}
|
|
10
|
+
}
|
|
11
|
+
function clearAttachmentParsers() {
|
|
12
|
+
_parsers.length = 0;
|
|
13
|
+
}
|
|
14
|
+
function getAttachmentParsers() {
|
|
15
|
+
return _parsers.slice();
|
|
16
|
+
}
|
|
17
|
+
function findAttachmentParser(name, mime) {
|
|
18
|
+
for (let i = 0; i < _parsers.length; i++) {
|
|
19
|
+
try {
|
|
20
|
+
if (_parsers[i].match({ name, mime })) return _parsers[i];
|
|
21
|
+
} catch {
|
|
22
|
+
}
|
|
23
|
+
}
|
|
24
|
+
return void 0;
|
|
25
|
+
}
|
|
26
|
+
async function parseAttachmentContent(file, name, mime) {
|
|
27
|
+
const parser = findAttachmentParser(name, mime);
|
|
28
|
+
if (!parser) return null;
|
|
29
|
+
let raw;
|
|
30
|
+
try {
|
|
31
|
+
raw = await parser.parse(file);
|
|
32
|
+
} catch (err) {
|
|
33
|
+
console.error(
|
|
34
|
+
`[chat-engine] attachment parser ${parser.name || "(unnamed)"} failed for ${name}:`,
|
|
35
|
+
err
|
|
36
|
+
);
|
|
37
|
+
return null;
|
|
38
|
+
}
|
|
39
|
+
let text = (raw == null ? "" : String(raw)).trim();
|
|
40
|
+
if (!text) return null;
|
|
41
|
+
if (text.length > MAX_PARSED_CONTENT_CHARS) {
|
|
42
|
+
text = text.slice(0, MAX_PARSED_CONTENT_CHARS) + `
|
|
43
|
+
...[truncated for length; original ${text.length} characters]`;
|
|
44
|
+
}
|
|
45
|
+
return text;
|
|
46
|
+
}
|
|
47
|
+
|
|
3
48
|
// src/engine/config.ts
|
|
4
49
|
var _config = null;
|
|
5
50
|
function configureChatEngine(config) {
|
|
6
51
|
_config = config;
|
|
52
|
+
if (config.attachmentParsers) {
|
|
53
|
+
for (const parser of config.attachmentParsers) registerAttachmentParser(parser);
|
|
54
|
+
}
|
|
7
55
|
}
|
|
8
56
|
function chatEngineConfig() {
|
|
9
57
|
if (!_config) {
|
|
@@ -30,13 +78,41 @@ var OFFICE_FILE_EXTENSIONS = /* @__PURE__ */ new Set([
|
|
|
30
78
|
"pptx",
|
|
31
79
|
"pptm",
|
|
32
80
|
"hwp",
|
|
33
|
-
"hwpx"
|
|
81
|
+
"hwpx",
|
|
82
|
+
"ods",
|
|
83
|
+
"odt",
|
|
84
|
+
"odp",
|
|
85
|
+
"epub"
|
|
86
|
+
]);
|
|
87
|
+
var WEB_FETCHABLE_TEXT_EXTENSIONS = /* @__PURE__ */ new Set([
|
|
88
|
+
"csv",
|
|
89
|
+
"tsv",
|
|
90
|
+
"tab",
|
|
91
|
+
"txt",
|
|
92
|
+
"text",
|
|
93
|
+
"md",
|
|
94
|
+
"markdown",
|
|
95
|
+
"json",
|
|
96
|
+
"ndjson",
|
|
97
|
+
"jsonl",
|
|
98
|
+
"xml",
|
|
99
|
+
"yaml",
|
|
100
|
+
"yml",
|
|
101
|
+
"log",
|
|
102
|
+
// RTF is a TEXT format (not a binary zip), so web_fetch can read it and the
|
|
103
|
+
// model decodes its control words. Pin it here so a `.rtf` reported as
|
|
104
|
+
// `application/msword` isn't misrouted to server-side extraction (which has no
|
|
105
|
+
// .rtf extractor → "unsupported format" note).
|
|
106
|
+
"rtf",
|
|
107
|
+
"htm",
|
|
108
|
+
"html"
|
|
34
109
|
]);
|
|
35
110
|
function isOfficeFile(name, mime) {
|
|
36
111
|
const ext = (name || "").split(".").pop()?.toLowerCase() || "";
|
|
37
112
|
if (OFFICE_FILE_EXTENSIONS.has(ext)) return true;
|
|
113
|
+
if (WEB_FETCHABLE_TEXT_EXTENSIONS.has(ext)) return false;
|
|
38
114
|
const m = (mime || "").toLowerCase();
|
|
39
|
-
return m.includes("officedocument") || m.includes("hwp") || m === "application/msword" || m === "application/vnd.ms-excel" || m === "application/vnd.ms-powerpoint";
|
|
115
|
+
return m.includes("officedocument") || m.includes("opendocument") || m.includes("hwp") || m.includes("epub") || m === "application/msword" || m === "application/vnd.ms-excel" || m === "application/vnd.ms-powerpoint";
|
|
40
116
|
}
|
|
41
117
|
var _extractPlaceholderSeq = 0;
|
|
42
118
|
function makeExtractPlaceholder(seed) {
|
|
@@ -157,6 +233,14 @@ File metadata:
|
|
|
157
233
|
` + (attachment.mime ? `- mime type: ${attachment.mime}
|
|
158
234
|
` : "") + (typeof attachment.size === "number" ? `- size (bytes): ${attachment.size}
|
|
159
235
|
` : "");
|
|
236
|
+
if (options?.inlineContent) {
|
|
237
|
+
return head + `
|
|
238
|
+
The file's content was parsed by the client and is provided inline below. Read it directly \u2014 do NOT fetch any URL for this file. Use the storage path above (not this content) for the "src::" unique_id.
|
|
239
|
+
|
|
240
|
+
----- BEGIN FILE CONTENT -----
|
|
241
|
+
${options.inlineContent}
|
|
242
|
+
----- END FILE CONTENT -----`;
|
|
243
|
+
}
|
|
160
244
|
if (options?.inlineContentPlaceholder) {
|
|
161
245
|
return head + `
|
|
162
246
|
The file's text content was extracted on the server and is provided inline below. Read it directly \u2014 do NOT fetch any URL for this file. Use the storage path above (not this content) for the "src::" unique_id.
|
|
@@ -655,14 +739,14 @@ async function callOpenAIWithPublicMcp(prompt, service, owner, messages, system,
|
|
|
655
739
|
});
|
|
656
740
|
}
|
|
657
741
|
async function notifyAgentSaveAttachment(info) {
|
|
658
|
-
const { platform, service, owner, attachment } = info;
|
|
659
|
-
const office = isOfficeFile(attachment.name, attachment.mime);
|
|
742
|
+
const { platform, service, owner, attachment, parsedContent } = info;
|
|
743
|
+
const office = !parsedContent && isOfficeFile(attachment.name, attachment.mime);
|
|
660
744
|
const placeholder = office ? makeExtractPlaceholder(attachment.storagePath) : void 0;
|
|
661
745
|
const extractContent = office && placeholder ? [{ path: attachment.storagePath, placeholder, name: attachment.name, mime: attachment.mime }] : void 0;
|
|
662
746
|
const skapiExtract = extractContent && extractContent.length ? { _skapi_extract: extractContent } : {};
|
|
663
747
|
const userMessage = buildIndexingUserMessage(
|
|
664
748
|
attachment,
|
|
665
|
-
placeholder ? { inlineContentPlaceholder: placeholder } : void 0
|
|
749
|
+
parsedContent ? { inlineContent: parsedContent } : placeholder ? { inlineContentPlaceholder: placeholder } : void 0
|
|
666
750
|
);
|
|
667
751
|
const systemPrompt = buildIndexingSystemPrompt({
|
|
668
752
|
service,
|
|
@@ -1864,44 +1948,49 @@ var ChatSession = class {
|
|
|
1864
1948
|
att.storagePath = member.storagePath;
|
|
1865
1949
|
}
|
|
1866
1950
|
var mime = member.file.type || self.host.getMimeType(member.file.name);
|
|
1867
|
-
return
|
|
1868
|
-
|
|
1869
|
-
|
|
1870
|
-
|
|
1871
|
-
|
|
1872
|
-
|
|
1873
|
-
|
|
1874
|
-
|
|
1875
|
-
|
|
1876
|
-
|
|
1877
|
-
|
|
1878
|
-
|
|
1879
|
-
|
|
1880
|
-
url
|
|
1881
|
-
}
|
|
1882
|
-
}).then(function(ack) {
|
|
1883
|
-
if (ack && typeof ack.id === "string") {
|
|
1884
|
-
self.bgTaskQueue.push({
|
|
1885
|
-
serviceId: id.serviceId,
|
|
1886
|
-
platform: id.platform,
|
|
1887
|
-
id: ack.id,
|
|
1888
|
-
filename: member.file.name,
|
|
1951
|
+
return Promise.resolve(
|
|
1952
|
+
parseAttachmentContent(member.file, member.file.name, mime || void 0)
|
|
1953
|
+
).then(function(parsedContent) {
|
|
1954
|
+
return notifyAgentSaveAttachment({
|
|
1955
|
+
platform: id.platform,
|
|
1956
|
+
model: id.model,
|
|
1957
|
+
service: id.serviceId,
|
|
1958
|
+
owner: id.owner,
|
|
1959
|
+
userId: id.userId || id.serviceId,
|
|
1960
|
+
serviceName: id.serviceName,
|
|
1961
|
+
serviceDescription: id.serviceDescription,
|
|
1962
|
+
attachment: {
|
|
1963
|
+
name: member.file.name,
|
|
1889
1964
|
storagePath: member.storagePath,
|
|
1890
|
-
isReindex: hadExists,
|
|
1891
1965
|
mime: mime || void 0,
|
|
1892
1966
|
size: member.file.size,
|
|
1893
|
-
|
|
1894
|
-
|
|
1895
|
-
|
|
1896
|
-
|
|
1897
|
-
|
|
1898
|
-
|
|
1899
|
-
|
|
1900
|
-
|
|
1901
|
-
|
|
1902
|
-
|
|
1903
|
-
|
|
1904
|
-
|
|
1967
|
+
url
|
|
1968
|
+
},
|
|
1969
|
+
parsedContent: parsedContent || void 0
|
|
1970
|
+
}).then(function(ack) {
|
|
1971
|
+
if (ack && typeof ack.id === "string") {
|
|
1972
|
+
self.bgTaskQueue.push({
|
|
1973
|
+
serviceId: id.serviceId,
|
|
1974
|
+
platform: id.platform,
|
|
1975
|
+
id: ack.id,
|
|
1976
|
+
filename: member.file.name,
|
|
1977
|
+
storagePath: member.storagePath,
|
|
1978
|
+
isReindex: hadExists,
|
|
1979
|
+
mime: mime || void 0,
|
|
1980
|
+
size: member.file.size,
|
|
1981
|
+
status: ack.status === "running" ? "running" : "pending",
|
|
1982
|
+
poll: ack.poll
|
|
1983
|
+
});
|
|
1984
|
+
self.drainBgTaskQueue();
|
|
1985
|
+
}
|
|
1986
|
+
}, function(e) {
|
|
1987
|
+
console.error("[chat-engine] indexing request failed", e);
|
|
1988
|
+
anyIndexFailed = true;
|
|
1989
|
+
if (!att.errorCode && !att.errorDetail) {
|
|
1990
|
+
att.errorCode = e && (e.code || e.body && e.body.code) || "";
|
|
1991
|
+
att.errorDetail = e && (e.message || e.body && e.body.message) || (typeof e === "string" ? e : "");
|
|
1992
|
+
}
|
|
1993
|
+
});
|
|
1905
1994
|
});
|
|
1906
1995
|
});
|
|
1907
1996
|
});
|
|
@@ -1993,6 +2082,7 @@ exports.EXPIRED_ATTACHMENT_URL_ORIGIN = EXPIRED_ATTACHMENT_URL_ORIGIN;
|
|
|
1993
2082
|
exports.HISTORY_TOKEN_BUDGET = HISTORY_TOKEN_BUDGET;
|
|
1994
2083
|
exports.LINK_LABEL_MAX_DISPLAY_CHARS = LINK_LABEL_MAX_DISPLAY_CHARS;
|
|
1995
2084
|
exports.MAX_HISTORY_MESSAGES = MAX_HISTORY_MESSAGES;
|
|
2085
|
+
exports.MAX_PARSED_CONTENT_CHARS = MAX_PARSED_CONTENT_CHARS;
|
|
1996
2086
|
exports.MCP_NAME = MCP_NAME;
|
|
1997
2087
|
exports.MIN_INPUT_TOKEN_BUDGET = MIN_INPUT_TOKEN_BUDGET;
|
|
1998
2088
|
exports.OUTPUT_TOKEN_RESERVE = OUTPUT_TOKEN_RESERVE;
|
|
@@ -2007,6 +2097,7 @@ exports.callClaudeWithMcp = callClaudeWithMcp;
|
|
|
2007
2097
|
exports.callClaudeWithPublicMcp = callClaudeWithPublicMcp;
|
|
2008
2098
|
exports.callOpenAIWithPublicMcp = callOpenAIWithPublicMcp;
|
|
2009
2099
|
exports.chatEngineConfig = chatEngineConfig;
|
|
2100
|
+
exports.clearAttachmentParsers = clearAttachmentParsers;
|
|
2010
2101
|
exports.composeUserMessage = composeUserMessage;
|
|
2011
2102
|
exports.configureChatEngine = configureChatEngine;
|
|
2012
2103
|
exports.createInlineLinkRegex = createInlineLinkRegex;
|
|
@@ -2018,6 +2109,8 @@ exports.extractLastUserTextFromRequest = extractLastUserTextFromRequest;
|
|
|
2018
2109
|
exports.extractOpenAIText = extractOpenAIText;
|
|
2019
2110
|
exports.extractRemotePathFromAttachmentHref = extractRemotePathFromAttachmentHref;
|
|
2020
2111
|
exports.filterListByClearHorizon = filterListByClearHorizon;
|
|
2112
|
+
exports.findAttachmentParser = findAttachmentParser;
|
|
2113
|
+
exports.getAttachmentParsers = getAttachmentParsers;
|
|
2021
2114
|
exports.getChatHistory = getChatHistory;
|
|
2022
2115
|
exports.getContextWindow = getContextWindow;
|
|
2023
2116
|
exports.getErrorMessage = getErrorMessage;
|
|
@@ -2034,6 +2127,8 @@ exports.mapHistoryListToMessages = mapHistoryListToMessages;
|
|
|
2034
2127
|
exports.normalizeAttachmentPathCandidate = normalizeAttachmentPathCandidate;
|
|
2035
2128
|
exports.normalizeTextContent = normalizeTextContent;
|
|
2036
2129
|
exports.notifyAgentSaveAttachment = notifyAgentSaveAttachment;
|
|
2130
|
+
exports.parseAttachmentContent = parseAttachmentContent;
|
|
2131
|
+
exports.registerAttachmentParser = registerAttachmentParser;
|
|
2037
2132
|
exports.safeDecodeURIComponent = safeDecodeURIComponent;
|
|
2038
2133
|
exports.sanitizeAttachmentLinksForHistory = sanitizeAttachmentLinksForHistory;
|
|
2039
2134
|
exports.stripFileBlocksFromHistory = stripFileBlocksFromHistory;
|