bunnyquery 1.15.0 → 1.15.1

package/README.md

@@ -140,7 +140,8 @@ Mounts the widget. Returns the `BunnyQuery` object.
 | `attachmentParsers`      | `array`   | `null`   | Client-side attachment parsers. See [Attachment parser plugins](#attachment-parser-plugins). |
 | `windowedIndexing`       | `boolean` | `true`   | Server-driven windowed indexing for text and grid files (see [file types](#supported-file-types)). Pass `false` to fall back to agent-driven paging, which keeps the traversal inside the model's turn budget and the tab open. |
 | `allowAnonymous`         | `boolean` | `null`   | Open the chat with no login for visitors without an account. `null` follows the project's own "Allow anonymous users" setting (`getConnectionInfo().conf.require_login`); `true`/`false` pins it. |
-| `webAccess`              | `boolean` | `null`   | Whether the chat's answers may reach the open web (Claude `web_fetch`, OpenAI `web_search`). `null` follows the project's own **Web access for answers** switch on its Project Settings page (the `web_access` key of the project's `bq::settings` record), which is **off** unless the owner turned it on: answers then come only from the project's files and database. `true`/`false` pins it for this page, both ways. Resolution order: this option, then the project setting, then off. Indexing is not affected. |
+| `webAccess`              | `boolean` | `null`   | Whether the chat's answers may reach the open web (Claude `web_fetch`, OpenAI `web_search`). The project's own **Web access for answers** switch on its Project Settings page (the `web_access` key of the project's `bq::settings` record, **off** unless the owner turned it on) is the **ceiling**, and this option may only restrict below it: `effective = projectSetting && webAccess !== false`. `null` or `true` follows the project; `false` turns the web off for this page whatever the project says; a value above the project setting has no effect. With the web off, answers come only from the project's files and database. Indexing is not affected. |
+| `skipDuplicateFiles`     | `boolean` | `null`   | Whether a chat attachment the signed-in user **already uploaded** to this project (same stored file name, same size in bytes, same uploader, found from the file's `src::` record) is skipped: it is neither uploaded nor indexed again, the chip reads "already uploaded", and the message uses the stored copy's url. The project's **Skip duplicate files** setting (the `skip_duplicate_files` key of `bq::settings`, **on** unless the owner turned it off) is the **ceiling**, and this option may only restrict below it: `effective = projectSetting && skipDuplicateFiles !== false`. `null` or `true` follows the project; `false` lets this page re-upload duplicates; a value above the project setting has no effect. The check never blocks an upload: a failed lookup uploads as usual. |
 | `liveStreaming`          | `boolean` | `true`   | Paint a chat answer into its bubble as it arrives, instead of at the end. On by default; pass `false` for buffered replies. Still a **request**, not a switch: the widget honours it only when your page's `skapi-js` actually carries skapi's half of the stream flag (it checks for `forwardRequestStream` and `forwardRequestFinalize`, or their older names `clientSecretRequestStream` and `clientSecretRequestFinalize` on an earlier `skapi-js`), and otherwise warns once and falls back to buffered replies. An older SDK silently drops the flag, which would leave the destination streaming SSE into a buffered row that reads back empty. It also needs a polling worker that relays the response bytes, which every deployed region now has; against a self-hosted worker that does not, pass `false`. |
 | `liveStreamingRealtime`  | `boolean` | `false`  | Deliver streamed chunks over skapi's websocket instead of waiting for the next poll tick. Requires `liveStreaming`. Off unless you ask for it: skapi's `joinRealtime` **replaces** the connection's group, so for the length of a turn it takes the room out from under whatever else your app uses realtime for. Purely an accelerator; with it off the reply still streams, on the poll's cadence. |
 
@@ -492,7 +493,8 @@ helpers. See the `.d.ts` shipped with `bunnyquery/engine`.
 | `attachmentParsers`           | `array?`   | Client-side attachment parsers, registered at configure time. More can be added later with `registerAttachmentParser()`. See [Attachment parser plugins](#attachment-parser-plugins). |
 | `windowedIndexing`            | `boolean?` | Opt in to **server-driven** windowed indexing for text and grid files (see [file types](#supported-file-types)). Off by default in the engine; the widget passes it as `true`. The deployed skapi workers support it; only leave it off against a self-hosted worker that does not yet strip the `_skapi_window` directive, where it would reach the provider as an unknown body field and fail the call terminally with no retry. |
 | `liveStreaming`               | `boolean?` | Opt in to **live streaming** of chat turns. Off by default in the engine (the widget passes it as `true`), and the backend ships first: a streamed row settles with a status and NO body (the answer was the stream), so against a worker that does not relay, the turn reads back empty. Pair it with `forwardRequestFinalize` and `forwardRequestStream`, and gate it on `skapiSupportsStreaming(skapi)`. |
-| `webAccess`                   | `boolean?` | Whether a chat turn may reach the open web: Claude's `web_fetch` and OpenAI's `web_search` are attached to a chat request only when this resolves to `true`. Leave it out to follow the project's own `web_access` setting (the `bq::settings` record, read through `configureProjectSettings`), which is **off** unless the project owner turned it on; a boolean here overrides the project for your host, both ways. `webAccessEnabled(serviceId)` and `resolveWebAccess(serviceId)` answer the resolved switch (sync from the cache, or awaiting the settings fetch). Indexing passes keep their tools regardless. Pass the same value to `buildChatSystemPrompt({ webAccess })` so the prompt matches the tool list. |
+| `webAccess`                   | `boolean?` | Whether a chat turn may reach the open web: Claude's `web_fetch` and OpenAI's `web_search` are attached to a chat request only when this resolves to `true`. The project's own `web_access` setting (the `bq::settings` record, read through `configureProjectSettings`; **off** unless the project owner turned it on) is the **ceiling** and this option may only restrict below it: `effective = projectSetting && webAccess !== false`. Leave it out or pass `true` to follow the project; `false` turns the web off for your host; a value above the project setting has no effect. `webAccessEnabled(serviceId)` and `resolveWebAccess(serviceId, override?)` answer the resolved switch (sync from the cache, or awaiting the settings fetch; `override` is a per-call host value that restricts the same way). Indexing passes keep their tools regardless. Pass the same value to `buildChatSystemPrompt({ webAccess })` so the prompt matches the tool list. |
+| `skipDuplicateFiles`          | `boolean?` | Whether an attachment the signed-in user already uploaded to the project (same stored name, same size, same uploader) is skipped by `ChatSession`'s upload path: not uploaded, not indexed, the turn using the stored copy's url. The project's `skip_duplicate_files` setting (**on** unless the owner turned it off) is the **ceiling** and this option may only restrict: `effective = projectSetting && skipDuplicateFiles !== false`. The check runs only when the host implements the optional `findDuplicateUpload(file)` hook; `findDuplicateUpload(skapi, { name, size }, userId, { service? })` from the barrel is the ready-made implementation (an ends-with `getUniqueId` lookup narrowed to `src::<name>` and `src::.../<name>`, then the first record whose `data.size_bytes` and `user_id` match; it never throws, a failed lookup answers `null` and the file uploads as usual). `skipDuplicateFilesEnabled(serviceId)` and `resolveSkipDuplicateFiles(serviceId)` answer the resolved switch. |
 | `forwardRequestFinalize`      | `function?` | `skapi.forwardRequestFinalize`, bound to your Skapi instance. Stores the version of a streamed turn that history keeps (the engine sends the assembled provider body, so it reads back exactly like a buffered turn) and releases that request's chunks. Without it a streamed turn is never finalized and its row stays empty. |
 | `forwardRequestStream`        | `function?` | `skapi.forwardRequestStream`, bound to your Skapi instance. The **second half of the durability guarantee**: a row that settles while no poll is attached (closed tab, discarded background tab, slept device) is never finalized, so its answer stays in the chunk store and its history row is terminal and empty. Given the request id this drains that turn's chunks in one pass; the engine parses them exactly as it parses a live stream and finalizes what it read, so each row is recovered at most once. Without it the engine mints no recovery marker at all and behaves as it did before streaming. |
 | `onLiveStreamUpdate`          | `function?` | Observation hook for a streaming turn (`{ serverItemId, ownerKey, phase, text, thinkingText, toolNames, complete, errored }`). The engine already paints the answer text itself, so this is only for affordances it does not decide the presentation of. Never throw from it. |

package/bunnyquery.js

@@ -61,6 +61,13 @@
   function webAccessFrom(data) {
     return data ? normalizeWebAccess(data.web_access) : DEFAULT_WEB_ACCESS;
   }
+  var DEFAULT_SKIP_DUPLICATE_FILES = true;
+  function normalizeSkipDuplicateFiles(value) {
+    return value !== false;
+  }
+  function skipDuplicateFilesFrom(data) {
+    return data ? normalizeSkipDuplicateFiles(data.skip_duplicate_files) : DEFAULT_SKIP_DUPLICATE_FILES;
+  }
   var CHAT_GREETING_MAX_LENGTH = 400;
   function normalizeChatGreeting(value) {
     if (typeof value !== "string") return "";
@@ -203,16 +210,25 @@
   function liveStreamingEnabled() {
     return _config?.liveStreaming === true;
   }
+  function underCeiling(projectSetting, option, override) {
+    if (!projectSetting) return false;
+    if (option === false) return false;
+    if (override === false) return false;
+    return true;
+  }
   function webAccessEnabled(service) {
     const opt = _config ? _config.webAccess : void 0;
-    if (typeof opt === "boolean") return opt;
-    return webAccessFrom(service ? cachedProjectSettings(service) : null);
+    return underCeiling(webAccessFrom(service ? cachedProjectSettings(service) : null), opt);
   }
-  function resolveWebAccess(service) {
+  function resolveWebAccess(service, override) {
     const opt = _config ? _config.webAccess : void 0;
-    if (typeof opt === "boolean") return Promise.resolve(opt);
-    if (!service) return Promise.resolve(false);
-    return loadProjectSettings(service).then(webAccessFrom, () => false);
+    if (!service) return Promise.resolve(underCeiling(webAccessFrom(null), opt, override));
+    return loadProjectSettings(service).then(webAccessFrom, () => false).then((setting) => underCeiling(setting, opt, override));
+  }
+  function resolveSkipDuplicateFiles(service) {
+    const opt = _config ? _config.skipDuplicateFiles : void 0;
+    if (!service) return Promise.resolve(underCeiling(skipDuplicateFilesFrom(null), opt));
+    return loadProjectSettings(service).then(skipDuplicateFilesFrom, () => skipDuplicateFilesFrom(null)).then((setting) => underCeiling(setting, opt));
   }
   function streamRecoveryEnabled() {
     if (!_config) return false;
@@ -228,6 +244,54 @@
     return p === void 0 ? {} : { poll: p };
   }
 
+  // src/engine/duplicates.ts
+  var DUPLICATE_LOOKUP_MAX_CANDIDATES = 25;
+  var SRC_PREFIX = "src::";
+  function isDuplicateCandidateId(uniqueId, name) {
+    if (typeof uniqueId !== "string" || !name) return false;
+    if (uniqueId.indexOf(SRC_PREFIX) !== 0) return false;
+    if (uniqueId === SRC_PREFIX + name) return true;
+    const tail = "/" + name;
+    return uniqueId.length > SRC_PREFIX.length + tail.length && uniqueId.slice(-tail.length) === tail;
+  }
+  function listOf(res) {
+    if (!res) return [];
+    if (Array.isArray(res)) return res;
+    return Array.isArray(res.list) ? res.list : [];
+  }
+  async function findDuplicateUpload(skapi, file, userId, opts) {
+    try {
+      if (!skapi || typeof skapi.getUniqueId !== "function" || typeof skapi.getRecords !== "function") return null;
+      const name = file && typeof file.name === "string" ? file.name : "";
+      const size = file && typeof file.size === "number" ? file.size : NaN;
+      if (!name || !(size >= 0) || !userId) return null;
+      const service = opts && opts.service ? { service: opts.service } : {};
+      const found = listOf(await skapi.getUniqueId(Object.assign({ unique_id: name, condition: "<=" }, service)));
+      const ids = [];
+      for (const item of found) {
+        const id = item && typeof item === "object" ? item.unique_id : item;
+        if (isDuplicateCandidateId(id, name) && ids.indexOf(id) === -1) ids.push(id);
+        if (ids.length >= DUPLICATE_LOOKUP_MAX_CANDIDATES) break;
+      }
+      for (const uniqueId of ids) {
+        let record = null;
+        try {
+          record = listOf(await skapi.getRecords(Object.assign({ unique_id: uniqueId }, service)))[0] || null;
+        } catch {
+          continue;
+        }
+        if (!record || typeof record !== "object") continue;
+        const data = record.data && typeof record.data === "object" ? record.data : null;
+        if (!data || data.size_bytes !== size) continue;
+        if (record.user_id !== userId) continue;
+        return { storagePath: uniqueId.slice(SRC_PREFIX.length), uniqueId, record };
+      }
+      return null;
+    } catch {
+      return null;
+    }
+  }
+
   // src/engine/office.ts
   var OFFICE_FILE_EXTENSIONS = /* @__PURE__ */ new Set([
     "doc",
@@ -637,6 +701,10 @@ Records for the earlier pages are ALREADY saved (they reference "${src}"). The N
 ` : "") + `- access group (use this for EVERY record you write for this file): ${indexingAccessGroup(attachment)}
 `;
   }
+  var PAGE_TEXT_TABLE = "page_texts";
+  var PAGE_TEXT_RULE = `PAGE TEXT for pages WITHOUT a text layer: for EVERY page whose label (its first label, when the page is split into strips) carries NO "Embedded text layer" block, ALSO post exactly ONE record in table "${PAGE_TEXT_TABLE}" with data {"page": <its document page number from the label, 1-based>, "page_text": "<the page's complete text exactly as printed, verbatim, in reading order, nothing summarised or left out>"}, reference "src::" + the storage path above, in addition to the records above. A page WITH an embedded text layer gets NO such record: its text is already stored.
+
+`;
   function buildRenderDatafy(placeholder) {
     return `
 ${placeholder}
@@ -647,7 +715,7 @@ Each image is preceded by a label giving its DOCUMENT PAGE number. That label is
 
 Transcribe COMPLETELY, not representatively. A table with twenty rows gets twenty records, not a sample of the first few - if a page has more rows than you can save comfortably, still save them all rather than summarising. Where a page carries an embedded text layer it is quoted above that page's image: it is the exact text and should be preferred over reading the pixels, with the image used for layout, tables, stamps and handwriting.
 
-Save records for THIS window of pages only, then stop and report what you saved. Do NOT try to read the rest of the file and do NOT worry about the pages after this window: if any remain, the next window is rendered and sent to you automatically. Report only the pages you were actually shown - never imply you have seen the whole document.`;
+` + PAGE_TEXT_RULE + `Save records for THIS window of pages only, then stop and report what you saved. Do NOT try to read the rest of the file and do NOT worry about the pages after this window: if any remain, the next window is rendered and sent to you automatically. Report only the pages you were actually shown - never imply you have seen the whole document.`;
   }
   function buildIndexingWindowMessage(attachment, placeholder, isContinuation, positionLabel) {
     const src = `src::${attachment.storagePath}`;
@@ -1151,7 +1219,7 @@ Index the REMAINING windows - one record per row/item, looking at any page image
   var FILE_KEYS = ["path", "storage_path", "storagePath", "file", "file_path", "filePath"];
   var TABLE_KEYS = ["table_name", "tableName"];
   var GROUP_KEYS = ["access_group", "accessGroup"];
-  var SRC_PREFIX = "src::";
+  var SRC_PREFIX2 = "src::";
   var RESULT_WALK_MAX_NODES = 400;
   var RESULT_WALK_MAX_DEPTH = 5;
   function isWebToolName(name) {
@@ -1180,8 +1248,8 @@ Index the REMAINING windows - one record per row/item, looking at any page image
   }
   function pathFromSrcId(value) {
     if (typeof value !== "string") return null;
-    if (value.indexOf(SRC_PREFIX) !== 0) return null;
-    var rest = value.slice(SRC_PREFIX.length).trim();
+    if (value.indexOf(SRC_PREFIX2) !== 0) return null;
+    var rest = value.slice(SRC_PREFIX2.length).trim();
     return rest ? rest : null;
   }
   function parseJsonLoose(value) {
@@ -2811,7 +2879,7 @@ Index the REMAINING windows - one record per row/item, looking at any page image
     fileUrls,
     webAccess
   }) {
-    const web = typeof webAccess === "boolean" ? webAccess : await resolveWebAccess(service);
+    const web = await resolveWebAccess(service, typeof webAccess === "boolean" ? webAccess : void 0);
     const mcpServerDefinition = {
       type: "url",
       name: mcpServer.name,
@@ -8341,11 +8409,26 @@ Index the REMAINING windows - one record per row/item, looking at any page image
       if (!total) return Promise.reject(new Error("Empty attachment"));
       var urls = [];
       var anyIndexFailed = false;
+      var reusedCount = 0;
+      att.reused = false;
       var chain = Promise.resolve();
+      var findDuplicate = function(member) {
+        if (typeof self.host.findDuplicateUpload !== "function") return Promise.resolve(null);
+        var storedName = String(member.storagePath || "").split("/").filter(Boolean).pop() || member.file.name;
+        return resolveSkipDuplicateFiles(id.projectId).then(function(on) {
+          if (!on) return null;
+          return Promise.resolve(self.host.findDuplicateUpload({ name: storedName, size: member.file.size })).then(function(hit) {
+            return hit && typeof hit.storagePath === "string" && hit.storagePath ? hit.storagePath : null;
+          });
+        }).catch(function() {
+          return null;
+        });
+      };
       members.forEach(function(member, idx) {
         chain = chain.then(function() {
           var hadExists = false;
           var skipped = false;
+          var duplicateOf = null;
           var existedBefore = false;
           var onProg = function(p) {
             if (p && p.total) {
@@ -8364,7 +8447,14 @@ Index the REMAINING windows - one record per row/item, looking at any page image
               }
             });
           };
-          return doMemberUpload(true).catch(function(err) {
+          return findDuplicate(member).then(function(dup) {
+            if (dup) {
+              duplicateOf = dup;
+              console.log("[chat-engine] skipping an upload the user already has:", member.storagePath, "->", dup);
+              return;
+            }
+            return doMemberUpload(true);
+          }).catch(function(err) {
             var code = err && (err.code || err.body && err.body.code);
             var msg = err && (err.message || err.body && err.body.message || (typeof err === "string" ? err : ""));
             var isExists = code === "EXISTS" || msg && /exist/i.test(msg);
@@ -8384,9 +8474,20 @@ Index the REMAINING windows - one record per row/item, looking at any page image
             });
           }).then(function() {
             if (skipped) return;
-            return self.host.getTemporaryUrl(member.storagePath);
+            return self.host.getTemporaryUrl(duplicateOf || member.storagePath);
           }).then(function(url) {
             if (skipped) return;
+            if (duplicateOf) {
+              urls.push({ name: member.relPath, url, storagePath: duplicateOf });
+              reusedCount += 1;
+              if (att.kind !== "folder") {
+                att.uploadedUrl = url;
+                att.storagePath = duplicateOf;
+                att.reused = true;
+              } else att.reusedCount = reusedCount;
+              self.host.renderAttachmentChips();
+              return;
+            }
             urls.push({ name: member.relPath, url, storagePath: member.storagePath });
             if (att.kind !== "folder") {
               att.uploadedUrl = url;
@@ -9018,7 +9119,7 @@ Index the REMAINING windows - one record per row/item, looking at any page image
   (function() {
     var MCP_PROD = "https://mcp.broadwayinc.computer";
     var MCP_DEV = "https://mcp-dev.broadwayinc.computer";
-    var BQ_VERSION = "1.15.0" ;
+    var BQ_VERSION = "1.15.1" ;
     var ATTACHMENT_URL_EXPIRES_SECONDS = 600;
     var GOOGLE_AUTH_URL = "https://accounts.google.com/o/oauth2/v2/auth";
     var GOOGLE_TOKEN_URL = "https://oauth2.googleapis.com/token";
@@ -10858,6 +10959,13 @@ Index the REMAINING windows - one record per row/item, looking at any page image
       uploadAccessGroup: function(path) {
         return resolveUploadAccessGroup(path);
       },
+      // "Already uploaded by this user?" from the src:: file records. The engine
+      // asks only while the skip-duplicates switch is on; null means upload.
+      findDuplicateUpload: function(file) {
+        var uid = S.user && S.user.user_id;
+        if (!uid) return Promise.resolve(null);
+        return findDuplicateUpload(S.skapi, file, uid, { service: S.projectId });
+      },
       storagePathFor: function(relPath) {
         return attachmentStoragePath(relPath);
       },
@@ -11839,7 +11947,7 @@ Index the REMAINING windows - one record per row/item, looking at any page image
         });
         chip.appendChild(h("span", { class: "bq-attachment-icon", html: isFolder ? FOLDER_ICON_SVG : FILE_ICON_SVG }));
         chip.appendChild(h("span", { class: "bq-attachment-name", text: att.name, title: att.name }));
-        var meta = att.status === "error" ? "(Failed)" : att.status === "indexError" ? "(Error)" : preparing ? "Preparing" : finalizing ? "Finalizing" : att.status === "uploading" ? att.progress + "%" : isFolder ? "(" + (att.files ? att.files.length : 0) + ")" : formatBytes(att.file ? att.file.size : att.size);
+        var meta = att.status === "error" ? "(Failed)" : att.status === "indexError" ? "(Error)" : preparing ? "Preparing" : finalizing ? "Finalizing" : att.status === "uploading" ? att.progress + "%" : isFolder ? "(" + (att.files ? att.files.length : 0) + ")" : att.reused ? "already uploaded" : formatBytes(att.file ? att.file.size : att.size);
         chip.appendChild(h("span", { class: "bq-attachment-meta", text: meta }));
         if (clickable) chip.appendChild(h("span", { class: "bq-attachment-arrow", text: "\u2197" }));
         if (att.status !== "uploading" && att.status !== "done") {
@@ -13663,6 +13771,15 @@ Index the REMAINING windows - one record per row/item, looking at any page image
         // true/false pins it for this page, both ways. Read at
         // configureChatEngine time and when each chat request is built.
         webAccess: null,
+        // Whether a chat attachment the signed-in user ALREADY uploaded to this
+        // project (same file name, same size, same uploader) is skipped: not
+        // uploaded again, not indexed again, the message using the stored
+        // copy. null = follow the project's own "Skip duplicate files" setting
+        // (`skip_duplicate_files` in the bq::settings record), which is ON
+        // unless the owner turned it off. The project setting is the ceiling:
+        // false here restricts below it for this page, true has no effect on
+        // a project that turned it off.
+        skipDuplicateFiles: null,
         // Server-driven windowed indexing; read at configureChatEngine time.
         // Listed here so the defaults object is the full opt surface.
         windowedIndexing: true,
@@ -13744,6 +13861,9 @@ Index the REMAINING windows - one record per row/item, looking at any page image
         // chat request (off unless the owner turned it on). Passing the null
         // default through would read as a decision.
         webAccess: typeof S.opts.webAccess === "boolean" ? S.opts.webAccess : void 0,
+        // Skip duplicate uploads. Same hand-over rule: only a pinned boolean is
+        // passed, and the project's setting is the ceiling the engine applies.
+        skipDuplicateFiles: typeof S.opts.skipDuplicateFiles === "boolean" ? S.opts.skipDuplicateFiles : void 0,
         // ---- live streaming (mirrored in agent.vue's ai_agent.ts) --------
         // On by default since every region's polling worker relays the
         // response bytes. It used to be off for the shipping-order reason

package/dist/engine.cjs

@@ -73,6 +73,13 @@ function normalizeWebAccess(value) {
 function webAccessFrom(data) {
   return data ? normalizeWebAccess(data.web_access) : DEFAULT_WEB_ACCESS;
 }
+var DEFAULT_SKIP_DUPLICATE_FILES = true;
+function normalizeSkipDuplicateFiles(value) {
+  return value !== false;
+}
+function skipDuplicateFilesFrom(data) {
+  return data ? normalizeSkipDuplicateFiles(data.skip_duplicate_files) : DEFAULT_SKIP_DUPLICATE_FILES;
+}
 var CHAT_GREETING_MAX_LENGTH = 400;
 function normalizeChatGreeting(value) {
   if (typeof value !== "string") return "";
@@ -159,6 +166,9 @@ function projectChatGreeting(service) {
 function projectWebAccess(service) {
   return webAccessFrom(cachedProjectSettings(service));
 }
+function projectSkipDuplicateFiles(service) {
+  return skipDuplicateFilesFrom(cachedProjectSettings(service));
+}
 function setProjectSettings(service, data) {
   if (!service) return;
   cache.set(service, { data: data || null, settled: true, inflight: null });
@@ -238,16 +248,29 @@ function liveStreamingRealtimeEnabled() {
 function liveStreamingEnabled() {
   return _config?.liveStreaming === true;
 }
+function underCeiling(projectSetting, option, override) {
+  if (!projectSetting) return false;
+  if (option === false) return false;
+  if (override === false) return false;
+  return true;
+}
 function webAccessEnabled(service) {
   const opt = _config ? _config.webAccess : void 0;
-  if (typeof opt === "boolean") return opt;
-  return webAccessFrom(service ? cachedProjectSettings(service) : null);
+  return underCeiling(webAccessFrom(service ? cachedProjectSettings(service) : null), opt);
 }
-function resolveWebAccess(service) {
+function resolveWebAccess(service, override) {
   const opt = _config ? _config.webAccess : void 0;
-  if (typeof opt === "boolean") return Promise.resolve(opt);
-  if (!service) return Promise.resolve(false);
-  return loadProjectSettings(service).then(webAccessFrom, () => false);
+  if (!service) return Promise.resolve(underCeiling(webAccessFrom(null), opt, override));
+  return loadProjectSettings(service).then(webAccessFrom, () => false).then((setting) => underCeiling(setting, opt, override));
+}
+function skipDuplicateFilesEnabled(service) {
+  const opt = _config ? _config.skipDuplicateFiles : void 0;
+  return underCeiling(skipDuplicateFilesFrom(service ? cachedProjectSettings(service) : null), opt);
+}
+function resolveSkipDuplicateFiles(service) {
+  const opt = _config ? _config.skipDuplicateFiles : void 0;
+  if (!service) return Promise.resolve(underCeiling(skipDuplicateFilesFrom(null), opt));
+  return loadProjectSettings(service).then(skipDuplicateFilesFrom, () => skipDuplicateFilesFrom(null)).then((setting) => underCeiling(setting, opt));
 }
 function streamRecoveryEnabled() {
   if (!_config) return false;
@@ -263,6 +286,54 @@ function pollOpt() {
   return p === void 0 ? {} : { poll: p };
 }
 
+// src/engine/duplicates.ts
+var DUPLICATE_LOOKUP_MAX_CANDIDATES = 25;
+var SRC_PREFIX = "src::";
+function isDuplicateCandidateId(uniqueId, name) {
+  if (typeof uniqueId !== "string" || !name) return false;
+  if (uniqueId.indexOf(SRC_PREFIX) !== 0) return false;
+  if (uniqueId === SRC_PREFIX + name) return true;
+  const tail = "/" + name;
+  return uniqueId.length > SRC_PREFIX.length + tail.length && uniqueId.slice(-tail.length) === tail;
+}
+function listOf(res) {
+  if (!res) return [];
+  if (Array.isArray(res)) return res;
+  return Array.isArray(res.list) ? res.list : [];
+}
+async function findDuplicateUpload(skapi, file, userId, opts) {
+  try {
+    if (!skapi || typeof skapi.getUniqueId !== "function" || typeof skapi.getRecords !== "function") return null;
+    const name = file && typeof file.name === "string" ? file.name : "";
+    const size = file && typeof file.size === "number" ? file.size : NaN;
+    if (!name || !(size >= 0) || !userId) return null;
+    const service = opts && opts.service ? { service: opts.service } : {};
+    const found = listOf(await skapi.getUniqueId(Object.assign({ unique_id: name, condition: "<=" }, service)));
+    const ids = [];
+    for (const item of found) {
+      const id = item && typeof item === "object" ? item.unique_id : item;
+      if (isDuplicateCandidateId(id, name) && ids.indexOf(id) === -1) ids.push(id);
+      if (ids.length >= DUPLICATE_LOOKUP_MAX_CANDIDATES) break;
+    }
+    for (const uniqueId of ids) {
+      let record = null;
+      try {
+        record = listOf(await skapi.getRecords(Object.assign({ unique_id: uniqueId }, service)))[0] || null;
+      } catch {
+        continue;
+      }
+      if (!record || typeof record !== "object") continue;
+      const data = record.data && typeof record.data === "object" ? record.data : null;
+      if (!data || data.size_bytes !== size) continue;
+      if (record.user_id !== userId) continue;
+      return { storagePath: uniqueId.slice(SRC_PREFIX.length), uniqueId, record };
+    }
+    return null;
+  } catch {
+    return null;
+  }
+}
+
 // src/engine/office.ts
 var OFFICE_FILE_EXTENSIONS = /* @__PURE__ */ new Set([
   "doc",
@@ -675,6 +746,10 @@ function buildRenderMeta(attachment) {
 ` : "") + `- access group (use this for EVERY record you write for this file): ${indexingAccessGroup(attachment)}
 `;
 }
+var PAGE_TEXT_TABLE = "page_texts";
+var PAGE_TEXT_RULE = `PAGE TEXT for pages WITHOUT a text layer: for EVERY page whose label (its first label, when the page is split into strips) carries NO "Embedded text layer" block, ALSO post exactly ONE record in table "${PAGE_TEXT_TABLE}" with data {"page": <its document page number from the label, 1-based>, "page_text": "<the page's complete text exactly as printed, verbatim, in reading order, nothing summarised or left out>"}, reference "src::" + the storage path above, in addition to the records above. A page WITH an embedded text layer gets NO such record: its text is already stored.
+
+`;
 function buildRenderDatafy(placeholder) {
   return `
 ${placeholder}
@@ -685,7 +760,7 @@ Each image is preceded by a label giving its DOCUMENT PAGE number. That label is
 
 Transcribe COMPLETELY, not representatively. A table with twenty rows gets twenty records, not a sample of the first few - if a page has more rows than you can save comfortably, still save them all rather than summarising. Where a page carries an embedded text layer it is quoted above that page's image: it is the exact text and should be preferred over reading the pixels, with the image used for layout, tables, stamps and handwriting.
 
-Save records for THIS window of pages only, then stop and report what you saved. Do NOT try to read the rest of the file and do NOT worry about the pages after this window: if any remain, the next window is rendered and sent to you automatically. Report only the pages you were actually shown - never imply you have seen the whole document.`;
+` + PAGE_TEXT_RULE + `Save records for THIS window of pages only, then stop and report what you saved. Do NOT try to read the rest of the file and do NOT worry about the pages after this window: if any remain, the next window is rendered and sent to you automatically. Report only the pages you were actually shown - never imply you have seen the whole document.`;
 }
 function buildIndexingWindowMessage(attachment, placeholder, isContinuation, positionLabel) {
   const src = `src::${attachment.storagePath}`;
@@ -1227,7 +1302,7 @@ var RETRIEVAL_NOTE_PREFIX = "[Retrieval note, not shown to the user";
 var FILE_KEYS = ["path", "storage_path", "storagePath", "file", "file_path", "filePath"];
 var TABLE_KEYS = ["table_name", "tableName"];
 var GROUP_KEYS = ["access_group", "accessGroup"];
-var SRC_PREFIX = "src::";
+var SRC_PREFIX2 = "src::";
 var RESULT_WALK_MAX_NODES = 400;
 var RESULT_WALK_MAX_DEPTH = 5;
 function isWebToolName(name) {
@@ -1256,8 +1331,8 @@ function pushHit(list, path, page) {
 }
 function pathFromSrcId(value) {
   if (typeof value !== "string") return null;
-  if (value.indexOf(SRC_PREFIX) !== 0) return null;
-  var rest = value.slice(SRC_PREFIX.length).trim();
+  if (value.indexOf(SRC_PREFIX2) !== 0) return null;
+  var rest = value.slice(SRC_PREFIX2.length).trim();
   return rest ? rest : null;
 }
 function parseJsonLoose(value) {
@@ -2926,7 +3001,7 @@ async function callClaudeWithMcp({
   fileUrls,
   webAccess
 }) {
-  const web = typeof webAccess === "boolean" ? webAccess : await resolveWebAccess(service);
+  const web = await resolveWebAccess(service, typeof webAccess === "boolean" ? webAccess : void 0);
   const mcpServerDefinition = {
     type: "url",
     name: mcpServer.name,
@@ -8515,11 +8590,26 @@ var ChatSession = class {
     if (!total) return Promise.reject(new Error("Empty attachment"));
     var urls = [];
     var anyIndexFailed = false;
+    var reusedCount = 0;
+    att.reused = false;
     var chain = Promise.resolve();
+    var findDuplicate = function(member) {
+      if (typeof self.host.findDuplicateUpload !== "function") return Promise.resolve(null);
+      var storedName = String(member.storagePath || "").split("/").filter(Boolean).pop() || member.file.name;
+      return resolveSkipDuplicateFiles(id.projectId).then(function(on) {
+        if (!on) return null;
+        return Promise.resolve(self.host.findDuplicateUpload({ name: storedName, size: member.file.size })).then(function(hit) {
+          return hit && typeof hit.storagePath === "string" && hit.storagePath ? hit.storagePath : null;
+        });
+      }).catch(function() {
+        return null;
+      });
+    };
     members.forEach(function(member, idx) {
       chain = chain.then(function() {
         var hadExists = false;
         var skipped = false;
+        var duplicateOf = null;
         var existedBefore = false;
         var onProg = function(p) {
           if (p && p.total) {
@@ -8538,7 +8628,14 @@ var ChatSession = class {
             }
           });
         };
-        return doMemberUpload(true).catch(function(err) {
+        return findDuplicate(member).then(function(dup) {
+          if (dup) {
+            duplicateOf = dup;
+            console.log("[chat-engine] skipping an upload the user already has:", member.storagePath, "->", dup);
+            return;
+          }
+          return doMemberUpload(true);
+        }).catch(function(err) {
           var code = err && (err.code || err.body && err.body.code);
           var msg = err && (err.message || err.body && err.body.message || (typeof err === "string" ? err : ""));
           var isExists = code === "EXISTS" || msg && /exist/i.test(msg);
@@ -8558,9 +8655,20 @@ var ChatSession = class {
           });
         }).then(function() {
           if (skipped) return;
-          return self.host.getTemporaryUrl(member.storagePath);
+          return self.host.getTemporaryUrl(duplicateOf || member.storagePath);
         }).then(function(url) {
           if (skipped) return;
+          if (duplicateOf) {
+            urls.push({ name: member.relPath, url, storagePath: duplicateOf });
+            reusedCount += 1;
+            if (att.kind !== "folder") {
+              att.uploadedUrl = url;
+              att.storagePath = duplicateOf;
+              att.reused = true;
+            } else att.reusedCount = reusedCount;
+            self.host.renderAttachmentChips();
+            return;
+          }
           urls.push({ name: member.relPath, url, storagePath: member.storagePath });
           if (att.kind !== "folder") {
             att.uploadedUrl = url;
@@ -9201,8 +9309,10 @@ exports.DATA_TOOL_NAMES = DATA_TOOL_NAMES;
 exports.DEFAULT_CLAUDE_MODEL = DEFAULT_CLAUDE_MODEL;
 exports.DEFAULT_CONTEXT_WINDOW = DEFAULT_CONTEXT_WINDOW;
 exports.DEFAULT_OPENAI_MODEL = DEFAULT_OPENAI_MODEL;
+exports.DEFAULT_SKIP_DUPLICATE_FILES = DEFAULT_SKIP_DUPLICATE_FILES;
 exports.DEFAULT_UPLOAD_ACCESS_GROUP = DEFAULT_UPLOAD_ACCESS_GROUP;
 exports.DEFAULT_WEB_ACCESS = DEFAULT_WEB_ACCESS;
+exports.DUPLICATE_LOOKUP_MAX_CANDIDATES = DUPLICATE_LOOKUP_MAX_CANDIDATES;
 exports.EMPTY_INDEXING_REPLY = EMPTY_INDEXING_REPLY;
 exports.EXPIRED_ATTACHMENT_URL_HOST = EXPIRED_ATTACHMENT_URL_HOST;
 exports.EXPIRED_ATTACHMENT_URL_ORIGIN = EXPIRED_ATTACHMENT_URL_ORIGIN;
@@ -9234,6 +9344,7 @@ exports.MIN_INPUT_TOKEN_BUDGET = MIN_INPUT_TOKEN_BUDGET;
 exports.MIN_PER_REQUEST_INPUT_CAP = MIN_PER_REQUEST_INPUT_CAP;
 exports.NO_DATA_TOOL_MARK = NO_DATA_TOOL_MARK;
 exports.OUTPUT_TOKEN_RESERVE = OUTPUT_TOKEN_RESERVE;
+exports.PAGE_TEXT_TABLE = PAGE_TEXT_TABLE;
 exports.POLL_INTERVAL = POLL_INTERVAL;
 exports.PRESIGN_SAFETY_MARGIN_MS = PRESIGN_SAFETY_MARGIN_MS;
 exports.PREVIEWABLE_IMAGE_CONTENT_TYPES = PREVIEWABLE_IMAGE_CONTENT_TYPES;
@@ -9322,6 +9433,7 @@ exports.fetchLiveIndexingKeys = fetchLiveIndexingKeys;
 exports.fillHistoryViewport = fillHistoryViewport;
 exports.filterListByClearHorizon = filterListByClearHorizon;
 exports.findAttachmentParser = findAttachmentParser;
+exports.findDuplicateUpload = findDuplicateUpload;
 exports.formatChatTimestamp = formatChatTimestamp;
 exports.formatDuration = formatDuration;
 exports.formatRetrievalNote = formatRetrievalNote;
@@ -9346,6 +9458,7 @@ exports.isAuthExpiredError = isAuthExpiredError;
 exports.isBgIndexingQueue = isBgIndexingQueue;
 exports.isCsrStatusEnvelope = isCsrStatusEnvelope;
 exports.isDataToolName = isDataToolName;
+exports.isDuplicateCandidateId = isDuplicateCandidateId;
 exports.isErrorResponseBody = isErrorResponseBody;
 exports.isHttpUrlLike = isHttpUrlLike;
 exports.isImageVisionFile = isImageVisionFile;
@@ -9379,6 +9492,7 @@ exports.normalizeAttachmentPathCandidate = normalizeAttachmentPathCandidate;
 exports.normalizeChatGreeting = normalizeChatGreeting;
 exports.normalizeExt = normalizeExt;
 exports.normalizeProjectAccessSetting = normalizeProjectAccessSetting;
+exports.normalizeSkipDuplicateFiles = normalizeSkipDuplicateFiles;
 exports.normalizeTextContent = normalizeTextContent;
 exports.normalizeTrailingInlineToken = normalizeTrailingInlineToken;
 exports.normalizeUploadAccessGroup = normalizeUploadAccessGroup;
@@ -9401,6 +9515,7 @@ exports.projectAccessSetting = projectAccessSetting;
 exports.projectAsksUploadAccess = projectAsksUploadAccess;
 exports.projectChatGreeting = projectChatGreeting;
 exports.projectSettingsSettled = projectSettingsSettled;
+exports.projectSkipDuplicateFiles = projectSkipDuplicateFiles;
 exports.projectUploadAccessGroup = projectUploadAccessGroup;
 exports.projectWebAccess = projectWebAccess;
 exports.providerBodyWasRead = providerBodyWasRead;
@@ -9412,6 +9527,7 @@ exports.renderInlineLinkHtml = renderInlineLinkHtml;
 exports.repairUrlEntities = repairUrlEntities;
 exports.repairUrlWhitespace = repairUrlWhitespace;
 exports.resolveImagePreviewUrl = resolveImagePreviewUrl;
+exports.resolveSkipDuplicateFiles = resolveSkipDuplicateFiles;
 exports.resolveWebAccess = resolveWebAccess;
 exports.retrievalNoteFromText = retrievalNoteFromText;
 exports.retrievalNoteOfAnswer = retrievalNoteOfAnswer;
@@ -9422,6 +9538,8 @@ exports.setProjectContextWindow = setProjectContextWindow;
 exports.setProjectSettings = setProjectSettings;
 exports.shouldRescueInFlightMessage = shouldRescueInFlightMessage;
 exports.skapiSupportsStreaming = skapiSupportsStreaming;
+exports.skipDuplicateFilesEnabled = skipDuplicateFilesEnabled;
+exports.skipDuplicateFilesFrom = skipDuplicateFilesFrom;
 exports.streamRecoveryEnabled = streamRecoveryEnabled;
 exports.streamRecoveryLabels = streamRecoveryLabels;
 exports.streamRecoveryPhase = streamRecoveryPhase;