gethstorage-sdk 5.0.2 → 5.0.3

package/README.md

@@ -1,208 +1,208 @@
-# gethstorage-sdk
-
-This SDK aims to standardize the interaction between applications and the EthStorage network to achieve reliable and
-efficient data management functionality.
-
-This SDK provides the `FlatDirectory` class, a higher-level data management tool that provides methods for uploading
-and downloading data of arbitrary size. Uploads are performed by sending the data through calldata.
-
-Click here to view [spec](./spec.md).
-
-# Installation
-
-Install the SDK using [npm](https://www.npmjs.com/package/gethstorage-sdk):
-
-```bash
-$ npm install gethstorage-sdk
-```
-
-# Example Usage
-
-## FlatDirectory
-
-### create
-
-Create a `FlatDirectory` instance.
-
-```js
-const { FlatDirectory } = require("gethstorage-sdk");
-
-const rpc = "https://rpc.beta.testnet.l2.quarkchain.io:8545";
-const ethStorageRpc = "https://rpc.beta.testnet.l2.ethstorage.io:9596";
-const privateKey = "0xabcd...";
-
-const flatDirectory = await FlatDirectory.create({
-    rpc: rpc,
-    ethStorageRpc: ethStorageRpc,
-    privateKey: privateKey,
-});
-```
-
-If `FlatDirectory` has been deployed, it can be set through the `address` field.
-
-```js
-const address = "0x987..."; // FlatDirectory address
-const flatDirectory = await FlatDirectory.create({
-    rpc: rpc,
-    ethStorageRpc: ethStorageRpc,
-    privateKey: privateKey,
-    address: address,
-});
-```
-
-If you only need the `download` function, you can create a `FlatDirectory` using the following pattern.
-
-```js
-const address = "0x987..."; // FlatDirectory address
-const flatDirectory = await FlatDirectory.create({
-    ethStorageRpc: 'https://rpc.beta.testnet.l2.ethstorage.io:9596',
-    address: address
-});
-```
-
-### deploy
-
-Deploy the implementation
-contract [FlatDirectory](https://github.com/ethstorage/evm-large-storage)
-for [EIP-5018](https://eips.ethereum.org/EIPS/eip-5018) standard.
-
-```js
-const contractAddress = await flatDirectory.deploy();
-console.log(`FlatDirectory address is ${contractAddress}.`);
-```
-
-### upload
-
-Upload `buffer | file` to the FlatDirectory. The content is sliced into chunks and each chunk is sent through calldata.
-
-```js
-const callback = {
-    onProgress: function (progress, count, isChange) {
-        console.log(`Uploaded ${progress} of ${count} chunks`);
-    },
-    onFail: function (err) {
-        console.log(err);
-    },
-    onFinish: function (totalUploadChunks, totalUploadSize, totalCost) {
-        console.log(`Total upload chunk count is ${totalUploadChunks}, size is ${totalUploadSize}, total cost is ${totalCost}`);
-    }
-};
-
-const request = {
-    key: "test.txt",
-    content: Buffer.from("big data"),
-    callback: callback
-}
-await flatDirectory.upload(request);
-```
-
-If you want to use `file`, it can be divided into browser and Node.js.
-
-Browser
-```javascript
-// <input id='fileToUpload' />
-const file = document.getElementById('fileToUpload').files[0];
-
-const request = {
-    key: "test.txt",
-    content: file,
-    callback: callback
-}
-await flatDirectory.upload(request);
-```
-
-Node.js
-```javascript
-const {NodeFile} = require("gethstorage-sdk/file");
-const file = new NodeFile("/usr/download/test.jpg");
-
-const request = {
-    key: "test.txt",
-    content: file,
-    callback: callback
-}
-await flatDirectory.upload(request);
-```
-
-### download
-
-Monitor the download progress by passing in a callback function.
-
-```js
-const key = "test.txt";
-await flatDirectory.download(key, {
-	onProgress: function (progress, count, chunk) {
-		console.log(`Download ${progress} of ${count} chunks, this chunk is ${Buffer.from(chunk).toString()}`);
-	},
-    onFail: function (error) {
-        console.error("Error download data:", error);
-    },
-    onFinish: function () {
-        console.log("Download success.");
-    }
-});
-```
-
-### estimateCost
-
-Estimate gas costs before uploading.
-
-```js
-const request = {
-    key: "example1.txt",
-    content: Buffer.from("large data to upload")
-}
-
-const cost = await flatDirectory.estimateCost(request);
-console.log(`Gas Cost: ${cost.gasCost}, Storage Cost: ${cost.storageCost}`);
-```
-
-Use `file`.
-
-Browser
-```javascript
-// <input id='fileToUpload' />
-const file = document.getElementById('fileToUpload').files[0];
-
-const request = {
-    key: "example1.txt",
-    content: file
-}
-const cost = await flatDirectory.estimateCost(request);
-console.log(`Gas Cost: ${cost.gasCost}, Storage Cost: ${cost.storageCost}`);
-```
-
-Node.js
-```javascript
-const {NodeFile} = require("gethstorage-sdk/file");
-const file = new NodeFile("/usr/download/test.jpg");
-
-const request = {
-    key: "example1.txt",
-    content: file
-}
-const cost = await flatDirectory.estimateCost(request);
-```
-
-
-### Chunk options
-
-Uploads are sliced into calldata chunks. Both `estimateCost` and `upload`
-requests accept two optional fields:
-
-| Field          | Type   | Default | Description                                              |
-|----------------|--------|---------|----------------------------------------------------------|
-| `maxChunkSize` | number | `64`    | data size of each chunk, in KB                            |
-| `maxChunks`    | number | `100`   | maximum number of chunks requested in a single call       |
-
-```js
-const request = {
-    key: "example1.txt",
-    content: Buffer.from("large data to upload"),
-    maxChunkSize: 32, // 32 KB per chunk
-    maxChunks: 120    // at most 120 chunks per request
-}
-await flatDirectory.upload({ ...request, callback });
-```
-
-`fetchHashes(keys, maxChunks)` accepts the same chunk limit as its second argument.
+# gethstorage-sdk
+
+This SDK aims to standardize the interaction between applications and the EthStorage network to achieve reliable and
+efficient data management functionality.
+
+This SDK provides the `FlatDirectory` class, a higher-level data management tool that provides methods for uploading
+and downloading data of arbitrary size. Uploads are performed by sending the data through calldata.
+
+Click here to view [spec](./spec.md).
+
+# Installation
+
+Install the SDK using [npm](https://www.npmjs.com/package/gethstorage-sdk):
+
+```bash
+$ npm install gethstorage-sdk
+```
+
+# Example Usage
+
+## FlatDirectory
+
+### create
+
+Create a `FlatDirectory` instance.
+
+```js
+const { FlatDirectory } = require("gethstorage-sdk");
+
+const rpc = "https://rpc.beta.testnet.l2.quarkchain.io:8545";
+const ethStorageRpc = "https://rpc.beta.testnet.l2.ethstorage.io:9596";
+const privateKey = "0xabcd...";
+
+const flatDirectory = await FlatDirectory.create({
+    rpc: rpc,
+    ethStorageRpc: ethStorageRpc,
+    privateKey: privateKey,
+});
+```
+
+If `FlatDirectory` has been deployed, it can be set through the `address` field.
+
+```js
+const address = "0x987..."; // FlatDirectory address
+const flatDirectory = await FlatDirectory.create({
+    rpc: rpc,
+    ethStorageRpc: ethStorageRpc,
+    privateKey: privateKey,
+    address: address,
+});
+```
+
+If you only need the `download` function, you can create a `FlatDirectory` using the following pattern.
+
+```js
+const address = "0x987..."; // FlatDirectory address
+const flatDirectory = await FlatDirectory.create({
+    ethStorageRpc: 'https://rpc.beta.testnet.l2.ethstorage.io:9596',
+    address: address
+});
+```
+
+### deploy
+
+Deploy the implementation
+contract [FlatDirectory](https://github.com/ethstorage/evm-large-storage)
+for [EIP-5018](https://eips.ethereum.org/EIPS/eip-5018) standard.
+
+```js
+const contractAddress = await flatDirectory.deploy();
+console.log(`FlatDirectory address is ${contractAddress}.`);
+```
+
+### upload
+
+Upload `buffer | file` to the FlatDirectory. The content is sliced into chunks and each chunk is sent through calldata.
+
+```js
+const callback = {
+    onProgress: function (progress, count, isChange) {
+        console.log(`Uploaded ${progress} of ${count} chunks`);
+    },
+    onFail: function (err) {
+        console.log(err);
+    },
+    onFinish: function (totalUploadChunks, totalUploadSize, totalCost) {
+        console.log(`Total upload chunk count is ${totalUploadChunks}, size is ${totalUploadSize}, total cost is ${totalCost}`);
+    }
+};
+
+const request = {
+    key: "test.txt",
+    content: Buffer.from("big data"),
+    callback: callback
+}
+await flatDirectory.upload(request);
+```
+
+If you want to use `file`, it can be divided into browser and Node.js.
+
+Browser
+```javascript
+// <input id='fileToUpload' />
+const file = document.getElementById('fileToUpload').files[0];
+
+const request = {
+    key: "test.txt",
+    content: file,
+    callback: callback
+}
+await flatDirectory.upload(request);
+```
+
+Node.js
+```javascript
+const {NodeFile} = require("gethstorage-sdk/file");
+const file = new NodeFile("/usr/download/test.jpg");
+
+const request = {
+    key: "test.txt",
+    content: file,
+    callback: callback
+}
+await flatDirectory.upload(request);
+```
+
+### download
+
+Monitor the download progress by passing in a callback function.
+
+```js
+const key = "test.txt";
+await flatDirectory.download(key, {
+	onProgress: function (progress, count, chunk) {
+		console.log(`Download ${progress} of ${count} chunks, this chunk is ${Buffer.from(chunk).toString()}`);
+	},
+    onFail: function (error) {
+        console.error("Error download data:", error);
+    },
+    onFinish: function () {
+        console.log("Download success.");
+    }
+});
+```
+
+### estimateCost
+
+Estimate gas costs before uploading.
+
+```js
+const request = {
+    key: "example1.txt",
+    content: Buffer.from("large data to upload")
+}
+
+const cost = await flatDirectory.estimateCost(request);
+console.log(`Gas Cost: ${cost.gasCost}, Storage Cost: ${cost.storageCost}`);
+```
+
+Use `file`.
+
+Browser
+```javascript
+// <input id='fileToUpload' />
+const file = document.getElementById('fileToUpload').files[0];
+
+const request = {
+    key: "example1.txt",
+    content: file
+}
+const cost = await flatDirectory.estimateCost(request);
+console.log(`Gas Cost: ${cost.gasCost}, Storage Cost: ${cost.storageCost}`);
+```
+
+Node.js
+```javascript
+const {NodeFile} = require("gethstorage-sdk/file");
+const file = new NodeFile("/usr/download/test.jpg");
+
+const request = {
+    key: "example1.txt",
+    content: file
+}
+const cost = await flatDirectory.estimateCost(request);
+```
+
+
+### Chunk options
+
+Uploads are sliced into calldata chunks. Both `estimateCost` and `upload`
+requests accept two optional fields:
+
+| Field          | Type   | Default | Description                                              |
+|----------------|--------|---------|----------------------------------------------------------|
+| `maxChunkSize` | number | `64`    | data size of each chunk, in KB                            |
+| `maxChunks`    | number | `100`   | maximum number of chunks requested in a single call       |
+
+```js
+const request = {
+    key: "example1.txt",
+    content: Buffer.from("large data to upload"),
+    maxChunkSize: 32, // 32 KB per chunk
+    maxChunks: 120    // at most 120 chunks per request
+}
+await flatDirectory.upload({ ...request, callback });
+```
+
+`fetchHashes(keys, maxChunks)` accepts the same chunk limit as its second argument.

package/dist/browser/index.d.mts

@@ -162,8 +162,44 @@ interface ChunkHashResult {
 }
 
 declare const stringToHex: (s: string) => string;
+/**
+ * EIP-3541 forbids deploying new contract code whose first byte is 0xEF.
+ * EthStorage stores each chunk as a contract via CREATE2, so a chunk whose
+ * first byte is 0xEF will revert. We avoid this by shifting chunk boundaries.
+ */
+declare const EF_MAGIC: number;
+/**
+ * Number of bytes reserved at the end of each chunk as extension headroom.
+ *
+ * The base chunk size is reduced by this many bytes; when a boundary would
+ * leave the next chunk starting with 0xEF, the current chunk is extended by
+ * up to this many bytes to absorb the run. The extended size never exceeds
+ * the original (unreserved) chunk size, so the contract's maxChunkSize limit
+ * is never violated.
+ *
+ * Probability justification (uniform random bytes):
+ *   P(16 consecutive 0xEF) = (1/256)^16 ≈ 1e-38, effectively zero.
+ *   Even across billions of boundaries the probability remains negligible.
+ *   If a longer run ever occurs, the shrink fallback handles it.
+ */
+declare const EF_RESERVED_BYTES: number;
+interface ChunkBoundary {
+    start: number;
+    end: number;
+}
 declare function getChainId(rpc: string): Promise<number>;
 declare function getContentChunk(content: ContentLike, start: number, end: number): Promise<Uint8Array<ArrayBuffer>>;
+/**
+ * Compute chunk boundaries for a calldata upload, shifting any boundary that
+ * would leave the next chunk starting with the 0xEF magic byte (EIP-3541).
+ *
+ * Strategy: for each boundary, if the byte at the boundary (first byte of the
+ * next chunk) is 0xEF, extend the current chunk to absorb it. If extension
+ * would exceed maxChunkSize, shrink the current chunk instead. In the extremely
+ * unlikely case that every byte in the adjustment window is 0xEF, the original
+ * fixed boundary is kept.
+ */
+declare function computeCalldataChunkBoundaries(content: ContentLike, baseChunkSize: number, maxChunkSize: number): Promise<ChunkBoundary[]>;
 declare function isBuffer(content: ContentLike): content is BufferLike;
 declare function isFile(content: ContentLike): content is FileLike;
 declare function isNodejs(): boolean;
@@ -212,9 +248,13 @@ declare function getChunkHashes(contract: Contract, batch: FileBatch[]): Promise
 
 declare function stableRetry<T>(fn: (...args: unknown[]) => Promise<T>, ...args: unknown[]): Promise<T>;
 
+type index_ChunkBoundary = ChunkBoundary;
+declare const index_EF_MAGIC: typeof EF_MAGIC;
+declare const index_EF_RESERVED_BYTES: typeof EF_RESERVED_BYTES;
 type index_TxSender = TxSender;
 declare const index_TxSender: typeof TxSender;
 declare const index_calcTxCost: typeof calcTxCost;
+declare const index_computeCalldataChunkBoundaries: typeof computeCalldataChunkBoundaries;
 declare const index_getChainId: typeof getChainId;
 declare const index_getChunkCounts: typeof getChunkCounts;
 declare const index_getChunkHashes: typeof getChunkHashes;
@@ -226,7 +266,7 @@ declare const index_isNodejs: typeof isNodejs;
 declare const index_stableRetry: typeof stableRetry;
 declare const index_stringToHex: typeof stringToHex;
 declare namespace index {
-  export { index_TxSender as TxSender, index_calcTxCost as calcTxCost, index_getChainId as getChainId, index_getChunkCounts as getChunkCounts, index_getChunkHashes as getChunkHashes, index_getContentChunk as getContentChunk, index_getUploadInfo as getUploadInfo, index_isBuffer as isBuffer, index_isFile as isFile, index_isNodejs as isNodejs, index_stableRetry as stableRetry, index_stringToHex as stringToHex };
+  export { type index_ChunkBoundary as ChunkBoundary, index_EF_MAGIC as EF_MAGIC, index_EF_RESERVED_BYTES as EF_RESERVED_BYTES, index_TxSender as TxSender, index_calcTxCost as calcTxCost, index_computeCalldataChunkBoundaries as computeCalldataChunkBoundaries, index_getChainId as getChainId, index_getChunkCounts as getChunkCounts, index_getChunkHashes as getChunkHashes, index_getContentChunk as getContentChunk, index_getUploadInfo as getUploadInfo, index_isBuffer as isBuffer, index_isFile as isFile, index_isNodejs as isNodejs, index_stableRetry as stableRetry, index_stringToHex as stringToHex };
 }
 
 declare class FlatDirectory {

package/dist/browser/index.mjs

@@ -7,8 +7,11 @@ var __export = (target, all) => {
 // src/utils/index.ts
 var utils_exports = {};
 __export(utils_exports, {
+  EF_MAGIC: () => EF_MAGIC,
+  EF_RESERVED_BYTES: () => EF_RESERVED_BYTES,
   TxSender: () => TxSender,
   calcTxCost: () => calcTxCost,
+  computeCalldataChunkBoundaries: () => computeCalldataChunkBoundaries,
   getChainId: () => getChainId,
   getChunkCounts: () => getChunkCounts,
   getChunkHashes: () => getChunkHashes,
@@ -24,6 +27,8 @@ __export(utils_exports, {
 // src/utils/util.ts
 import { ethers } from "ethers";
 var stringToHex = (s) => ethers.hexlify(ethers.toUtf8Bytes(s));
+var EF_MAGIC = 239;
+var EF_RESERVED_BYTES = 16;
 async function getChainId(rpc) {
   const provider = new ethers.JsonRpcProvider(rpc);
   const network = await provider.getNetwork();
@@ -38,6 +43,50 @@ async function getContentChunk(content, start, end) {
     return new Uint8Array(data);
   }
 }
+async function computeCalldataChunkBoundaries(content, baseChunkSize, maxChunkSize) {
+  const size = isFile(content) ? content.size : content.length;
+  const boundaries = [];
+  let pos = 0;
+  while (pos < size) {
+    let end = Math.min(pos + baseChunkSize, size);
+    if (end < size) {
+      const maxExtendEnd = Math.min(pos + maxChunkSize, size);
+      if (maxExtendEnd > end) {
+        const extWindow = await getContentChunk(content, end, maxExtendEnd);
+        let safeOffset = -1;
+        for (let i = 0; i < extWindow.length; i++) {
+          if (extWindow[i] !== EF_MAGIC) {
+            safeOffset = i;
+            break;
+          }
+        }
+        if (safeOffset >= 0) {
+          end = end + safeOffset;
+        }
+      }
+      if (end < size) {
+        const firstByte = await getContentChunk(content, end, end + 1);
+        if (firstByte.length > 0 && firstByte[0] === EF_MAGIC) {
+          const shrinkStart = Math.max(pos, end - 256);
+          const shrinkWindow = await getContentChunk(content, shrinkStart, end);
+          let lastSafe = -1;
+          for (let i = shrinkWindow.length - 1; i >= 0; i--) {
+            if (shrinkWindow[i] !== EF_MAGIC) {
+              lastSafe = shrinkStart + i;
+              break;
+            }
+          }
+          if (lastSafe > pos) {
+            end = lastSafe;
+          }
+        }
+      }
+    }
+    boundaries.push({ start: pos, end });
+    pos = end;
+  }
+  return boundaries;
+}
 function isBuffer(content) {
   return content instanceof Uint8Array;
 }
@@ -762,10 +811,13 @@ var FlatDirectory = class _FlatDirectory {
     if (toChunkSizeBytes(maxChunkSize) > CONTRACT_MAX_CHUNK_SIZE) {
       throw new Error(`FlatDirectory: Chunk size ${maxChunkSize} KB exceeds the on-chain max payload of ${CONTRACT_MAX_CHUNK_SIZE} bytes.`);
     }
-    const { chunkDataSize, calldataChunkCount } = this.#calculateCalldataChunkDetails(content, maxChunkSize);
+    const { chunkDataSize } = this.#calculateCalldataChunkDetails(content, maxChunkSize);
     if (chunkDataSize === -1) {
       throw new Error(`FlatDirectory: Invalid upload content!`);
     }
+    const baseChunkSize = Math.max(1, chunkDataSize - EF_RESERVED_BYTES);
+    const boundaries = await computeCalldataChunkBoundaries(content, baseChunkSize, chunkDataSize);
+    const calldataChunkCount = boundaries.length;
     const hexName = stringToHex(key);
     const fileContract = new ethers3.Contract(this.#contractAddrChecked, FlatDirectoryAbi, this.#walletChecked);
     const [{ oldChunkCount, fileMode }, resolvedFees] = await Promise.all([
@@ -791,7 +843,8 @@ var FlatDirectory = class _FlatDirectory {
       () => fileContract["writeChunkByCalldata"].estimateGas(hexName, chunkId, hexData)
     );
     for (let i = 0; i < calldataChunkCount; i++) {
-      const chunk = await getContentChunk(content, i * chunkDataSize, (i + 1) * chunkDataSize);
+      const { start, end } = boundaries[i];
+      const chunk = await getContentChunk(content, start, end);
       if (i < chunkHashes.length && ethers3.keccak256(chunk) === chunkHashes[i]) {
         continue;
       }
@@ -830,12 +883,15 @@ var FlatDirectory = class _FlatDirectory {
       callback.onFinish(totalUploadChunks, totalUploadSize, totalCost);
       return;
     }
-    const { chunkDataSize, calldataChunkCount } = this.#calculateCalldataChunkDetails(content, maxChunkSize);
+    const { chunkDataSize } = this.#calculateCalldataChunkDetails(content, maxChunkSize);
     if (chunkDataSize === -1) {
       callback.onFail(new Error(`FlatDirectory: Invalid upload content!`));
       callback.onFinish(totalUploadChunks, totalUploadSize, totalCost);
       return;
     }
+    const baseChunkSize = Math.max(1, chunkDataSize - EF_RESERVED_BYTES);
+    const boundaries = await computeCalldataChunkBoundaries(content, baseChunkSize, chunkDataSize);
+    const calldataChunkCount = boundaries.length;
     const hexName = stringToHex(key);
     const fileContract = new ethers3.Contract(this.#contractAddrChecked, FlatDirectoryAbi, this.#walletChecked);
     const { oldChunkCount, fileMode } = await getUploadInfo(fileContract, hexName);
@@ -876,7 +932,8 @@ var FlatDirectory = class _FlatDirectory {
       }
     }
     for (let i = 0; i < calldataChunkCount; i++) {
-      const chunk = await getContentChunk(content, i * chunkDataSize, (i + 1) * chunkDataSize);
+      const { start, end } = boundaries[i];
+      const chunk = await getContentChunk(content, start, end);
       if (i < chunkHashes.length && ethers3.keccak256(chunk) === chunkHashes[i]) {
         callback.onProgress(i, calldataChunkCount, false);
         continue;

package/dist/node/index.cjs

@@ -52,8 +52,11 @@ module.exports = __toCommonJS(index_exports);
 // src/utils/index.ts
 var utils_exports = {};
 __export(utils_exports, {
+  EF_MAGIC: () => EF_MAGIC,
+  EF_RESERVED_BYTES: () => EF_RESERVED_BYTES,
   TxSender: () => TxSender,
   calcTxCost: () => calcTxCost,
+  computeCalldataChunkBoundaries: () => computeCalldataChunkBoundaries,
   getChainId: () => getChainId,
   getChunkCounts: () => getChunkCounts,
   getChunkHashes: () => getChunkHashes,
@@ -69,6 +72,8 @@ __export(utils_exports, {
 // src/utils/util.ts
 var import_ethers = require("ethers");
 var stringToHex = (s) => import_ethers.ethers.hexlify(import_ethers.ethers.toUtf8Bytes(s));
+var EF_MAGIC = 239;
+var EF_RESERVED_BYTES = 16;
 async function getChainId(rpc) {
   const provider = new import_ethers.ethers.JsonRpcProvider(rpc);
   const network = await provider.getNetwork();
@@ -83,6 +88,50 @@ async function getContentChunk(content, start, end) {
     return new Uint8Array(data);
   }
 }
+async function computeCalldataChunkBoundaries(content, baseChunkSize, maxChunkSize) {
+  const size = isFile(content) ? content.size : content.length;
+  const boundaries = [];
+  let pos = 0;
+  while (pos < size) {
+    let end = Math.min(pos + baseChunkSize, size);
+    if (end < size) {
+      const maxExtendEnd = Math.min(pos + maxChunkSize, size);
+      if (maxExtendEnd > end) {
+        const extWindow = await getContentChunk(content, end, maxExtendEnd);
+        let safeOffset = -1;
+        for (let i = 0; i < extWindow.length; i++) {
+          if (extWindow[i] !== EF_MAGIC) {
+            safeOffset = i;
+            break;
+          }
+        }
+        if (safeOffset >= 0) {
+          end = end + safeOffset;
+        }
+      }
+      if (end < size) {
+        const firstByte = await getContentChunk(content, end, end + 1);
+        if (firstByte.length > 0 && firstByte[0] === EF_MAGIC) {
+          const shrinkStart = Math.max(pos, end - 256);
+          const shrinkWindow = await getContentChunk(content, shrinkStart, end);
+          let lastSafe = -1;
+          for (let i = shrinkWindow.length - 1; i >= 0; i--) {
+            if (shrinkWindow[i] !== EF_MAGIC) {
+              lastSafe = shrinkStart + i;
+              break;
+            }
+          }
+          if (lastSafe > pos) {
+            end = lastSafe;
+          }
+        }
+      }
+    }
+    boundaries.push({ start: pos, end });
+    pos = end;
+  }
+  return boundaries;
+}
 function isBuffer(content) {
   return content instanceof Uint8Array;
 }
@@ -807,10 +856,13 @@ var FlatDirectory = class _FlatDirectory {
     if (toChunkSizeBytes(maxChunkSize) > CONTRACT_MAX_CHUNK_SIZE) {
       throw new Error(`FlatDirectory: Chunk size ${maxChunkSize} KB exceeds the on-chain max payload of ${CONTRACT_MAX_CHUNK_SIZE} bytes.`);
     }
-    const { chunkDataSize, calldataChunkCount } = this.#calculateCalldataChunkDetails(content, maxChunkSize);
+    const { chunkDataSize } = this.#calculateCalldataChunkDetails(content, maxChunkSize);
     if (chunkDataSize === -1) {
       throw new Error(`FlatDirectory: Invalid upload content!`);
     }
+    const baseChunkSize = Math.max(1, chunkDataSize - EF_RESERVED_BYTES);
+    const boundaries = await computeCalldataChunkBoundaries(content, baseChunkSize, chunkDataSize);
+    const calldataChunkCount = boundaries.length;
     const hexName = stringToHex(key);
     const fileContract = new import_ethers3.ethers.Contract(this.#contractAddrChecked, FlatDirectoryAbi, this.#walletChecked);
     const [{ oldChunkCount, fileMode }, resolvedFees] = await Promise.all([
@@ -836,7 +888,8 @@ var FlatDirectory = class _FlatDirectory {
       () => fileContract["writeChunkByCalldata"].estimateGas(hexName, chunkId, hexData)
     );
     for (let i = 0; i < calldataChunkCount; i++) {
-      const chunk = await getContentChunk(content, i * chunkDataSize, (i + 1) * chunkDataSize);
+      const { start, end } = boundaries[i];
+      const chunk = await getContentChunk(content, start, end);
       if (i < chunkHashes.length && import_ethers3.ethers.keccak256(chunk) === chunkHashes[i]) {
         continue;
       }
@@ -875,12 +928,15 @@ var FlatDirectory = class _FlatDirectory {
       callback.onFinish(totalUploadChunks, totalUploadSize, totalCost);
       return;
     }
-    const { chunkDataSize, calldataChunkCount } = this.#calculateCalldataChunkDetails(content, maxChunkSize);
+    const { chunkDataSize } = this.#calculateCalldataChunkDetails(content, maxChunkSize);
     if (chunkDataSize === -1) {
       callback.onFail(new Error(`FlatDirectory: Invalid upload content!`));
       callback.onFinish(totalUploadChunks, totalUploadSize, totalCost);
       return;
     }
+    const baseChunkSize = Math.max(1, chunkDataSize - EF_RESERVED_BYTES);
+    const boundaries = await computeCalldataChunkBoundaries(content, baseChunkSize, chunkDataSize);
+    const calldataChunkCount = boundaries.length;
     const hexName = stringToHex(key);
     const fileContract = new import_ethers3.ethers.Contract(this.#contractAddrChecked, FlatDirectoryAbi, this.#walletChecked);
     const { oldChunkCount, fileMode } = await getUploadInfo(fileContract, hexName);
@@ -921,7 +977,8 @@ var FlatDirectory = class _FlatDirectory {
       }
     }
     for (let i = 0; i < calldataChunkCount; i++) {
-      const chunk = await getContentChunk(content, i * chunkDataSize, (i + 1) * chunkDataSize);
+      const { start, end } = boundaries[i];
+      const chunk = await getContentChunk(content, start, end);
       if (i < chunkHashes.length && import_ethers3.ethers.keccak256(chunk) === chunkHashes[i]) {
         callback.onProgress(i, calldataChunkCount, false);
         continue;

package/dist/node/index.d.mts

@@ -162,8 +162,44 @@ interface ChunkHashResult {
 }
 
 declare const stringToHex: (s: string) => string;
+/**
+ * EIP-3541 forbids deploying new contract code whose first byte is 0xEF.
+ * EthStorage stores each chunk as a contract via CREATE2, so a chunk whose
+ * first byte is 0xEF will revert. We avoid this by shifting chunk boundaries.
+ */
+declare const EF_MAGIC: number;
+/**
+ * Number of bytes reserved at the end of each chunk as extension headroom.
+ *
+ * The base chunk size is reduced by this many bytes; when a boundary would
+ * leave the next chunk starting with 0xEF, the current chunk is extended by
+ * up to this many bytes to absorb the run. The extended size never exceeds
+ * the original (unreserved) chunk size, so the contract's maxChunkSize limit
+ * is never violated.
+ *
+ * Probability justification (uniform random bytes):
+ *   P(16 consecutive 0xEF) = (1/256)^16 ≈ 1e-38, effectively zero.
+ *   Even across billions of boundaries the probability remains negligible.
+ *   If a longer run ever occurs, the shrink fallback handles it.
+ */
+declare const EF_RESERVED_BYTES: number;
+interface ChunkBoundary {
+    start: number;
+    end: number;
+}
 declare function getChainId(rpc: string): Promise<number>;
 declare function getContentChunk(content: ContentLike, start: number, end: number): Promise<Uint8Array<ArrayBuffer>>;
+/**
+ * Compute chunk boundaries for a calldata upload, shifting any boundary that
+ * would leave the next chunk starting with the 0xEF magic byte (EIP-3541).
+ *
+ * Strategy: for each boundary, if the byte at the boundary (first byte of the
+ * next chunk) is 0xEF, extend the current chunk to absorb it. If extension
+ * would exceed maxChunkSize, shrink the current chunk instead. In the extremely
+ * unlikely case that every byte in the adjustment window is 0xEF, the original
+ * fixed boundary is kept.
+ */
+declare function computeCalldataChunkBoundaries(content: ContentLike, baseChunkSize: number, maxChunkSize: number): Promise<ChunkBoundary[]>;
 declare function isBuffer(content: ContentLike): content is BufferLike;
 declare function isFile(content: ContentLike): content is FileLike;
 declare function isNodejs(): boolean;
@@ -212,9 +248,13 @@ declare function getChunkHashes(contract: Contract, batch: FileBatch[]): Promise
 
 declare function stableRetry<T>(fn: (...args: unknown[]) => Promise<T>, ...args: unknown[]): Promise<T>;
 
+type index_ChunkBoundary = ChunkBoundary;
+declare const index_EF_MAGIC: typeof EF_MAGIC;
+declare const index_EF_RESERVED_BYTES: typeof EF_RESERVED_BYTES;
 type index_TxSender = TxSender;
 declare const index_TxSender: typeof TxSender;
 declare const index_calcTxCost: typeof calcTxCost;
+declare const index_computeCalldataChunkBoundaries: typeof computeCalldataChunkBoundaries;
 declare const index_getChainId: typeof getChainId;
 declare const index_getChunkCounts: typeof getChunkCounts;
 declare const index_getChunkHashes: typeof getChunkHashes;
@@ -226,7 +266,7 @@ declare const index_isNodejs: typeof isNodejs;
 declare const index_stableRetry: typeof stableRetry;
 declare const index_stringToHex: typeof stringToHex;
 declare namespace index {
-  export { index_TxSender as TxSender, index_calcTxCost as calcTxCost, index_getChainId as getChainId, index_getChunkCounts as getChunkCounts, index_getChunkHashes as getChunkHashes, index_getContentChunk as getContentChunk, index_getUploadInfo as getUploadInfo, index_isBuffer as isBuffer, index_isFile as isFile, index_isNodejs as isNodejs, index_stableRetry as stableRetry, index_stringToHex as stringToHex };
+  export { type index_ChunkBoundary as ChunkBoundary, index_EF_MAGIC as EF_MAGIC, index_EF_RESERVED_BYTES as EF_RESERVED_BYTES, index_TxSender as TxSender, index_calcTxCost as calcTxCost, index_computeCalldataChunkBoundaries as computeCalldataChunkBoundaries, index_getChainId as getChainId, index_getChunkCounts as getChunkCounts, index_getChunkHashes as getChunkHashes, index_getContentChunk as getContentChunk, index_getUploadInfo as getUploadInfo, index_isBuffer as isBuffer, index_isFile as isFile, index_isNodejs as isNodejs, index_stableRetry as stableRetry, index_stringToHex as stringToHex };
 }
 
 declare class FlatDirectory {

package/dist/node/index.d.ts

@@ -162,8 +162,44 @@ interface ChunkHashResult {
 }
 
 declare const stringToHex: (s: string) => string;
+/**
+ * EIP-3541 forbids deploying new contract code whose first byte is 0xEF.
+ * EthStorage stores each chunk as a contract via CREATE2, so a chunk whose
+ * first byte is 0xEF will revert. We avoid this by shifting chunk boundaries.
+ */
+declare const EF_MAGIC: number;
+/**
+ * Number of bytes reserved at the end of each chunk as extension headroom.
+ *
+ * The base chunk size is reduced by this many bytes; when a boundary would
+ * leave the next chunk starting with 0xEF, the current chunk is extended by
+ * up to this many bytes to absorb the run. The extended size never exceeds
+ * the original (unreserved) chunk size, so the contract's maxChunkSize limit
+ * is never violated.
+ *
+ * Probability justification (uniform random bytes):
+ *   P(16 consecutive 0xEF) = (1/256)^16 ≈ 1e-38, effectively zero.
+ *   Even across billions of boundaries the probability remains negligible.
+ *   If a longer run ever occurs, the shrink fallback handles it.
+ */
+declare const EF_RESERVED_BYTES: number;
+interface ChunkBoundary {
+    start: number;
+    end: number;
+}
 declare function getChainId(rpc: string): Promise<number>;
 declare function getContentChunk(content: ContentLike, start: number, end: number): Promise<Uint8Array<ArrayBuffer>>;
+/**
+ * Compute chunk boundaries for a calldata upload, shifting any boundary that
+ * would leave the next chunk starting with the 0xEF magic byte (EIP-3541).
+ *
+ * Strategy: for each boundary, if the byte at the boundary (first byte of the
+ * next chunk) is 0xEF, extend the current chunk to absorb it. If extension
+ * would exceed maxChunkSize, shrink the current chunk instead. In the extremely
+ * unlikely case that every byte in the adjustment window is 0xEF, the original
+ * fixed boundary is kept.
+ */
+declare function computeCalldataChunkBoundaries(content: ContentLike, baseChunkSize: number, maxChunkSize: number): Promise<ChunkBoundary[]>;
 declare function isBuffer(content: ContentLike): content is BufferLike;
 declare function isFile(content: ContentLike): content is FileLike;
 declare function isNodejs(): boolean;
@@ -212,9 +248,13 @@ declare function getChunkHashes(contract: Contract, batch: FileBatch[]): Promise
 
 declare function stableRetry<T>(fn: (...args: unknown[]) => Promise<T>, ...args: unknown[]): Promise<T>;
 
+type index_ChunkBoundary = ChunkBoundary;
+declare const index_EF_MAGIC: typeof EF_MAGIC;
+declare const index_EF_RESERVED_BYTES: typeof EF_RESERVED_BYTES;
 type index_TxSender = TxSender;
 declare const index_TxSender: typeof TxSender;
 declare const index_calcTxCost: typeof calcTxCost;
+declare const index_computeCalldataChunkBoundaries: typeof computeCalldataChunkBoundaries;
 declare const index_getChainId: typeof getChainId;
 declare const index_getChunkCounts: typeof getChunkCounts;
 declare const index_getChunkHashes: typeof getChunkHashes;
@@ -226,7 +266,7 @@ declare const index_isNodejs: typeof isNodejs;
 declare const index_stableRetry: typeof stableRetry;
 declare const index_stringToHex: typeof stringToHex;
 declare namespace index {
-  export { index_TxSender as TxSender, index_calcTxCost as calcTxCost, index_getChainId as getChainId, index_getChunkCounts as getChunkCounts, index_getChunkHashes as getChunkHashes, index_getContentChunk as getContentChunk, index_getUploadInfo as getUploadInfo, index_isBuffer as isBuffer, index_isFile as isFile, index_isNodejs as isNodejs, index_stableRetry as stableRetry, index_stringToHex as stringToHex };
+  export { type index_ChunkBoundary as ChunkBoundary, index_EF_MAGIC as EF_MAGIC, index_EF_RESERVED_BYTES as EF_RESERVED_BYTES, index_TxSender as TxSender, index_calcTxCost as calcTxCost, index_computeCalldataChunkBoundaries as computeCalldataChunkBoundaries, index_getChainId as getChainId, index_getChunkCounts as getChunkCounts, index_getChunkHashes as getChunkHashes, index_getContentChunk as getContentChunk, index_getUploadInfo as getUploadInfo, index_isBuffer as isBuffer, index_isFile as isFile, index_isNodejs as isNodejs, index_stableRetry as stableRetry, index_stringToHex as stringToHex };
 }
 
 declare class FlatDirectory {

package/dist/node/index.mjs

@@ -7,8 +7,11 @@ var __export = (target, all) => {
 // src/utils/index.ts
 var utils_exports = {};
 __export(utils_exports, {
+  EF_MAGIC: () => EF_MAGIC,
+  EF_RESERVED_BYTES: () => EF_RESERVED_BYTES,
   TxSender: () => TxSender,
   calcTxCost: () => calcTxCost,
+  computeCalldataChunkBoundaries: () => computeCalldataChunkBoundaries,
   getChainId: () => getChainId,
   getChunkCounts: () => getChunkCounts,
   getChunkHashes: () => getChunkHashes,
@@ -24,6 +27,8 @@ __export(utils_exports, {
 // src/utils/util.ts
 import { ethers } from "ethers";
 var stringToHex = (s) => ethers.hexlify(ethers.toUtf8Bytes(s));
+var EF_MAGIC = 239;
+var EF_RESERVED_BYTES = 16;
 async function getChainId(rpc) {
   const provider = new ethers.JsonRpcProvider(rpc);
   const network = await provider.getNetwork();
@@ -38,6 +43,50 @@ async function getContentChunk(content, start, end) {
     return new Uint8Array(data);
   }
 }
+async function computeCalldataChunkBoundaries(content, baseChunkSize, maxChunkSize) {
+  const size = isFile(content) ? content.size : content.length;
+  const boundaries = [];
+  let pos = 0;
+  while (pos < size) {
+    let end = Math.min(pos + baseChunkSize, size);
+    if (end < size) {
+      const maxExtendEnd = Math.min(pos + maxChunkSize, size);
+      if (maxExtendEnd > end) {
+        const extWindow = await getContentChunk(content, end, maxExtendEnd);
+        let safeOffset = -1;
+        for (let i = 0; i < extWindow.length; i++) {
+          if (extWindow[i] !== EF_MAGIC) {
+            safeOffset = i;
+            break;
+          }
+        }
+        if (safeOffset >= 0) {
+          end = end + safeOffset;
+        }
+      }
+      if (end < size) {
+        const firstByte = await getContentChunk(content, end, end + 1);
+        if (firstByte.length > 0 && firstByte[0] === EF_MAGIC) {
+          const shrinkStart = Math.max(pos, end - 256);
+          const shrinkWindow = await getContentChunk(content, shrinkStart, end);
+          let lastSafe = -1;
+          for (let i = shrinkWindow.length - 1; i >= 0; i--) {
+            if (shrinkWindow[i] !== EF_MAGIC) {
+              lastSafe = shrinkStart + i;
+              break;
+            }
+          }
+          if (lastSafe > pos) {
+            end = lastSafe;
+          }
+        }
+      }
+    }
+    boundaries.push({ start: pos, end });
+    pos = end;
+  }
+  return boundaries;
+}
 function isBuffer(content) {
   return content instanceof Uint8Array;
 }
@@ -762,10 +811,13 @@ var FlatDirectory = class _FlatDirectory {
     if (toChunkSizeBytes(maxChunkSize) > CONTRACT_MAX_CHUNK_SIZE) {
       throw new Error(`FlatDirectory: Chunk size ${maxChunkSize} KB exceeds the on-chain max payload of ${CONTRACT_MAX_CHUNK_SIZE} bytes.`);
     }
-    const { chunkDataSize, calldataChunkCount } = this.#calculateCalldataChunkDetails(content, maxChunkSize);
+    const { chunkDataSize } = this.#calculateCalldataChunkDetails(content, maxChunkSize);
     if (chunkDataSize === -1) {
       throw new Error(`FlatDirectory: Invalid upload content!`);
     }
+    const baseChunkSize = Math.max(1, chunkDataSize - EF_RESERVED_BYTES);
+    const boundaries = await computeCalldataChunkBoundaries(content, baseChunkSize, chunkDataSize);
+    const calldataChunkCount = boundaries.length;
     const hexName = stringToHex(key);
     const fileContract = new ethers3.Contract(this.#contractAddrChecked, FlatDirectoryAbi, this.#walletChecked);
     const [{ oldChunkCount, fileMode }, resolvedFees] = await Promise.all([
@@ -791,7 +843,8 @@ var FlatDirectory = class _FlatDirectory {
       () => fileContract["writeChunkByCalldata"].estimateGas(hexName, chunkId, hexData)
     );
     for (let i = 0; i < calldataChunkCount; i++) {
-      const chunk = await getContentChunk(content, i * chunkDataSize, (i + 1) * chunkDataSize);
+      const { start, end } = boundaries[i];
+      const chunk = await getContentChunk(content, start, end);
       if (i < chunkHashes.length && ethers3.keccak256(chunk) === chunkHashes[i]) {
         continue;
       }
@@ -830,12 +883,15 @@ var FlatDirectory = class _FlatDirectory {
       callback.onFinish(totalUploadChunks, totalUploadSize, totalCost);
       return;
     }
-    const { chunkDataSize, calldataChunkCount } = this.#calculateCalldataChunkDetails(content, maxChunkSize);
+    const { chunkDataSize } = this.#calculateCalldataChunkDetails(content, maxChunkSize);
     if (chunkDataSize === -1) {
       callback.onFail(new Error(`FlatDirectory: Invalid upload content!`));
       callback.onFinish(totalUploadChunks, totalUploadSize, totalCost);
       return;
     }
+    const baseChunkSize = Math.max(1, chunkDataSize - EF_RESERVED_BYTES);
+    const boundaries = await computeCalldataChunkBoundaries(content, baseChunkSize, chunkDataSize);
+    const calldataChunkCount = boundaries.length;
     const hexName = stringToHex(key);
     const fileContract = new ethers3.Contract(this.#contractAddrChecked, FlatDirectoryAbi, this.#walletChecked);
     const { oldChunkCount, fileMode } = await getUploadInfo(fileContract, hexName);
@@ -876,7 +932,8 @@ var FlatDirectory = class _FlatDirectory {
       }
     }
     for (let i = 0; i < calldataChunkCount; i++) {
-      const chunk = await getContentChunk(content, i * chunkDataSize, (i + 1) * chunkDataSize);
+      const { start, end } = boundaries[i];
+      const chunk = await getContentChunk(content, start, end);
       if (i < chunkHashes.length && ethers3.keccak256(chunk) === chunkHashes[i]) {
         callback.onProgress(i, calldataChunkCount, false);
         continue;

package/package.json

@@ -1,65 +1,65 @@
-{
-  "name": "gethstorage-sdk",
-  "version": "5.0.2",
-  "description": "EthStorage flat directory upload sdk (calldata only)",
-  "files": [
-    "dist"
-  ],
-  "main": "./dist/node/index.cjs",
-  "types": "./dist/node/index.d.ts",
-  "exports": {
-    ".": {
-      "browser": "./dist/browser/index.mjs",
-      "node": {
-        "import": "./dist/node/index.mjs",
-        "require": "./dist/node/index.cjs"
-      },
-      "types": "./dist/node/index.d.ts",
-      "default": "./dist/node/index.mjs"
-    },
-    "./file": {
-      "node": {
-        "import": "./dist/node/file.mjs",
-        "require": "./dist/node/file.cjs"
-      },
-      "types": "./dist/node/file.d.ts"
-    }
-  },
-  "scripts": {
-    "build": "node -e \"require('fs').rmSync('dist',{recursive:true,force:true})\" && tsup",
-    "dev": "tsup --watch --env.NODE_ENV development",
-    "prepublishOnly": "npm run build"
-  },
-  "dependencies": {
-    "dotenv": "^16.4.5",
-    "ethers": "^6.17.0",
-    "p-limit": "^7.2.0"
-  },
-  "repository": {
-    "type": "git",
-    "url": "git+https://github.com/0xCampanula/gethstorage-sdk.git"
-  },
-  "bugs": {
-    "url": "https://github.com/0xCampanula/gethstorage-sdk/issues"
-  },
-  "homepage": "https://github.com/0xCampanula/gethstorage-sdk/main/",
-  "keywords": [
-    "File",
-    "Upload"
-  ],
-  "author": "ethsorage",
-  "license": "ISC",
-  "devDependencies": {
-    "tsup": "^8.5.1",
-    "typescript": "^5.9.3"
-  },
-  "resolutions": {
-    "strip-ansi": "7.1.0",
-    "color-convert": "2.0.1",
-    "color-name": "1.1.4"
-  },
-  "overrides": {
-    "rollup": ">=4.59.0",
-    "picomatch": ">=4.0.4"
-  }
-}
+{
+  "name": "gethstorage-sdk",
+  "version": "5.0.3",
+  "description": "EthStorage flat directory upload sdk (calldata only)",
+  "files": [
+    "dist"
+  ],
+  "main": "./dist/node/index.cjs",
+  "types": "./dist/node/index.d.ts",
+  "exports": {
+    ".": {
+      "browser": "./dist/browser/index.mjs",
+      "node": {
+        "import": "./dist/node/index.mjs",
+        "require": "./dist/node/index.cjs"
+      },
+      "types": "./dist/node/index.d.ts",
+      "default": "./dist/node/index.mjs"
+    },
+    "./file": {
+      "node": {
+        "import": "./dist/node/file.mjs",
+        "require": "./dist/node/file.cjs"
+      },
+      "types": "./dist/node/file.d.ts"
+    }
+  },
+  "scripts": {
+    "build": "node -e \"require('fs').rmSync('dist',{recursive:true,force:true})\" && tsup",
+    "dev": "tsup --watch --env.NODE_ENV development",
+    "prepublishOnly": "npm run build"
+  },
+  "dependencies": {
+    "dotenv": "^16.4.5",
+    "ethers": "^6.17.0",
+    "p-limit": "^7.2.0"
+  },
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/0xCampanula/gethstorage-sdk.git"
+  },
+  "bugs": {
+    "url": "https://github.com/0xCampanula/gethstorage-sdk/issues"
+  },
+  "homepage": "https://github.com/0xCampanula/gethstorage-sdk/main/",
+  "keywords": [
+    "File",
+    "Upload"
+  ],
+  "author": "ethsorage",
+  "license": "ISC",
+  "devDependencies": {
+    "tsup": "^8.5.1",
+    "typescript": "^5.9.3"
+  },
+  "resolutions": {
+    "strip-ansi": "7.1.0",
+    "color-convert": "2.0.1",
+    "color-name": "1.1.4"
+  },
+  "overrides": {
+    "rollup": ">=4.59.0",
+    "picomatch": ">=4.0.4"
+  }
+}