@ckbfs/api 1.5.0 → 2.0.0

package/README.md

@@ -11,7 +11,8 @@ CKBFS is a file system protocol built on top of the CKB blockchain that enables
 - **File Publishing**: Store files of any type on the CKB blockchain
 - **File Appending**: Add content to existing files with automatic checksum updates
 - **Content Integrity**: Adler32 checksum verification for all file operations
-- **Protocol Versions**: Support for both V1 and V2 CKBFS protocol versions
+- **Protocol Versions**: Support for V1, V2, and V3 CKBFS protocol versions
+- **Witnesses-based Storage**: V3 uses cost-efficient witness storage with backlink tracking
 - **Network Support**: Compatible with CKB mainnet and testnet
 - **Chunked Storage**: Automatic file chunking for large files
 
@@ -31,7 +32,7 @@ const ckbfs = new CKBFS(
   'your-private-key-here',
   NetworkType.Testnet,
   {
-    version: ProtocolVersion.V2,
+    version: ProtocolVersion.V3, // V3 is now default
     chunkSize: 30 * 1024, // 30KB chunks
     useTypeID: false
   }
@@ -65,7 +66,7 @@ new CKBFS(signerOrPrivateKey, networkOrOptions?, options?)
 ```typescript
 interface CKBFSOptions {
   chunkSize?: number;     // Default: 30KB
-  version?: string;       // Default: V2
+  version?: string;       // Default: V3
   useTypeID?: boolean;    // Default: false
   network?: NetworkType;  // Default: Testnet
 }
@@ -260,7 +261,7 @@ console.log(`Type: ${parsed.type}`); // "typeId" or "outPoint"
 // Get file content using any identifier format (follows backlinks automatically)
 const fileData = await getFileContentFromChainByIdentifier(client, ckbfsTypeIdUri, {
   network: 'testnet',
-  version: ProtocolVersion.V2,
+  version: ProtocolVersion.V3, // V3 supports witness-based backlinks
   useTypeID: false
 });
 
@@ -445,11 +446,31 @@ CKBFS uses a molecule-encoded data structure stored in cell output data:
 }
 ```
 
+**V3 Format:**
+```
+{
+  index: number,           // Single witness index (first content witness)
+  checksum: number,        // Adler32 checksum
+  contentType: string,     // MIME type
+  filename: string,        // Original filename
+  // Note: No backLinks in cell data - moved to witnesses for cost efficiency
+}
+```
+
 ### Witness Format
 
-CKBFS witnesses contain:
+**V1/V2 Witnesses:**
+- 5-byte header: "CKBFS" (0x43, 0x4B, 0x42, 0x46, 0x53)
+- 1-byte version: 0x00 or 0x02
+- Variable-length content data
+
+**V3 Witnesses (Enhanced with Backlinks):**
 - 5-byte header: "CKBFS" (0x43, 0x4B, 0x42, 0x46, 0x53)
-- 1-byte version: 0x00
+- 1-byte version: 0x03
+- 32-byte previous transaction hash
+- 4-byte previous witness index
+- 4-byte previous checksum
+- 4-byte next witness index (or 0x00000000 for tail)
 - Variable-length content data
 
 ### Checksum Algorithm
@@ -482,6 +503,10 @@ npm run example:append -- --txhash=0x123456...
 # Retrieve example (demonstrates witness decoding and generic identifier APIs)
 npm run example:retrieve -- --txhash=0x123456...
 
+# V3 specific examples
+npm run example:publish-v3
+npm run example:append-v3 -- --txhash=0x123456...
+
 # All examples
 npm run example
 ```

package/RFC.v3.md

@@ -0,0 +1,210 @@
+# CKBFS Protocol V3
+
+This is a next generation for CKBFS Protocol, aimed to provide a more affordable storage price -- comparing to similar solutions like IPFS.
+
+The most important change of V3 is that there's no longer `Backlinks` inside the Cell data. Instead, it was moved into witnesses.
+
+
+## Protocol Standard
+
+### Data Structure
+
+#### CKBFS v3 Cell
+
+CKBFS Cell is a cell that stores metadata of the file:
+
+```yaml
+Data:
+  content_type: Bytes # String Bytes
+  filename: Bytes # String Bytes
+  index: Uint32 # Reference of the first witnesses index.
+  checksum: Uint32 # Adler32 checksum
+
+Type:
+  hash_type: "data1"
+  code_hash: CKBFS_V3_TYPE_DATA_HASH
+  args: <TypeID, 32 bytes>,[<hasher_code_hash>, optional]
+Lock:
+  <user_defined>
+```
+
+The following rules should be met in a CKBFS cell:
+
+- Rule 1: data structure of a CKBFS cell is molecule encoded. See [Molecule](https://github.com/nervosnetwork/molecule) definitions below.
+- Rule 2: checksum must match with specified witnesses. Default checksum algorithm will be Alder32 if not specify `hasher_code_hash` in Type script args.
+- Rule 3: if `hasher_code_hash` is specified, then it will use hasher binary from CellDeps that matches `code_hash`, with same input parameter.
+- Rule 4: Once created, a CKBFS cell can only be updated/transfered, which means it can not be destroyed.
+- Rule 5: **`index` is the first witness index of the stored CKBFS structured contents in splited witnesses.**
+
+
+### Witnesses
+
+File contents are stored in witnesses. In a single transaction, witnesses can be splitted into multiple parts and concat together while verification. 
+
+```yaml
+Witnesses:
+  <"CKBFS"> <0x03> <PREVIOUS_POSITION(TX_HASH, Witness_index)> <PREVIOUS_CHECKSUM(4 bytes) | 0x00000000> <NEXT_INDEX | 0x00000000> [CONTENT_BYTES_PART_1]
+  <NEXT_INDEX | 0x00000000> <CONTENT_BYTES_PART_2>
+  <NEXT_INDEX | 0x00000000> <CONTENT_BYTES_PART_3> ...
+  <0x00000000> <CONTENT_BYTES_PART_N>
+```
+
+The following rules should be met for witnesses used in CKBFS:
+- Rule 6: Witnesses are different in `Head Witness`, `Middle Witnesses`, and `Tail Witness`, their form should follow bellow's rules
+- Rule 7: The first 5 bytes of `Head Witness` must be UTF8 coded string bytes of `CKBFS`, which should be: `0x434b424653`
+- Rule 8: The 6th byte of `Head Witness` must be the version of CKBFS protocol, which should be: `0x03`.
+- Rule 9: Previous position of this CKBFS content stores from 7th bytes to 42th bytes in `Head Witness`. it should be previous transaction hash(H256) and previous Head Witness index(Uint32).
+- Rule 10: Previous checksum value must stored in `Head Witness`, position from 43rd to 46th bytes. If there's no previous status, then it should be `0x00000000`.
+- Rule 10: File contents bytes are stored from:
+    - 51st byte from the Head Witness.
+    - 5th byte from the `Middle Witnesses` and `Tail Witness`
+
+----
+
+### Operations
+
+This section describes operations and restrictions in CKBFS v3 implementation.
+
+#### Publish
+
+Publish operation creates one or more new CKBFS v3 cell.
+
+```yaml
+Witnesses:
+  <...>
+  <0x434b424653, 0x03, 32bytes 0x00, 0x00000000, 0x00000000, 0x00000002,CKBFS_CONTENT_BYTES_PART_1>
+  <0x00000003, CKBFS_CONTENT_BYTES_PART_2>
+  <...>
+  <0x00000000, CKBFS_CONTENT_BYTES_PART_TAIL>
+  <...>
+Inputs:
+  <...>
+Outputs:
+  <vec> CKBFS_V3_CELL
+    Data:
+      content-type: string
+      filename: string
+      indexes: uint32
+      checksum: uint32
+    Type:
+      code_hash: ckbfs v3 type script
+      args: 32 bytes type_id, (...)
+  <...>
+```
+
+Publish operation must satisfy following rule:
+
+- Rule 11: in a publish operation, checksum in cell data must be equal with `hash(Witnesses[ALL_CONTENT_PARTS])`.
+- Rule 12: Previous position value, previous checksum value should be all zero in `Head Witnesses`
+
+
+---
+
+#### Append
+
+Append operation updates exist live CKBFS v3 cell, validates the latest checksum.
+
+```yaml
+// Append
+Witnesses:
+  <...>
+  <0x434b424653, 0x03, PREVIOUS_TX_HASH_VALUE, PREVIOUS_INDEX_IN_CKBFS_V3_CELL, PREIVOUS_CHECKSUM, 0x00000002,CKBFS_CONTENT_BYTES_PART_1>
+  <0x00000003, CKBFS_CONTENT_BYTES_PART_2>
+  <...>
+  <0x00000000, CKBFS_CONTENT_BYTES_PART_TAIL>
+  <...>
+Inputs:
+  <...>
+  CKBFS_V3_CELL
+    Data:
+      content-type: string
+      filename: string
+      index: uint32
+      checksum: uint32 # previous checksum
+    Type:
+      code_hash: ckbfs v3 type script
+      args: 32 bytes type_id, (...)
+  <...>
+Outputs:
+  <...>
+  CKBFS_V3_CELL:
+    Data:
+      content-type: string
+      filename: string
+      index: uint32
+      checksum: uint32 # updated checksum
+    Type:
+      code_hash: ckbfs v3 type script
+      args: 32 bytes type_id, (...)
+```
+
+- Rule 13: new checksum of updated CKBFS cell should be equal to:  `hasher.recover_from(previous_checksum).update(new_content_bytes)`
+- Rule 14: `content-type`, `filename`, and Type args of a CKBFS cell CAN NOT be updated in ANY condition
+
+
+---
+
+#### Transfer
+
+Transfer operation transfers ownership of a CKBFS cell, and ensure it did not lost tracking of backlinks.
+
+```yaml
+// Transfer
+Witnesses:
+  <...>
+  <0x434b424653, 0x03, PREVIOUS_TX_HASH_VALUE, PREVIOUS_INDEX_IN_CKBFS_V3_CELL, PREIVOUS_CHECKSUM, 0x00000000>
+  <...>
+
+Inputs:
+  <...>
+  CKBFS_V3_CELL
+    Data:
+      content-type: string
+      filename: string
+      index: uint32
+      checksum: uint32
+    Type:
+      code_hash: ckbfs type script
+      args: 32 bytes type_id, (...)
+    Lock:
+      <USER_DEFINED>
+  <...>
+Outputs:
+  <...>
+  CKBFS_V3_CELL:
+    Data:
+      content-type: string
+      filename: string
+      index: uint32
+      checksum: uint32
+    Type:
+      code_hash: ckbfs type script
+      args: 32 bytes type_id, (...)
+    Lock:
+      <USER_DEFINED>
+```
+
+- Rule 15: The `Head Witness` should not contain any content part bytes
+- Rule 16: in a transfer operation, `checksum` CAN NOT be updated
+
+
+---
+
+## Other Notes
+
+### Molecule Definitions:
+
+Here’s molecule definitions of CKBFS data structures
+
+```jsx
+vector Bytes <byte>;
+option BytesOpt (Bytes);
+option Uint32Opt (Uint32);
+
+table CKBFSData {
+  index: Uint32,
+  checksum: Uint32,
+  content_type: Bytes,
+  filename: Bytes,
+}
+```

package/dist/index.d.ts

@@ -1,8 +1,8 @@
 import { Script, Signer, Transaction } from "@ckb-ccc/core";
-import { calculateChecksum, verifyChecksum, updateChecksum, verifyWitnessChecksum } from "./utils/checksum";
-import { createCKBFSCell, createPublishTransaction as utilCreatePublishTransaction, createAppendTransaction as utilCreateAppendTransaction, publishCKBFS as utilPublishCKBFS, appendCKBFS as utilAppendCKBFS, CKBFSCellOptions, PublishOptions, AppendOptions } from "./utils/transaction";
-import { readFile, readFileAsText, readFileAsUint8Array, writeFile, getContentType, splitFileIntoChunks, combineChunksToFile, getFileContentFromChain, saveFileFromChain, getFileContentFromChainByTypeId, saveFileFromChainByTypeId, decodeFileFromChainByTypeId, getFileContentFromChainByIdentifier, saveFileFromChainByIdentifier, decodeFileFromChainByIdentifier, parseIdentifier, IdentifierType, decodeWitnessContent, decodeMultipleWitnessContents, extractFileFromWitnesses, decodeFileFromWitnessData, saveFileFromWitnessData } from "./utils/file";
-import { createCKBFSWitness, createTextCKBFSWitness, extractCKBFSWitnessContent, isCKBFSWitness, createChunkedCKBFSWitnesses } from "./utils/witness";
+import { calculateChecksum, verifyChecksum, updateChecksum, verifyWitnessChecksum, verifyV3WitnessChecksum, verifyV3WitnessChain } from "./utils/checksum";
+import { createCKBFSCell, createPublishTransaction as utilCreatePublishTransaction, preparePublishTransaction, createAppendTransaction as utilCreateAppendTransaction, prepareAppendTransaction, createAppendTransactionDry, publishCKBFS as utilPublishCKBFS, appendCKBFS as utilAppendCKBFS, CKBFSCellOptions, PublishOptions, AppendOptions, AppendV3Options, TransferV3Options } from "./utils/transaction";
+import { readFile, readFileAsText, readFileAsUint8Array, writeFile, getContentType, splitFileIntoChunks, combineChunksToFile, getFileContentFromChain, saveFileFromChain, getFileContentFromChainByTypeId, saveFileFromChainByTypeId, decodeFileFromChainByTypeId, getFileContentFromChainByIdentifier, saveFileFromChainByIdentifier, decodeFileFromChainByIdentifier, parseIdentifier, IdentifierType, decodeWitnessContent, decodeMultipleWitnessContents, extractFileFromWitnesses, decodeFileFromWitnessData, saveFileFromWitnessData, getFileContentFromChainV3, getFileContentFromChainByIdentifierV3, saveFileFromChainByIdentifierV3 } from "./utils/file";
+import { createCKBFSWitness, createTextCKBFSWitness, extractCKBFSWitnessContent, isCKBFSWitness, createChunkedCKBFSWitnesses, createCKBFSV3Witness, createChunkedCKBFSV3Witnesses, extractCKBFSV3WitnessContent, isCKBFSV3Witness, CKBFSV3WitnessOptions } from "./utils/witness";
 import { CKBFSData, BackLinkV1, BackLinkV2, CKBFSDataType, BackLinkType, CKBFS_HEADER, CKBFS_HEADER_STRING } from "./utils/molecule";
 import { NetworkType, ProtocolVersion, ProtocolVersionType, DEFAULT_NETWORK, DEFAULT_VERSION, CKBFS_CODE_HASH, CKBFS_TYPE_ID, ADLER32_CODE_HASH, ADLER32_TYPE_ID, DEP_GROUP_TX_HASH, DEPLOY_TX_HASH, getCKBFSScriptConfig, CKBFSScriptConfig } from "./utils/constants";
 /**
@@ -28,6 +28,9 @@ export type PublishContentOptions = Omit<FileOptions, "capacity" | "contentType"
  */
 export type AppendContentOptions = Omit<FileOptions, "contentType" | "filename" | "capacity"> & {
     capacity?: bigint;
+    previousTxHash?: string;
+    previousWitnessIndex?: number;
+    previousChecksum?: number;
 };
 /**
  * Configuration options for the CKBFS SDK
@@ -92,7 +95,11 @@ export declare class CKBFS {
      * @param options Additional options for the append operation
      * @returns Promise resolving to the transaction hash
      */
-    appendFile(filePath: string, ckbfsCell: AppendOptions["ckbfsCell"], options?: Omit<FileOptions, "contentType" | "filename">): Promise<string>;
+    appendFile(filePath: string, ckbfsCell: AppendOptions["ckbfsCell"], options?: Omit<FileOptions, "contentType" | "filename"> & {
+        previousTxHash?: string;
+        previousWitnessIndex?: number;
+        previousChecksum?: number;
+    }): Promise<string>;
     /**
      * Appends content (string or Uint8Array) directly to an existing CKBFS file
      * @param content The content string or byte array to append
@@ -122,7 +129,11 @@ export declare class CKBFS {
      * @param options Additional options for the append operation
      * @returns Promise resolving to the unsigned transaction
      */
-    createAppendTransaction(filePath: string, ckbfsCell: AppendOptions["ckbfsCell"], options?: Omit<FileOptions, "contentType" | "filename">): Promise<Transaction>;
+    createAppendTransaction(filePath: string, ckbfsCell: AppendOptions["ckbfsCell"], options?: Omit<FileOptions, "contentType" | "filename"> & {
+        previousTxHash?: string;
+        previousWitnessIndex?: number;
+        previousChecksum?: number;
+    }): Promise<Transaction>;
     /**
      * Creates a new transaction for appending content (string or Uint8Array) directly, but doesn't sign or send it
      * @param content The content string or byte array to append
@@ -131,5 +142,59 @@ export declare class CKBFS {
      * @returns Promise resolving to the unsigned transaction
      */
     createAppendContentTransaction(content: string | Uint8Array, ckbfsCell: AppendOptions["ckbfsCell"], options?: AppendContentOptions): Promise<Transaction>;
+    /**
+     * Publishes a file to CKBFS v3
+     * @param filePath The path to the file to publish
+     * @param options Options for publishing the file
+     * @returns Promise resolving to the transaction hash
+     */
+    publishFileV3(filePath: string, options?: Omit<FileOptions, 'version'>): Promise<string>;
+    /**
+     * Publishes content (string or Uint8Array) directly to CKBFS v3
+     * @param content The content string or byte array to publish
+     * @param options Options for publishing the content (contentType and filename are required)
+     * @returns Promise resolving to the transaction hash
+     */
+    publishContentV3(content: string | Uint8Array, options: PublishContentOptions): Promise<string>;
+    /**
+     * Appends content from a file to an existing CKBFS v3 file
+     * @param filePath The path to the file containing the content to append
+     * @param ckbfsCell The CKBFS cell to append to
+     * @param previousTxHash The previous transaction hash for backlink
+     * @param previousWitnessIndex The previous witness index for backlink
+     * @param previousChecksum The previous checksum for backlink
+     * @param options Additional options for the append operation
+     * @returns Promise resolving to the transaction hash
+     */
+    appendFileV3(filePath: string, ckbfsCell: AppendV3Options["ckbfsCell"], previousTxHash: string, previousWitnessIndex: number, previousChecksum: number, options?: Omit<FileOptions, "contentType" | "filename" | "version">): Promise<string>;
+    /**
+     * Appends content (string or Uint8Array) directly to an existing CKBFS v3 file
+     * @param content The content string or byte array to append
+     * @param ckbfsCell The CKBFS cell to append to
+     * @param previousTxHash The previous transaction hash for backlink
+     * @param previousWitnessIndex The previous witness index for backlink
+     * @param previousChecksum The previous checksum for backlink
+     * @param options Additional options for the append operation
+     * @returns Promise resolving to the transaction hash
+     */
+    appendContentV3(content: string | Uint8Array, ckbfsCell: AppendV3Options["ckbfsCell"], previousTxHash: string, previousWitnessIndex: number, previousChecksum: number, options?: AppendContentOptions): Promise<string>;
+    /**
+     * Transfers ownership of a CKBFS v3 file to a new lock script
+     * @param ckbfsCell The CKBFS cell to transfer
+     * @param newLock The new lock script for the transferred file
+     * @param previousTxHash The previous transaction hash for backlink
+     * @param previousWitnessIndex The previous witness index for backlink
+     * @param previousChecksum The previous checksum for backlink
+     * @param options Additional options for the transfer operation
+     * @returns Promise resolving to the transaction hash
+     */
+    transferFileV3(ckbfsCell: TransferV3Options["ckbfsCell"], newLock: Script, previousTxHash: string, previousWitnessIndex: number, previousChecksum: number, options?: Omit<FileOptions, "contentType" | "filename" | "version">): Promise<string>;
+    /**
+     * Creates a new transaction for publishing a file to CKBFS v3 but doesn't sign or send it
+     * @param filePath The path to the file to publish
+     * @param options Options for publishing the file
+     * @returns Promise resolving to the unsigned transaction
+     */
+    createPublishV3Transaction(filePath: string, options?: Omit<FileOptions, 'version'>): Promise<Transaction>;
 }
-export { calculateChecksum, verifyChecksum, updateChecksum, verifyWitnessChecksum, createCKBFSCell, utilCreatePublishTransaction as createPublishTransaction, utilCreateAppendTransaction as createAppendTransaction, utilPublishCKBFS as publishCKBFS, utilAppendCKBFS as appendCKBFS, readFile, readFileAsText, readFileAsUint8Array, writeFile, getContentType, splitFileIntoChunks, combineChunksToFile, getFileContentFromChain, saveFileFromChain, getFileContentFromChainByTypeId, saveFileFromChainByTypeId, decodeFileFromChainByTypeId, getFileContentFromChainByIdentifier, saveFileFromChainByIdentifier, decodeFileFromChainByIdentifier, parseIdentifier, IdentifierType, decodeWitnessContent, decodeMultipleWitnessContents, extractFileFromWitnesses, decodeFileFromWitnessData, saveFileFromWitnessData, createCKBFSWitness, createTextCKBFSWitness, extractCKBFSWitnessContent, isCKBFSWitness, createChunkedCKBFSWitnesses, CKBFSData, BackLinkV1, BackLinkV2, CKBFSDataType, BackLinkType, CKBFSCellOptions, PublishOptions, AppendOptions, CKBFS_HEADER, CKBFS_HEADER_STRING, NetworkType, ProtocolVersion, ProtocolVersionType, DEFAULT_NETWORK, DEFAULT_VERSION, CKBFS_CODE_HASH, CKBFS_TYPE_ID, ADLER32_CODE_HASH, ADLER32_TYPE_ID, DEP_GROUP_TX_HASH, DEPLOY_TX_HASH, getCKBFSScriptConfig, CKBFSScriptConfig, };
+export { calculateChecksum, verifyChecksum, updateChecksum, verifyWitnessChecksum, verifyV3WitnessChecksum, verifyV3WitnessChain, createCKBFSCell, utilCreatePublishTransaction as createPublishTransaction, preparePublishTransaction, utilCreateAppendTransaction as createAppendTransaction, prepareAppendTransaction, utilPublishCKBFS as publishCKBFS, utilAppendCKBFS as appendCKBFS, createAppendTransactionDry, readFile, readFileAsText, readFileAsUint8Array, writeFile, getContentType, splitFileIntoChunks, combineChunksToFile, getFileContentFromChain, saveFileFromChain, getFileContentFromChainByTypeId, saveFileFromChainByTypeId, decodeFileFromChainByTypeId, getFileContentFromChainByIdentifier, saveFileFromChainByIdentifier, decodeFileFromChainByIdentifier, parseIdentifier, IdentifierType, decodeWitnessContent, decodeMultipleWitnessContents, extractFileFromWitnesses, decodeFileFromWitnessData, saveFileFromWitnessData, getFileContentFromChainV3, getFileContentFromChainByIdentifierV3, saveFileFromChainByIdentifierV3, createCKBFSWitness, createTextCKBFSWitness, extractCKBFSWitnessContent, isCKBFSWitness, createChunkedCKBFSWitnesses, createCKBFSV3Witness, createChunkedCKBFSV3Witnesses, extractCKBFSV3WitnessContent, isCKBFSV3Witness, CKBFSV3WitnessOptions, CKBFSData, BackLinkV1, BackLinkV2, CKBFSDataType, BackLinkType, CKBFSCellOptions, PublishOptions, AppendOptions, CKBFS_HEADER, CKBFS_HEADER_STRING, NetworkType, ProtocolVersion, ProtocolVersionType, DEFAULT_NETWORK, DEFAULT_VERSION, CKBFS_CODE_HASH, CKBFS_TYPE_ID, ADLER32_CODE_HASH, ADLER32_TYPE_ID, DEP_GROUP_TX_HASH, DEPLOY_TX_HASH, getCKBFSScriptConfig, CKBFSScriptConfig, };