@zkim-platform/file-format 1.1.0

package/CHANGELOG.md

@@ -0,0 +1,100 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [1.0.0] - 2026-01-09
+
+### Added
+- **Initial Release - v1.0.0:**
+  - Secure file format using NIST-standardized post-quantum cryptography
+  - Magic bytes: `"ZKIM"` (4 bytes)
+  - Version: `1` (2 bytes)
+  - Signature type: `1` (ML-DSA-65, 3,309 bytes, FIPS 204)
+- **Post-Quantum Cryptography:**
+  - ML-DSA-65 (FIPS 204) for digital signatures (3,309 bytes)
+  - ML-KEM-768 (FIPS 203) for key encapsulation mechanism and key derivation
+  - NIST-standardized algorithms for long-term security
+- **ML-KEM-768 Key Derivation**: Platform and user layer keys are derived from ML-KEM-768 shared secrets using BLAKE3. Content layer keys are cryptographically random for per-file perfect forward secrecy.
+- **Self-Encryption Helpers**: `zkimSelfEncrypt()` and `zkimSelfDecrypt()` helper functions for ML-KEM-768 self-encryption pattern
+- **KEM Ciphertext in Wire Format**: Wire format includes ML-KEM-768 ciphertext (1,088 bytes) for key exchange
+- **Key Storage Infrastructure**: ML-KEM-768 secret keys are stored encrypted with user keys
+- **Enhanced Security:**
+  - Post-quantum resistant signatures and key exchange
+  - Future-proof cryptography for store-now-decrypt-later scenarios
+  - Explicit threat model documentation
+- **Core Features:**
+  - Three-layer encryption with XChaCha20-Poly1305 (platform, user, content layers)
+  - Privacy-preserving searchable encryption using OPRF-based trapdoors
+  - Integrity validation with BLAKE3 hashing and ML-DSA-65 signatures
+  - Error detection and recovery mechanisms for corrupted files
+  - Optional GZIP/Brotli compression for efficient storage
+  - Performance monitoring and optimization
+  - Constant-time security operations to prevent timing attacks
+- **Documentation:**
+  - README with essential information
+  - Comprehensive GitHub Wiki with 9 detailed pages:
+    - Getting Started guide
+    - Storage Integration guide (critical for custom backends)
+    - Complete API Reference
+    - Code Examples and patterns
+    - Security documentation (post-quantum details)
+    - Architecture specification
+    - Troubleshooting guide
+    - Contributing guidelines
+- **Build Attestation:**
+  - npm Provenance support (SLSA Level 2)
+  - Verifiable build information on npm package page
+  - Public ledger transparency log entries
+
+### Changed
+- Updated package name to `@zkim-platform/file-format` to match npm organization scope
+- Updated all code examples and documentation to use correct package name
+
+### Fixed
+- Fixed install command in documentation (`npm install @zkim-platform/file-format`)
+- Fixed build status badge URL format for GitHub Actions
+- Updated all source file comments and JSDoc to reference correct package name
+- Fixed platform key usage - now included in encryption key derivation for tenant isolation
+
+### Security
+- **Post-Quantum Security:**
+  - ML-DSA-65 signatures (FIPS 204) for long-term authenticity
+  - ML-KEM-768 key exchange (FIPS 203) for post-quantum key encapsulation
+  - ML-KEM-768 key derivation for platform and user layer keys (post-quantum secure)
+  - Content layer keys are cryptographically random (per-file perfect forward secrecy)
+  - BLAKE3 hashing (256-bit output) for integrity verification and key derivation
+  - XChaCha20-Poly1305 symmetric encryption (AEAD) with ML-KEM-768 derived keys
+- **Post-Quantum Key Derivation**: Platform and user layer encryption keys are derived from ML-KEM-768 shared secrets, ensuring post-quantum security for key derivation. Content layer keys are random for perfect forward secrecy.
+- **Self-Encryption Pattern**: ML-KEM-768 self-encryption pattern for data encrypted by and for the same user/device
+- **Key Derivation Pattern**: 
+  - Platform key: `blake3([sharedSecret, ...platformKey], { dkLen: 32 })`
+  - User key: `blake3([sharedSecret, ...userKey], { dkLen: 32 })`
+  - Content key: Cryptographically random (per-file perfect forward secrecy)
+- **Note:** This package uses NIST-standardized algorithms (FIPS 203/204) but is not FIPS 140-3 validated by an accredited laboratory.
+
+### Technical Details
+- **Dependencies:**
+  - libsodium-wrappers-sumo ^0.7.15 - Cryptographic operations (XChaCha20-Poly1305)
+  - @noble/hashes ^2.0.1 - BLAKE3 hashing and key derivation
+  - @noble/post-quantum ^0.2.1 - ML-DSA-65 and ML-KEM-768
+  - @noble/curves ^2.0.1 - Ristretto255 for searchable encryption
+- **Algorithm Suite**: `ALG_SUITE_ID = 0x01` represents post-quantum secure suite (ML-KEM-768 for key exchange, ML-DSA-65 for signatures, XChaCha20-Poly1305 for symmetric encryption with post-quantum key derivation, BLAKE3 for hashing)
+- **Default Encryption Type**: `"ML-KEM-768+XChaCha20-Poly1305"` (post-quantum key derivation)
+- **KEM Ciphertext Size**: 1,088 bytes (ML-KEM-768 standard)
+- **Wire Format**: Includes KEM ciphertext for ML-KEM-768 key exchange
+- **Backward Compatibility**: Maintained for legacy data formats without ML-KEM-768 components
+- **Platform Support:**
+  - Node.js 18+ with ES Modules
+  - Modern browsers with TypedArray and WebAssembly support
+  - Browser builds rely on WebAssembly-backed libsodium (via `libsodium-wrappers-sumo`)
+- **Build:**
+  - TypeScript 5.9+ with strict mode
+  - ESM and CommonJS dual package support
+  - Source maps and declaration files included
+  - npm Provenance attestation enabled
+
+---
+

package/LICENSE

@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) 2025 ZKIM Team
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+

package/README.md

@@ -0,0 +1,176 @@
+# @zkim-platform/file-format
+
+![npm version](https://img.shields.io/npm/v/@zkim-platform/file-format)
+![npm downloads](https://img.shields.io/npm/dm/@zkim-platform/file-format)
+![License](https://img.shields.io/badge/license-MIT-blue)
+![TypeScript](https://img.shields.io/badge/TypeScript-5.9-blue)
+![Node.js](https://img.shields.io/badge/Node.js-18%2B-green)
+![Test Coverage](https://img.shields.io/badge/coverage-92%25-brightgreen)
+![Build Status](https://github.com/zkdotim/zkim-file-format/actions/workflows/ci.yml/badge.svg?branch=main)
+
+Post-quantum secure file format using ML-KEM-768 (FIPS 203) for key exchange and ML-DSA-65 (FIPS 204) for signatures, with XChaCha20-Poly1305 symmetric encryption protected by post-quantum key derivation. Three-layer encryption with privacy-preserving search capabilities. Platform and user layer keys are derived from ML-KEM-768 shared secrets using BLAKE3. Content layer keys are cryptographically random for per-file perfect forward secrecy.
+
+## Installation
+
+```bash
+npm install @zkim-platform/file-format
+```
+
+### Requirements
+
+- Node.js 18+ (for Node.js environments)
+- Modern browser with TypedArray and ES2020+ support (for browser environments)
+  - Browser builds rely on WebAssembly-backed libsodium (via `libsodium-wrappers-sumo`)
+- TypeScript 5.0+ (recommended for type safety)
+
+## Quick Start
+
+```typescript
+import { ZKIMFileService, InMemoryStorage } from "@zkim-platform/file-format";
+import sodium from "libsodium-wrappers-sumo";
+
+async function main() {
+  await sodium.ready;
+
+  // ⚠️ NOTE: This example uses random keys for simplicity.
+  // In production, derive keys from actual user authentication.
+  // See Authentication Integration guide for proper key derivation.
+  
+  // Platform key (store securely, same for all users)
+  const platformKey = sodium.randombytes_buf(32);
+  
+  // User key (in production, derive from user authentication)
+  // Example: const userKey = await deriveKeyFromWallet(walletAddress, signature);
+  const userKey = sodium.randombytes_buf(32);
+  const userId = "example-user";
+
+  // Create storage backend
+  const storage = new InMemoryStorage();
+
+  // Initialize the file service
+  const fileService = new ZKIMFileService(
+    {
+      enableCompression: true,
+      enableSearchableEncryption: false,
+      enableIntegrityValidation: true,
+    },
+    undefined,
+    storage
+  );
+
+  await fileService.initialize();
+
+  try {
+    // Create encrypted ZKIM file
+    const testData = new TextEncoder().encode("Hello, ZKIM File Format!");
+    const result = await fileService.createZkimFile(
+      testData,
+      userId,
+      platformKey,
+      userKey,
+      {
+        fileName: "example.txt",
+        mimeType: "text/plain",
+      }
+    );
+
+    if (result.success && result.file) {
+      console.log("File created:", result.file.header.fileId);
+    }
+  } catch (error) {
+    console.error("Failed to create file:", error);
+  }
+
+  await fileService.cleanup();
+}
+
+main().catch(console.error);
+```
+
+## Who is this for?
+
+- Developers building secure file storage or sharing systems
+- Applications requiring long-term confidentiality (store-now-decrypt-later resistance)
+- Projects that need post-quantum signatures and crypto agility
+- Teams concerned about supply-chain security and provenance
+
+## Documentation
+
+📖 **[Full Documentation → Wiki](https://github.com/zkdotim/zkim-file-format/wiki)**
+
+- [Getting Started](https://github.com/zkdotim/zkim-file-format/wiki/Getting-Started)
+- [Authentication Integration](https://github.com/zkdotim/zkim-file-format/wiki/Authentication-Integration) ⭐ **CRITICAL**
+- [Storage Integration](https://github.com/zkdotim/zkim-file-format/wiki/Storage-Integration) ⭐ **CRITICAL**
+- [API Reference](https://github.com/zkdotim/zkim-file-format/wiki/API-Reference)
+- [Examples](https://github.com/zkdotim/zkim-file-format/wiki/Examples)
+- [Security & Post-Quantum](https://github.com/zkdotim/zkim-file-format/wiki/Security)
+- [Architecture](https://github.com/zkdotim/zkim-file-format/wiki/Architecture)
+- [Troubleshooting](https://github.com/zkdotim/zkim-file-format/wiki/Troubleshooting)
+
+## Features
+
+- 🔐 **Three-Layer Encryption**: XChaCha20-Poly1305 with platform, user, and content layers
+- 🔍 **Privacy-Preserving Search**: OPRF-based trapdoors with rotation
+- ✅ **Post-Quantum Signatures**: ML-DSA-65 (FIPS 204) for long-term authenticity
+- 📦 **Compression**: Optional GZIP/Brotli compression
+- 🛡️ **Integrity Validation**: BLAKE3-based content hashing and integrity verification
+- 🌐 **Cross-Platform**: Works in browser and Node.js environments
+
+**⚠️ FIPS Validation Disclaimer:** This package uses NIST-standardized algorithms (FIPS 203/204) but is **NOT FIPS 140-3 validated** by an accredited laboratory. The implementation follows NIST specifications but is not certified for government use requiring FIPS validation. See [Security Documentation](https://github.com/zkdotim/zkim-file-format/wiki/Security) for details.
+
+## Key Technologies
+
+- **Encryption**: XChaCha20-Poly1305 (AEAD) via libsodium
+- **Hashing**: BLAKE3 (256-bit output) via @noble/hashes
+- **Signatures**: ML-DSA-65 (FIPS 204, post-quantum) via @noble/post-quantum
+- **Key Exchange**: ML-KEM-768 (FIPS 203, post-quantum) via @noble/post-quantum
+- **Searchable Encryption**: OPRF (Oblivious Pseudorandom Function) via @noble/curves
+
+## Design Philosophy
+
+- **Post-quantum by default** - Future-proof cryptography from day one
+- **Crypto agility** - Algorithm choices are explicit and configurable
+- **Explicit threat models** - Security assumptions are documented
+- **Auditability over obscurity** - Open design and verifiable builds
+
+## License
+
+MIT License - see [LICENSE](./LICENSE) for details.
+
+---
+
+## Provenance
+
+This package is published with **npm Provenance** (build attestation) to ensure authenticity and integrity.
+
+**What is Provenance?**
+Provenance provides verifiable information about how and where this package was built:
+- ✅ **Built and signed on:** GitHub Actions
+- ✅ **Source Commit:** Links to exact GitHub commit
+- ✅ **Build File:** Links to GitHub Actions workflow
+- ✅ **Public Ledger:** Transparency log entry (immutable record)
+
+**Why it matters:**
+- Verifies package authenticity
+- Shows exact source code used
+- Provides build environment details
+- Creates immutable audit trail
+- Enhances supply chain security
+
+**View Provenance:**
+Visit the package page on npm and scroll to the "Provenance" section at the bottom to see:
+- Build environment details
+- Source commit link
+- Build workflow file
+- Public ledger entry
+
+**Verify locally:**
+```bash
+npm audit signatures
+```
+
+For more information, see [npm Provenance Documentation](https://docs.npmjs.com/generating-provenance-statements/).
+
+---
+
+**Made with ❤️ by the ZKIM Team**