@nightowne/tas-cli 2.0.0 → 2.1.0

package/README.md

@@ -1,14 +1,25 @@
-# Telegram as Storage
+<p align="center">
+  <img src="assets/demo.gif" alt="TAS Demo" width="600">
+</p>
+
+<h1 align="center">Telegram as Storage</h1>
 
-[![CI](https://github.com/ixchio/tas/actions/workflows/ci.yml/badge.svg)](https://github.com/ixchio/tas/actions/workflows/ci.yml)
-[![npm](https://img.shields.io/npm/v/@nightowne/tas-cli)](https://www.npmjs.com/package/@nightowne/tas-cli)
-[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+<p align="center">
+  <strong>Free, encrypted, unlimited cloud storage — inside Telegram.</strong>
+</p>
 
 <p align="center">
-  <img src="assets/demo.gif" alt="Demo" width="600">
+  <a href="https://github.com/ixchio/tas/actions/workflows/ci.yml"><img src="https://github.com/ixchio/tas/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
+  <a href="https://www.npmjs.com/package/@nightowne/tas-cli"><img src="https://img.shields.io/npm/v/@nightowne/tas-cli" alt="npm"></a>
+  <a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License: MIT"></a>
+  <a href="https://www.npmjs.com/package/@nightowne/tas-cli"><img src="https://img.shields.io/npm/dm/@nightowne/tas-cli" alt="Downloads"></a>
 </p>
 
-A CLI tool that uses your Telegram bot as encrypted file storage. Files are compressed, encrypted locally, then uploaded to your private bot chat.
+---
+
+I built this because I wanted encrypted cloud storage that's actually free. Google Drive reads your files. Dropbox costs money. Telegram gives you unlimited storage with a bot API — so I wrote a CLI that turns it into a proper encrypted drive.
+
+**What this does:** Compresses, encrypts (AES-256-GCM), and uploads your files to your own private Telegram bot chat. Mount it as a folder, sync directories, or share files with expiring links.
 
 ```
 ┌─────────────┐     ┌───────────────┐     ┌──────────────┐
@@ -22,162 +33,130 @@ A CLI tool that uses your Telegram bot as encrypted file storage. Files are comp
 └─────────────┘     └───────────────┘     └──────────────┘
 ```
 
-## Why TAS?
-
-| Feature | TAS | Session-based tools (e.g. teldrive) |
-|---------|:---:|:-----------------------------------:|
-| Account ban risk | None (Bot API) | High (session hijack detection) |
-| Encryption | AES-256-GCM | Usually none |
-| Dependencies | SQLite only | Rclone, external DB |
-| Setup complexity | 2 minutes | Docker + multiple services |
-
-Key differences:
-- Uses Bot API, not session-based auth - Telegram can't ban your account
-- Encryption by default - files encrypted before leaving your machine
-- Local-first - SQLite index, no cloud dependencies
-- FUSE mount - use Telegram like a folder
-
-## Security Model
+## ⚡ Quick Start
 
-| Component | Implementation |
-|-----------|----------------|
-| Cipher | AES-256-GCM |
-| Key derivation | PBKDF2-SHA512, 100,000 iterations |
-| Salt | 32 bytes, random per file |
-| IV | 12 bytes, random per file |
-| Auth tag | 16 bytes (integrity) |
-
-Your password never leaves your machine. Telegram stores encrypted blobs.
+```bash
+npm install -g @nightowne/tas-cli
+tas init          # 2-minute setup wizard
+tas push secret.pdf
+tas pull secret.pdf
+```
 
-## Limitations
+## 🔥 Features
 
-- Not a backup - Telegram can delete content without notice
-- No versioning - overwriting a file deletes the old version
-- 49MB chunks - files split due to Bot API limits
-- FUSE required - mount feature needs libfuse on Linux/macOS
-- Single user - designed for personal use, not multi-tenant
+### Mount as a folder
+```bash
+tas mount ~/cloud      # FUSE mount — drag & drop files
+tas unmount ~/cloud
+```
+Requires `libfuse` (`apt install fuse libfuse-dev` on Debian/Ubuntu, `brew install macfuse` on macOS).
 
-## Quick Start
+### Auto-sync folders
+```bash
+tas sync add ~/Documents    # Register
+tas sync start              # Watch for changes, auto-upload
+tas sync pull               # Download everything back
+```
 
+### Share files with expiring links
 ```bash
-# Install
-npm install -g @nightowne/tas-cli
+tas share create secret.pdf --expire 1h --max-downloads 3
+# → http://localhost:3000/d/a1b2c3d4...
 
-# Setup (creates bot connection + encryption password)
-tas init
+tas share list              # Active shares
+tas share revoke a1b2c3d4   # Revoke
+```
+Spins up a local HTTP server. File is downloaded from Telegram, decrypted, and served. Dark-themed download page with file info.
 
-# Upload a file
-tas push secret.pdf
+## 🛡️ Security
 
-# Download a file
-tas pull secret.pdf
+| Component | Implementation |
+|-----------|----------------|
+| Cipher | AES-256-GCM |
+| Key derivation | PBKDF2-SHA512, 100k iterations |
+| Salt | 32 bytes, random per file |
+| IV | 12 bytes, random per file |
+| Auth tag | 16 bytes (integrity verification) |
 
-# Mount as folder (requires libfuse)
-tas mount ~/cloud
-```
+Your password **never** leaves your machine. Telegram only sees encrypted blobs. Even if someone accesses your bot chat, they get meaningless data without your password.
 
-### Prerequisites
-- Node.js ≥18
-- Telegram account + bot token from [@BotFather](https://t.me/BotFather)
-- `libfuse` for mount feature:
-  ```bash
-  # Debian/Ubuntu
-  sudo apt install fuse libfuse-dev
-  
-  # Fedora
-  sudo dnf install fuse fuse-devel
-  
-  # macOS
-  brew install macfuse
-  ```
-
-## CLI Reference
+## 📋 Full CLI Reference
 
 ```bash
 # Core
 tas init                    # Setup wizard
-tas push <file>             # Upload file
-tas pull <file|hash>        # Download file
-tas list [-l]               # List files (long format)
-tas delete <file|hash>      # Remove file
-tas status                  # Show stats
-
-# Search & Resume (v1.1.0)
-tas search <query>          # Search by filename
-tas search -t <query>       # Search by tag
+tas push <file>             # Upload
+tas pull <file|hash>        # Download
+tas list [-l]               # List files
+tas delete <file|hash>      # Delete
+tas status                  # Stats
+tas search <query>          # Search files
 tas resume                  # Resume interrupted uploads
+tas verify                  # Check integrity
 
 # FUSE Mount
 tas mount <path>            # Mount as folder
 tas unmount <path>          # Unmount
 
-# Tags
-tas tag add <file> <tags...>    # Add tags
-tas tag remove <file> <tags...> # Remove tags
-tas tag list [tag]              # List tags or files by tag
-
-# Sync (Dropbox-style)
-tas sync add <folder>       # Register folder for sync
+# Sync
+tas sync add <folder>       # Register folder
 tas sync start              # Start watching
-tas sync pull               # Download all to sync folders
-tas sync status             # Show sync status
+tas sync pull               # Download all
+tas sync status             # Show status
 
-# Share (temporary links)
+# Share
 tas share create <file>     # Create download link
-tas share create <file> --expire 1h --max-downloads 3
-tas share list              # Show active shares
-tas share revoke <token>    # Revoke a share
+tas share list              # Active shares
+tas share revoke <token>    # Revoke
 
-# Verification
-tas verify                  # Check file integrity
+# Tags
+tas tag add <file> <tags...>
+tas tag remove <file> <tags...>
+tas tag list [tag]
 ```
 
-## Automation
-
-Skip password prompts for scripts and CI/CD:
+## 🤖 Automation
 
 ```bash
-# Environment variable
+# Skip password prompts
 export TAS_PASSWORD="your-password"
 tas push file.pdf
-tas sync start
 
-# Or use CLI flag (recommended for CI/CD)
+# Or inline
 tas push -p "password" file.pdf
 ```
 
-Works great with:
-- Cron jobs for automated backups
-- GitHub Actions and GitLab CI
-- Docker containers
-- Shell scripts for batch operations
+Works with cron, GitHub Actions, Docker, or any CI/CD.
 
-## Auto-Start (systemd)
+## ⚠️ Limitations
 
-See [systemd/README.md](systemd/README.md) for running sync as a service.
+- **Not a backup** — Telegram can delete content without notice
+- **No versioning** — overwriting deletes the old version
+- **49MB chunks** — files split due to Bot API limits
+- **Single user** — personal use, not multi-tenant
+- **FUSE required** — mount feature needs libfuse
 
-## Development
+## 🛠️ Development
 
 ```bash
 git clone https://github.com/ixchio/tas
-cd tas
-npm install
-npm test  # 28 tests
+cd tas && npm install
+npm test  # 43 tests
 ```
 
-### Project Structure
 ```
 src/
-├── cli.js           # Command definitions
+├── cli.js           # Commands
 ├── index.js         # Upload/download pipeline
-├── crypto/          # AES-256-GCM encryption
-├── db/              # SQLite file index
-├── fuse/            # FUSE filesystem mount
+├── crypto/          # AES-256-GCM
+├── db/              # SQLite index
+├── fuse/            # FUSE filesystem
+├── share/           # HTTP share server
 ├── sync/            # Folder sync engine
 ├── telegram/        # Bot API client
 └── utils/           # Compression, chunking
 ```
 
-## License
+## 📄 License
 
-MIT
+MIT — do whatever you want with it.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@nightowne/tas-cli",
-  "version": "2.0.0",
+  "version": "2.1.0",
   "description": "Telegram as Storage - Automated encrypted cloud backup. Free, encrypted, scriptable. Mount as folder or use with cron/Docker.",
   "type": "module",
   "main": "src/index.js",

package/src/cli.js

@@ -674,19 +674,38 @@ syncCmd
     .command('start')
     .description('Start syncing all registered folders')
     .option('-p, --password <password>', 'Encryption password (uses TAS_PASSWORD env var if not provided)')
+    .option('-l, --limit <limit>', 'Bandwidth limit (e.g. 500k, 1m)')
     .action(async (options) => {
         console.log(chalk.cyan('\n🔄 Starting folder sync...\n'));
 
         const config = requireConfig(DATA_DIR);
         const password = await getAndVerifyPassword(options.password, DATA_DIR);
 
+        let limitRate = null;
+        if (options.limit) {
+            const match = options.limit.match(/^(\d+)([kmg]?)$/i);
+            if (!match) {
+                console.error(chalk.red('Invalid limit format. Use e.g. 500{}, 1m'));
+                process.exit(1);
+            }
+            const val = parseInt(match[1]);
+            const unit = match[2].toLowerCase();
+            if (unit === 'k') limitRate = val * 1024;
+            else if (unit === 'm') limitRate = val * 1024 * 1024;
+            else if (unit === 'g') limitRate = val * 1024 * 1024 * 1024;
+            else limitRate = val;
+
+            console.log(chalk.dim(`   Bandwidth limit: ${options.limit}/s`));
+        }
+
         try {
             const { SyncEngine } = await import('./sync/sync.js');
 
             const syncEngine = new SyncEngine({
                 dataDir: DATA_DIR,
                 password,
-                config
+                config,
+                limitRate
             });
 
             await syncEngine.initialize();

package/src/crypto/encryption.js

@@ -3,6 +3,7 @@
  */
 
 import crypto from 'crypto';
+import { Transform } from 'stream';
 
 const ALGORITHM = 'aes-256-gcm';
 const KEY_LENGTH = 32; // 256 bits
@@ -66,6 +67,41 @@ export class Encryptor {
         return Buffer.concat([salt, iv, encrypted, authTag]);
     }
 
+    /**
+     * Get an encryption transform stream
+     * Needs to append the salt/iv to the stream begin, and authTag to the stream end
+     */
+    getEncryptStream() {
+        const salt = crypto.randomBytes(SALT_LENGTH);
+        const iv = crypto.randomBytes(IV_LENGTH);
+        const key = this.deriveKey(salt);
+        const cipher = crypto.createCipheriv(ALGORITHM, key, iv);
+
+        let headerWritten = false;
+
+        return new Transform({
+            transform(chunk, encoding, callback) {
+                if (!headerWritten) {
+                    this.push(Buffer.concat([salt, iv]));
+                    headerWritten = true;
+                }
+                const encrypted = cipher.update(chunk);
+                if (encrypted.length > 0) {
+                    this.push(encrypted);
+                }
+                callback();
+            },
+            flush(callback) {
+                const final = cipher.final();
+                if (final.length > 0) {
+                    this.push(final);
+                }
+                this.push(cipher.getAuthTag());
+                callback();
+            }
+        });
+    }
+
     /**
      * Decrypt data
      * Input: Buffer containing [salt (32) | iv (12) | ciphertext | authTag (16)]
@@ -90,6 +126,98 @@ export class Encryptor {
             decipher.final()
         ]);
     }
+
+    /**
+     * Get a decryption transform stream
+     * Expects [salt (32) | iv (12) | ciphertext | authTag (16)]
+     */
+    getDecryptStream() {
+        let salt = null;
+        let iv = null;
+        let authTag = null;
+        let key = null;
+        let decipher = null;
+
+        // Buffer for storing the salt and iv during the first few chunks
+        let headerBuffer = Buffer.alloc(0);
+        let headerRead = false;
+
+        // We must buffer the last 16 bytes across chunks because it's the authTag
+        let tailBuffer = Buffer.alloc(0);
+
+        const self = this;
+
+        return new Transform({
+            transform(chunk, encoding, callback) {
+                try {
+                    // 1. Read the header (salt + iv)
+                    if (!headerRead) {
+                        headerBuffer = Buffer.concat([headerBuffer, chunk]);
+
+                        if (headerBuffer.length >= SALT_LENGTH + IV_LENGTH) {
+                            salt = headerBuffer.subarray(0, SALT_LENGTH);
+                            iv = headerBuffer.subarray(SALT_LENGTH, SALT_LENGTH + IV_LENGTH);
+                            key = self.deriveKey(salt);
+                            decipher = crypto.createDecipheriv(ALGORITHM, key, iv);
+
+                            // The rest of the header buffer is ciphertext
+                            const remaining = headerBuffer.subarray(SALT_LENGTH + IV_LENGTH);
+                            headerBuffer = null; // free memory
+                            headerRead = true;
+
+                            // Push remaining into tailBuffer for processing
+                            if (remaining.length > 0) {
+                                tailBuffer = Buffer.concat([tailBuffer, remaining]);
+                            }
+                        }
+                    } else {
+                        tailBuffer = Buffer.concat([tailBuffer, chunk]);
+                    }
+
+                    // 2. Process ciphertext, keeping exactly TAG_LENGTH bytes in tailBuffer
+                    if (headerRead && tailBuffer.length > TAG_LENGTH) {
+                        const processLength = tailBuffer.length - TAG_LENGTH;
+                        const toProcess = tailBuffer.subarray(0, processLength);
+
+                        const decrypted = decipher.update(toProcess);
+                        if (decrypted.length > 0) {
+                            this.push(decrypted);
+                        }
+
+                        // Keep only the end
+                        tailBuffer = tailBuffer.subarray(processLength);
+                    }
+
+                    callback();
+                } catch (err) {
+                    callback(err);
+                }
+            },
+            flush(callback) {
+                try {
+                    if (!headerRead) {
+                        return callback(new Error('Invalid encrypted data stream: too short'));
+                    }
+
+                    if (tailBuffer.length !== TAG_LENGTH) {
+                        return callback(new Error(`Invalid encrypted data stream: missing auth tag. Got ${tailBuffer.length} bytes, expected ${TAG_LENGTH}`));
+                    }
+
+                    authTag = tailBuffer;
+                    decipher.setAuthTag(authTag);
+
+                    const final = decipher.final();
+                    if (final.length > 0) {
+                        this.push(final);
+                    }
+
+                    callback();
+                } catch (err) {
+                    callback(new Error(`Decryption failed: wrong password or corrupt data (${err.message})`));
+                }
+            }
+        });
+    }
 }
 
 /**

package/src/fuse/mount.js

@@ -130,26 +130,35 @@ export class TelegramFS {
     }
 
     /**
-     * Read file contents
+     * Read file contents from disk cache
      */
     async read(filepath, fd, buffer, length, position, cb) {
         const filename = path.basename(filepath);
 
         try {
+            // Check write buffers first
+            const wb = this.writeBuffers.get(filename);
+            if (wb) {
+                const slice = wb.data.subarray(position, position + length);
+                slice.copy(buffer);
+                return cb(slice.length);
+            }
+
             // Check cache first
-            let data = this.getCached(filename);
+            let cachedPath = this.getCached(filename);
 
-            if (!data) {
-                // Download and decrypt
-                data = await this.downloadFile(filename);
-                this.setCache(filename, data);
+            if (!cachedPath) {
+                // Download, decrypt, and save to disk cache
+                cachedPath = await this.downloadFileToCache(filename);
+                this.setCache(filename, cachedPath);
             }
 
-            // Copy requested portion to buffer
-            const slice = data.subarray(position, position + length);
-            slice.copy(buffer);
+            // Copy requested portion to buffer from disk
+            const fdDisk = fs.openSync(cachedPath, 'r');
+            const bytesRead = fs.readSync(fdDisk, buffer, 0, length, position);
+            fs.closeSync(fdDisk);
 
-            return cb(slice.length);
+            return cb(bytesRead);
         } catch (err) {
             console.error('Read error:', err.message);
             return cb(Fuse.EIO);
@@ -305,10 +314,10 @@ export class TelegramFS {
 
         // Get or load into write buffer
         if (!this.writeBuffers.has(filename)) {
-            const cached = this.getCached(filename);
-            if (cached) {
+            const cachedPath = this.getCached(filename);
+            if (cachedPath) {
                 this.writeBuffers.set(filename, {
-                    data: Buffer.from(cached),
+                    data: fs.readFileSync(cachedPath), // Note: RAM buffer here could be big, but it's okay for truncate/writes right now
                     modified: true
                 });
             } else {
@@ -335,38 +344,81 @@ export class TelegramFS {
 
     // ============== Helper Methods ==============
 
-    async downloadFile(filename) {
+    async downloadFileToCache(filename) {
         const file = this.db.findByName(filename);
         if (!file) throw new Error('File not found');
 
+        const cacheDir = path.join(this.dataDir, 'cache');
+        if (!fs.existsSync(cacheDir)) {
+            fs.mkdirSync(cacheDir, { recursive: true });
+        }
+
+        const outputPath = path.join(cacheDir, file.hash);
+
+        // If it's already fully downloaded and cached on disk, return path
+        if (fs.existsSync(outputPath)) {
+            const stats = fs.statSync(outputPath);
+            if (stats.size === file.original_size) {
+                return outputPath;
+            }
+        }
+
         const chunks = this.db.getChunks(file.id);
         if (chunks.length === 0) throw new Error('No chunks found');
 
-        // Download all chunks
-        const downloadedChunks = [];
-
-        for (const chunk of chunks) {
-            const data = await this.client.downloadFile(chunk.file_telegram_id);
-            const header = parseHeader(data);
-            const payload = data.subarray(HEADER_SIZE);
+        // Pre-sort chunks
+        chunks.sort((a, b) => a.chunk_index - b.chunk_index);
+
+        const firstChunkData = await this.client.downloadFile(chunks[0].file_telegram_id);
+        const header = parseHeader(firstChunkData);
+        let wasCompressed = header.compressed;
+
+        const decryptStream = this.encryptor.getDecryptStream();
+        const decompressStream = this.compressor.getDecompressStream(wasCompressed);
+
+        const { Readable } = await import('stream');
+        const { pipeline } = await import('stream/promises');
+
+        const self = this;
+        let currentChunkIndex = 0;
+        let preloadedFirstChunk = firstChunkData;
+
+        const downloadStream = new Readable({
+            async read() {
+                try {
+                    if (currentChunkIndex >= chunks.length) {
+                        this.push(null);
+                        return;
+                    }
+
+                    const chunk = chunks[currentChunkIndex];
+                    let data;
+                    if (currentChunkIndex === 0 && preloadedFirstChunk) {
+                        data = preloadedFirstChunk;
+                        preloadedFirstChunk = null;
+                    } else {
+                        data = await self.client.downloadFile(chunk.file_telegram_id);
+                    }
+
+                    const payload = data.subarray(HEADER_SIZE);
+                    this.push(payload);
+                    currentChunkIndex++;
+                } catch (err) {
+                    this.destroy(err);
+                }
+            }
+        });
 
-            downloadedChunks.push({
-                index: header.chunkIndex,
-                data: payload,
-                compressed: header.compressed
-            });
-        }
+        const tmpOutputPath = outputPath + '.tmp';
+        const writeStream = fs.createWriteStream(tmpOutputPath);
 
-        // Reassemble
-        downloadedChunks.sort((a, b) => a.index - b.index);
-        const encryptedData = Buffer.concat(downloadedChunks.map(c => c.data));
+        // Pipeline: Telegram -> Decrypt -> Decompress -> Disk Cache
+        await pipeline(downloadStream, decryptStream, decompressStream, writeStream);
 
-        // Decrypt
-        const compressedData = this.encryptor.decrypt(encryptedData);
+        // Rename to final atomic path
+        fs.renameSync(tmpOutputPath, outputPath);
 
-        // Decompress
-        const wasCompressed = downloadedChunks[0].compressed;
-        return await this.compressor.decompress(compressedData, wasCompressed);
+        return outputPath;
     }
 
     async uploadFile(filename, data) {
@@ -399,7 +451,7 @@ export class TelegramFS {
         const encryptedData = this.encryptor.encrypt(compressedData);
 
         // Create temp file with header
-        const tempDir = path.join(this.dataDir, 'tmp');
+        const tempDir = process.env.TAS_TMP_DIR || path.join(this.dataDir, 'tmp');
         if (!fs.existsSync(tempDir)) {
             fs.mkdirSync(tempDir, { recursive: true });
         }
@@ -438,22 +490,34 @@ export class TelegramFS {
         if (!entry) return null;
 
         if (Date.now() - entry.timestamp > CACHE_TTL) {
+            // Expired, delete the file if possible
+            try {
+                if (fs.existsSync(entry.path)) fs.unlinkSync(entry.path);
+            } catch (e) { }
             fileCache.delete(filename);
             return null;
         }
 
-        return entry.data;
+        // Extend cache TTL on read
+        entry.timestamp = Date.now();
+        return entry.path;
     }
 
-    setCache(filename, data) {
+    setCache(filename, cachePath) {
         fileCache.set(filename, {
-            data,
+            path: cachePath,
             timestamp: Date.now()
         });
     }
 
     invalidateCache(filename) {
-        fileCache.delete(filename);
+        const entry = fileCache.get(filename);
+        if (entry) {
+            try {
+                if (fs.existsSync(entry.path)) fs.unlinkSync(entry.path);
+            } catch (e) { }
+            fileCache.delete(filename);
+        }
     }
 
     /**