telstore 0.1.10 → 0.1.11

package/README.md

@@ -37,6 +37,7 @@ npx telstore restore telstore-20260905-7f3a91
 | `telstore restore <backup-id> -- <command>...` | Restore onto the command's stdin instead of a file. |
 | `telstore verify <backup-id>...` | Check that every chunk of a backup is still in the chat. Downloads nothing. |
 | `telstore delete <backup-id>...` | Remove a backup's chunks and manifest from the chat, for good. Several ids are listed and confirmed once. |
+| `telstore join <manifest.json>` | Reassemble chunks you downloaded by hand. Offline: no login, no connection. |
 | `telstore status` | Account, destination, and unfinished uploads and restores. |
 | `telstore config` | Show or change settings. |
 | `telstore token` | Print a session token for a machine you do not trust. |
@@ -180,6 +181,25 @@ one. That is also how your data is compressed or encrypted **before** it reaches
 stderr is left as it is, so one that fails explains itself in its own words and telstore adds
 only the exit code and what it did about it.
 
+## Encrypting a backup
+
+```bash
+npx telstore photos.tar --encrypt
+npx telstore tarc photos.tar.gz ./photos --encrypt
+```
+
+telstore asks for a password twice and for an optional hint, then encrypts every chunk before
+it leaves the machine (AES-256-CTR, the key derived with scrypt, the manifest sealed with
+AES-256-GCM). `restore`, `tarx` and `join` see that a backup is encrypted, print its hint, and
+ask for the password. The hint is shown in the chat as plain text — telstore refuses one that
+contains the password. If telstore ever restores a backup you encrypted without asking for its
+password, the manifest in the chat has been replaced: do not trust the result.
+
+What stays readable to anyone who can read the chat: the file name, the note, the size, the
+number of chunks, the dates and the hint. **A forgotten password is a lost backup** — nothing
+can recover it. The password is only ever typed at a terminal; for unattended encrypted
+backups, use `--` with a key-based tool such as `age`.
+
 ## Checking a backup is still there
 
 A backup is a set of messages in a chat, and messages can be deleted by hand. `list` reads
@@ -210,6 +230,26 @@ Chunk 3/12 is gone: message 1042 is no longer in @my_backups.
 12 chunks checked, 1 damaged. This backup cannot be restored.
 ```
 
+## Joining chunks downloaded by hand
+
+A backup is ordinary files in a chat, so it can be fetched without telstore — from Telegram
+web, on a machine where you cannot or would rather not log in. Download the manifest
+(`<backupId>.manifest.json`) and every chunk (`<backupId>.part0001`, `.part0002`, …) into one
+folder, keeping the names they have in the chat, then:
+
+```bash
+npx telstore join ~/Downloads/telstore-20260905-7f3a91.manifest.json
+npx telstore join ~/Downloads/telstore-20260905-7f3a91.manifest.json --out data.tar
+```
+
+`join` makes the same promise `restore` does, from the manifest rather than the chat: every
+chunk is checked for its length before anything is written, every missing or short one is
+named in one go, each chunk's sha256 is checked as it is copied, and the file takes its real
+name only after all of it passes. Until then it is `<target>.joining`, which a failure or a
+Ctrl-C removes — the chunks are still in the folder, so there is nothing worth keeping. A
+browser that saves a name twice tends to add ` (1)` to it; `join` does not guess past that,
+so rename the file back.
+
 ## Several backups at once
 
 `restore`, `verify` and `delete` take a list of ids the same way, over one connection, with a
@@ -269,9 +309,9 @@ There is no expiry and no revocation: to end a session for good, terminate it un
 
 ## Limits worth knowing
 
-- **Your data is not encrypted.** Don't upload anything you would mind sitting on someone
-  else's infrastructure — the one thing telstore encrypts is a session token, and that
-  protects your login rather than your files.
+- **Your data is not encrypted unless you pass `--encrypt`.** Without it, don't upload anything
+  you would mind sitting on someone else's infrastructure. With it, the contents are encrypted
+  but the file name, note, size and hint are not, and a forgotten password cannot be recovered.
 - A chunk cannot exceed 1950MB: Telegram accepts at most 4000 parts of 512KB per file.
 - **A backup made from a command cannot be resumed**, so a run that fails or is interrupted
   removes the chunks it had already sent instead of keeping them. Ctrl-C takes a moment longer

package/bin/telstore.js

@@ -38,8 +38,9 @@ let settled = false
 // This process is already on its way out, and has already said why.
 let leaving = false
 
-// The chunk file a stream upload is buffering into right now, or null. The run removes its
-// own on every ending it gets to run code for; this exists for the one ending it does not.
+// The chunk file a stream upload is buffering into right now, or the half-joined file a join
+// is writing, or null. The run removes its own on every ending it gets to run code for; this
+// exists for the one ending it does not.
 let tempChunk = null
 
 // The command a streaming restore is feeding, if one is running. A stream upload unwinds
@@ -99,8 +100,8 @@ function dropTempChunk() {
     if (err.code === 'ENOENT') return ''
 
     return (
-      `\nThe chunk telstore was buffering is still on this machine: ${file} ` +
-      `(${err.message}). It holds up to one chunk — remove it by hand.\n`
+      `\nThe temporary file telstore was writing is still on this machine: ${file} ` +
+      `(${err.message}). Nothing needs it — remove it by hand.\n`
     )
   }
 }
@@ -410,6 +411,19 @@ async function main() {
       return
     }
 
+    case 'join': {
+      const { runJoin } = await import('../src/commands/join.js')
+
+      // The half-joined file rides the same seam as a stream upload's buffered chunk, so the
+      // SIGINT handler removes it on the way out rather than leaving it for someone to find.
+      await runJoin(parsed.args[0], parsed.options, {
+        onTempChunk: (file) => {
+          tempChunk = file
+        },
+      })
+      return
+    }
+
     default:
       throw new Error(`Unknown command: ${parsed.command}`)
   }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "telstore",
-  "version": "0.1.10",
+  "version": "0.1.11",
   "description": "Split large files into chunks and store them on Telegram",
   "keywords": [
     "telegram",

package/src/caption.js

@@ -14,6 +14,23 @@ function oneLine(name) {
   return String(name).replace(/\s+/g, ' ').trim()
 }
 
+// C0 and C1, DEL included. What makes a string an instruction to the terminal rather than text
+// on it: an escape sequence can clear the screen, move the cursor, or rewrite the line above.
+const CONTROL_CHARACTERS = /[\x00-\x1f\x7f-\x9f]/g
+
+export function hasControlCharacter(text) {
+  return /[\x00-\x1f\x7f-\x9f]/.test(String(text))
+}
+
+// A hint is printed before anything can prove it genuine: the seal covers it, but only a password
+// that opens the seal says so, and the hint is what someone reads while deciding what to type. So
+// everywhere it reaches a terminal it goes through here first — whitespace folded the way oneLine
+// folds it, every other control character dropped — and the most a hint someone else wrote can do
+// is say something unhelpful, never repaint the screen around the password prompt.
+export function terminalSafe(text) {
+  return oneLine(String(text).replace(/\s/g, ' ').replace(CONTROL_CHARACTERS, ''))
+}
+
 // Telegram takes 1024 characters in a caption, and the card around the note already spends
 // some of them — a file name alone may be 255. 500 leaves both room to spare.
 export const MAX_NOTE_LENGTH = 500
@@ -45,6 +62,42 @@ export function parseNote(raw) {
   return note
 }
 
+// What the card has left once every other line has had its share. Measured 2026-09-14: the
+// worst card telstore writes without encryption — a 255-character name, a 500-character note,
+// 10000 chunks — is 899 of Telegram's 1024 characters, and the lock and hint lines leave 108.
+export const MAX_HINT_LENGTH = 100
+
+const LOCK_LINE = '🔒 encrypted'
+
+// Written at the password prompt and shown in the open: on the card, in `list`, and above the
+// password prompt at restore time. So a hint holding the password is a password in the chat.
+//
+// Stripped of control characters here, once, rather than only when printed: parseManifest refuses
+// a hint carrying one, so a hint that kept it would be an honest backup restore turns away.
+export function parseHint(raw, password = null) {
+  if (raw === undefined || raw === null) return null
+
+  const hint = terminalSafe(raw)
+
+  if (hint === '') return null
+
+  if (hint.length > MAX_HINT_LENGTH) {
+    throw new Error(
+      `The hint is ${hint.length} characters, and the card in the chat has room for ` +
+        `${MAX_HINT_LENGTH}. Shorten it: telstore will not cut it short by itself.`,
+    )
+  }
+
+  if (password && hint.toLowerCase().includes(String(password).toLowerCase())) {
+    throw new Error(
+      'The hint contains the password itself, and the hint is shown in the chat as plain ' +
+        'text. Write something only you would connect with it.',
+    )
+  }
+
+  return hint
+}
+
 function utcMinutes(createdAt) {
   return `${new Date(createdAt).toISOString().slice(0, 16).replace('T', ' ')} UTC`
 }
@@ -58,7 +111,7 @@ export function chunkCaption({ id, number, total }) {
     : `📦 ${id} · ${number}/${total}`
 }
 
-export function manifestCaption({ id, name, size, chunks, createdAt, note = null }) {
+export function manifestCaption({ id, name, size, chunks, createdAt, note = null, encrypted = false, hint = null }) {
   return [
     `📄 ${oneLine(name)}`,
     `💾 ${formatBytes(size)} · ${chunks} chunk${chunks === 1 ? '' : 's'}`,
@@ -67,6 +120,10 @@ export function manifestCaption({ id, name, size, chunks, createdAt, note = null
     // Below the facts telstore knows, above the line that says how to get the file back:
     // the note is the one part of the card a person wrote, so it reads last of the four.
     ...(note ? [`📝 ${oneLine(note)}`] : []),
+    // Below the note and above the restore line: the lock is a fact about the backup a person
+    // needs before they try to restore it, and the hint is what they will need at the prompt.
+    ...(encrypted ? [LOCK_LINE] : []),
+    ...(encrypted && hint ? [`💡 ${oneLine(hint)}`] : []),
     '',
     `↩ npx telstore restore ${id}`,
     MANIFEST_TAG,
@@ -93,11 +150,16 @@ export function parseManifestCaption(text) {
   // one marker whose absence means "there is no note" rather than "this is not a card".
   const note = marker(lines, '📝')
 
+  // Both optional, like the note: every card telstore wrote before encryption existed is a
+  // complete card with neither.
+  const encrypted = lines.includes(LOCK_LINE)
+  const hint = marker(lines, '💡')
+
   if (!name || !totals || !id || !createdAt) return null
 
   const match = /^(.+) · (\d+) chunks?$/.exec(totals)
 
   if (!match) return null
 
-  return { id, name, size: match[1], chunks: Number(match[2]), createdAt, note }
+  return { id, name, size: match[1], chunks: Number(match[2]), createdAt, note, encrypted, hint }
 }

package/src/cipher.js

@@ -0,0 +1,211 @@
+import { createCipheriv, createDecipheriv, createHash, createHmac, hkdfSync, randomBytes, scrypt } from 'node:crypto'
+import { promisify } from 'node:util'
+
+const derive = promisify(scrypt)
+
+// Pinned to manifest version 2 and never read from a manifest, for the reason src/token.js
+// pins its own: a manifest naming its own N would let a stranger decide how much memory this
+// machine allocates, and whoever holds a manifest can try passwords offline as fast as their
+// hardware allows — 64MB per attempt is what makes that expensive.
+const SCRYPT = { N: 65536, r: 8, p: 1, maxmem: 128 * 1024 * 1024 }
+const KEY_BYTES = 32
+const SALT_BYTES = 16
+const IV_BYTES = 8
+const NONCE_BYTES = 12
+const TAG_BYTES = 16
+const BLOCK = 16
+const SHA256 = /^[0-9a-f]{64}$/
+
+// Large enough that a 1800MB chunk is a couple of hundred reads, and a multiple of the AES
+// block so a full pass never needs the partial-block path.
+const IN_PLACE_BLOCK = 8 * 1024 * 1024
+
+export function newSalt() {
+  return randomBytes(SALT_BYTES).toString('hex')
+}
+
+// Fresh for every attempt at a chunk, never derived from its index: a run that dies halfway
+// through chunk 3 leaves parts of it on Telegram's servers, and a derived nonce would put the
+// next run's chunk 3 — possibly different bytes, if the file changed — under the same keystream.
+export function newIv() {
+  return randomBytes(IV_BYTES).toString('hex')
+}
+
+// Two keys, because GCM is CTR inside: one key used for both would let a counter block of the
+// manifest seal coincide with a counter block of some chunk.
+export async function deriveKeys(password, salt) {
+  const saltBytes = Buffer.from(salt, 'hex')
+  const master = await derive(String(password).normalize('NFC'), saltBytes, KEY_BYTES, SCRYPT)
+
+  return {
+    chunkKey: Buffer.from(hkdfSync('sha256', master, saltBytes, 'telstore v2 chunk key', KEY_BYTES)),
+    manifestKey: Buffer.from(hkdfSync('sha256', master, saltBytes, 'telstore v2 manifest key', KEY_BYTES)),
+  }
+}
+
+// What an unfinished upload keeps on disk instead of the password: enough to refuse a resume
+// under a different one, which would put two keys into one backup.
+export function passwordCheck(keys) {
+  return createHmac('sha256', keys.manifestKey).update('telstore v2 password check').digest('hex')
+}
+
+// CTR at any offset inside a chunk: the counter block for byte `offset` is the chunk's iv
+// followed by the 64-bit block number, and the bytes before `offset` inside that block are
+// discarded. A chunk is at most 1950MB, about 1.3e8 blocks, so the counter never carries into
+// the iv.
+export function chunkCipher(chunkKey, iv) {
+  const prefix = Buffer.from(iv, 'hex')
+
+  return {
+    apply(bytes, offset) {
+      const counter = Buffer.alloc(BLOCK)
+      prefix.copy(counter, 0)
+      counter.writeBigUInt64BE(BigInt(Math.floor(offset / BLOCK)), IV_BYTES)
+
+      const cipher = createCipheriv('aes-256-ctr', chunkKey, counter)
+      const skip = offset % BLOCK
+
+      if (skip > 0) cipher.update(Buffer.alloc(skip))
+
+      return Buffer.concat([cipher.update(bytes), cipher.final()])
+    },
+  }
+}
+
+// Built from the fields, in one fixed order, and never by serializing the parsed object again:
+// an array has one serialization, while an object's depends on the key order of a file a person
+// can edit. Every field is covered, the readable ones included. That proves the hint genuine only
+// once a password opens the seal, and it is printed before then, so it is also made safe to print
+// (terminalSafe in src/caption.js) rather than trusted because of this.
+export function additionalData(manifest) {
+  return Buffer.from(
+    JSON.stringify([
+      'telstore-enc-v2',
+      manifest.id,
+      manifest.name,
+      manifest.size,
+      manifest.chunkSize,
+      manifest.createdAt,
+      manifest.note ?? null,
+      manifest.enc.salt,
+      manifest.enc.hint ?? null,
+      manifest.chunks.map((chunk) => [chunk.i, chunk.msgId, chunk.size, chunk.sha256, chunk.iv]),
+    ]),
+    'utf8',
+  )
+}
+
+export function sealManifest(manifest, keys, plainSha256) {
+  const nonce = randomBytes(NONCE_BYTES)
+  const cipher = createCipheriv('aes-256-gcm', keys.manifestKey, nonce)
+
+  cipher.setAAD(additionalData(manifest))
+
+  const body = Buffer.concat([cipher.update(JSON.stringify({ plainSha256 }), 'utf8'), cipher.final()])
+
+  return {
+    ...manifest,
+    enc: {
+      ...manifest.enc,
+      sealed: Buffer.concat([nonce, cipher.getAuthTag(), body]).toString('base64'),
+    },
+  }
+}
+
+// Null when the tag fails, which is one failure with two causes — a wrong password or an
+// altered manifest — and the caller is the one that decides whether to ask again. Past the tag
+// the contents are ours, so anything malformed there is a telstore that disagreed with this one
+// about the format, and that throws rather than reading as a wrong password.
+export async function openManifest(manifest, password) {
+  const keys = await deriveKeys(password, manifest.enc.salt)
+  const sealed = Buffer.from(manifest.enc.sealed, 'base64')
+
+  if (sealed.length <= NONCE_BYTES + TAG_BYTES) return null
+
+  const decipher = createDecipheriv('aes-256-gcm', keys.manifestKey, sealed.subarray(0, NONCE_BYTES))
+  decipher.setAAD(additionalData(manifest))
+  decipher.setAuthTag(sealed.subarray(NONCE_BYTES, NONCE_BYTES + TAG_BYTES))
+
+  let text
+  try {
+    text = Buffer.concat([
+      decipher.update(sealed.subarray(NONCE_BYTES + TAG_BYTES)),
+      decipher.final(),
+    ]).toString('utf8')
+  } catch {
+    return null
+  }
+
+  let inside = null
+  try {
+    inside = JSON.parse(text)
+  } catch {
+    // Falls through to the refusal below.
+  }
+
+  const hashes = inside?.plainSha256
+
+  if (
+    !Array.isArray(hashes) ||
+    hashes.length !== manifest.chunks.length ||
+    !hashes.every((hash) => typeof hash === 'string' && SHA256.test(hash))
+  ) {
+    throw new Error(
+      `The manifest of ${manifest.id} opened, but what is sealed inside it is not what telstore ` +
+        'writes. Refusing to restore from it.',
+    )
+  }
+
+  return { keys, plainSha256: hashes }
+}
+
+async function readFully(handle, buffer, length, position) {
+  let filled = 0
+
+  while (filled < length) {
+    const { bytesRead } = await handle.read(buffer, filled, length - filled, position + filled)
+
+    if (bytesRead === 0) {
+      throw new Error(`Short read: needed ${length} bytes at offset ${position} but the file ended.`)
+    }
+
+    filled += bytesRead
+  }
+}
+
+async function writeFully(handle, buffer, position) {
+  let written = 0
+
+  while (written < buffer.length) {
+    const { bytesWritten } = await handle.write(buffer, written, buffer.length - written, position + written)
+
+    // readFully's guard, for the same reason: a write that makes no progress would spin here
+    // forever rather than fail, and a .partial half-decrypted with nobody told is the worst end.
+    if (bytesWritten === 0) {
+      throw new Error(`Short write: ${buffer.length - written} bytes at offset ${position + written} would not go to disk.`)
+    }
+
+    written += bytesWritten
+  }
+}
+
+// Reads a range that holds a verified chunk's ciphertext, writes its plaintext back over it,
+// and hashes the plaintext on the way. Called only after the ciphertext sha256 has matched, so
+// the hash it returns is the second check, not the first.
+export async function decryptInPlace(handle, offset, length, cipher, { blockSize = IN_PLACE_BLOCK } = {}) {
+  const hash = createHash('sha256')
+  const buffer = Buffer.allocUnsafe(Math.min(blockSize, Math.max(length, 1)))
+
+  for (let at = 0; at < length; at += blockSize) {
+    const size = Math.min(blockSize, length - at)
+
+    await readFully(handle, buffer, size, offset + at)
+
+    const clear = cipher.apply(buffer.subarray(0, size), at)
+
+    hash.update(clear)
+    await writeFully(handle, clear, offset + at)
+  }
+
+  return hash.digest('hex')
+}

package/src/cli.js

@@ -23,6 +23,7 @@ const SUBCOMMANDS = new Set([
   'restore',
   'verify',
   'delete',
+  'join',
   'status',
   'config',
   'token',
@@ -37,6 +38,7 @@ export const OPTIONS = {
   'download-concurrency': { type: 'string' },
   out: { type: 'string' },
   note: { type: 'string' },
+  encrypt: { type: 'boolean' },
   limit: { type: 'string' },
   search: { type: 'string' },
   verbose: { type: 'boolean' },
@@ -60,6 +62,7 @@ Usage:
   npx telstore restore <backup-id>...     Download the chunks and reassemble the files
   npx telstore verify <backup-id>...      Check that a backup's chunks are all still in the chat
   npx telstore delete <backup-id>...      Remove backups' chunks and manifests from the chat
+  npx telstore join <manifest.json>       Reassemble chunks you downloaded by hand, offline
   npx telstore status                     Show the account, the destination and unfinished uploads and restores
   npx telstore config                     Show every setting and where its value comes from
   npx telstore logout                     Remove the saved session
@@ -106,6 +109,11 @@ an exit code that reports any that failed. delete shows everything it is about t
 asks once. verify downloads nothing: it asks the chat whether every chunk message is still
 there at the length the manifest records, which is what restore would need.
 
+join is restore without Telegram, for chunks you downloaded yourself — from Telegram web, say.
+Put the manifest and every <id>.partNNNN file in one folder under the names they have in the
+chat, and point join at the manifest. It needs no login and opens no connection, and it checks
+each chunk's size and sha256 against the manifest before the file takes its real name.
+
 Settings:
   npx telstore config <name>              Print one setting's value
   npx telstore config <name> <value>      Change it for good
@@ -124,14 +132,20 @@ Options apply to one run and are never saved. Use config to change a setting for
                               size it started with.
   --upload-concurrency <n>    512KB parts in parallel while uploading, this run only.
   --download-concurrency <n>  8MB slices in parallel while restoring, this run only.
-  --out <path>                Where to write the restored file. Defaults to the basename in
-                              the manifest, and works with one backup id only.
+  --out <path>                Where to write the restored or joined file. Defaults to the
+                              basename in the manifest, and works with one backup id only.
   --note <text>               A note to store with the upload. It goes into the manifest and
                               onto the manifest message, where Telegram's own search can find
                               it, and every file of a batch gets the same one. A note with
                               spaces in it has to be quoted — --note "march archive" — or the
                               shell hands the words after the first to telstore as more files
                               to upload.
+  --encrypt                   Encrypt the contents with a password before they leave this
+                              machine. Asks for the password twice and for an optional hint,
+                              which is shown in the chat as plain text. restore, tarx and join
+                              see from the manifest that a backup is encrypted and ask for the
+                              password themselves. The file name, note and size stay readable,
+                              and a forgotten password is a lost backup.
   --limit <n>                 How many backups list shows this run.
   --search <text>             List only the backups whose file name, note, backup id or
                               creation day contains this text. Telegram's own index does
@@ -271,6 +285,13 @@ export function interruptMessage(
     )
   }
 
+  // A join writes nothing under the real name until the end, and the half-joined file is
+  // removed on the way out, so there is nothing to carry on from — and nothing in the chat
+  // it could have touched.
+  if (command === 'join') {
+    return '\nStopped. Nothing was joined — run the same command again to start over.\n'
+  }
+
   return '\nStopped.\n'
 }
 
@@ -409,7 +430,7 @@ function requireOneBackupId(ids, what) {
   }
 }
 
-export function route(argv) {
+function routeLine(argv) {
   const { head, childArgv } = splitAtTerminator(argv)
 
   const { values, positionals, tokens } = parseArgs({
@@ -514,6 +535,22 @@ export function route(argv) {
   const shortcut = SHORTCUTS.get(first)
   if (shortcut) return shortcut(rest, values, filesAfterNote)
 
+  // --out names one file, and each manifest is one file: a second manifest on the line would
+  // have to be written somewhere nobody said.
+  if (first === 'join') {
+    if (rest.length === 0) {
+      throw new Error(
+        'Missing the manifest to join. Example: npx telstore join ./telstore-20260905-7f3a91.manifest.json',
+      )
+    }
+
+    if (rest.length > 1) {
+      throw new Error(
+        `join takes one manifest and got ${rest.length}: ${rest.join(', ')}. Run it once per backup.`,
+      )
+    }
+  }
+
   if (SUBCOMMANDS.has(first)) {
     return { command: first, args: rest, options: values, filesAfterNote, childArgv, shortcut: null }
   }
@@ -522,3 +559,19 @@ export function route(argv) {
   // rest without a word, which is the one thing this project never does.
   return { command: 'upload', args: positionals, options: values, filesAfterNote, childArgv, shortcut: null }
 }
+
+// Encryption is decided when a backup is made; every command that reads one learns it from the
+// manifest. A flag beside a restore would read as "decrypt with this", which it would not be,
+// and a flag that silently does nothing is one nobody can predict without the source.
+export function route(argv) {
+  const parsed = routeLine(argv)
+
+  if (parsed.options.encrypt && parsed.command !== 'upload' && parsed.command !== 'help') {
+    throw new Error(
+      '--encrypt applies to uploads only. restore, tarx and join see from the manifest that a ' +
+        'backup is encrypted and ask for its password by themselves.',
+    )
+  }
+
+  return parsed
+}