whalibmob 5.29.6 → 5.32.0

package/README.md

@@ -194,6 +194,9 @@ npm install -g whalibmob
     - [One-time Pre-keys](#one-time-pre-keys)
     - [Where the Folder Comes From](#where-the-folder-comes-from)
     - [Working Out the Paths Yourself](#working-out-the-paths-yourself)
+    - [Where a Session Is Kept](#where-a-session-is-kept)
+    - [One Database Instead of 834 Files](#one-database-instead-of-834-files)
+    - [Moving a Session Between Backends](#moving-a-session-between-backends)
   - [Signal Store Utilities](#signal-store-utilities)
     - [makeCacheableSignalKeyStore](#makecacheablesignalkeystore)
     - [addTransactionCapability](#addtransactioncapability)
@@ -1845,6 +1848,9 @@ Both are optional and both keep working when omitted — you get an error naming
 > [!NOTE]
 > Registration reports the screens it passes through to WhatsApp's `/client_log`, the way the phone clients do — a client that registers in total silence does something no real installation does. It is fire-and-forget and every failure is swallowed, so it can never take a registration down. Set `WA_FUNNEL_LOG=0` to send none of it.
 
+> [!NOTE]
+> Those events carry timestamps, and registration waits between them the way a person would: a few seconds to type the number in, a moment on the confirmation sheet, longer before asking again after a refusal. Without the waits the whole funnel leaves inside one millisecond, which no handset does. It adds a handful of seconds to a registration. Set `WA_REG_PACING=0` to remove them.
+
 ### Registering as Android
 
 **There is nothing to do first.** Name the platform and register:
@@ -3061,6 +3067,171 @@ migrateSession(base, '919634847671')
 `SESSION_SUFFIXES` is every per-number file the library writes — the list to
 copy or delete against if you are moving an account by hand.
 
+### Where a Session Is Kept
+
+Everything above describes files, because files are what whalibmob writes and
+what it will go on writing unless you say otherwise. **Nothing in this section
+is something you have to do.** Leave it alone and sessions stay exactly where
+they have always been, in the JSON files named above.
+
+What is new is that the place is now a choice. A **backend** is four
+synchronous operations over a flat key space:
+
+```js
+read(key)          // the stored text, or null when the key was never written
+write(key, value)  // put it there, replacing whatever was there before
+remove(key)        // take it away; a key that is not there is not an error
+list(prefix)       // every key present that starts with prefix
+```
+
+The keys are logical names rather than file names — `auth`, `signal`,
+`sender-key`, `tc-token`, `device-cache`, `lid-mapping`,
+`lid-reverse-mapping`, `history`, `messages`, `app-state`, `app-state-keys`,
+and `` `pre-key/${id}` `` for each of the 812 one-time pre-keys.
+
+Three implementations ship with the package:
+
+| Backend | Where the state goes | Needs |
+|---|---|---|
+| `FileBackend` | JSON files — what the library has always written | nothing; this is the default |
+| `SqliteBackend` | one database file | Node 22.5+, or `better-sqlite3` |
+| `MemoryBackend` | nowhere; gone when the process ends | nothing |
+
+```js
+const { FileBackend } = require('whalibmob')
+
+const backend = new FileBackend({
+  dir:   '/home/you/.waSession/919634847671',
+  phone: '919634847671'
+})
+
+backend.write('auth', JSON.stringify(creds))
+backend.read('auth')          // the text, or null
+backend.list()                // ['auth', 'pre-key/1', 'signal', …]
+backend.list('pre-key/')      // just the pre-keys
+backend.remove('pre-key/42')
+backend.fileFor('signal')     // …/919634847671.signal.json
+```
+
+`FileBackend` writes the same names in the same places as every release before
+it, so a session written by whalibmob 5.30 opens through it untouched and one
+written through it opens in 5.30. The only change is that writes now go to a
+temporary file and are renamed into place, so a process that dies mid-write
+leaves the previous state intact instead of half a file.
+
+> [!IMPORTANT]
+> **A number has two halves, and they are separate sessions.** The one
+> registered over the Mobile API and the companion linked over the Web API
+> never share state — in particular they have separate pre-key id spaces, and
+> mixing them hands out two different keys under one id and breaks decryption.
+> A backend covers **one** half, chosen by `web`:
+>
+> ```js
+> const mobile = new FileBackend({ dir, phone, web: false })  // default
+> const web    = new FileBackend({ dir, phone, web: true })
+> ```
+>
+> Both take the same keys and keep entirely separate values.
+
+> [!NOTE]
+> **The client does not accept a backend yet.** `new WhalibmobClient({ … })`
+> takes `sessionDir` and writes files, as it always has; the modules that hold
+> session state still do their own file I/O. The backends are usable on their
+> own — for reading, inspecting, copying or moving a session — and wiring them
+> through the client is the next step. Nothing here changes how a session is
+> created or connected today.
+
+### One Database Instead of 834 Files
+
+A number's state is 22 named files plus a file for each of its 812 one-time
+pre-keys. On a laptop nobody notices. On a phone under Termux, on a container
+with a small inode budget, or with fifty numbers in one folder, it is 40 000
+files whose directory has to be read every time the pre-key pool is counted.
+
+`SqliteBackend` is the same state as a handful of rows:
+
+```js
+const { SqliteBackend } = require('whalibmob')
+
+const db = new SqliteBackend({
+  path:  '/home/you/.waSession/sessions.sqlite',
+  phone: '919634847671'
+})
+
+db.write('auth', JSON.stringify(creds))
+db.read('auth')
+db.list('pre-key/')
+db.driver          // 'node:sqlite' or 'better-sqlite3'
+db.close()         // let go of the file
+```
+
+**What it needs.** Node ships SQLite of its own from **22.5.0** as
+`node:sqlite`, and that is what this uses when it is there — nothing to
+install, nothing for node-gyp to fail at, and Termux stays a place whalibmob
+runs. On older Node it falls back to `better-sqlite3` if that is installed:
+
+```sh
+npm install better-sqlite3      # only on Node older than 22.5
+```
+
+Neither is a dependency of this package. On a runtime with neither, the
+constructor throws and says which of the two to reach for — and `FileBackend`
+goes on needing nothing at all.
+
+One file holds as many numbers and halves as you like, while each backend sees
+only its own slice:
+
+```js
+const mobile = new SqliteBackend({ path: file, phone: '919634847671' })
+const web    = new SqliteBackend({ path: file, phone: '919634847671', web: true })
+const other  = new SqliteBackend({ path: file, phone: '40712345678' })
+
+SqliteBackend.sessionsIn(file)
+// [ { phone: '40712345678',  half: 'mobile', web: false },
+//   { phone: '919634847671', half: 'mobile', web: false },
+//   { phone: '919634847671', half: 'web',    web: true  } ]
+```
+
+The file handle is shared between the backends opened on it and closed once
+the last of them calls `close()`, so closing one does not pull the file out
+from under the others. Calling `close()` twice is harmless.
+
+The database runs in WAL mode, so a client flushing its Signal store and
+another reading the pre-key pool do not block each other. Values are stored as
+blobs rather than text: a credential blob carrying a NUL byte is truncated by
+a text binding and the session then loads with keys that are wrong from that
+byte on, which is the worst way for a bug to present.
+
+### Moving a Session Between Backends
+
+Nothing is removed and the source is never modified, so if the result is not
+what you wanted the old files are still there to go back to:
+
+```js
+const { FileBackend, SqliteBackend, copySession, compareSessions } = require('whalibmob')
+
+const files = new FileBackend({ dir: '/home/you/.waSession/919634847671',
+                                phone: '919634847671' })
+const db    = new SqliteBackend({ path: '/home/you/.waSession/sessions.sqlite',
+                                  phone: '919634847671' })
+
+const moved = copySession(files, db)
+// { copied: ['auth', 'pre-key/1', …], skipped: [], bytes: 1284 }
+
+const check = compareSessions(files, db)
+// { ok: true, missing: [], differing: [], extra: [] }
+
+db.close()
+```
+
+`compareSessions` reads both sides back rather than trusting that the copy
+said so — run it before deleting anything. A key the destination already holds
+is left alone, so an interrupted copy is safe to run again; pass
+`{ overwrite: true }` when you do mean to replace what is there.
+
+Do both halves of a number separately — `web: false` and `web: true` are two
+sessions and a copy of one is not a copy of the other.
+
 ## Signal Store Utilities
 
 `auth-utils` is a collection of optional helpers for power users who manage their own `SignalStore` instances directly (e.g. custom storage backends, multi-account servers).

package/index.d.ts

@@ -1297,6 +1297,180 @@ export declare const SessionPaths: {
   SHARED_FILES: string[];
 };
 
+// ────────────────────────────────────────────────────────────────────────────
+// Session storage
+// ────────────────────────────────────────────────────────────────────────────
+
+/**
+ * A key under which a piece of session state is kept — one of the fixed names
+ * (`'auth'`, `'signal'`, …) or `` `pre-key/${id}` `` for one one-time pre-key.
+ */
+export type StorageKey = string;
+
+/**
+ * Where a session's state is kept.
+ *
+ * Four synchronous operations over a flat key space. `FileBackend` is the
+ * implementation the library has always used and remains the default; anything
+ * satisfying this interface can take its place.
+ *
+ * Synchronous on purpose: the Signal store flushes from `exit` and `SIGTERM`
+ * handlers, where an awaited write does not land.
+ */
+export interface StorageBackend {
+  /** The stored text, or `null` when the key has never been written. */
+  read(key: StorageKey): string | null;
+  /** Store `value` under `key`, replacing whatever was there. */
+  write(key: StorageKey, value: string): void;
+  /** Remove `key`. A key that is not there is not an error. */
+  remove(key: StorageKey): void;
+  /** Every key present that starts with `prefix` (all of them when omitted). */
+  list(prefix?: string): StorageKey[];
+}
+
+/** Options for {@link FileBackend}. */
+export interface FileBackendOptions {
+  /** The directory this session's files live in. */
+  dir: string;
+  /** The number. Non-digits are stripped. */
+  phone: string;
+  /** The companion (Web API) half rather than the mobile one. Default `false`. */
+  web?: boolean;
+}
+
+/**
+ * The session on disk as JSON files — what whalibmob has always written, under
+ * the same names, in the same place.
+ */
+export declare class FileBackend implements StorageBackend {
+  constructor(opts: FileBackendOptions);
+  readonly dir: string;
+  readonly phone: string;
+  readonly web: boolean;
+  /** The absolute path a key is kept at, whether or not it exists yet. */
+  fileFor(key: StorageKey): string;
+  read(key: StorageKey): string | null;
+  write(key: StorageKey, value: string): void;
+  remove(key: StorageKey): void;
+  list(prefix?: string): StorageKey[];
+  /** key → the part of the file name that follows the number. */
+  static KEY_SUFFIX: Record<string, string>;
+  /** What sits between the stem and a pre-key's id. */
+  static PRE_KEY_INFIX: string;
+}
+
+/** Options for {@link SqliteBackend}. */
+export interface SqliteBackendOptions {
+  /** The database file. Created, with its directory, if missing. */
+  path: string;
+  /** The number. Non-digits are stripped. */
+  phone: string;
+  /** The companion (Web API) half rather than the mobile one. Default `false`. */
+  web?: boolean;
+  /** Demand one driver rather than taking whichever is available. */
+  driver?: 'node' | 'better-sqlite3';
+}
+
+/** One number and half of it, as held in a database file. */
+export interface SqliteSessionRow {
+  phone: string;
+  half: 'mobile' | 'web';
+  web: boolean;
+}
+
+/**
+ * The session in one database instead of 834 files.
+ *
+ * Uses Node's built-in `node:sqlite` (22.5.0 and later) when it is there, and
+ * `better-sqlite3` when it is not. Neither is a dependency of this package —
+ * on a runtime with neither, the constructor throws saying what to install,
+ * and {@link FileBackend} goes on needing nothing.
+ *
+ * One file holds any number of sessions; a backend addresses one number's one
+ * half of it. The file handle is shared between the backends opened on it and
+ * released when the last of them calls `close()`.
+ */
+export declare class SqliteBackend implements StorageBackend {
+  constructor(opts: SqliteBackendOptions);
+  readonly path: string;
+  readonly phone: string;
+  readonly web: boolean;
+  /** Which driver this backend actually opened the file with. */
+  readonly driver: 'node:sqlite' | 'better-sqlite3';
+  read(key: StorageKey): string | null;
+  write(key: StorageKey, value: string): void;
+  remove(key: StorageKey): void;
+  list(prefix?: string): StorageKey[];
+  /** Let go of the database; the file closes once nobody else holds it. */
+  close(): void;
+  /** Every number and half the database file holds. */
+  static sessionsIn(file: string, opts?: { driver?: 'node' | 'better-sqlite3' }): SqliteSessionRow[];
+  /** The schema version this release writes and understands. */
+  static SCHEMA_VERSION: number;
+  /** The table the state lives in. */
+  static TABLE: string;
+}
+
+/** What {@link copySession} did. */
+export interface CopySessionResult {
+  /** Keys written to the destination. */
+  copied: StorageKey[];
+  /** Keys left alone — already present, or gone from the source mid-copy. */
+  skipped: StorageKey[];
+  /** How much was copied, in UTF-8 bytes. */
+  bytes: number;
+}
+
+/** What {@link compareSessions} found. */
+export interface CompareSessionsResult {
+  /** True when both hold the same keys with the same values. */
+  ok: boolean;
+  /** Keys the first has and the second does not. */
+  missing: StorageKey[];
+  /** Keys both have, holding different values. */
+  differing: StorageKey[];
+  /** Keys the second has and the first does not. */
+  extra: StorageKey[];
+}
+
+/**
+ * Copy every key from one backend to another. The source is not modified, and
+ * a key the destination already holds is left alone unless `overwrite` is set —
+ * so an interrupted copy is safe to run again.
+ */
+export declare function copySession(
+  from: StorageBackend,
+  to: StorageBackend,
+  opts?: { overwrite?: boolean }
+): CopySessionResult;
+
+/**
+ * Check that two backends hold the same state, reading both sides rather than
+ * trusting that a copy said so. Worth running before deleting the original.
+ */
+export declare function compareSessions(
+  a: StorageBackend,
+  b: StorageBackend
+): CompareSessionsResult;
+
+/**
+ * A session held only for the life of the process.
+ *
+ * Nothing survives the run: a number registered against this backend cannot be
+ * recovered, because the keys that proved the registration are gone with it.
+ */
+export declare class MemoryBackend implements StorageBackend {
+  constructor();
+  read(key: StorageKey): string | null;
+  write(key: StorageKey, value: string): void;
+  remove(key: StorageKey): void;
+  list(prefix?: string): StorageKey[];
+  /** How many keys are held. Not part of the contract. */
+  readonly size: number;
+  /** Drop everything. Not part of the contract. */
+  clear(): void;
+}
+
 // ────────────────────────────────────────────────────────────────────────────
 // Device
 // ────────────────────────────────────────────────────────────────────────────
@@ -1432,6 +1606,35 @@ export interface LibModule {
   [name: string]: any;
 }
 
+/**
+ * `whalibmob/lib/store/Backend` — the storage contract itself: the key names a
+ * backend has to honour, and the checks that go with them. The implementations
+ * are {@link FileBackend} and {@link MemoryBackend}, declared above.
+ */
+export declare const StoreBackend: {
+  /** Every key that is a fixed name rather than one generated per record. */
+  KEYS: readonly string[];
+  /** The prefix the per-record pre-key keys are built on: `'pre-key/'`. */
+  PRE_KEY_PREFIX: string;
+  /** The key one pre-key id is stored under. Throws on a non-integer id. */
+  preKeyKey(id: number): StorageKey;
+  /** The id back out of a pre-key key, or `null` when the key is not one. */
+  preKeyId(key: StorageKey): number | null;
+  /** Whether a string is a key every backend must accept. */
+  isValidKey(key: unknown): boolean;
+  /** Throw unless `backend` implements the contract. Returns it when it does. */
+  assertBackend<T>(backend: T, what?: string): T;
+};
+
+/**
+ * `whalibmob/lib/store/migrate` — moving a session from one backend to another
+ * and checking that it landed. Both members are also exported flat, above.
+ */
+export declare const StoreMigrate: {
+  copySession: typeof copySession;
+  compareSessions: typeof compareSessions;
+};
+
 /**
  * `whalibmob/lib/MediaService` — the encryption, upload and download beneath
  * `client.downloadMedia()`. Everything it exports is typed.

package/index.js

@@ -102,6 +102,16 @@ const {
 
 const { encodeWAM, BinaryInfo, WEB_EVENTS, WEB_GLOBALS } = WAM;
 
+// Where a session's state goes. FileBackend is what the library has always
+// done — JSON files, same names — and is the default; anything implementing
+// the same four methods can take its place. See lib/store/Backend.js.
+const StoreBackend    = require('./lib/store/Backend');
+const { FileBackend }   = require('./lib/store/FileBackend');
+const { MemoryBackend } = require('./lib/store/MemoryBackend');
+const { SqliteBackend } = require('./lib/store/SqliteBackend');
+const StoreMigrate      = require('./lib/store/migrate');
+const { copySession, compareSessions } = StoreMigrate;
+
 // ─── The namespaces ──────────────────────────────────────────────────────────
 //
 // Every module of lib/, whole, under a name of its own — see the note at the
@@ -229,6 +239,20 @@ module.exports = {
   webStoreFileFor,
   listSessions,
   migrateSession,
+  // Where a session's state is kept. FileBackend is the default and writes the
+  // JSON files whalibmob has always written; MemoryBackend keeps a session only
+  // for the life of the process. Both satisfy the contract in StoreBackend.
+  StoreBackend,
+  FileBackend,
+  MemoryBackend,
+  // One database instead of 834 files. Needs Node 22.5+ for its built-in
+  // node:sqlite, or better-sqlite3 installed; neither is a dependency, and
+  // FileBackend stays the default that needs nothing.
+  SqliteBackend,
+  // Moving a session from one backend to another, and checking that it landed.
+  StoreMigrate,
+  copySession,
+  compareSessions,
   // Device config — reads WA_OS / WA_DEVICE / WA_DEVICE_* from process.env
   getDeviceConfig,
   // Store helpers

package/lib/Registration.js

@@ -1339,10 +1339,17 @@ function getAppPid(store) {
   return store._appPid;
 }
 
-function buildClientMetrics(attempt) {
+// is_sim_absent used to be the constant false while the same request could send
+// sim_mcc/sim_mnc as 000/000 — which is precisely what a handset reports when
+// there is no SIM in it. One request then said both "a SIM is present" and "no
+// operator", which no real handset ever says. Reading it off the operator the
+// request actually declares keeps the two halves telling one story.
+function buildClientMetrics(attempt, meta) {
+  const mcc = meta && meta.mcc != null ? String(meta.mcc).trim() : '';
+  const simAbsent = !/^\d+$/.test(mcc) || /^0+$/.test(mcc);
   const json = '{"attempts":' + (attempt || 1)
     + ',"app_campaign_download_source":"google-play|unknown"'
-    + ',"is_sim_absent":false}';
+    + ',"is_sim_absent":' + (simAbsent ? 'true' : 'false') + '}';
   return encodeURIComponent(json);
 }
 
@@ -1383,7 +1390,7 @@ function getRequestVerificationCodeParameters(store, method, meta, device, attem
       'prefer_sms_over_flash',      wantsFlash ? 'false' : 'true',
       'simnum',                     '0',
       'airplane_mode_type',         '0',
-      'client_metrics',             buildClientMetrics(attempt),
+      'client_metrics',             buildClientMetrics(attempt, meta),
       'mistyped',                   '7',
       'advertising_id',             store.advertisingId || '',
       'hasinrc',                    '1',
@@ -1641,6 +1648,45 @@ async function sendEncrypted(path, plaintext, store, waVersion) {
   return result;
 }
 
+// ---------- Pacing ----------
+//
+// The funnel events say a person walked through the screens. The timestamps on
+// them said otherwise: session_start, the number lookup and the code request
+// left within a few milliseconds of each other, because nothing in between was
+// waiting on a person. Nobody opens WhatsApp and has a phone number typed,
+// checked and submitted inside one millisecond, and that gap is visible to the
+// server on every event it receives.
+//
+// So the waits a person actually causes are put back. Each one is a range
+// rather than a number — a fixed delay is its own signature — and each is
+// named after the thing being waited for:
+//
+//   enter_number    typing the number in before the lookup fires
+//   confirm_number  the "is this your number?" sheet, and tapping through it
+//   retry_code      after a refused request, before asking again
+//   switch_method   picking a different delivery method out of the list
+//
+// Set WA_REG_PACING=0 to drop all of it, for anything that wants the request
+// shapes without the waiting.
+const PACING_RANGES_MS = {
+  enter_number:   [1800, 5200],
+  confirm_number: [1200, 3600],
+  retry_code:     [2800, 7500],
+  switch_method:  [2000, 5000]
+};
+
+function pacingEnabled() {
+  return process.env.WA_REG_PACING !== '0';
+}
+
+function humanPause(kind) {
+  const range = PACING_RANGES_MS[kind];
+  if (!range || !pacingEnabled()) return Promise.resolve();
+  const ms = range[0] + Math.floor(Math.random() * (range[1] - range[0] + 1));
+  _whaDbg('[DBG] REG pacing ' + kind + ' ' + ms + 'ms');
+  return new Promise((resolve) => setTimeout(resolve, ms));
+}
+
 // ---------- Funnel telemetry ----------
 //
 // The native client reports every screen it moves through — the number entry,
@@ -2042,10 +2088,13 @@ async function checkNumberStatus(phoneNumber) {
 async function assertRegistrationKeys(store, waVersion) {
   // The session-start event the native client fires before it knows the number.
   await sendPrePnFunnelLog(store, waVersion, 'session_start', 'registration_session_start');
+  // The number gets typed between those two events.
+  await humanPause('enter_number');
   await sendFunnelLog(store, waVersion, 'enter_number', 'exist_check', 'exist_attempt');
 
   for (let attempt = 0; attempt < 2; attempt++) {
     try {
+      if (attempt > 0) await humanPause('retry_code');
       logDeviceIdentity('before /exist', store, waVersion);
       const result = await sendRequest('/exist', store, waVersion, false, null);
       // reason === 'incorrect' → keys not found → fresh
@@ -2140,6 +2189,8 @@ async function requestSmsCode(store, method, opts) {
     //   7. Unknown error first time → retry once
     let lastReason = null;
     let attemptNum = 1;
+    // The confirmation sheet the app puts up before it will ask for a code.
+    await humanPause('confirm_number');
     while (true) {
       // Rebuilt per attempt so client_metrics carries the current attempt count.
       const extra = getRequestVerificationCodeParameters(store, m, _regMeta, _device, attemptNum);
@@ -2204,6 +2255,9 @@ async function requestSmsCode(store, method, opts) {
       if (attemptNum > MAX_CODE_REQUEST_ATTEMPTS) {
         throw new Error(`Registration error (${m}): giving up after ${MAX_CODE_REQUEST_ATTEMPTS} attempts — raw: ${JSON.stringify(result)}`);
       }
+
+      // Nobody re-taps the button the instant the error lands.
+      await humanPause('retry_code');
     }
   }
 
@@ -2214,6 +2268,8 @@ async function requestSmsCode(store, method, opts) {
   if (result && result._noRoutes && !autoFallbackDone && method !== 'email') {
     autoFallbackDone = true;
     process.stderr.write(`[REG] ${method} returned no_routes — auto-trying ${fallbackMethod}\n`);
+    // Reading the refusal and picking another method off the list.
+    await humanPause('switch_method');
     result = await _tryMethod(fallbackMethod);
   }
 
@@ -2373,3 +2429,6 @@ module.exports._verify = {
   flashCodeFromCallerId, flashCodeLength, codeForSubmission,
   getRequestVerificationCodeParameters
 };
+
+// Pacing internals, exposed for tests. Not part of the public API.
+module.exports._pacing = { pacingEnabled, humanPause, buildClientMetrics, PACING_RANGES_MS };