@wirecat/cli-core 0.549.0 → 0.550.0

package/CHANGELOG.md

@@ -7,6 +7,53 @@ shape of this file.
 
 ## Unreleased
 
+## 0.550.0 — 02.11.2026
+
+### Changed — may break callers
+
+- **cli-core registers `mail.search-refused`.** A CLI that listed it in its own reasons gets "listed twice" from
+  `defineReasons` and must drop its entry; mail does.
+- **`MailStore` has `markPartial(accountId, externalId, mark)`, and `partialEmails` answers `stuck` and leaves out an
+  email marked `partial.stuck`.** A store of its own must add both. `PartialOutcome` (`syncMail`'s `partial`) gains
+  `stuck`, and `partial` is answered whenever any of `completed`, `left` or `stuck` is not 0.
+- **cli-core now needs `@wirecat/cli-base` 0.74.0**, which holds the `--compact` switch's name. A CLI that pins
+  cli-base bumps it to 0.74.0 together with cli-core.
+
+### Fixed
+
+- **A partial email the server no longer gives at its stored place is given up on instead of read whole on every
+  sync.** `completePartial` reads it at its newest stored location under that location's UIDVALIDITY (the new optional
+  `Quick.validity`, which a source checks before reading), counts each failed read in `metadata.partial.tries`, and
+  after 3, or at once when the place holds another message, marks it `partial.stuck`: it stays readable as it is, is
+  counted in `partial.stuck`, and no longer takes one of the 100 places a run reads, so it never holds back newer ones.
+  An email with no stored location is given up on at once.
+- **A server search never stores a partial copy over an email already stored whole.** A sync in another process could
+  save the email between the search's check and its write; the write then dropped every attachment. The store now
+  keeps a live whole email as it is when a copy marked `partial` is saved over it.
+- **`--brief` now cuts a messenger search hit.** It only knew mail's camelCase hits, so `search messages --brief` in
+  tg, max and wa printed every field; it now keeps `ref`, `chat_title`, `timestamp`, `sender_name`, `excerpt` and
+  `score`. The default and `--full` answers are unchanged. Commands that print through `outputFor` now honour
+  `--brief` and `--full` too, and refuse them together as the others do.
+
+### Added
+
+- **`--compact`: one line per item for an agent, from the same answer `--json` prints.** `messages list <chat>
+  --compact` prints `#<id> · 10-01 09:12 · Анна · <text>` under one header line that names the shared ref prefix
+  (`ref: msg:max/500/-1001/<id>`), what every row has in common (the chat's title), the date span and the zone. Rows
+  from several chats keep their whole refs. Text stays on one line with control and direction characters made
+  visible, and is cut at 400 characters with `…(+N)`; `--full` keeps it whole. The paging, `missing`, the next steps
+  and an incomplete answer follow as `# more:`, `# missing:`, `# next:` and `# complete: false` lines on stdout, and
+  each command line in them carries `--compact` again. On 1 000 messages of one chat it is about 48 % of the JSON
+  bytes; the texts alone are 40 %. `--compact` with `--json`, `--json=…`, `--jsonl` or `--brief` fails with
+  `validation_error` (`output.conflict`); it never prompts, and exits with the code JSON would. MCP keeps JSON.
+  - **Every command that answers through `renderAnswer` gets it.** A command draws with its own row
+    (`answerEntity(cmd, name, schema, { row })`), its app's (`rows: { name: shape }` in the program definition,
+    which also draws an app entity listed inside one thing), the shared one in `SHARED_ROWS` (`message`,
+    `message.summary`, `message.hit`, `chat`), or the generic row: key, ref or id · title, name or path · state,
+    status or due date · text. A record or a list is never printed as a column's value. `RowShape` is data, and `compactText(body, shape,
+    options)` is pure, for an app that prints its own answers.
+  - What a command prints without `renderAnswer` stays JSON under `--compact`.
+
 ## 0.549.0 — 02.11.2026
 
 ### Added
@@ -293,22 +340,6 @@ shape of this file.
   **`knowledge.decideSuggestedLink(id, decision)`** an older unconfirmed link, each in one transaction: the link and
   the decision land together or not at all. A decision's `reason` is kept: on a rejected proposal as its result, on a
   fact in `metadata.decision`, on an evidence findings receipt.
-- **`--compact`: one line per item for an agent, from the same answer `--json` prints.** `messages list <chat>
-  --compact` prints `#<id> · 10-01 09:12 · Анна · <text>` under one header line that names the shared ref prefix
-  (`ref: msg:max/500/-1001/<id>`), what every row has in common (the chat's title), the date span and the zone. Rows
-  from several chats keep their whole refs. Text stays on one line with control and direction characters made
-  visible, and is cut at 400 characters with `…(+N)`; `--full` keeps it whole. The paging, `missing`, the next steps
-  and an incomplete answer follow as `# more:`, `# missing:`, `# next:` and `# complete: false` lines on stdout, and
-  each command line in them carries `--compact` again. On 1 000 messages of one chat it is about 48 % of the JSON
-  bytes; the texts alone are 40 %. `--compact` with `--json`, `--json=…`, `--jsonl` or `--brief` fails with
-  `validation_error` (`output.conflict`); it never prompts, and exits with the code JSON would. MCP keeps JSON.
-  - **Every command that answers through `renderAnswer` gets it.** A command draws with its own row
-    (`answerEntity(cmd, name, schema, { row })`), its app's (`rows: { name: shape }` in the program definition,
-    which also draws an app entity listed inside one thing), the shared one in `SHARED_ROWS` (`message`,
-    `message.summary`, `message.hit`, `chat`), or the generic row: key, ref or id · title, name or path · state,
-    status or due date · text. A record or a list is never printed as a column's value. `RowShape` is data, and `compactText(body, shape,
-    options)` is pure, for an app that prints its own answers.
-  - What a command prints without `renderAnswer` stays JSON under `--compact`.
 
 ### Changed — may break callers
 
@@ -351,8 +382,6 @@ shape of this file.
   the agent's bot and reason otherwise.
   An agent's confirmation of a relationship is recorded in the link's provenance, and of a fact or proposal as its
   decider.
-- **cli-core now needs `@wirecat/cli-base` 0.74.0**, which holds the `--compact` switch's name. A CLI that pins
-  cli-base bumps it to 0.74.0 together with cli-core.
 
 ### Fixed
 
@@ -366,10 +395,6 @@ shape of this file.
   `starred:`.
 - **An inclusive start keeps a message sent exactly at it.** `messages list 'date>=2026-10-01'` skipped a message
   sent at the first millisecond of the day, and `date<="…"` one sent at that moment.
-- **`--brief` now cuts a messenger search hit.** It only knew mail's camelCase hits, so `search messages --brief` in
-  tg, max and wa printed every field; it now keeps `ref`, `chat_title`, `timestamp`, `sender_name`, `excerpt` and
-  `score`. The default and `--full` answers are unchanged. Commands that print through `outputFor` now honour
-  `--brief` and `--full` too, and refuse them together as the others do.
 
 ### Security
 

package/dist/errors/reasons.js

@@ -50,6 +50,7 @@ const OTHERS = [
     validation("extraction.text-too-long", "the text is too long for memory extraction"),
     validation("file.locked", "another process holds the file's lock"),
     ["mail.source-failed", "provider_error", "the mail server refused or did not complete a request"],
+    ["mail.search-refused", "provider_error", "the mail server refused to search a folder"],
     ["mirror.partial", "provider_error", "some mirrored files were not synced"],
     ["profile.wrong-account", "authentication_error", "the signed-in source account does not match this profile"],
     validation("push.too-many-files", "the folder holds more files than one push takes"),

package/dist/mail/partial.d.ts

@@ -6,16 +6,22 @@ import type { MailSource } from "./source.js";
 export declare const PARTIAL = "partial";
 /** How many partial emails one sync reads whole at most; the rest wait for the next run. */
 export declare const PARTIAL_PER_RUN = 100;
+/** Failed reads after which a partial email is given up on (`partial.stuck`) and stays as it is. */
+export declare const PARTIAL_TRIES = 3;
 export interface PartialOutcome {
     /** Read whole this run, the mark cleared. */
     completed: number;
-    /** Still partial after this run. */
+    /** Still partial after this run, tried again by the next. */
     left: number;
+    /** Given up on: the server no longer gives this email at its stored place. They stay readable as they are. */
+    stuck: number;
 }
 /**
  * Reads whole the emails a server search stored from their first bytes, so the archive holds their attachments too.
- * Sync never reads stored mail again, so without this step a partial email would stay partial. Gmail's are found by
- * their id; elsewhere at a stored location, trusted only when the bytes read there carry the stored id.
+ * Sync never reads stored mail again, so without this step a partial email would stay partial. A source with `find`
+ * finds them by id; elsewhere they are read at their newest stored location under its UIDVALIDITY, trusted only when
+ * `identify` gives the stored id for the bytes read there. Each failed read is counted, and after `PARTIAL_TRIES`, or
+ * at once when the place holds another message, the email is left partial for good.
  */
 export declare const completePartial: ({ store, source, accountId, address, folders, maxBodyChars, maxAttachmentChars, loadEngine, now, take, signal, }: {
     store: MessageStore;

package/dist/mail/partial.js

@@ -6,15 +6,29 @@ import { parseMail } from "./parse.js";
 export const PARTIAL = "partial";
 /** How many partial emails one sync reads whole at most; the rest wait for the next run. */
 export const PARTIAL_PER_RUN = 100;
+/** Failed reads after which a partial email is given up on (`partial.stuck`) and stays as it is. */
+export const PARTIAL_TRIES = 3;
 /**
  * Reads whole the emails a server search stored from their first bytes, so the archive holds their attachments too.
- * Sync never reads stored mail again, so without this step a partial email would stay partial. Gmail's are found by
- * their id; elsewhere at a stored location, trusted only when the bytes read there carry the stored id.
+ * Sync never reads stored mail again, so without this step a partial email would stay partial. A source with `find`
+ * finds them by id; elsewhere they are read at their newest stored location under its UIDVALIDITY, trusted only when
+ * `identify` gives the stored id for the bytes read there. Each failed read is counted, and after `PARTIAL_TRIES`, or
+ * at once when the place holds another message, the email is left partial for good.
  */
 export const completePartial = async ({ store, source, accountId, address, folders, maxBodyChars, maxAttachmentChars, loadEngine, now, take, signal, }) => {
-    const { emails, total } = await store.mail.partialEmails(accountId, PARTIAL_PER_RUN);
+    const { emails, total, stuck } = await store.mail.partialEmails(accountId, PARTIAL_PER_RUN);
     if (emails.length === 0)
-        return { completed: 0, left: 0 };
+        return { completed: 0, left: 0, stuck };
+    let given = 0;
+    // A failed read is counted, so an email the server no longer gives is not read whole on every sync forever.
+    const failed = async (email, forGood = false) => {
+        const mark = (email.metadata?.[PARTIAL] ?? {});
+        const tries = Number(mark.tries ?? 0) + 1;
+        const gaveUp = forGood || tries >= PARTIAL_TRIES;
+        await store.mail.markPartial(accountId, email.externalId, { ...mark, tries, ...(gaveUp ? { stuck: true } : {}) });
+        if (gaveUp)
+            given++;
+    };
     const byId = new Map(emails.map((email) => [email.externalId, email]));
     const items = new Map();
     if (source.find) {
@@ -23,21 +37,27 @@ export const completePartial = async ({ store, source, accountId, address, folde
             if (left.length === 0)
                 break;
             if (take?.())
-                return { completed: 0, left: total };
+                return { completed: 0, left: total, stuck };
             for (const found of await source.find(left, folder)) {
                 const email = byId.get(found.messageId);
                 if (email && !items.has(email))
                     items.set(email, found);
             }
         }
+        for (const email of emails)
+            if (!items.has(email))
+                await failed(email);
     }
     else if (source.identify)
         for (const email of emails) {
             const place = email.locations[0];
-            if (place)
+            if (!place)
+                await failed(email, true);
+            else
                 items.set(email, {
                     uid: String(place.uid),
                     folder: place.folder,
+                    validity: place.uidvalidity,
                     receivedAt: new Date(email.receivedAt ?? 0).toISOString(),
                 });
         }
@@ -58,11 +78,15 @@ export const completePartial = async ({ store, source, accountId, address, folde
         const attachments = [];
         for (const [index, [email, item]] of chunk.entries()) {
             const raw = raws[index];
-            if (!(raw instanceof Uint8Array))
+            if (!(raw instanceof Uint8Array)) {
+                await failed(email);
                 continue;
-            // A UID means another message after a UIDVALIDITY change; only the stored id proves it is this one.
-            if (!source.find && source.identify?.(raw, item).messageId !== email.externalId)
+            }
+            // Only the stored id proves the bytes at a stored UID are this email; another one there will not change back.
+            if (!source.find && source.identify?.(raw, item).messageId !== email.externalId) {
+                await failed(email, true);
                 continue;
+            }
             try {
                 const mail = await parseMail(raw, maxBodyChars);
                 const files = await readAttachments(mail, email.externalId, { loadEngine, maxAttachmentChars, at: now, ...(signal ? { signal } : {}) }, attachments);
@@ -87,8 +111,9 @@ export const completePartial = async ({ store, source, accountId, address, folde
             catch {
                 if (signal?.aborted)
                     break;
+                await failed(email);
             }
         }
     }
-    return { completed, left: total - completed };
+    return { completed, left: total - completed - given, stuck: stuck + given };
 };

package/dist/mail/source.d.ts

@@ -24,7 +24,13 @@ export interface Listed {
     headers?: MailHeaders;
 }
 /** A message as `quick` lists it: everything but the ids, which a source without a cheap one reads from the raw bytes. */
-export type Quick = Omit<Listed, "messageId" | "threadId">;
+export type Quick = Omit<Listed, "messageId" | "threadId"> & {
+    /**
+     * The folder's UIDVALIDITY the UID was stored under. `fetchRaw` reads nothing for it when the folder's differs, and
+     * `identify` works the ids out under it.
+     */
+    validity?: string;
+};
 export interface Searched {
     /** Absent when the server does not say; then no UID is trusted across runs. */
     validity?: string;

package/dist/mail/sync.js

@@ -54,7 +54,7 @@ export const syncMail = async (source, store, options) => {
         ...(options.budget ? { take: () => options.budget?.take() ?? null } : {}),
         ...(options.signal ? { signal: options.signal } : {}),
     });
-    return partial.completed || partial.left ? { ...result, partial } : result;
+    return partial.completed || partial.left || partial.stuck ? { ...result, partial } : result;
 };
 const syncRounds = async (source, store, options) => {
     const { budget, index } = options;

package/dist/store/sqlite/email-locations.d.ts

@@ -23,7 +23,8 @@ export declare const locate: (database: CacheDatabase, accountId: number, emailI
 export declare const folderEmails: (database: CacheDatabase, accountId: number, folder: string, uidvalidity: string) => FolderEmail[];
 /**
  * An email stored from its first bytes only (`metadata.partial`): a sync reads it whole and clears the mark.
- * `partialEmails` lists an account's live ones newest first, with how many there are in all.
+ * `partialEmails` lists an account's live ones newest first, with how many there are in all; one given up on
+ * (`partial.stuck`) is left out and only counted, so it never holds back newer ones. Its newest location comes first.
  */
 export interface PartialEmail {
     externalId: string;
@@ -35,6 +36,26 @@ export interface PartialEmail {
 export declare const partialEmails: (database: CacheDatabase, accountId: number, limit: number) => {
     emails: PartialEmail[];
     total: number;
+    stuck: number;
 };
+/** Replaces a partial email's `metadata.partial`, as a failed read records itself; a whole email is left alone. */
+export declare const markPartial: (database: CacheDatabase, accountId: number, externalId: string, mark: Metadata) => {
+    changes: number;
+};
+export interface PartialMailStore {
+    partialEmails(accountId: number, limit: number): Promise<{
+        emails: PartialEmail[];
+        total: number;
+        stuck: number;
+    }>;
+    /** Records a failed read of a partial email in its `metadata.partial`; a whole email is left alone. */
+    markPartial(accountId: number, externalId: string, mark: Metadata): Promise<void>;
+}
+export declare const partialMail: (database: CacheDatabase) => PartialMailStore;
+/**
+ * A live email stored whole, which a partial copy must not replace: a server search checks what is held before its
+ * slow read, and a sync in another process may store the email whole in between.
+ */
+export declare const storedWhole: (database: CacheDatabase, emailId: number) => boolean;
 export declare const setLocations: (database: CacheDatabase, accountId: number, folder: string, uidvalidity: string, found: ReadonlyMap<string, number[]>) => number;
 export declare const forgetLocations: (database: CacheDatabase, accountId: number, folder: string, uidvalidity: string, uids: number[]) => void;

package/dist/store/sqlite/email-locations.js

@@ -1,4 +1,5 @@
 import { fromJson } from "./events.js";
+import { write } from "./transaction.js";
 export const locate = (database, accountId, emailId, where) => database
     .prepare(`INSERT OR REPLACE INTO email_locations (mailbox_id, uidvalidity, uid, email_id)
          SELECT id, ?, ?, ? FROM mailboxes WHERE account_id = ? AND external_id = ?`)
@@ -26,14 +27,22 @@ export const partialEmails = (database, accountId, limit) => {
     const rows = database
         .prepare(`SELECT e.external_id, t.external_id AS thread, coalesce(e.received_at, e.sent_at) AS at, e.metadata,
          count(*) OVER () AS total,
-         (SELECT json_group_array(json_object('folder', b.external_id, 'uidvalidity', l.uidvalidity, 'uid', l.uid))
-            FROM email_locations l JOIN mailboxes b ON b.id = l.mailbox_id WHERE l.email_id = e.id) AS places
+         (SELECT json_group_array(json_object('folder', folder, 'uidvalidity', uidvalidity, 'uid', uid))
+            FROM (SELECT b.external_id AS folder, l.uidvalidity, l.uid
+                    FROM email_locations l JOIN mailboxes b ON b.id = l.mailbox_id
+                    WHERE l.email_id = e.id ORDER BY l.rowid DESC)) AS places
        FROM emails e JOIN email_threads t ON t.id = e.email_thread_id
        WHERE e.account_id = ? AND e.deleted_at IS NULL AND json_extract(e.metadata, '$.partial') IS NOT NULL
+         AND json_extract(e.metadata, '$.partial.stuck') IS NULL
        ORDER BY at DESC, e.id DESC LIMIT ?`)
         .all(accountId, limit);
+    const stuck = database
+        .prepare(`SELECT count(*) AS n FROM emails WHERE account_id = ? AND deleted_at IS NULL
+         AND json_extract(metadata, '$.partial.stuck') IS NOT NULL`)
+        .get(accountId);
     return {
         total: rows.length ? Number(rows[0]?.total) : 0,
+        stuck: Number(stuck?.n ?? 0),
         emails: rows.map((row) => ({
             externalId: String(row.external_id),
             threadExternalId: String(row.thread),
@@ -47,6 +56,24 @@ export const partialEmails = (database, accountId, limit) => {
         })),
     };
 };
+/** Replaces a partial email's `metadata.partial`, as a failed read records itself; a whole email is left alone. */
+export const markPartial = (database, accountId, externalId, mark) => database
+    .prepare(`UPDATE emails SET metadata = json_set(metadata, '$.partial', json(?)) WHERE account_id = ? AND external_id = ?
+         AND json_extract(metadata, '$.partial') IS NOT NULL`)
+    .run(JSON.stringify(mark), accountId, externalId);
+export const partialMail = (database) => ({
+    partialEmails: async (accountId, limit) => partialEmails(database, accountId, limit),
+    markPartial: async (accountId, externalId, mark) => {
+        write(database, () => markPartial(database, accountId, externalId, mark));
+    },
+});
+/**
+ * A live email stored whole, which a partial copy must not replace: a server search checks what is held before its
+ * slow read, and a sync in another process may store the email whole in between.
+ */
+export const storedWhole = (database, emailId) => database
+    .prepare("SELECT 1 FROM emails WHERE id = ? AND deleted_at IS NULL AND json_extract(metadata, '$.partial') IS NULL")
+    .get(emailId) !== undefined;
 export const setLocations = (database, accountId, folder, uidvalidity, found) => {
     database
         .prepare(`DELETE FROM email_locations WHERE uidvalidity <> ?

package/dist/store/sqlite/emails.d.ts

@@ -3,13 +3,13 @@ import type { Attachment, Metadata } from "../../meetings/index.js";
 import type { Stemmer, Stemmers } from "../../search/stem.js";
 import type { CacheDatabase } from "../driver.js";
 import type { ConversationSummary, StoredHit } from "../types.js";
-import { type EmailLocationInput, type FolderEmail, type PartialEmail } from "./email-locations.js";
+import { type EmailLocationInput, type FolderEmail, type PartialMailStore } from "./email-locations.js";
 import { type MailRulesStore } from "./mail-rules.js";
 import { type AttachmentInput } from "./meetings.js";
 import type { StoreContext } from "./open.js";
 import { type SyncStateSave } from "./sync.js";
 import { type ThreadFilter } from "./thread-columns.js";
-export type { EmailLocationInput, FolderEmail, PartialEmail };
+export type { EmailLocationInput, FolderEmail, PartialEmail } from "./email-locations.js";
 export interface EmailAddress {
     address: string;
     name: string | null;
@@ -205,7 +205,7 @@ export interface SenderScore {
     demoted?: boolean;
 }
 /** Mail in tables of its own: threads, emails, recipients and mailboxes, keyed by account and Message-ID. */
-export interface MailStore {
+export interface MailStore extends PartialMailStore {
     saveThread(input: ThreadSave): Promise<ThreadDetails>;
     /** Several threads and sync state in one transaction: a slice of mail commits with its resume point, or not at all. */
     saveThreads(inputs: ThreadSave[], states?: SyncStateSave[]): Promise<ThreadDetails[]>;
@@ -217,10 +217,6 @@ export interface MailStore {
     emails(filter?: EmailFilter): Promise<Email[]>;
     /** Each matching email's Message-ID and mailboxes, without text or people: cheap over a whole mailbox. */
     emailRefs(filter?: Omit<EmailFilter, "limit" | "offset">): Promise<EmailRef[]>;
-    partialEmails(accountId: number, limit: number): Promise<{
-        emails: PartialEmail[];
-        total: number;
-    }>;
     /** Each matching email's sender, recipients, list, size, flag and mailboxes, oldest first; no text. */
     emailFacts(filter?: Omit<EmailFilter, "limit" | "offset">): Promise<EmailFacts[]>;
     /**

package/dist/store/sqlite/emails.js

@@ -2,7 +2,7 @@ import { formatLocator } from "../../domain/locator.js";
 import { formatReference } from "../../domain/references.js";
 import { fold, normalize } from "../normalize.js";
 import { chunkAttachments, drainEmailIndex, dropAttachmentChunks, MAIL_PIECES, queueThreads, threadPieces, titleOf, } from "./email-index.js";
-import { folderEmails, forgetLocations, locate, partialEmails, setLocations, } from "./email-locations.js";
+import { folderEmails, forgetLocations, locate, partialMail, setLocations, storedWhole, } from "./email-locations.js";
 import { markEmailsDeleted, markEmailsRestored, recountEmailThread as recount, setEmailFlags } from "./email-state.js";
 import { bool, flag, fromJson, int, page, str, toJson } from "./events.js";
 import { folded } from "./folded.js";
@@ -204,6 +204,8 @@ const saveEmail = (database, orm, accountId, threadId, mail, now) => {
     const found = database
         .prepare("SELECT id, email_thread_id FROM emails WHERE account_id = ? AND external_id = ?")
         .get(accountId, mail.externalId);
+    if (found && mail.metadata?.partial !== undefined && storedWhole(database, Number(found.id)))
+        return null;
     let emailId;
     if (found) {
         emailId = Number(found.id);
@@ -469,7 +471,7 @@ export const mailStoreOver = ({ database, orm, now }, stemmerFor = stemmerCache(
             mailboxes: JSON.parse(String(row.boxes)),
         }));
     },
-    partialEmails: async (accountId, limit) => partialEmails(database, accountId, limit),
+    ...partialMail(database),
     async emailFacts(filter = {}) {
         const [where, params] = emailWhere(filter);
         return database

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@wirecat/cli-core",
-  "version": "0.549.0",
+  "version": "0.550.0",
   "description": "The messenger-neutral half of a messaging CLI: domain model, message locators, rendering, name resolution, the SQLite seam, the command skeleton and shared read commands, the send guard, run records and the message store",
   "license": "Apache-2.0",
   "author": "Viacheslav Ptsarev",