@everdeep/pubmed 0.1.0 → 0.1.1

package/README.md

@@ -75,6 +75,27 @@ if (first.nextCursor) {
 
 Cursors are versioned, base64url-encoded, unsigned, implementation-specific continuation state backed by NCBI search history. Treat them as untrusted values: they are validated when consumed but are not encrypted or authenticated, and their decoded shape is not a public API. They contain no client credentials. Cursors are temporary and can produce `CursorExpiredError`; malformed values produce `CursorInvalidError`.
 
+### Count changes during pagination
+
+NCBI can return HTTP 200 with valid IDs but a different total on a later page. By default, the client stops with `PaginationConsistencyError` (`code: "PAGINATION_INCONSISTENT"`, `retryable: false`), not `INVALID_RESPONSE` or `SEARCH_LIMIT`. This is a consistency signal, **not provider unavailability**. Retrying may succeed, but does not establish consistent results; the client does not automatically retry or restart such searches.
+
+If best-effort pagination is acceptable, opt in explicitly at client construction:
+
+```ts
+const client = new PubMedClient({ email, tool, totalDriftPolicy: "warn" });
+const first = await client.search({ query: "cancer", pageSize: 2 });
+if (first.nextCursor) {
+  const second = await client.search({ cursor: first.nextCursor });
+  console.log(second.diagnostics); // safe count-drift metadata, when present
+}
+```
+
+In both modes, `SearchBatch.total` and the pagination bound remain the **initial** total. Growth never extends the original target; shrinkage never silently shortens it. Continuations retain the original history reference and cursor expiry. Short, empty, oversized, duplicate-ID, or count-contradictory pages still fail before record retrieval, even in `"warn"` mode. The 10,000-ID retrieval window and `searchAll.maxResults` remain enforced.
+
+Warn mode returns a `diagnostics` entry on each drifted page and emits a `search-total-drift` event. A consistency error exposes the same metadata through `.diagnostic` and `toJSON()`: `reason`, `originalTotal`, `observedTotal`, `offset`, `requestedIds`, and `returnedIds`. No queries, PMIDs, cursors, or history tokens are included. Record parse `warnings` remain separate.
+
+Neither mode guarantees snapshot enumeration: equal counts do not prove stable membership or ordering, and page-local uniqueness does not detect cross-page duplicates or omissions. Warn mode explicitly accepts that risk; it does not silently deduplicate or claim complete results. Consumers requiring exact enumeration should not opt in.
+
 For progressive consumption, use `searchAll()`. `maxResults` is required so a caller must make the retrieval bound explicit:
 
 ```ts
@@ -173,9 +194,9 @@ A process-shared FIFO limiter stays below NCBI ceilings: approximately 2.8 reque
 
 The client retries network failures, timeouts, HTTP 408, 429, and 5xx responses with exponential full jitter. Other 4xx responses and XML/JSON parse failures are not retried. Every request uses an `application/x-www-form-urlencoded` POST to a fixed NCBI endpoint; parameters and credentials are never placed in the URL. Response bodies are capped while streaming.
 
-There is no default logging. An optional `onEvent` callback receives sanitized events for correlation IDs, cache hits/misses, in-flight coalescing, requests, response byte counts, retries, queue delays, cooldowns, parse warnings, and terminal failures. Each logical transport request gets a generated opaque correlation ID. A `request-coalesced` event links a joining request's ID to the shared operation's ID; IDs are not derived from request content. Request and retry events also carry correlation IDs when they belong to an HTTP operation. Terminal failures expose only a stable error code, not an error message.
+There is no default logging. An optional `onEvent` callback receives sanitized events for correlation IDs, cache hits/misses, in-flight coalescing, requests, response byte counts, retries, queue delays, cooldowns, parse warnings, tolerated search-total drift, and terminal failures. Each logical transport request gets a generated opaque correlation ID. A `request-coalesced` event links a joining request's ID to the shared operation's ID; IDs are not derived from request content. Request and retry events also carry correlation IDs when they belong to an HTTP operation. Terminal failures expose only a stable error code, not an error message.
 
-Callback exceptions are ignored. Events and typed errors never include API keys, email addresses, queries, request bodies, raw responses, or internal cache keys. Event payloads contain only bounded operational metadata such as endpoint, status, attempt, timing, byte count, error code, and opaque correlation IDs.
+Callback exceptions are ignored. Events and typed errors never include API keys, email addresses, queries, request bodies, raw responses, or internal cache keys. Event payloads contain only bounded operational metadata such as endpoint, status, attempt, timing, byte count, error code, search totals/page counts/offsets, and opaque correlation IDs.
 
 ## Errors
 
@@ -186,6 +207,7 @@ All library failures extend `PubMedError` and have stable `code` and `retryable`
 - `TimeoutError` and `NetworkError`
 - `ResponseTooLargeError` and `QueueFullError`
 - `ParseError` and `InvalidResponseError`
+- `PaginationConsistencyError` (count drift; includes safe `.diagnostic` metadata)
 - `CursorExpiredError` and `CursorInvalidError`
 - `AbortedError`
 - `SearchLimitError`

package/dist/index.cjs

@@ -27,6 +27,7 @@ __export(index_exports, {
   InvalidResponseError: () => InvalidResponseError,
   MemoryCache: () => MemoryCache,
   NetworkError: () => NetworkError,
+  PaginationConsistencyError: () => PaginationConsistencyError,
   ParseError: () => ParseError,
   PubMedClient: () => PubMedClient,
   PubMedError: () => PubMedError,
@@ -118,6 +119,24 @@ var InvalidResponseError = class extends PubMedError {
     this.name = "InvalidResponseError";
   }
 };
+var PaginationConsistencyError = class extends PubMedError {
+  diagnostic;
+  constructor(diagnostic) {
+    super("PubMed search total changed during pagination", "PAGINATION_INCONSISTENT");
+    this.name = "PaginationConsistencyError";
+    this.diagnostic = Object.freeze({
+      reason: "total-changed",
+      originalTotal: diagnostic.originalTotal,
+      observedTotal: diagnostic.observedTotal,
+      offset: diagnostic.offset,
+      requestedIds: diagnostic.requestedIds,
+      returnedIds: diagnostic.returnedIds
+    });
+  }
+  toJSON() {
+    return { ...super.toJSON(), diagnostic: this.diagnostic };
+  }
+};
 var CursorExpiredError = class extends PubMedError {
   constructor() {
     super("The PubMed search cursor has expired", "CURSOR_EXPIRED");
@@ -1788,9 +1807,8 @@ function parseSearchResponse(body, cursorContext = false) {
   }
   return { total: count, webEnv, queryKey, ids };
 }
-function validateSearchState(state, requestedIds, expectedTotal) {
-  const expectedIds = Math.min(requestedIds, state.total);
-  if (expectedTotal !== void 0 && state.total !== expectedTotal || state.ids.length !== expectedIds || new Set(state.ids).size !== state.ids.length) {
+function validateSearchState(state, expectedIds, offset = 0) {
+  if (state.total < offset + state.ids.length || state.ids.length !== expectedIds || new Set(state.ids).size !== state.ids.length) {
     throw new InvalidResponseError("PubMed returned an incomplete search ID page");
   }
 }
@@ -1876,10 +1894,15 @@ function withLinks(record, extra) {
 var PubMedClient = class {
   #transport;
   #maxBatchSize;
+  #totalDriftPolicy;
   #onEvent;
   constructor(options) {
     if (typeof options !== "object" || options === null) throw new ValidationError("PubMedClient options are required");
     validateAdapterShapes(options);
+    if (options.totalDriftPolicy !== void 0 && options.totalDriftPolicy !== "error" && options.totalDriftPolicy !== "warn") {
+      throw new ValidationError('totalDriftPolicy must be "error" or "warn"');
+    }
+    this.#totalDriftPolicy = options.totalDriftPolicy ?? "error";
     if (typeof options.email !== "string" || options.email.trim() === "") throw new ValidationError("email is required");
     if (typeof options.tool !== "string" || options.tool.trim() === "") throw new ValidationError("tool is required");
     if (options.apiKey !== void 0 && (typeof options.apiKey !== "string" || options.apiKey.trim() === "")) {
@@ -2040,7 +2063,7 @@ var PubMedClient = class {
         key: "esearch-initial-v1",
         decode: (body) => {
           const parsed = parseSearchResponse(body);
-          validateSearchState(parsed, pageSize);
+          validateSearchState(parsed, Math.min(pageSize, parsed.total));
           return parsed;
         }
       },
@@ -2074,7 +2097,7 @@ var PubMedClient = class {
     const remainingWindow = SEARCH_WINDOW - cursor.offset;
     const requested = maxExpected === void 0 ? cursor.pageSize : Math.min(cursor.pageSize, positiveInteger(maxExpected, "maxResults"));
     const retmax = Math.min(requested, cursor.total - cursor.offset, remainingWindow);
-    const state = await this.#transport.request(
+    const { state, diagnostic } = await this.#transport.request(
       "esearch",
       {
         term: `#${cursor.queryKey}`,
@@ -2086,22 +2109,35 @@ var PubMedClient = class {
         usehistory: "y"
       },
       {
-        key: `esearch-cursor-v1:${cursor.total}`,
+        key: `esearch-cursor-v2:${cursor.total}:${this.#totalDriftPolicy}`,
         decode: (body) => {
           const parsed = parseSearchResponse(body, true);
-          validateSearchState(parsed, retmax, cursor.total);
-          return parsed;
+          validateSearchState(parsed, retmax, cursor.offset);
+          const diagnostic2 = parsed.total === cursor.total ? void 0 : {
+            reason: "total-changed",
+            originalTotal: cursor.total,
+            observedTotal: parsed.total,
+            offset: cursor.offset,
+            requestedIds: retmax,
+            returnedIds: parsed.ids.length
+          };
+          if (diagnostic2 !== void 0 && this.#totalDriftPolicy === "error") {
+            throw new PaginationConsistencyError(diagnostic2);
+          }
+          return { state: parsed, diagnostic: diagnostic2 };
         }
       },
       { cache: false, ...signal === void 0 ? {} : { signal } }
     );
+    if (diagnostic !== void 0) safeEvent(this.#onEvent, { type: "search-total-drift", ...diagnostic });
     const batch = await this.getMany(state.ids, { includeLinkOuts, ...signal === void 0 ? {} : { signal } });
     const offset = cursor.offset + state.ids.length;
     return {
       batch: {
         ...batch,
         total: cursor.total,
-        nextCursor: offset < cursor.total ? encodeCursor({ ...cursor, offset }) : null
+        nextCursor: offset < cursor.total ? encodeCursor({ ...cursor, offset }) : null,
+        ...diagnostic === void 0 ? {} : { diagnostics: [{ ...diagnostic }] }
       },
       expectedPmids: state.ids
     };
@@ -2545,6 +2581,7 @@ function formatCitations(sources, format) {
   InvalidResponseError,
   MemoryCache,
   NetworkError,
+  PaginationConsistencyError,
   ParseError,
   PubMedClient,
   PubMedError,

package/dist/index.d.cts

@@ -1,4 +1,4 @@
-type PubMedErrorCode = "VALIDATION_ERROR" | "HTTP_ERROR" | "RATE_LIMIT_ERROR" | "TIMEOUT_ERROR" | "NETWORK_ERROR" | "RESPONSE_TOO_LARGE" | "QUEUE_FULL" | "PARSE_ERROR" | "INVALID_RESPONSE" | "CURSOR_EXPIRED" | "CURSOR_INVALID" | "ABORTED" | "SEARCH_LIMIT";
+type PubMedErrorCode = "VALIDATION_ERROR" | "HTTP_ERROR" | "RATE_LIMIT_ERROR" | "TIMEOUT_ERROR" | "NETWORK_ERROR" | "RESPONSE_TOO_LARGE" | "QUEUE_FULL" | "PARSE_ERROR" | "INVALID_RESPONSE" | "PAGINATION_INCONSISTENT" | "CURSOR_EXPIRED" | "CURSOR_INVALID" | "ABORTED" | "SEARCH_LIMIT";
 declare class PubMedError extends Error {
     readonly code: PubMedErrorCode;
     readonly retryable: boolean;
@@ -40,6 +40,13 @@ declare class ParseError extends PubMedError {
 declare class InvalidResponseError extends PubMedError {
     constructor(message?: string, options?: ErrorOptions);
 }
+declare class PaginationConsistencyError extends PubMedError {
+    readonly diagnostic: SearchTotalDriftDiagnostic;
+    constructor(diagnostic: SearchTotalDriftDiagnostic);
+    toJSON(): ReturnType<PubMedError["toJSON"]> & Readonly<{
+        diagnostic: SearchTotalDriftDiagnostic;
+    }>;
+}
 declare class CursorExpiredError extends PubMedError {
     constructor();
 }
@@ -281,9 +288,21 @@ interface BatchResult {
     readonly missingPmids: readonly string[];
     readonly warnings: readonly PubMedWarning[];
 }
+/** Safe metadata only; never includes queries, PMIDs, cursors, or history tokens. */
+interface SearchTotalDriftDiagnostic {
+    readonly reason: "total-changed";
+    readonly originalTotal: number;
+    readonly observedTotal: number;
+    readonly offset: number;
+    readonly requestedIds: number;
+    readonly returnedIds: number;
+}
 interface SearchBatch extends BatchResult {
+    /** Initial search total and fixed pagination bound, not a refreshed count. */
     readonly total: number;
     readonly nextCursor: string | null;
+    /** Present when this page tolerated count drift under totalDriftPolicy: "warn". */
+    readonly diagnostics?: readonly SearchTotalDriftDiagnostic[];
 }
 interface RequestOptions {
     readonly includeLinkOuts?: boolean;
@@ -326,6 +345,8 @@ interface RateLimitCoordinator {
 }
 type PubMedEndpoint = "esearch" | "esummary" | "efetch" | "elink";
 type PubMedEvent = Readonly<{
+    type: "search-total-drift";
+} & SearchTotalDriftDiagnostic> | Readonly<{
     type: "correlation-id";
     endpoint: PubMedEndpoint;
     correlationId: string;
@@ -387,6 +408,8 @@ interface PubMedClientOptions {
     readonly maxResponseBytes?: number;
     readonly maxQueuedRequests?: number;
     readonly maxBatchSize?: number;
+    /** Default "error". "warn" explicitly permits best-effort, non-snapshot pagination. */
+    readonly totalDriftPolicy?: "error" | "warn";
 }
 
 declare class PubMedClient {
@@ -429,4 +452,4 @@ interface ParsedPubMedXml {
 /** Parse an entire PubmedArticleSet while retaining each direct child byte-for-byte. */
 declare function parsePubMedXml(xml: string): ParsedPubMedXml;
 
-export { AbortedError, type AbstractSection, type Affiliation, type AuthorIdentifier, type BasePubMedRecord, type BatchResult, type CacheAdapter, type CitationFormat, type CitationSource, type CollectiveAuthor, CursorExpiredError, CursorInvalidError, HttpError, InvalidResponseError, type JournalCitation, type JsonObject, type JsonPrimitive, type JsonValue, type Keyword, MemoryCache, type MemoryCacheOptions, type MeshHeading, NetworkError, ParseError, type ParsedPubMedXml, type PartialDate, type PersonalAuthor, type PubMedArticleRecord, type PubMedAuthor, type PubMedBookRecord, PubMedClient, type PubMedClientOptions, type PubMedEndpoint, PubMedError, type PubMedErrorCode, type PubMedEvent, type PubMedIdentifier, type PubMedLink, type PubMedRecord, type PubMedSummary, type PubMedSummaryAuthor, type PubMedSummaryBook, type PubMedSummaryHistoryEntry, type PubMedSummaryIdentifier, type PubMedSummaryJournal, type PubMedWarning, type PublicationDates, type PublicationHistoryEntry, QueueFullError, type RateLimitBucket, type RateLimitCoordinator, RateLimitError, type RequestOptions, ResponseTooLargeError, type SearchAllOptions, type SearchBatch, type SearchCursorOptions, SearchLimitError, type SearchOptions, type SearchQueryOptions, type SummaryBatchResult, type SummaryRequestOptions, TimeoutError, type UnknownPubMedRecord, ValidationError, formatCitation, formatCitations, parsePubMedXml };
+export { AbortedError, type AbstractSection, type Affiliation, type AuthorIdentifier, type BasePubMedRecord, type BatchResult, type CacheAdapter, type CitationFormat, type CitationSource, type CollectiveAuthor, CursorExpiredError, CursorInvalidError, HttpError, InvalidResponseError, type JournalCitation, type JsonObject, type JsonPrimitive, type JsonValue, type Keyword, MemoryCache, type MemoryCacheOptions, type MeshHeading, NetworkError, PaginationConsistencyError, ParseError, type ParsedPubMedXml, type PartialDate, type PersonalAuthor, type PubMedArticleRecord, type PubMedAuthor, type PubMedBookRecord, PubMedClient, type PubMedClientOptions, type PubMedEndpoint, PubMedError, type PubMedErrorCode, type PubMedEvent, type PubMedIdentifier, type PubMedLink, type PubMedRecord, type PubMedSummary, type PubMedSummaryAuthor, type PubMedSummaryBook, type PubMedSummaryHistoryEntry, type PubMedSummaryIdentifier, type PubMedSummaryJournal, type PubMedWarning, type PublicationDates, type PublicationHistoryEntry, QueueFullError, type RateLimitBucket, type RateLimitCoordinator, RateLimitError, type RequestOptions, ResponseTooLargeError, type SearchAllOptions, type SearchBatch, type SearchCursorOptions, SearchLimitError, type SearchOptions, type SearchQueryOptions, type SearchTotalDriftDiagnostic, type SummaryBatchResult, type SummaryRequestOptions, TimeoutError, type UnknownPubMedRecord, ValidationError, formatCitation, formatCitations, parsePubMedXml };

package/dist/index.d.ts

@@ -1,4 +1,4 @@
-type PubMedErrorCode = "VALIDATION_ERROR" | "HTTP_ERROR" | "RATE_LIMIT_ERROR" | "TIMEOUT_ERROR" | "NETWORK_ERROR" | "RESPONSE_TOO_LARGE" | "QUEUE_FULL" | "PARSE_ERROR" | "INVALID_RESPONSE" | "CURSOR_EXPIRED" | "CURSOR_INVALID" | "ABORTED" | "SEARCH_LIMIT";
+type PubMedErrorCode = "VALIDATION_ERROR" | "HTTP_ERROR" | "RATE_LIMIT_ERROR" | "TIMEOUT_ERROR" | "NETWORK_ERROR" | "RESPONSE_TOO_LARGE" | "QUEUE_FULL" | "PARSE_ERROR" | "INVALID_RESPONSE" | "PAGINATION_INCONSISTENT" | "CURSOR_EXPIRED" | "CURSOR_INVALID" | "ABORTED" | "SEARCH_LIMIT";
 declare class PubMedError extends Error {
     readonly code: PubMedErrorCode;
     readonly retryable: boolean;
@@ -40,6 +40,13 @@ declare class ParseError extends PubMedError {
 declare class InvalidResponseError extends PubMedError {
     constructor(message?: string, options?: ErrorOptions);
 }
+declare class PaginationConsistencyError extends PubMedError {
+    readonly diagnostic: SearchTotalDriftDiagnostic;
+    constructor(diagnostic: SearchTotalDriftDiagnostic);
+    toJSON(): ReturnType<PubMedError["toJSON"]> & Readonly<{
+        diagnostic: SearchTotalDriftDiagnostic;
+    }>;
+}
 declare class CursorExpiredError extends PubMedError {
     constructor();
 }
@@ -281,9 +288,21 @@ interface BatchResult {
     readonly missingPmids: readonly string[];
     readonly warnings: readonly PubMedWarning[];
 }
+/** Safe metadata only; never includes queries, PMIDs, cursors, or history tokens. */
+interface SearchTotalDriftDiagnostic {
+    readonly reason: "total-changed";
+    readonly originalTotal: number;
+    readonly observedTotal: number;
+    readonly offset: number;
+    readonly requestedIds: number;
+    readonly returnedIds: number;
+}
 interface SearchBatch extends BatchResult {
+    /** Initial search total and fixed pagination bound, not a refreshed count. */
     readonly total: number;
     readonly nextCursor: string | null;
+    /** Present when this page tolerated count drift under totalDriftPolicy: "warn". */
+    readonly diagnostics?: readonly SearchTotalDriftDiagnostic[];
 }
 interface RequestOptions {
     readonly includeLinkOuts?: boolean;
@@ -326,6 +345,8 @@ interface RateLimitCoordinator {
 }
 type PubMedEndpoint = "esearch" | "esummary" | "efetch" | "elink";
 type PubMedEvent = Readonly<{
+    type: "search-total-drift";
+} & SearchTotalDriftDiagnostic> | Readonly<{
     type: "correlation-id";
     endpoint: PubMedEndpoint;
     correlationId: string;
@@ -387,6 +408,8 @@ interface PubMedClientOptions {
     readonly maxResponseBytes?: number;
     readonly maxQueuedRequests?: number;
     readonly maxBatchSize?: number;
+    /** Default "error". "warn" explicitly permits best-effort, non-snapshot pagination. */
+    readonly totalDriftPolicy?: "error" | "warn";
 }
 
 declare class PubMedClient {
@@ -429,4 +452,4 @@ interface ParsedPubMedXml {
 /** Parse an entire PubmedArticleSet while retaining each direct child byte-for-byte. */
 declare function parsePubMedXml(xml: string): ParsedPubMedXml;
 
-export { AbortedError, type AbstractSection, type Affiliation, type AuthorIdentifier, type BasePubMedRecord, type BatchResult, type CacheAdapter, type CitationFormat, type CitationSource, type CollectiveAuthor, CursorExpiredError, CursorInvalidError, HttpError, InvalidResponseError, type JournalCitation, type JsonObject, type JsonPrimitive, type JsonValue, type Keyword, MemoryCache, type MemoryCacheOptions, type MeshHeading, NetworkError, ParseError, type ParsedPubMedXml, type PartialDate, type PersonalAuthor, type PubMedArticleRecord, type PubMedAuthor, type PubMedBookRecord, PubMedClient, type PubMedClientOptions, type PubMedEndpoint, PubMedError, type PubMedErrorCode, type PubMedEvent, type PubMedIdentifier, type PubMedLink, type PubMedRecord, type PubMedSummary, type PubMedSummaryAuthor, type PubMedSummaryBook, type PubMedSummaryHistoryEntry, type PubMedSummaryIdentifier, type PubMedSummaryJournal, type PubMedWarning, type PublicationDates, type PublicationHistoryEntry, QueueFullError, type RateLimitBucket, type RateLimitCoordinator, RateLimitError, type RequestOptions, ResponseTooLargeError, type SearchAllOptions, type SearchBatch, type SearchCursorOptions, SearchLimitError, type SearchOptions, type SearchQueryOptions, type SummaryBatchResult, type SummaryRequestOptions, TimeoutError, type UnknownPubMedRecord, ValidationError, formatCitation, formatCitations, parsePubMedXml };
+export { AbortedError, type AbstractSection, type Affiliation, type AuthorIdentifier, type BasePubMedRecord, type BatchResult, type CacheAdapter, type CitationFormat, type CitationSource, type CollectiveAuthor, CursorExpiredError, CursorInvalidError, HttpError, InvalidResponseError, type JournalCitation, type JsonObject, type JsonPrimitive, type JsonValue, type Keyword, MemoryCache, type MemoryCacheOptions, type MeshHeading, NetworkError, PaginationConsistencyError, ParseError, type ParsedPubMedXml, type PartialDate, type PersonalAuthor, type PubMedArticleRecord, type PubMedAuthor, type PubMedBookRecord, PubMedClient, type PubMedClientOptions, type PubMedEndpoint, PubMedError, type PubMedErrorCode, type PubMedEvent, type PubMedIdentifier, type PubMedLink, type PubMedRecord, type PubMedSummary, type PubMedSummaryAuthor, type PubMedSummaryBook, type PubMedSummaryHistoryEntry, type PubMedSummaryIdentifier, type PubMedSummaryJournal, type PubMedWarning, type PublicationDates, type PublicationHistoryEntry, QueueFullError, type RateLimitBucket, type RateLimitCoordinator, RateLimitError, type RequestOptions, ResponseTooLargeError, type SearchAllOptions, type SearchBatch, type SearchCursorOptions, SearchLimitError, type SearchOptions, type SearchQueryOptions, type SearchTotalDriftDiagnostic, type SummaryBatchResult, type SummaryRequestOptions, TimeoutError, type UnknownPubMedRecord, ValidationError, formatCitation, formatCitations, parsePubMedXml };

package/dist/index.js

@@ -74,6 +74,24 @@ var InvalidResponseError = class extends PubMedError {
     this.name = "InvalidResponseError";
   }
 };
+var PaginationConsistencyError = class extends PubMedError {
+  diagnostic;
+  constructor(diagnostic) {
+    super("PubMed search total changed during pagination", "PAGINATION_INCONSISTENT");
+    this.name = "PaginationConsistencyError";
+    this.diagnostic = Object.freeze({
+      reason: "total-changed",
+      originalTotal: diagnostic.originalTotal,
+      observedTotal: diagnostic.observedTotal,
+      offset: diagnostic.offset,
+      requestedIds: diagnostic.requestedIds,
+      returnedIds: diagnostic.returnedIds
+    });
+  }
+  toJSON() {
+    return { ...super.toJSON(), diagnostic: this.diagnostic };
+  }
+};
 var CursorExpiredError = class extends PubMedError {
   constructor() {
     super("The PubMed search cursor has expired", "CURSOR_EXPIRED");
@@ -1744,9 +1762,8 @@ function parseSearchResponse(body, cursorContext = false) {
   }
   return { total: count, webEnv, queryKey, ids };
 }
-function validateSearchState(state, requestedIds, expectedTotal) {
-  const expectedIds = Math.min(requestedIds, state.total);
-  if (expectedTotal !== void 0 && state.total !== expectedTotal || state.ids.length !== expectedIds || new Set(state.ids).size !== state.ids.length) {
+function validateSearchState(state, expectedIds, offset = 0) {
+  if (state.total < offset + state.ids.length || state.ids.length !== expectedIds || new Set(state.ids).size !== state.ids.length) {
     throw new InvalidResponseError("PubMed returned an incomplete search ID page");
   }
 }
@@ -1832,10 +1849,15 @@ function withLinks(record, extra) {
 var PubMedClient = class {
   #transport;
   #maxBatchSize;
+  #totalDriftPolicy;
   #onEvent;
   constructor(options) {
     if (typeof options !== "object" || options === null) throw new ValidationError("PubMedClient options are required");
     validateAdapterShapes(options);
+    if (options.totalDriftPolicy !== void 0 && options.totalDriftPolicy !== "error" && options.totalDriftPolicy !== "warn") {
+      throw new ValidationError('totalDriftPolicy must be "error" or "warn"');
+    }
+    this.#totalDriftPolicy = options.totalDriftPolicy ?? "error";
     if (typeof options.email !== "string" || options.email.trim() === "") throw new ValidationError("email is required");
     if (typeof options.tool !== "string" || options.tool.trim() === "") throw new ValidationError("tool is required");
     if (options.apiKey !== void 0 && (typeof options.apiKey !== "string" || options.apiKey.trim() === "")) {
@@ -1996,7 +2018,7 @@ var PubMedClient = class {
         key: "esearch-initial-v1",
         decode: (body) => {
           const parsed = parseSearchResponse(body);
-          validateSearchState(parsed, pageSize);
+          validateSearchState(parsed, Math.min(pageSize, parsed.total));
           return parsed;
         }
       },
@@ -2030,7 +2052,7 @@ var PubMedClient = class {
     const remainingWindow = SEARCH_WINDOW - cursor.offset;
     const requested = maxExpected === void 0 ? cursor.pageSize : Math.min(cursor.pageSize, positiveInteger(maxExpected, "maxResults"));
     const retmax = Math.min(requested, cursor.total - cursor.offset, remainingWindow);
-    const state = await this.#transport.request(
+    const { state, diagnostic } = await this.#transport.request(
       "esearch",
       {
         term: `#${cursor.queryKey}`,
@@ -2042,22 +2064,35 @@ var PubMedClient = class {
         usehistory: "y"
       },
       {
-        key: `esearch-cursor-v1:${cursor.total}`,
+        key: `esearch-cursor-v2:${cursor.total}:${this.#totalDriftPolicy}`,
         decode: (body) => {
           const parsed = parseSearchResponse(body, true);
-          validateSearchState(parsed, retmax, cursor.total);
-          return parsed;
+          validateSearchState(parsed, retmax, cursor.offset);
+          const diagnostic2 = parsed.total === cursor.total ? void 0 : {
+            reason: "total-changed",
+            originalTotal: cursor.total,
+            observedTotal: parsed.total,
+            offset: cursor.offset,
+            requestedIds: retmax,
+            returnedIds: parsed.ids.length
+          };
+          if (diagnostic2 !== void 0 && this.#totalDriftPolicy === "error") {
+            throw new PaginationConsistencyError(diagnostic2);
+          }
+          return { state: parsed, diagnostic: diagnostic2 };
         }
       },
       { cache: false, ...signal === void 0 ? {} : { signal } }
     );
+    if (diagnostic !== void 0) safeEvent(this.#onEvent, { type: "search-total-drift", ...diagnostic });
     const batch = await this.getMany(state.ids, { includeLinkOuts, ...signal === void 0 ? {} : { signal } });
     const offset = cursor.offset + state.ids.length;
     return {
       batch: {
         ...batch,
         total: cursor.total,
-        nextCursor: offset < cursor.total ? encodeCursor({ ...cursor, offset }) : null
+        nextCursor: offset < cursor.total ? encodeCursor({ ...cursor, offset }) : null,
+        ...diagnostic === void 0 ? {} : { diagnostics: [{ ...diagnostic }] }
       },
       expectedPmids: state.ids
     };
@@ -2500,6 +2535,7 @@ export {
   InvalidResponseError,
   MemoryCache,
   NetworkError,
+  PaginationConsistencyError,
   ParseError,
   PubMedClient,
   PubMedError,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@everdeep/pubmed",
-  "version": "0.1.0",
+  "version": "0.1.1",
   "description": "Safe, typed TypeScript client for the NCBI PubMed E-utilities API",
   "type": "module",
   "main": "./dist/index.cjs",