@devrev/airsync-sdk 2.0.0-beta.1 → 2.0.0-beta.11

package/docs/MIGRATION.md

@@ -0,0 +1,751 @@
+# Migrating connectors from v1 (`@devrev/ts-adaas`) to v2 (`@devrev/airsync-sdk`)
+
+This is the complete list of connector-facing changes from `@devrev/ts-adaas`
+1.x to `@devrev/airsync-sdk` 2.x, written so an AI agent can apply it on its
+own:
+
+> Read `MIGRATION.md` from `@devrev/airsync-sdk` and migrate this connector.
+
+It ships in the package at `node_modules/@devrev/airsync-sdk/docs/MIGRATION.md`
+(read it from disk; the `exports` map only blocks `import`/`require`). Before
+v2 is installed, use
+`https://raw.githubusercontent.com/devrev/adaas-sdk/v2/docs/MIGRATION.md`.
+
+**The wire protocol is unchanged.** Event payloads, routes, headers, the
+artifact flow and every event-type and `ERROR_CODE=…` string are identical to
+v1. Only the TypeScript API changed, all at once, with no compatibility shims.
+
+---
+
+## Runbook
+
+**Rules.** Work autonomously; when a rewrite is ambiguous, pick the option
+that preserves runtime behavior and note it in `MIGRATION_REVIEW.md`. Migrate
+source **and** tests. Never change a runtime string value. Don't add comments
+about the migration; the `// v1` / `// v2` labels here are reading aids. Leave
+no compile errors or orphaned imports. A section with no hits is a no-op.
+Existing comments and commented-out code that describe v1 behavior (`emit`,
+`{ id }` / `{ error }` returns, `.data.sync_mapper_record`) are rewritten to
+v2; templates are copied into new connectors, so stale comments spread.
+
+1. **Locate the package.** Find the `package.json` that depends on
+   `@devrev/ts-adaas` (generated connectors keep it in `code/`). Run everything
+   from there. If it already depends on `@devrev/airsync-sdk` only, report
+   "already on v2" and stop.
+2. **Start from green latest v1.** If behind the latest stable 1.x, upgrade and
+   apply the release notes of every release in between that mentions a
+   migration (the last ones were 1.16.0–1.18.0), then get tsc/lint/build/test
+   green on v1. If the upgrade alone breaks tsc with axios type mismatches,
+   bump the connector's `axios` to the version that SDK release depends on.
+   Record failures that remain as **Pre-existing**.
+3. **Install v2.**
+   ```bash
+   npm uninstall @devrev/ts-adaas
+   npm install @devrev/airsync-sdk@beta   # @latest after GA
+   ```
+   Rename or remove `overrides` / `resolutions` entries keyed on
+   `@devrev/ts-adaas`; `npm uninstall` leaves them behind. If the installed
+   `docs/MIGRATION.md` differs from this copy, switch to it.
+4. **Apply §1–§14 in order** across source and tests. After §1, search by
+   symbol (`\.emit\(`, `WorkerAdapter`, `sync_mapper_record`, …), not by the old
+   package name; test files that import nothing from the SDK still need
+   changes.
+5. **Verify**: `npm install && npx tsc --noEmit && npm run lint && npm run build && npm test`.
+   If tsconfig excludes folders the connector still runs (`test/`, a local
+   runner), type-check them too: use the repo's `tsconfig.test.json` if it has
+   one, otherwise a temporary tsconfig that extends `tsconfig.json` and
+   overrides `"include": ["src", "test"]`, `"exclude": ["node_modules", "dist"]`,
+   `"rootDir": "."` and `"noEmit": true` (the base `exclude` and `rootDir` still
+   apply otherwise), then `npx tsc -p` it. Fix and rerun,
+   at most 4 attempts; the migration must add no new failures compared to
+   step 2.
+6. **Sweep.** This must return nothing except legitimate hits (an
+   `EventEmitter.emit`, the connector's own `axiosClient` from §9,
+   `@devrev/typescript-sdk` in `package.json` from §1, `git config --local`,
+   legacy event names in fields the SDK doesn't read, such as
+   `event_type_adaas`):
+   ```bash
+   grep -rnE "@devrev/ts-adaas|airsync-sdk/dist|\bprocessTask\b|\bWorkerAdapter\b|\.emit\(|hasWorkerEmitted|\bAirdrop(Event|Message)\b|axiosClient|(format|serialize)AxiosError|workerPath:|overridenOptions|skipConfirmation|isLocalDevelopment|--local|-- local\b|@devrev/typescript-sdk|(Extractor|Loader)EventType|ExtractionCommonError|ERROR_CODE=|sdkState|onUpload|uploadedArtifacts|currentEventDataLength|EXTRACTION_(DATA|METADATA|ATTACHMENTS|EXTERNAL_SYNC_UNITS)_" \
+     --include='*.ts' --include='*.js' --include='*.json' --include='*.yml' --include='*.yaml' --include='*.md' --include='*.mdc' --include='*.sh' . \
+     | grep -vE 'node_modules|/dist/|/coverage/|package-lock.json|MIGRATION_REVIEW.md'
+   ```
+   The sweep covers `test/` and other folders that tsc, lint and jest often
+   exclude. A hit there is real: `@devrev/typescript-sdk` must be in
+   `package.json` (§1), and a `local` argv must become the env var (§3).
+   Also check `package.json` scripts and shell scripts for a bare `local`
+   argument (`npm run test:server -- local`, `main.ts local`).
+   Also grep `getByExternalId|getByTargetId` and check each call against §10:
+   existence checks there compile but silently invert.
+   Then rewrite any connector docs (`README`, `AGENTS.md`, `CLAUDE.md`, `docs/`)
+   that describe v1 behavior, including text that paraphrases DevRev docs with
+   legacy event names.
+7. **Write `MIGRATION_REVIEW.md`** at the repo root: one line per section that
+   changed code with `high` / `medium` / `low confidence`, a sentence on what
+   to verify for medium/low, and the final check results with an
+   **Unresolved** list. Use medium/low for anything judged: emits bubbled out of
+   helpers (§4), paths that emitted nothing (§4), `onTimeout` bodies kept (§4),
+   delay values that look like milliseconds or headers (§4), the cursor decision
+   (§8), ESU repo rewrites (§7), mapper error handling (§10), and a v1 baseline
+   upgrade. Record the exact installed versions.
+   ```markdown
+   # v2 Migration Review
+   Migrated `@devrev/ts-adaas` 1.20.2 → `@devrev/airsync-sdk` 2.0.0.
+   - §4 emit→return — high confidence
+   - §8 state / cursor — low confidence: repointed `lastSuccessfulSyncStarted`
+     to `extract_from`; confirm the connector didn't own this cursor.
+   ## Verification
+   tsc ✓ · lint ✓ · build ✓ · test ✓ (48 passed)
+   ```
+8. **Report** the start and end versions, which sections fired, the check
+   results and how many review entries need a second look.
+
+---
+
+## 1. Package, imports and runtime
+
+- Replace `@devrev/ts-adaas` with `@devrev/airsync-sdk` **everywhere the
+  string appears**: imports, `jest.mock` / `jest.requireActual` arguments and
+  `moduleNameMapper`.
+- **Import only from the root.** `exports` exposes nothing else, so every
+  `@devrev/airsync-sdk/dist/**` import fails with
+  `ERR_PACKAGE_PATH_NOT_EXPORTED`. Everything public is on the root, including
+  `Mappers`, `Item`, `ItemTypeToLoad`, `InitialSyncScope`, the mapper
+  `*Params` / `*Response` types and `SyncMapperRecord*`. Deep imports with no
+  root equivalent are listed in §11. Never import the SDK's internal HTTP client
+  (`dist/http/axios-client-internal`): use the connector's own client (§9).
+- **Node.js ≥ 20.** Bump the connector's `engines`, CI image and `.nvmrc` if
+  they pin older. `@types/node` can stay as it is. Dependabot / CI configs that name `@devrev/ts-adaas` switch to
+  `@devrev/airsync-sdk`.
+- `ts-node` and `@devrev/typescript-sdk` are no longer SDK dependencies. A
+  connector that runs `.ts` workers directly needs `ts-node` in its own
+  `devDependencies`. A connector that uses `@devrev/typescript-sdk` must list
+  it. The template's local test server (`code/test/runner.ts`, and the copy
+  `airsync start-worker` writes to `code/.snapin-test/`) imports
+  `@devrev/typescript-sdk/dist/snap-ins`, so template connectors add it to
+  `devDependencies` (`1.1.78`, the version the template pins). Most connectors exclude `test/` from tsc, lint and jest,
+  so the gates pass and the local worker then fails with
+  `Cannot find module '@devrev/typescript-sdk/dist/snap-ins'`.
+  `InputData` (the type of `event.input_data`) is now exported from the
+  root.
+
+## 2. Event types
+
+| v1 | v2 |
+|----|----|
+| `AirdropEvent` | `AirSyncEvent` |
+| `AirdropMessage` | `AirSyncMessage` |
+
+The shapes are unchanged. `AirSyncEvent.context` now also declares `user_id`,
+`dev_oid`, `source_id` and `service_account_id` (on `event.context`, not on
+`payload.event_context`). Delete a hand-rolled context type that added these
+(e.g. `interface CustomAirdropEvent extends AirdropEvent { context: CustomContext }`):
+a narrower `context` no longer satisfies `AirSyncEvent`. Keep only the fields
+the SDK type lacks. These fields are required, so typed mock events must set
+them. `execution_metadata` is typed `{ devrev_endpoint }`; a custom type that
+widened it with `@devrev/typescript-sdk` fields goes too, along with mock values
+for fields nothing reads.
+
+Deprecated `EventContext` aliases are removed. Use the current names:
+
+| Removed | Use |
+|---------|-----|
+| `dev_org` | `dev_oid` |
+| `dev_user`, `dev_user_id` | `dev_uid` |
+| `external_sync_unit` | `external_sync_unit_id` |
+| `external_system` | `external_system_id` |
+| `sync_run`, `sync_run_id` | `run_id` |
+| `uuid` | `request_id_adaas` |
+
+Rename these keys in fixture and test `event_context` objects too. When an
+object already has the new key (or has two old keys for one new one, like
+`dev_user` and `dev_user_id`), keep the existing new-name value, otherwise the
+first old one, and drop the rest. Untyped
+JSON still compiles, but the SDK reads only the current names (e.g. state
+requests carry `request_id_adaas`).
+
+`reset_extraction` and `reset_extract_from` remain (the SDK never read them);
+leave connector logic that uses them alone.
+
+## 3. `spawn`
+
+| v1 | v2 |
+|----|----|
+| `workerPath` | removed; use `baseWorkerPath` |
+| `baseWorkerPath?` | **required**, the directory containing `workers/` (usually `__dirname`) |
+| `options.workerPathOverrides` | top-level `workerPathOverrides` |
+| `options: WorkerAdapterOptions` | `options?: SpawnOptions` (`{ timeout?, batchSize? }`) |
+| `options.isLocalDevelopment`, `local` / `--local` argv | removed; set `AIRSYNC_LOCAL_DEVELOPMENT=1` |
+
+```ts
+// v1
+spawn({
+  event,
+  initialState,
+  workerPath: __dirname + '/workers/data-extraction',
+  options: { isLocalDevelopment: true, workerPathOverrides: { [EventType.StartExtractingData]: '/workers/custom' } },
+});
+// v2
+spawn({
+  event,
+  initialState,
+  baseWorkerPath: __dirname,
+  workerPathOverrides: { [EventType.StartExtractingData]: '/workers/custom' },
+});
+```
+
+- If `workerPath` came from a dispatcher helper (e.g.
+  `getWorkerPerLoadingPhase(event)`), delete the helper and its now-unused
+  imports.
+- **Fixture and test events** must use the modern event-type strings; v1
+  translated legacy ones and v2 doesn't. Replace `EXTRACTION_EXTERNAL_SYNC_UNITS_START`,
+  `EXTRACTION_METADATA_START`, `EXTRACTION_DATA_START` / `_CONTINUE` / `_DELETE`
+  and `EXTRACTION_ATTACHMENTS_START` / `_CONTINUE` / `_DELETE` with
+  `START_EXTRACTING_EXTERNAL_SYNC_UNITS`, `START_EXTRACTING_METADATA`,
+  `START_EXTRACTING_DATA` / `CONTINUE_EXTRACTING_DATA` /
+  `START_DELETING_EXTRACTOR_STATE` and `START_EXTRACTING_ATTACHMENTS` /
+  `CONTINUE_EXTRACTING_ATTACHMENTS` / `START_DELETING_EXTRACTOR_ATTACHMENTS_STATE`.
+- **Local mode** comes only from the `AIRSYNC_LOCAL_DEVELOPMENT=1` environment
+  variable, and is ignored on Lambda. v1 also enabled it from a bare `local`
+  argument, which the template test server receives (`main.ts local --port …`,
+  also used by `airsync start-worker`). v2 ignores both `local` and `--local`
+  without an error: logs come out as production JSON and no copies land in
+  `extracted_files/`. A local runner that did `process.argv.push('--local')`
+  or relied on the `local` argument sets
+  `process.env.AIRSYNC_LOCAL_DEVELOPMENT = '1'` before calling `spawn` instead,
+  or documents running the worker with `AIRSYNC_LOCAL_DEVELOPMENT=1`.
+
+## 4. Workers return a `TaskResult` instead of emitting
+
+`adapter.emit` is gone (a compile error). A task reports how the phase ended by
+returning a result; the SDK picks the platform event and emits it exactly once.
+
+```ts
+export type TaskResult =
+  | { status: 'success' }
+  | { status: 'progress' }
+  | { status: 'delay'; delaySeconds: number }
+  | { status: 'error'; error: ErrorRecord }; // { message: string; code?: AirSyncErrorCode }
+```
+
+Replace `processTask` with `processExtractionTask` or `processLoadingTask`,
+depending on the worker (extraction workers use repos or `streamAttachments`;
+loading workers use `loadItemTypes` or `loadAttachments`). Their
+generic stays the connector state: `processTask<State>` →
+`processExtractionTask<State>`.
+
+```ts
+// v1
+processTask({
+  task: async ({ adapter }) => {
+    // ...
+    await adapter.emit(ExtractorEventType.DataExtractionDone);
+  },
+  onTimeout: async ({ adapter }) => {
+    await adapter.postState();
+    await adapter.emit(ExtractorEventType.DataExtractionProgress);
+  },
+});
+// v2
+processExtractionTask({
+  task: async ({ adapter }) => {
+    // ...
+    return { status: 'success' };
+  },
+});
+```
+
+| v1 | v2 |
+|----|----|
+| `emit(XxxDone)` | `return { status: 'success' }` |
+| `emit(XxxProgress, { progress })` | `return { status: 'progress' }` |
+| `emit(XxxDelayed, { delay })` | `return { status: 'delay', delaySeconds: delay }` — keep the same number; the platform always read it as seconds. A raw `Retry-After` header stays `Number(header)`. If it is clearly milliseconds (a `*_MS` constant, a value in the thousands, "ms" in the message), keep the number, flag it and list it under **Unresolved**: v1 waited that many seconds too, and fixing it changes behavior |
+| `emit(XxxDelayed)` with no `delay` | `delaySeconds` is required; use the connector's existing default retry constant, else `0` (what v1 effectively sent), and flag it |
+| `emit(XxxError, { error })` | `return { status: 'error', error }` — `error` is required; fall back to `{ message: '<phase> failed' }` |
+| `emit(XxxDone, { error })` (finished with warnings) | `return { status: 'success' }` and log the error. Returning `error` would fail the phase |
+| `emit(ExternalSyncUnitExtractionDone, { external_sync_units })` | push to a repo, then `success` (§7) |
+| `emit(MetadataExtractionDone, { pre_extraction_item_counts })` | `adapter.preExtractionItemCounts = counts`, then `success` |
+| a path that **emitted nothing** (a `catch` that only logs, a branch that falls through) | `{ status: 'error', error: { message } }`, or rethrow. In v1 the SDK reported these as a phase error ("Worker exited without emitting"); never turn them into `success` or `progress` |
+| any `emit(XxxDone)` after `loadItemTypes` / `loadAttachments` / `streamAttachments`, guarded (`!adapter.hasWorkerEmitted && !adapter.isTimeout`) or not | `return adapter.loadItemTypes(...)` (or the other call). Never discard their result and return `success`: v1 emitted their `delay` / `error` itself. Drop the guard and any module-level "already emitted" flags. If v1 did more work after the call and then emitted `Done` with an `error`, return the call's result and log that error |
+
+In non-resumable phases (external sync units, state deletion), `progress` and
+`delay` become `*Error`.
+
+**Emits inside helpers.** Only the task's return value reaches the SDK, so a
+helper that emitted must hand its result back up. Either return
+`TaskResult | null` up the call chain (`null` = keep going), or record it and
+return it once the loop ends:
+
+```ts
+private result: TaskResult | undefined;
+private onPermissionDenied() {
+  this.result = { status: 'error', error: { message: '...' } };
+}
+// at the end of the task: a recorded result, else continue only on timeout
+return this.result ?? (adapter.isTimeout
+  ? { status: 'progress' }
+  : { status: 'error', error: { message: 'Extraction ended without a result' } });
+```
+
+If a shared `handleError` emitted from many `catch` blocks that rethrow, v1
+kept only the first emit. Record the first result (`firstError ??= …`) and
+return it from the task's outermost `catch`.
+
+If a helper **emitted and then kept going** (no throw), the caller often
+emitted again later, e.g. `DataExtractionDone` after the loop, and v1 dropped
+every emit after the first. Record the first result the same way, stop the
+work that followed it (return out of the loop or extractor), and return it
+from the task. Unawaited emits inside loops count too. When the emitting helper
+is deep in the call stack, throw an error class that carries the `TaskResult`
+instead (`class TaskResultError extends Error { constructor(readonly result: TaskResult) { super(); } }`),
+stop the extractor, and return `e.result` from the task's outermost `catch`.
+A free-function handler that must keep only the first result stores it per
+adapter (`WeakMap<adapter, TaskResult>`) and the task reads it back.
+
+A helper that returned `{ delay?, error? }` and emitted only when one was set
+needs an explicit result when neither is set: `success` if the caller carried
+on, otherwise `error`.
+
+If a helper (often the HTTP client's 429 handler) **emitted and then threw a
+sentinel error** that call sites rethrow, keep the throw, drop the emit, and map
+it in the task's outermost `catch`:
+`if (e instanceof RateLimitDelayedError) return { status: 'delay', delaySeconds: e.retryAfterSeconds };`.
+Don't let it reach a generic error branch, or every rate limit fails the sync.
+In external sync units (non-resumable) the SDK still sends `delay` as an
+error; that matches nothing better, so keep it and flag it in the review.
+
+Tests of helpers that now record a result read it back
+(`expect(extractor['result']).toEqual(…)`) or, for `TaskResult | null`
+helpers, assert on the return value.
+
+**Timeouts.**
+
+- In the **task body**, `if (adapter.isTimeout) return;` becomes
+  `return { status: 'progress' };`. The same applies to a `return;` in a
+  `catch` that handles a timeout (`if (adapter.isTimeout || e instanceof
+  MyTimeout) return;`). Early `return`s inside `Promise<void>` helpers stay as
+  they are, as long as the task checks `isTimeout` after calling them.
+- After the soft timeout, a returned `progress` is replaced by the
+  `onTimeout` result, and so is a returned `success` in resumable phases
+  (metadata, data, attachments, loading). A returned `error` or `delay`, and
+  a `success` in external sync units or state deletion, are emitted as is.
+- `onTimeout` is optional and defaults to `progress` (or a timeout error in
+  external sync units and state deletion). v1 ran `onTimeout` only when the
+  task finished **without emitting**; v2 also runs it when a resumable task
+  returns `success` after the timeout, which costs one extra `PROGRESS` →
+  `CONTINUE` round instead of `DONE`. So:
+  - **Delete** it when it only emitted the usual event or computed a
+    `progress` number, when the v1 task emitted on every path, or when the task
+    only returns `loadItemTypes` / `loadAttachments` / `streamAttachments`
+    (they handled the timeout themselves).
+  - **Keep** it, returning a `TaskResult`, only when it could actually run in
+    v1 (the task had a path that didn't emit) **and** it did real work or set a
+    custom error. Make its body safe to run after a completed task. A custom
+    "Lambda timeout" `onTimeout` on a task that emits on every path is
+    deleted; v1 never ran it.
+  - `onTimeout` runs only after the task has returned, in v1 and v2, so
+    cleanup there (cancelling a rate limiter, aborting requests) never
+    interrupted a stuck task. Move it into the task's `isTimeout` /
+    `timeoutSignal` handling if it matters, otherwise delete it.
+- Don't call `process.exit()` or `postState()` in `onTimeout`; the SDK saves
+  state and exits.
+
+**Throws.** A task that throws emits the phase's `*Error` with the message,
+along with the artifacts already uploaded. `repo.push()` and `repo.upload()`
+resolve `void` and throw if an upload fails. **Any use of `push()`'s value is
+now wrong**, and `Boolean(await repo.push(x))` or `if (!(await …push(x)))`
+compiles but is always falsy. Make wrappers `await repo.push(x)` and return
+`true` (or `void`). Keep a top-level `try/catch` that maps an error to `delay`
+(e.g. a thrown 429); otherwise it's only needed for a custom message.
+`loadItemTypes` and `loadAttachments` don't throw: a `create` / `update` that
+throws becomes their `error` result, so a 429 must be returned as
+`{ status: 'delay', delaySeconds }` from the callback (§6). A `catch` around
+them that checks `isAxiosError(e) && status === 429` is dead; delete it.
+
+`ProcessTaskInterface<T>` and `TaskAdapterInterface<T>` now take the
+**adapter** type: `ProcessTaskInterface<ExtractionAdapter<State>>`, not
+`<State>`.
+
+## 5. Adapters
+
+The `WorkerAdapter` class is gone. Annotate with the phase's adapter:
+
+```ts
+async function extractList(adapter: ExtractionAdapter<State>) { ... }
+```
+
+| Members | On |
+|---------|----|
+| `initializeRepos`, `getRepo`, `streamAttachments`, `processAttachment`, `shouldExtract`, `artifacts`, `preExtractionItemCounts` | `ExtractionAdapter` |
+| `loadItemTypes`, `loadAttachments`, `reports`, `processedFiles` | `LoadingAdapter` |
+| `event`, `state`, `mappers`, `postState`, `uploadArtifact`, `downloadArtifact`, `isTimeout`, `extractionScope` | both |
+
+- The adapters and `Repo` are **type-only**: `new ExtractionAdapter(...)` and
+  `class X extends LoadingAdapter<S>` or `extends Repo` (test doubles that
+  subclassed `Repo` through `dist/`) don't compile; use a plain object with
+  the methods the code calls. `interface X extends …`
+  does. SDK internals (`emit`, `sdkState`, `options`, constructors, …) aren't
+  public. `isTimeout` and `artifacts` are read-only.
+- `adapter.mappers` works in extraction as in v1; keep those calls and only
+  apply the §10 return-shape changes.
+- **Integration tests** that built an adapter by hand use the root factories:
+
+  ```ts
+  // v1
+  const adapter = new WorkerAdapter({ event, adapterState: new State({ event, initialState }) });
+  adapterState.state.fromDevRev = { filesToLoad };
+  // v2
+  const adapter = createTestExtractionAdapter({ event, initialState });
+  const loader = createTestLoadingAdapter({ event, initialState, filesToLoad });
+  ```
+
+  Platform calls still go to the event's URLs. Build the event with
+  `createMockEvent(mockServer.baseUrl, …)` and serve the URLs with
+  `MockServer`. Artifact downloads need two routes: `GET
+  /internal/airdrop.artifacts.download-url` returning `{ download_url }`, and
+  that URL serving the gzipped JSONL. Assert on the returned `TaskResult`,
+  `adapter.reports` and `adapter.processedFiles`, not on `emit` or SDK state;
+  to test a resume, build a second adapter with `filesToLoad` at the cursor you
+  expect. To simulate a timeout, use a plain mock object, not a real adapter.
+  Test repos are plain objects (`{ itemType, push: jest.fn() }`).
+- `adapter.postState()` takes no argument and **throws** on failure (v1 exited
+  the process). State is saved on every emit, so keep a call only to save
+  state before an external write that must not be repeated. v1
+  `postState(next)` becomes `adapter.state = next; await adapter.postState();`.
+
+## 6. Loading, attachments and callback results
+
+`loadItemTypes`, `loadAttachments` and `streamAttachments` return a
+`TaskResult`. Return it directly:
+
+```ts
+processLoadingTask({ task: async ({ adapter }) => adapter.loadItemTypes({ itemTypesToLoad }) });
+processExtractionTask({ task: async ({ adapter }) => adapter.streamAttachments({ stream }) });
+```
+
+The result has no `reports` / `processed_files`. To add a report, push it to
+`adapter.reports` before returning. Tests assert on `adapter.reports`.
+
+**Connector callbacks** return a `status` union, like `TaskResult`:
+
+| Callback | v1 | v2 |
+|----------|----|----|
+| attachment `stream` | `{ httpStream }` / `{ delay }` / `{ error: { message, statusCode? } }` | `{ status: 'success', httpStream }` / `{ status: 'delay', delaySeconds }` / `{ status: 'error', error: { message } }` |
+| loader `create` / `update` | `{ id, modifiedDate? }` / `{ delay }` / `{ error: 'msg' \| { message } }` / neither (counted as failed) | `{ status: 'success', id, modifiedDate? }` / `{ status: 'delay', delaySeconds }` / `{ status: 'error', error: { message } }` |
+| `adapter.processAttachment`, custom `iterator` | `{ delay? , error? } \| undefined` | `{ status: 'success' }` / `{ status: 'delay', delaySeconds }` / `{ status: 'error', error }` |
+
+```ts
+// custom iterator, v2
+const response = await adapter.processAttachment(attachment, stream);
+if (response.status === 'delay') return response;
+if (response.status === 'error') console.warn(response.error.message);
+// ...
+return { status: 'success' }; // was a bare `return;`
+```
+
+- Loader `error` results still count as failed and loading continues. A v1
+  result with **both** `delay` and `error` becomes `delay` (v1 honored the delay).
+- `success` needs a real `id`: a v1 `{ id }` whose value may be `undefined`
+  (from an untyped response) was counted as failed, so guard it:
+  `if (!id) return { status: 'error', error: { message } }`. Extra fields such
+  as `skip` are dropped; every `success` creates or updates a mapper record.
+- Shared helpers typed as these responses change too. Callers that read
+  `result.delay` or interpolate `${result.error}` (now an object) switch on
+  `result.status`. A helper that only builds delays and errors returns the
+  narrowed type (`Extract<…, { status: 'delay' | 'error' }>`), so callers and
+  tests can read `delaySeconds` and `error` without a check.
+- An error `code` returned from the stream callback is only logged; the pool
+  skips that attachment.
+- `httpStream` is typed `HttpStreamResponse` (`{ data, headers }`); an axios
+  response still satisfies it. Types derived by indexing the old interface
+  (`ExternalSystemAttachmentStreamingResponse['httpStream']`) use the root
+  `HttpStreamResponse` instead.
+- `statusCode` on stream errors is gone; put it in the message if you need it.
+
+**`processAttachment` works in any extraction phase.** SSOR artifacts created
+outside the attachments phase are held by the SDK and reported with
+`ATTACHMENT_EXTRACTION_DONE`. **Delete any workaround** for this, along with
+its state fields and tests. Workarounds typically override the
+`ssor_attachment` repo's `onUpload`, splice `uploadedArtifacts`, adjust
+`currentEventDataLength`, or stash artifact refs in state.
+
+- A worker whose only job was re-injecting those refs returns
+  `{ status: 'success' }` (or its remaining work); keep its phase wired.
+- A connector's own `{ itemType: 'ssor_attachment' }` repo entry can stay.
+- Tests that mocked the deep `Uploader` use adapter doubles instead:
+  `{ uploadArtifact: jest.fn(), downloadArtifact: jest.fn() }`.
+- If the workaround stashed refs in connector state, syncs between data done and
+  attachments done lose them after the upgrade; drain them first.
+
+## 7. External sync units go through a repo
+
+`EventData.external_sync_units` is removed. Push them to a repo:
+
+```ts
+adapter.initializeRepos([
+  // matches the batching v1 used for the inline path
+  { itemType: AirSyncDefaultItemTypes.EXTERNAL_SYNC_UNITS, overriddenOptions: { batchSize: 25000 } },
+]);
+await adapter.getRepo(AirSyncDefaultItemTypes.EXTERNAL_SYNC_UNITS)?.push(externalSyncUnits);
+return { status: 'success' };
+```
+
+If the connector already pushed ESUs to a repo, keep its options as they are
+(apart from the §14 rename and removing `skipConfirmation`) and only convert
+the emit.
+
+## 8. State: connector state only
+
+- `adapter.state` is the connector's own state (the `initialState` shape);
+  reads and writes work as before. SDK bookkeeping (`toDevRev`, `fromDevRev`,
+  `workers*`, `pendingWorkers*`, `lastSyncStarted`,
+  `lastSuccessfulSyncStarted`, `snapInVersionId`) is internal. The
+  `AdapterState<S>` alias is removed; use your own `State` type.
+- **Persisted state** becomes `{ stateVersion: 2, connectorState, sdkState }`.
+  A v1 blob is migrated automatically on first read: known SDK keys move to
+  `sdkState` and everything else becomes connector state. **Rolling back to v1
+  after a v2 worker has run is not supported**; drain or reset syncs first. A
+  v1 state with a top-level key named `connectorState` or `sdkState` is
+  misread; rename it before upgrading.
+- **Sync window**: read
+  `adapter.event.payload.event_context.extract_from` / `extract_to`. The SDK
+  resolves them from the platform's time values, or from the last successful
+  sync on platforms that don't send them.
+- Remove SDK fields (`toDevRev`, `fromDevRev`, …) from the connector's own
+  `State` interface, `getInitialState` and fixtures.
+
+**`lastSyncStarted` / `lastSuccessfulSyncStarted`: rename or repoint.**
+Getting this wrong compiles but silently breaks incremental sync.
+
+These rules cover **top-level** state keys only; the v1 migration moves only
+those. A cursor nested in the connector's own object (`state.extractor.lastSuccessfulSyncStarted`)
+is connector-owned: leave it alone.
+
+- **Not in the connector's `State` interface**, or declared but never written
+  by the connector (it came from the SDK): delete writes, and repoint reads to
+  `extract_from`.
+- **Declared and used as the connector's own cursor** (derived from data, or
+  read alongside `extract_from`): **rename** it (e.g. →
+  `lastSuccessfulWindowStart`) everywhere, including fixtures. Otherwise the v1
+  auto-migration moves it into SDK state and the connector loses it.
+- **Declared but only mirrors the SDK**: its only writes are "now" when data
+  extraction starts and a copy on finish. Treat it as not declared, **even if
+  it is read next to `extract_from`**. Renaming would leave it empty after the
+  upgrade, because the v1 value moves into SDK state.
+
+After repointing, branches that chose between the two
+(`reset_extract_from ? extract_from : lastSuccessfulSyncStarted`) read the same
+value; collapse them and drop the `reset_extract_from` read. If the connector
+already used `extract_from` for something else (e.g. a skip filter) and the
+cursor for a second window (e.g. "re-emit and bump the floor"), repointing
+merges the two, so code below the window may become dead. Flag that low
+confidence.
+
+`extract_from` equals the last successful sync start only when the platform
+sends no time window. If a repointed read feeds logic beyond "skip items older
+than this" (e.g. distinguishing first syncs), flag it as low confidence; there
+is no public "last successful sync" value. Fixtures move from
+`adapter.state.lastSuccessfulSyncStarted = X` to
+`event.payload.event_context.extract_from = X`.
+
+## 9. HTTP: bring your own axios client
+
+`axios`, `axiosClient`, `formatAxiosError`, `serializeAxiosError` and
+`HTTPResponse` are no longer exported. Add `axios` (and `axios-retry` for a
+retrying client) to the connector's dependencies.
+
+- **`axios`** (v1 re-exported plain axios, without retries) →
+  `import axios from 'axios'`. Calls stay as they are; don't route them through
+  a retrying client.
+- **`axiosClient`** → the connector's own client, in its own module (e.g.
+  `src/http/axios-client.ts`), so tests can mock that module:
+
+```ts
+import axios from 'axios';
+import axiosRetry from 'axios-retry';
+
+export const axiosClient = axios.create();
+axiosRetry(axiosClient as Parameters<typeof axiosRetry>[0], {
+  retries: 5,
+  retryDelay: (retryCount, error) => axiosRetry.exponentialDelay(retryCount, error, 1000),
+  retryCondition: (error) =>
+    (axiosRetry.isNetworkOrIdempotentRequestError(error) && error.response?.status !== 429) ||
+    (error.response?.status ?? 0) >= 500,
+  onMaxRetryTimesExceeded: (error) => {
+    delete error.config?.headers?.authorization;
+    delete error.config?.headers?.Authorization;
+    delete error.request?._header;
+  },
+});
+```
+
+The cast is needed under ts-jest, which can resolve axios's `.d.ts` and
+`.d.cts` to different types.
+
+A deep import of the SDK's internal client (`dist/http/axios-client-internal`)
+relied on more: a 30 s timeout, retries of 429s that send `Retry-After` (waiting
+that long) and of `ECONNABORTED`. Reproduce what the connector depends on with
+`axios.create({ timeout: 30_000 })`, `shouldResetTimeout: true`, and a
+`retryCondition` / `retryDelay` that honor `Retry-After`.
+
+For logging, use the root `serializeError(error)`, which returns a string.
+For an axios error it is JSON of the request method, URL and params plus the
+response status and body, without headers. Interpolating `serializeAxiosError`
+printed `[object Object]` in v1, so such messages now carry these details.
+Where `serializeAxiosError(e)` was used as an object (spread, or `.response`),
+build the fields you need from the axios error yourself, leaving out headers.
+
+## 10. Mappers
+
+| Method | v1 | v2 |
+|--------|----|----|
+| `getByTargetId`, `getByExternalId` | `AxiosResponse<T>`; rejects with 404 when missing | `T \| null`; `null` when no record exists |
+| `create`, `update` | `AxiosResponse<T>` | `T` |
+| any other failure | axios error | `AirSyncHttpError` (`message`, `status?`), exported from the root |
+
+```ts
+// v1
+try {
+  const res = await mappers.getByTargetId({ sync_unit, target });
+  return res.data.sync_mapper_record.external_ids[0];
+} catch (e) {
+  if (axios.isAxiosError(e) && e.response?.status === 404) return undefined;
+  throw e;
+}
+// v2
+const record = await mappers.getByTargetId({ sync_unit, target });
+return record?.sync_mapper_record.external_ids[0];
+```
+
+- Drop `.data`; loosely typed code would otherwise read `undefined`.
+- **Existence checks invert silently.** `try { await mappers.getByExternalId(…); return true; } catch { return false; }`
+  now always returns `true`, because a missing record resolves `null`. Rewrite
+  the `try` body as `return (await mappers.getByExternalId(…)) !== null;` and
+  **keep the `catch`** if v1 had one, so other failures still mean what they
+  meant before. Find these with
+  `grep -n "getByExternalId\|getByTargetId"`, not by searching for 404.
+- Hand-written structural types (`{ getByTargetId(): Promise<{ data?: … }> }`)
+  keep `.data` compiling. Replace them with `Pick<Mappers, 'getByTargetId'>`.
+- Replace 404 `catch` branches with a `null` check.
+- A **generic `catch`** around a lookup (no 404 test) also handled "not
+  found": it reported an item error, logged a warning or tried the next ID.
+  With `null` that path no longer runs, so add a `null` branch that does what
+  the `catch` did, and keep the `catch` for other failures.
+- Replace `isAxiosError` checks on mapper calls with
+  `e instanceof AirSyncHttpError` / `e.status`. Checks on the connector's own
+  HTTP calls stay.
+- Test doubles: `mockResolvedValue({ data: { sync_mapper_record } })` →
+  `mockResolvedValue({ sync_mapper_record })`; a 404 `mockRejectedValue(...)`
+  or a "not found" `mockResolvedValue({ data: {} })` → `mockResolvedValue(null)`.
+  Find them with `grep sync_mapper_record`.
+
+## 11. Removed symbols
+
+| Removed | Replacement |
+|---------|-------------|
+| `ExtractorEventType`, `LoaderEventType` | none; return a `TaskResult` (§4), and in tests assert on it. Compare against the raw wire string if really needed |
+| `ExtractionCommonError` | `AirSyncErrorCode` on `error.code` (§12) |
+| `WorkerAdapterOptions` | `SpawnOptions` (spawn), `RepoOptions` (`overriddenOptions`) |
+| `WorkerAdapterInterface`, `SpawnInterface`, `StreamResponse`, `StreamAttachmentsResponse` | none; drop the import |
+| `ArtifactsPrepareResponse` | none; if the connector uses it as its own type, declare it locally (`{ url: string; id: string; form_data: { key: string; value: string }[] }`) |
+| worker IPC types (`WorkerEvent`, `WorkerMessage*`, `WorkerData`, `GetWorkerPathInterface`) | none; assert on the returned `TaskResult` |
+| `State`, `createAdapterState`, `ToDevRev`, `AdapterState` | `createTest*Adapter` (§5); drop SDK state types (§8) |
+| `Uploader` (deep import of `dist/uploader/uploader`, used to store and read back the connector's own artifacts) | `adapter.uploadArtifact(itemType, data)` → `{ id }` and `adapter.downloadArtifact<T>(id)` → `T[]`; both throw instead of returning `{ error }` |
+| `Adapter`, `createAdapter`, `DemoExtractor`, `HTTPClient`, `defaultResponse`, deprecated `Uploader` | the v2 worker contract, your own axios client, repos |
+| `ExtractionMode` | `SyncMode` |
+| `EventContextIn`, `EventContextOut` | `EventContext` |
+| `DomainObjectState`, `ErrorLevel`, `LogRecord`, `AdapterUpdateParams` | none |
+| `translate*EventType` helpers | none; the platform sends modern strings |
+| `EventType.UnknownEventType` | the string `'UNKNOWN_EVENT_TYPE'` |
+| `adapter.hasWorkerEmitted`, `adapter.emit` spies | none; return / assert on the `TaskResult` (§4, §13) |
+| `EventData.progress`, `EventData.external_sync_units` | none (§4, §7) |
+
+Deprecated `EventType` members are removed. They had **different** string
+values from their replacements:
+
+| Removed | Use |
+|---------|-----|
+| `ExtractionExternalSyncUnitsStart` | `StartExtractingExternalSyncUnits` |
+| `ExtractionMetadataStart` | `StartExtractingMetadata` |
+| `ExtractionDataStart` / `ExtractionDataContinue` | `StartExtractingData` / `ContinueExtractingData` |
+| `ExtractionDataDelete` | `StartDeletingExtractorState` |
+| `ExtractionAttachmentsStart` / `ExtractionAttachmentsContinue` | `StartExtractingAttachments` / `ContinueExtractingAttachments` |
+| `ExtractionAttachmentsDelete` | `StartDeletingExtractorAttachmentsState` |
+
+This also applies to `workerPathOverrides` keys. `EventType` can gain members
+in minor versions, so give `switch` statements over it a `default`.
+
+## 12. Error codes
+
+`ExtractionCommonError` is now `AirSyncErrorCode`. Its values are bare codes,
+and you set them on `error.code`:
+
+```ts
+// v1
+return { status: 'error', error: { message: ExtractionCommonError.EXTERNAL_SYNC_UNIT_DELETED } };
+// v2
+return {
+  status: 'error',
+  error: { message: `Project ${id} was deleted.`, code: AirSyncErrorCode.EXTERNAL_SYNC_UNIT_DELETED },
+};
+```
+
+The platform only recognizes a code when the message is exactly
+`ERROR_CODE=<code>`, so the SDK sends just that and logs your `message`. Write
+a useful `message`. Raw `'ERROR_CODE=…'` strings and connector helpers that
+return them (`HISTORICAL_DONE`, `authErrorCode(e)`) still work at runtime but
+are migrated too: make the helper return `AirSyncErrorCode | undefined`, set it
+as `code`, and update tests that pin the literal. The wire string doesn't
+change.
+
+## 13. Tests
+
+- Mock factories: `processTask` → `processExtractionTask` /
+  `processLoadingTask`. Drop the `WorkerAdapter`, `axios`, `axiosClient` and
+  `emit: jest.fn()` keys; if nothing is left, or the file under test never
+  imports the SDK, delete the `jest.mock` entirely. Keep `jest.requireActual`
+  only for enums that still exist (`EventType`, `SyncMode`, `AirSyncErrorCode`).
+- HTTP mocks: a test that mocked the SDK's `axiosClient` mocks the connector's
+  client module instead (`jest.mock('../http/axios-client', () => ({ axiosClient: { get: jest.fn(), post: jest.fn() } }))`).
+  Don't automock `axios` when the source calls `axios.create()`, because the
+  client becomes `undefined`. For plain `axios` use
+  `jest.mock('axios', () => ({ __esModule: true, default: { get: jest.fn(), isAxiosError: jest.fn() }, AxiosError: jest.requireActual('axios').AxiosError }))`.
+  Patching a client's `defaults.adapter` now affects only connector traffic.
+- ESU tests assert on the repo: give the mock adapter
+  `initializeRepos: jest.fn()` and `getRepo: () => ({ push: mockPush })`, then
+  check `mockPush.mock.calls[0][0]`.
+- Replace emit assertions with assertions on the returned result. This applies
+  to helpers you call directly too.
+  ```ts
+  const result = await mockProcessTask.mock.calls[0][0].task({ adapter });
+  expect(result).toEqual({ status: 'success' });
+  ```
+- Casts like `as unknown as WorkerAdapter<S>` become `ExtractionAdapter<S>` /
+  `LoadingAdapter<S>`.
+- Update doubles and fixtures to the v2 shapes: `delaySeconds`, callback
+  `status` unions (§6), mapper `null` / unwrapped bodies (§10), `code` on
+  errors (§12), renamed state fields and the removed SDK fields (§8).
+- For a deleted `onTimeout`, drop the tests of its body and assert it's
+  `undefined`. A factory like `processExtractionTask: (_config) => jest.fn()`
+  discards the config; use `processExtractionTask: jest.fn()` and read
+  `mocked.mock.calls[0][0].onTimeout`. Capture it in a module-level `const`
+  right after importing the worker: `jest.clearAllMocks()` in a `beforeEach`
+  wipes the calls. Tests of a deleted SSOR workaround go with it.
+
+## 14. Smaller changes
+
+- `RepoInterface.overridenOptions` → `overriddenOptions`.
+- `skipConfirmation` is removed; the SDK handles the ESU repo itself.
+- `NormalizedAttachment.grand_parent_id` is `string`; use `String(id)`.
+
+---
+
+## Rare cases
+
+- **Attachment syncs started on SDK < 1.15.2 and still in flight** re-upload
+  their already-processed attachments once. Drain them before upgrading if
+  that matters.
+- **`is_sdk_log`** is no longer on log lines; update dashboards that filter on
+  it.
+- **Connectors migrated against a beta (≤ `2.0.0-beta.6`)** must re-apply
+  §3–§14 against GA, because most of the API above changed after those betas.
+  State they wrote is still read.