@devrev/airsync-sdk 2.0.0-beta.6 → 2.0.0-beta.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (89) hide show
  1. package/README.md +8 -12
  2. package/dist/attachments-streaming/attachments-streaming-pool.d.ts.map +1 -1
  3. package/dist/attachments-streaming/attachments-streaming-pool.js +8 -9
  4. package/dist/common/constants.d.ts +0 -7
  5. package/dist/common/constants.d.ts.map +1 -1
  6. package/dist/common/constants.js +1 -28
  7. package/dist/common/helpers.d.ts.map +1 -1
  8. package/dist/common/helpers.js +1 -1
  9. package/dist/http/errors.d.ts +8 -0
  10. package/dist/http/errors.d.ts.map +1 -0
  11. package/dist/http/errors.js +31 -0
  12. package/dist/index.d.ts +18 -13
  13. package/dist/index.d.ts.map +1 -1
  14. package/dist/index.js +14 -11
  15. package/dist/logger/logger.d.ts.map +1 -1
  16. package/dist/logger/logger.interfaces.d.ts +2 -2
  17. package/dist/logger/logger.interfaces.d.ts.map +1 -1
  18. package/dist/mappers/mappers.d.ts +9 -3
  19. package/dist/mappers/mappers.d.ts.map +1 -1
  20. package/dist/mappers/mappers.interfaces.d.ts +0 -2
  21. package/dist/mappers/mappers.interfaces.d.ts.map +1 -1
  22. package/dist/mappers/mappers.js +33 -13
  23. package/dist/multithreading/adapters/base-adapter.d.ts +16 -35
  24. package/dist/multithreading/adapters/base-adapter.d.ts.map +1 -1
  25. package/dist/multithreading/adapters/base-adapter.js +89 -23
  26. package/dist/multithreading/adapters/extraction-adapter.d.ts +24 -20
  27. package/dist/multithreading/adapters/extraction-adapter.d.ts.map +1 -1
  28. package/dist/multithreading/adapters/extraction-adapter.js +175 -122
  29. package/dist/multithreading/adapters/loading-adapter.d.ts +13 -23
  30. package/dist/multithreading/adapters/loading-adapter.d.ts.map +1 -1
  31. package/dist/multithreading/adapters/loading-adapter.js +176 -301
  32. package/dist/multithreading/emit.d.ts.map +1 -1
  33. package/dist/multithreading/process-task.d.ts.map +1 -1
  34. package/dist/multithreading/process-task.js +24 -6
  35. package/dist/multithreading/spawn/spawn.d.ts +6 -1
  36. package/dist/multithreading/spawn/spawn.d.ts.map +1 -1
  37. package/dist/multithreading/spawn/spawn.helpers.d.ts.map +1 -1
  38. package/dist/multithreading/spawn/spawn.helpers.js +17 -52
  39. package/dist/multithreading/spawn/spawn.js +30 -26
  40. package/dist/multithreading/worker.js +15 -1
  41. package/dist/repo/repo.d.ts +4 -5
  42. package/dist/repo/repo.d.ts.map +1 -1
  43. package/dist/repo/repo.interfaces.d.ts +5 -4
  44. package/dist/repo/repo.interfaces.d.ts.map +1 -1
  45. package/dist/repo/repo.js +13 -15
  46. package/dist/state/base-state.d.ts +10 -5
  47. package/dist/state/base-state.d.ts.map +1 -1
  48. package/dist/state/base-state.js +39 -36
  49. package/dist/state/extraction-state.d.ts.map +1 -1
  50. package/dist/state/extraction-state.js +39 -4
  51. package/dist/state/state.interfaces.d.ts +13 -7
  52. package/dist/state/state.interfaces.d.ts.map +1 -1
  53. package/dist/state/state.interfaces.js +6 -8
  54. package/dist/state/time-value-resolver.d.ts.map +1 -1
  55. package/dist/state/time-value-resolver.js +3 -2
  56. package/dist/testing/mock-event.d.ts.map +1 -1
  57. package/dist/testing/mock-event.js +0 -8
  58. package/dist/testing/mock-server.d.ts +2 -0
  59. package/dist/testing/mock-server.d.ts.map +1 -1
  60. package/dist/testing/mock-server.interfaces.d.ts +6 -0
  61. package/dist/testing/mock-server.interfaces.d.ts.map +1 -1
  62. package/dist/testing/mock-server.js +37 -6
  63. package/dist/testing/test-adapters.d.ts +22 -0
  64. package/dist/testing/test-adapters.d.ts.map +1 -0
  65. package/dist/testing/test-adapters.js +32 -0
  66. package/dist/types/common.d.ts +4 -1
  67. package/dist/types/common.d.ts.map +1 -1
  68. package/dist/types/common.js +1 -1
  69. package/dist/types/errors.d.ts +13 -7
  70. package/dist/types/errors.d.ts.map +1 -1
  71. package/dist/types/errors.js +24 -0
  72. package/dist/types/extraction.d.ts +60 -44
  73. package/dist/types/extraction.d.ts.map +1 -1
  74. package/dist/types/extraction.js +10 -1
  75. package/dist/types/loading.d.ts +11 -30
  76. package/dist/types/loading.d.ts.map +1 -1
  77. package/dist/types/workers.d.ts +27 -23
  78. package/dist/types/workers.d.ts.map +1 -1
  79. package/dist/uploader/uploader.d.ts +1 -2
  80. package/dist/uploader/uploader.d.ts.map +1 -1
  81. package/dist/uploader/uploader.helpers.d.ts +3 -0
  82. package/dist/uploader/uploader.helpers.d.ts.map +1 -1
  83. package/dist/uploader/uploader.helpers.js +16 -0
  84. package/dist/uploader/uploader.interfaces.d.ts +4 -29
  85. package/dist/uploader/uploader.interfaces.d.ts.map +1 -1
  86. package/dist/uploader/uploader.js +4 -23
  87. package/docs/MIGRATION.md +651 -0
  88. package/package.json +24 -10
  89. package/MIGRATION.md +0 -1028
package/MIGRATION.md DELETED
@@ -1,1028 +0,0 @@
1
- # Migrating connectors from v1 (`@devrev/ts-adaas`) to v2 (`@devrev/airsync-sdk`)
2
-
3
- Covers every connector-facing breaking change between `@devrev/ts-adaas` v1.x
4
- and `@devrev/airsync-sdk` v2.0.0, with before/after examples from real
5
- connectors. Self-contained: the numbered sections, applied **in order** (see
6
- execution plan), are the complete migration — executable top to bottom without
7
- any other reference. It ships in the npm package, so after installing v2 the
8
- version-matched copy is at `node_modules/@devrev/airsync-sdk/MIGRATION.md`.
9
-
10
- **The wire protocol is unchanged.** Event payloads, API routes, headers, the
11
- artifact/upload flow, and every surviving event-type **string value** are
12
- byte-for-byte identical to v1. Only the connector-facing **TypeScript API**
13
- changed.
14
-
15
- > **Why a hard break with no deprecation shims?** v2 is a single bundled-pain
16
- > major: the package was renamed, so there are no existing consumers to keep
17
- > backward-compatible. Everything changed at once in one guide.
18
-
19
- ---
20
-
21
- ## TL;DR — what breaks
22
-
23
- | # | Change | Who it affects |
24
- |---|--------|----------------|
25
- | 1 | Package renamed `@devrev/ts-adaas` → `@devrev/airsync-sdk` | every import |
26
- | 2 | `AirdropEvent` → `AirSyncEvent`, `AirdropMessage` → `AirSyncMessage` | type annotations (all connectors) |
27
- | 3 | `processTask` → `processExtractionTask` / `processLoadingTask` | every worker file |
28
- | 4 | `adapter.emit(...)` is gone — tasks **return** a `TaskResult` instead | every worker file |
29
- | 5 | `WorkerAdapter` **class** removed → `ExtractionAdapter` / `LoadingAdapter` | helper signatures |
30
- | 6 | `loadItemTypes` / `loadAttachments` / `streamAttachments` now **return** a `TaskResult` | loading + attachment workers |
31
- | 7 | `EventData.external_sync_units` **removed** — external sync units must be pushed to a repo | ESU workers |
32
- | 8 | `adapter.state` is connector-state only; SDK fields readable via `adapter.sdkState`. `lastSyncStarted` / `lastSuccessfulSyncStarted` **removed** | connectors reading/writing SDK bookkeeping |
33
- | 9 | `axios`, `axiosClient`, `formatAxiosError`, `serializeAxiosError`, `HTTPResponse` no longer exported | anyone importing the SDK's HTTP/axios surface |
34
- | 10 | `Mappers` methods return the unwrapped body (`Promise<T>`), not `Promise<AxiosResponse<T>>` | loaders calling `adapter.mappers.*` |
35
- | 11 | Deprecated v1 modules deleted (`Adapter`, `createAdapter`, `DemoExtractor`, `HTTPClient`, `defaultResponse`, deprecated `Uploader`) | only legacy code |
36
- | 12 | Deprecated event-type enum **members** deleted; deprecated types/enums (`ExtractionMode`, `EventContextIn`/`Out`, `DomainObjectState`, `ErrorLevel`, `LogRecord`, `AdapterUpdateParams`, `AdapterState`) **removed**; internal worker-IPC types (`WorkerEvent`, `WorkerMessageSubject`, `WorkerMessage*`, `WorkerData`, `GetWorkerPathInterface`) off the root barrel | only if you used the old members/types or the internal IPC types |
37
- | 13 | Several deep `dist/**` import paths moved/removed | deep-importers |
38
- | 14 | Legacy `string[]` attachment-dedup migration dropped | in-flight attachment syncs started on SDK **< 1.15.2** |
39
- | 15 | `spawn`'s deprecated `workerPath` option **removed** — use `baseWorkerPath: __dirname` | connectors still passing `workerPath` |
40
- | 16 | Jest mocks of the SDK hardcode the v1 shape; emit-called assertions become return-value assertions | every test suite mocking the SDK |
41
-
42
- (Also: `EventData.progress` was removed — a no-op since v1; the backend computes
43
- progress. Covered with the other `EventData` removals in §12.)
44
-
45
- (Also: emitted logs no longer carry the `is_sdk_log` field — an
46
- observability-only change that breaks no connector code; see §12.)
47
-
48
- What does **not** change: `spawn(...)` and its surviving options (`initialState`,
49
- `initialDomainMapping`, `workerPathOverrides`, `baseWorkerPath`,
50
- `options.batchSize`, `timeout`, `isLocalDevelopment`, …) — the one exception is
51
- the long-deprecated `workerPath`, removed (§15); default worker paths
52
- (`/workers/data-extraction`, etc.); repos (`initializeRepos`, `getRepo`,
53
- `push`); the normalization interfaces (`NormalizedItem`, `NormalizedAttachment`,
54
- `RepoInterface`, `ExternalSyncUnit`); `installInitialDomainMapping`;
55
- `createMockEvent` / `MockServer`; HTTP retry behavior;
56
- `event_context.extract_from` / `extract_to`; and every surviving event-type
57
- string value on the wire.
58
-
59
- ### Execution plan (for migrating a whole connector in one pass)
60
-
61
- Apply sections **in numbered order** — later ones assume earlier ones ran (e.g.
62
- §3–§6 rewrite emit sites that §5's adapter types must already annotate). For each
63
- section: search the whole connector — source **and** tests — apply every hit,
64
- then move on. Verify at the end, not per-section (a half-applied section never
65
- compiles). Two search rules:
66
-
67
- - **After §1, the old specifier is gone.** Every later search must target the new
68
- `@devrev/airsync-sdk` specifier or a bare symbol name (`\bWorkerAdapter\b`,
69
- `\baxiosClient\b`, `\.emit\(`, `lastSuccessfulSyncStarted`, …) — searching
70
- `@devrev/ts-adaas` after §1 finds nothing and makes a section look like a false
71
- no-op.
72
- - **A section with zero hits is a verified no-op** — confirm and move on; many
73
- connectors already use `baseWorkerPath`, push ESUs to a repo, and have no deep
74
- imports.
75
- - **Don't narrate the migration in code.** The `// v1` / `// v2` labels here are
76
- reading aids — never copy them into the connector. Apply each edit cleanly, no
77
- comment explaining what changed. Add a comment only where the resulting logic
78
- is genuinely non-obvious independent of this migration.
79
-
80
- Final verification, from the connector's package dir: `npm install` (regenerates
81
- the lockfile), `npx tsc --noEmit`, `npm run lint` (unused imports/locals orphaned
82
- by rewrites fail here — clean them up), `npm run build`, and the repo's test
83
- command if tests exist. A leftover `adapter.emit`, a removed symbol, a missing
84
- `delaySeconds`, a destructured `{ reports }`, or an un-renamed test fixture is
85
- the usual failure cause.
86
-
87
- ---
88
-
89
- ## 1. Package rename
90
-
91
- ```bash
92
- npm uninstall @devrev/ts-adaas
93
- npm install @devrev/airsync-sdk@beta
94
- ```
95
-
96
- > Until GA ships, install via the **`beta` dist-tag** — it always resolves to the
97
- > newest published 2.x beta. A plain `npm install @devrev/airsync-sdk` may resolve
98
- > an older beta, and `@2.0.0` does not exist yet and 404s. Once GA is published,
99
- > plain install (`@latest`) is correct.
100
-
101
- > **Start from the latest stable v1.** This guide describes the delta from the
102
- > latest stable `@devrev/ts-adaas` (1.x). If your connector is on an older 1.x
103
- > that skipped an intra-v1 migration, do that first so the examples match your
104
- > code. Check **https://github.com/devrev/adaas-sdk/releases** for any release
105
- > between your version and latest whose notes call out a migration or breaking
106
- > change (the last such were **1.18.0**, **1.17.0**, **1.16.0**; **1.19.0 →
107
- > 1.20.0** were migration-free). Bring the connector to a green latest-stable v1
108
- > build before starting.
109
-
110
- Then global-replace the import specifier **everywhere the string appears**: every
111
- import, every `jest.mock('...')` / `jest.requireActual('...')` first argument, and
112
- any jest `moduleNameMapper`. This also rewrites deep-import path prefixes
113
- (`@devrev/ts-adaas/dist/x` → `@devrev/airsync-sdk/dist/x`); §13 handles the paths
114
- that moved. Do not rename any symbols in this step.
115
-
116
- ```ts
117
- // v1
118
- import { spawn, EventType } from '@devrev/ts-adaas';
119
- // v2
120
- import { spawn, EventType } from '@devrev/airsync-sdk';
121
- ```
122
-
123
- > **Deep imports** like `@devrev/ts-adaas/dist/...` are fragile — several paths
124
- > moved or were removed in v2 (§13). Prefer root imports; the v2 barrel now exports
125
- > several symbols that previously required a deep import (`Mappers`, `Item`,
126
- > `ItemTypeToLoad`).
127
-
128
- ## 2. Type renames: `AirdropEvent` → `AirSyncEvent`
129
-
130
- Hard rename, no compatibility alias. The payload **shape** is identical — only the
131
- type name changed.
132
-
133
- | v1 | v2 |
134
- |----|----|
135
- | `AirdropEvent` | `AirSyncEvent` |
136
- | `AirdropMessage` | `AirSyncMessage` |
137
-
138
- E.g. `(events: AirdropEvent[])` → `(events: AirSyncEvent[])`.
139
-
140
- No other public type was renamed — `ConnectionData`, `EventContext`, `EventData`,
141
- `ExtractorEvent`, and `ExternalSyncUnit` keep their v1 names. Platform-owned
142
- strings (`/internal/airdrop.*` routes, the `'ADaaS'` external system type, the
143
- `adaas_library_version` metadata key, `airdrop_*` mapping enum values) are
144
- intentionally unchanged.
145
-
146
- ### `AirSyncEvent.context` gained identity fields
147
-
148
- `AirSyncEvent.context` now declares the identity fields the platform already
149
- sends, in addition to `secrets`, `snap_in_id`, `snap_in_version_id`:
150
-
151
- ```ts
152
- // v2 — AirSyncEvent.context
153
- context: {
154
- secrets: { service_account_token: string };
155
- snap_in_version_id: string;
156
- snap_in_id: string;
157
- user_id: string; // new
158
- dev_oid: string; // new
159
- source_id: string; // new
160
- service_account_id: string; // new
161
- };
162
- ```
163
-
164
- If you previously extended the event to read these (e.g. a hand-rolled
165
- `CustomAirdropEvent` that added `user_id`), drop the extension and read
166
- `adapter.event.context.user_id` directly. (`snap_in_id` was already in v1; only
167
- the four fields above are new.)
168
-
169
- > Note: these four fields live on the **top-level** `AirSyncEvent.context`, not on
170
- > the `EventContext` inside `payload.event_context` (a different, unchanged
171
- > object).
172
-
173
- ## 3 + 4. The new worker contract: **return a `TaskResult`** instead of emitting
174
-
175
- The core change of v2. In v1 the connector decided *which event* to emit and
176
- called `adapter.emit(...)`. In v2 it only reports *how the phase ended* by
177
- **returning** a `TaskResult`; the SDK maps it to the correct platform event for
178
- the current phase and emits it exactly once.
179
-
180
- ```ts
181
- // the exact union (exported from @devrev/airsync-sdk)
182
- export type TaskResult =
183
- | { status: 'success' }
184
- | { status: 'progress' }
185
- | { status: 'delay'; delaySeconds: number } // note: delaySeconds, not delay
186
- | { status: 'error'; error: ErrorRecord }; // ErrorRecord = { message: string }
187
- ```
188
-
189
- `processTask` is split into two typed entry points — pick the one matching the
190
- worker's phase. Phase is per-file: workers under `extraction/` (or whose body uses
191
- `initializeRepos`/`getRepo`/`streamAttachments`) are extraction; workers under
192
- `loading/` (or using `loadItemTypes`/`loadAttachments`/`mappers`) are loading.
193
-
194
- ### Before (v1)
195
-
196
- ```ts
197
- import { processTask, ExtractorEventType } from '@devrev/ts-adaas';
198
-
199
- processTask({
200
- task: async ({ adapter }) => {
201
- // ... extract ...
202
- await adapter.emit(ExtractorEventType.DataExtractionDone);
203
- },
204
- onTimeout: async ({ adapter }) => {
205
- await adapter.postState();
206
- await adapter.emit(ExtractorEventType.DataExtractionProgress, { progress: 50 });
207
- },
208
- });
209
- ```
210
-
211
- ### After (v2)
212
-
213
- ```ts
214
- import { processExtractionTask } from '@devrev/airsync-sdk';
215
-
216
- processExtractionTask({
217
- task: async ({ adapter }) => {
218
- // ... extract ...
219
- return { status: 'success' };
220
- },
221
- // onTimeout can be omitted entirely for resumable phases:
222
- // the SDK emits a progress (continuation) result by default.
223
- });
224
- ```
225
-
226
- Loading workers use `processLoadingTask` the same way.
227
-
228
- > `adapter.emit(...)` is now **`protected`** — calling `adapter.emit(...)` or
229
- > `this.adapter.emit(...)` is a hard **compile error** in v2, not a deprecation.
230
- > Every emit site must be converted.
231
-
232
- ### `emit()` → `return` translation table
233
-
234
- | v1 emit call | v2 return |
235
- |--------------|-----------|
236
- | `await adapter.emit(XxxDone)` | `return { status: 'success' }` |
237
- | `await adapter.emit(XxxProgress, { progress })` | `return { status: 'progress' }` |
238
- | `await adapter.emit(XxxDelayed, { delay })` | `return { status: 'delay', delaySeconds: delay }` |
239
- | `await adapter.emit(XxxError, { error })` | `return { status: 'error', error }` |
240
- | `await adapter.emit(ExternalSyncUnitExtractionDone, { external_sync_units })` | push to repo + `return { status: 'success' }` — see §7 |
241
-
242
- `progress` (`{ progress: n }`) carried no semantic value in v1 beyond "not done
243
- yet" and is dropped; the platform tracks progress itself.
244
-
245
- > **A `Done` emit that carried a non-fatal error summary** (v1
246
- > `emit(XxxDone, { error })` — "finished, but here's what went wrong") maps to
247
- > `return { status: 'success' }` plus surfacing the error some other way (a report
248
- > entry or `console.warn`) — **not** to `{ status: 'error' }`, which would emit
249
- > `*Error` and flip a successful phase to failed.
250
-
251
- > **`ProcessTaskInterface` / `TaskAdapterInterface` changed their generic.** Both
252
- > survive, but the type parameter now names the **adapter**, not the connector
253
- > state: v1 `ProcessTaskInterface<State>` / `TaskAdapterInterface<State>` → v2
254
- > `ProcessTaskInterface<ExtractionAdapter<State>>` (or `LoadingAdapter<State>`),
255
- > same for `TaskAdapterInterface`. A bare `<State>` compiles in loosely-typed
256
- > spots but silently types `adapter` as the connector state. `onTimeout` is now
257
- > optional, and both callbacks return `Promise<TaskResult>` not `Promise<void>`.
258
-
259
- ### Status → emitted event, per phase
260
-
261
- The SDK picks the platform event from the **current phase** and the returned
262
- status:
263
-
264
- | status | Resumable phases — data/attachment **extraction**, data/attachment **loading** | Non-resumable — external sync units, metadata, state deletion |
265
- |--------|------------------------------------------------------------------------------|---------------------------------------------------------------|
266
- | `'success'` | `*Done` | `*Done` |
267
- | `'progress'` | `*Progress` (continuation) | **`*Error`** (illegal for these phases; emitted with a generated message) |
268
- | `'delay'` | `*Delayed` (with `delaySeconds`) | **`*Error`** (illegal) |
269
- | `'error'` | `*Error` (with the error record) | `*Error` |
270
-
271
- ### Emits buried inside helper functions
272
-
273
- A common v1 pattern: emit deep inside a helper, return a boolean telling the
274
- caller to stop. Since only the **task's return value** reaches the SDK in v2, the
275
- helper must **bubble the outcome up**.
276
-
277
- A clean way (used by the migrated google-drive connector): store the terminal
278
- result on a field and return it once the loop unwinds:
279
-
280
- ```ts
281
- // v1 — helper emits, returns false to abort
282
- private async handlePermissionDeniedError(error: unknown) {
283
- await this.adapter.emit(ExtractorEventType.DataExtractionError, {
284
- error: { message: '...' },
285
- });
286
- }
287
- ```
288
-
289
- ```ts
290
- // v2 — helper records the result; the task returns it
291
- private result: TaskResult | undefined;
292
-
293
- private handlePermissionDeniedError(error: unknown) {
294
- this.result = { status: 'error', error: { message: '...' } };
295
- }
296
-
297
- // ... and where the loop ends:
298
- private finalResult(): TaskResult {
299
- // a stored error/delay set by a helper, else progress (continuation)
300
- return this.result ?? { status: 'progress' };
301
- }
302
- ```
303
-
304
- For a simple "stop iterating" signal, return the `TaskResult` directly up the call
305
- chain:
306
-
307
- ```ts
308
- // v2
309
- async function extractList(adapter: ExtractionAdapter<State>): Promise<TaskResult | null> {
310
- if (rateLimited) return { status: 'delay', delaySeconds: retryAfter };
311
- // ...
312
- return null; // keep going
313
- }
314
-
315
- processExtractionTask<State>({
316
- task: async ({ adapter }) => {
317
- for (const itemType of itemTypes) {
318
- const stop = await extractList(adapter);
319
- if (stop) return stop;
320
- }
321
- return { status: 'success' };
322
- },
323
- });
324
- ```
325
-
326
- ### Timeout handling
327
-
328
- Checking `adapter.isTimeout` in your extraction loop still works as in v1 — but
329
- instead of emitting progress and exiting, **return** progress:
330
-
331
- ```ts
332
- if (adapter.isTimeout) {
333
- return { status: 'progress' }; // platform sends CONTINUE_* next
334
- }
335
- ```
336
-
337
- Two behaviors matter when migrating:
338
-
339
- - **The timeout outcome always wins.** Once the soft timeout fires, the SDK emits
340
- the `onTimeout` result (or its default) and **ignores whatever the task
341
- returned** — a phase that ran out of time must hand off for continuation, not
342
- report itself complete.
343
- - **Omit `onTimeout` unless it does real work.** Its default is phase-aware:
344
- `progress` (continuation) for resumable phases, a timeout **error** for
345
- non-resumable ones (external sync units, metadata, state deletion) where
346
- continuation is impossible. So a v1 `onTimeout` that only emitted the
347
- phase-appropriate event (optionally after `postState()`) is now redundant —
348
- **delete it**. Keep an explicit `onTimeout` only when its v1 body did real work
349
- that must survive a timeout (cancelling rate limiting, clearing upload
350
- bookkeeping, a custom error message) — preserve that body, replacing the
351
- trailing emit with a returned `TaskResult`:
352
-
353
- ```ts
354
- onTimeout: async () => ({
355
- status: 'error',
356
- error: { message: 'Custom timeout message.' },
357
- }),
358
- ```
359
-
360
- > Do **not** call `process.exit()` yourself in v2 — the SDK owns the single worker
361
- > exit after it emits your `TaskResult`. Likewise drop `adapter.postState()` inside
362
- > `onTimeout`; the SDK persists state around the timeout emit.
363
-
364
- ## 5. `WorkerAdapter` → `ExtractionAdapter` / `LoadingAdapter`
365
-
366
- The `WorkerAdapter` **class** is gone. Replace the type annotation on your helpers
367
- with the mode-specific adapter:
368
-
369
- ```ts
370
- // v1
371
- async function extractList(adapter: WorkerAdapter<State>) { ... }
372
- // v2
373
- import { ExtractionAdapter } from '@devrev/airsync-sdk';
374
- async function extractList(adapter: ExtractionAdapter<State>) { ... }
375
- ```
376
-
377
- | Surface | Lives on |
378
- |---------|----------|
379
- | `initializeRepos`, `getRepo`, `streamAttachments`, `shouldExtract`, `artifacts` | `ExtractionAdapter` |
380
- | `loadItemTypes`, `loadAttachments`, `mappers`, `reports`, `processedFiles` | `LoadingAdapter` |
381
- | `event`, `state`, `sdkState`, `postState`, `isTimeout`, `extractionScope` | both (shared `BaseAdapter`) |
382
-
383
- > Only the **class** was removed. The **types** `WorkerAdapterInterface` and
384
- > `WorkerAdapterOptions` still exist and are still exported — don't blindly rename
385
- > every `WorkerAdapter` token; replace the `WorkerAdapter<T>` annotations and
386
- > `new WorkerAdapter(...)` constructions only.
387
-
388
- ### Hand-constructed adapters (integration tests)
389
-
390
- Some connectors construct the adapter directly in integration tests
391
- (`new WorkerAdapter({ event, adapterState })`). The same construction works with
392
- the phase-specific classes — but `adapterState` must be a v2 state class, which
393
- replaced the v1 `State` class / `createAdapterState` factory (both removed; §13
394
- for the import path):
395
-
396
- ```ts
397
- // v1
398
- const adapterState = new State({ event, initialState });
399
- const adapter = new WorkerAdapter({ event, adapterState });
400
-
401
- // v2 — same shape, phase-specific classes
402
- import { LoadingState } from '@devrev/airsync-sdk/dist/state/state';
403
- const adapterState = new LoadingState({ event, initialState });
404
- const adapter = new LoadingAdapter({ event, adapterState });
405
- ```
406
-
407
- Sync `new State(...)` maps to sync `new ExtractionState(...)` /
408
- `new LoadingState(...)` (same `{ event, initialState, initialDomainMapping?,
409
- options? }` params). The async `createAdapterState(...)` (fetched persisted state)
410
- maps to the async `createExtractionState(...)` / `createLoadingState(...)`
411
- factories at the same import path. A test that poked SDK fields via
412
- `adapterState.state = { fromDevRev: ... }` must move them onto the `sdkState`
413
- envelope (§8).
414
-
415
- > **`mappers` / `reports` / `processedFiles` moved to `LoadingAdapter` only.** In
416
- > v1 they were on the single `WorkerAdapter`, reachable in any phase. In v2 they
417
- > are not on `ExtractionAdapter`. Code that touched `adapter.mappers` (etc.)
418
- > during an extraction phase must move to the loading path.
419
-
420
- ## 6. Loading & attachment methods return a `TaskResult`
421
-
422
- `loadItemTypes`, `loadAttachments`, and `streamAttachments` no longer emit or exit
423
- mid-flight — they **return** a `TaskResult` you pass straight through. Rate limits
424
- (→ `delay`), timeouts (→ `progress`), errors (→ `error`), completion (→ `success`)
425
- are all encoded in the result; `reports` / `processed_files` (loading) and
426
- artifacts (extraction) are attached to the emitted event automatically.
427
-
428
- ### Loading — before (v1)
429
-
430
- ```ts
431
- import { LoaderEventType, processTask } from '@devrev/ts-adaas';
432
-
433
- processTask({
434
- task: async ({ adapter }) => {
435
- await adapter.loadItemTypes({ itemTypesToLoad });
436
- await adapter.emit(LoaderEventType.DataLoadingDone);
437
- },
438
- onTimeout: async ({ adapter }) => {
439
- await adapter.postState();
440
- await adapter.emit(LoaderEventType.DataLoadingProgress);
441
- },
442
- });
443
- ```
444
-
445
- ### Loading — after (v2)
446
-
447
- ```ts
448
- import { processLoadingTask } from '@devrev/airsync-sdk';
449
-
450
- processLoadingTask({
451
- task: async ({ adapter }) => {
452
- return adapter.loadItemTypes({ itemTypesToLoad });
453
- },
454
- });
455
- ```
456
-
457
- ### Attachment streaming — before (v1)
458
-
459
- ```ts
460
- const response = await adapter.streamAttachments({ stream: getFileStream, batchSize: 50 });
461
- if (response?.delay) {
462
- await adapter.emit(ExtractorEventType.AttachmentExtractionDelayed, { delay: response.delay });
463
- } else if (response?.error) {
464
- await adapter.emit(ExtractorEventType.AttachmentExtractionError, { error: response.error });
465
- } else {
466
- await adapter.emit(ExtractorEventType.AttachmentExtractionDone);
467
- }
468
- ```
469
-
470
- ### Attachment streaming — after (v2)
471
-
472
- ```ts
473
- return adapter.streamAttachments({ stream: getFileStream, batchSize: 50 });
474
- ```
475
-
476
- > **Never destructure the returned `TaskResult`.** The union has **no** `reports` /
477
- > `processed_files` members, so v1 code like
478
- > `const { reports, processed_files } = await adapter.loadAttachments(...)` — or
479
- > pushing a synthetic report into the returned array — is a compile error. A
480
- > connector that augmented the reports pushes onto the **live getter before**
481
- > returning instead:
482
- >
483
- > ```ts
484
- > adapter.reports.push(buildNotesReport(...));
485
- > return await adapter.loadAttachments({ create });
486
- > ```
487
- >
488
- > A defensive outer try/catch with bespoke rate-limit handling can stay:
489
- > `return await adapter.loadItemTypes(...)` in the `try`, map escaped throws to
490
- > `{ status: 'delay' | 'error' }` in the `catch`.
491
-
492
- Custom attachment processors (reducer/iterator) keep the same call signatures;
493
- only their `adapter` parameter type changes from `WorkerAdapter<C>` to
494
- `ExtractionAdapter<C>` (§5). The `getAttachmentStream` function you implement still
495
- returns `{ httpStream }` / `{ delay }` / `{ error }` — but see §9 for the
496
- `httpStream` type change.
497
-
498
- ## 7. External sync units go through a repo
499
-
500
- In v1 the SDK accepted `external_sync_units` in emit data and uploaded them
501
- internally. With emit gone, push them to the `EXTERNAL_SYNC_UNITS` repo yourself.
502
- The `EventData.external_sync_units` field — deprecated in v1 — is **removed
503
- entirely** in v2: there is no inline ESU path, and any code still referencing
504
- `external_sync_units` in emit data is now a compile error. External sync units
505
- leave the worker only as repo artifacts.
506
-
507
- ### Before (v1)
508
-
509
- ```ts
510
- await adapter.emit(ExtractorEventType.ExternalSyncUnitExtractionDone, {
511
- external_sync_units: externalSyncUnits,
512
- });
513
- ```
514
-
515
- ### After (v2)
516
-
517
- ```ts
518
- import { AirSyncDefaultItemTypes, processExtractionTask } from '@devrev/airsync-sdk';
519
-
520
- adapter.initializeRepos([
521
- {
522
- itemType: AirSyncDefaultItemTypes.EXTERNAL_SYNC_UNITS,
523
- // mirror the batching the v1 SDK used internally for ESUs
524
- overridenOptions: { batchSize: 25000, skipConfirmation: true },
525
- },
526
- ]);
527
- await adapter.getRepo(AirSyncDefaultItemTypes.EXTERNAL_SYNC_UNITS)?.push(externalSyncUnits);
528
-
529
- return { status: 'success' };
530
- ```
531
-
532
- The repo is uploaded automatically before the `Done` event is emitted.
533
-
534
- > If your connector already pushed ESUs to a repo in v1 (some do), this section is
535
- > a no-op — just convert the trailing emit to `return`.
536
-
537
- ## 8. State split: `adapter.state` vs `adapter.sdkState`
538
-
539
- In v1, connector state and SDK bookkeeping lived in one flat blob and
540
- `adapter.state` exposed both. In v2:
541
-
542
- - **`adapter.state` is connector state only** — exactly the shape of the
543
- `initialState` you pass to `spawn`. Getter and setter both survive, so in-place
544
- reads/writes work unchanged:
545
-
546
- ```ts
547
- // works identically in v1 and v2
548
- adapter.state[itemType].cursor = nextCursor;
549
- adapter.state[itemType].complete = true;
550
- ```
551
-
552
- - **SDK bookkeeping** (`workersOldest`/`workersNewest`, `pendingWorkers*`,
553
- `toDevRev`, `fromDevRev`, `snapInVersionId`) moved to **`adapter.sdkState`**
554
- (read-only getter).
555
- - On disk, state is persisted as a `{ connectorState, sdkState }` envelope. The SDK
556
- **migrates a v1 flat blob automatically on first read** (recognized SDK keys
557
- split into `sdkState`, the rest into `connectorState`), so in-flight syncs
558
- survive the upgrade.
559
-
560
- > **`lastSyncStarted` / `lastSuccessfulSyncStarted` are gone.** Both were
561
- > `@deprecated` in v1 and are **removed from `SdkState` in v2** — the SDK no longer
562
- > sets, reads, or declares them. They were wall-clock sync-start timestamps; the
563
- > SDK now resolves the incremental window from extraction-data boundaries instead
564
- > (`workersOldest`/`workersNewest` and the resolved `extract_from`/`extract_to`).
565
- > The two old key names are still recognized as SDK-owned by the v1-blob migration,
566
- > so they route into `sdkState` (where they sit unused) rather than leaking into
567
- > your connector state.
568
-
569
- ### If your v1 connector mixed SDK fields into its own `State` interface
570
-
571
- Some connectors declared SDK-owned fields (e.g. `toDevRev`,
572
- `lastSuccessfulSyncStarted`) on their hand-written `State` type and seeded them in
573
- `getInitialState()`. Remove those — `toDevRev`/`fromDevRev`/`workers*` are
574
- SDK-managed now, and `lastSyncStarted`/`lastSuccessfulSyncStarted` no longer exist
575
- at all:
576
-
577
- ```ts
578
- // v1 — State interface mixing SDK fields (remove these)
579
- export interface State {
580
- // ... your fields ...
581
- lastSyncStarted?: string; // removed in v2 — no replacement field
582
- lastSuccessfulSyncStarted?: string; // removed in v2 — read workersNewest / extract_to instead
583
- toDevRev?: ToDevRev; // also drop the dist/state/state.interfaces import
584
- }
585
- ```
586
-
587
- ### Cursor decision rule: rename vs repoint
588
-
589
- For `lastSyncStarted` / `lastSuccessfulSyncStarted` specifically, what to do
590
- depends on **whether the field is declared in your own `State` interface /
591
- `getInitialState`** — getting this backwards compiles but silently breaks
592
- incremental sync:
593
-
594
- - **Not declared** (only accessed loosely as `adapter.state.<field>`, absent from
595
- your interface): it was SDK-supplied via the removed v1 `AdapterState<T>`, and
596
- the access is now a hard type error. **Delete** a bare write; **repoint** a read
597
- to `adapter.event.payload.event_context.extract_from` (the resolved window start
598
- — usually already destructured nearby). There is no field to preserve, so do not
599
- rename.
600
- - **Declared** (your connector writes and reads it as its own cross-sync cursor,
601
- often consuming `extract_from` separately): **rename** it off the reserved SDK
602
- key (e.g. `lastSuccessfulSyncStarted` → `lastSuccessfulWindowStart`) throughout
603
- the interface, `getInitialState`, all read/write sites, and test fixtures. The
604
- rename is mandatory because the v1-blob auto-migration routes the reserved key
605
- into `sdkState` — a connector-owned cursor left on that name would be stripped
606
- from your state. Substituting `extract_from` here would destroy the cursor
607
- mechanism.
608
-
609
- ### Reading the incremental-sync window
610
-
611
- In v1 you read `lastSuccessfulSyncStarted` to decide an incremental cursor. That
612
- field is gone (above). Read the window the SDK resolves for you instead:
613
-
614
- ```ts
615
- // preferred — the resolved window for this invocation, on the event context
616
- const { extract_from, extract_to } = adapter.event.payload.event_context;
617
-
618
- // the persisted cross-sync high-water mark (committed at end of each cycle)
619
- const lastExtractedTo = adapter.sdkState.workersNewest;
620
- ```
621
-
622
- `extract_from`/`extract_to` are the per-invocation window the SDK computes from the
623
- platform's `extraction_start_time`/`extraction_end_time`;
624
- `workersOldest`/`workersNewest` are the persisted boundaries committed at the end
625
- of a completed cycle — the true replacement for the old "last successful sync"
626
- resume point.
627
-
628
- > **`AdapterState<ConnectorState>` is removed.** The deprecated flat
629
- > `ConnectorState & SdkState` alias no longer exists. If you annotated anything
630
- > `AdapterState<S>`, drop the import — use your own connector `State` type for
631
- > `adapter.state`, and read SDK fields from `adapter.sdkState` (or the event
632
- > context). The v2 on-disk shape is the `AdapterStateEnvelope`
633
- > (`{ connectorState, sdkState }`), which the SDK manages for you.
634
-
635
- > **Edge regression (old in-flight incremental syncs):** the SDK used to fall back
636
- > to `lastSuccessfulSyncStarted` when resolving a `WORKERS_NEWEST` window on state
637
- > predating `workersNewest` (SDK **< 1.17.1**). With the field removed, that
638
- > fallback is gone: such a window now resolves to the unbounded epoch, so an
639
- > incremental sync still mid-flight across the upgrade re-extracts from the
640
- > beginning **once**. The next completed cycle commits `workersNewest` and the sync
641
- > self-heals; the platform deduplicates downstream. New syncs and any sync whose
642
- > first full cycle completed on ≥ 1.17.1 are unaffected. Drain in-flight
643
- > incremental syncs before upgrading if the one-time re-extract matters.
644
-
645
- > **Edge case:** if your v1 connector state had a top-level key literally named
646
- > `connectorState` or `sdkState`, the auto-migration mis-reads it (the envelope is
647
- > detected by those key names). Rename such a key **before** upgrading.
648
-
649
- ## 9. HTTP / axios surface removed from the SDK
650
-
651
- The SDK no longer re-exports any axios surface from its public entry point:
652
-
653
- | v1 export | v2 |
654
- |-----------|----|
655
- | `axios` (raw instance) | **removed** — import `axios` in your connector |
656
- | `axiosClient` (retry-wrapped instance) | **removed from the public API** (still exists internally, but not exported and not at the old deep path) |
657
- | `formatAxiosError` | **removed** (deleted from source) |
658
- | `serializeAxiosError` | **removed from the public API** — use `serializeError` |
659
- | `HTTPResponse` | **removed** |
660
-
661
- `axios` and `axios-retry` are still runtime **dependencies** of the SDK (so
662
- they're available transitively), but you must construct your own client:
663
-
664
- ### Before (v1)
665
-
666
- ```ts
667
- import { axios, axiosClient } from '@devrev/ts-adaas';
668
-
669
- const res = await axiosClient.get(url, { responseType: 'stream' });
670
- ```
671
-
672
- ### After (v2)
673
-
674
- ```ts
675
- import axios from 'axios';
676
- import axiosRetry from 'axios-retry';
677
-
678
- const axiosClient = axios.create();
679
- axiosRetry(axiosClient, { retries: 5, retryDelay: axiosRetry.exponentialDelay });
680
-
681
- const res = await axiosClient.get(url, { responseType: 'stream' });
682
- ```
683
-
684
- Add `axios` and `axios-retry` to your `package.json` **dependencies** if missing —
685
- `axios-retry` in particular is usually not yet a direct dependency. The minimal
686
- client above does not replicate three behaviors of v1's public `axiosClient`:
687
- (1) its retry condition excluded 429s (letting your rate-limit handling see them);
688
- (2) it stripped the `Authorization` header from logged errors after max retries;
689
- (3) it used a **1000 ms** exponential-backoff base
690
- (`exponentialDelay(retryCount, error, 1000)` → ~2s, 4s, 8s, 16s, 32s), whereas
691
- `axios-retry`'s bare `exponentialDelay` defaults to a 100 ms factor (~0.2s … 3.2s)
692
- — 10× more aggressive. To match v1's backoff, pass the base explicitly:
693
-
694
- ```ts
695
- axiosRetry(axiosClient, {
696
- retries: 5,
697
- retryDelay: (retryCount, error) =>
698
- axiosRetry.exponentialDelay(retryCount, error, 1000),
699
- });
700
- ```
701
-
702
- Replicate any of these only if your connector relied on the SDK client's behavior.
703
-
704
- > If you previously deep-imported `@devrev/ts-adaas/dist/http/axios-client`, that
705
- > path **no longer exists** (the file was renamed to `http/client` and is
706
- > internal). Bring your own axios instance as above.
707
-
708
- For error logging, replace `formatAxiosError` / `serializeAxiosError` with the
709
- exported `serializeError`:
710
-
711
- ```ts
712
- import { serializeError } from '@devrev/airsync-sdk';
713
- console.error(serializeError(error));
714
- ```
715
-
716
- > **`serializeAxiosError` is conditional on how you used the result.**
717
- > `serializeError` returns a **string**; `serializeAxiosError` returned an
718
- > **object**. Where the result was used as a string (logging, concatenation), swap
719
- > to `serializeError`. Where it was **spread or property-accessed as an object**
720
- > (`{ ...serializeAxiosError(e) }`, `serializeAxiosError(e).message`), swapping
721
- > produces a TS2698 "spread types" error — instead keep the function via its deep
722
- > path (it still exists internally and still returns the object):
723
- > `import { serializeAxiosError } from '@devrev/airsync-sdk/dist/logger/logger';`
724
-
725
- ### `httpStream` type changed
726
-
727
- The connector-implemented attachment-stream function returns
728
- `ExternalSystemAttachmentStreamingResponse`, whose `httpStream` field changed from
729
- axios's `AxiosResponse` to the new public `HttpStreamResponse`
730
- (`{ data: any; headers: Record<string, any> }`). An axios stream response still
731
- satisfies it structurally, but if you annotated the stream with `AxiosResponse`
732
- imported *from the SDK*, switch to `HttpStreamResponse` (or import `AxiosResponse`
733
- from `axios` directly).
734
-
735
- > **`error.statusCode` dropped from the streaming response.** v1.20 briefly widened
736
- > `ExternalSystemAttachmentStreamingResponse.error` to
737
- > `ErrorRecord & { statusCode?: number }`; v2 narrows it back to a plain
738
- > `ErrorRecord`. The SDK never read `statusCode`, so this only affects a
739
- > `getAttachmentStream` that returned `{ error: { message, statusCode } }` — drop
740
- > the extra property (TS excess-property checking will flag it). Encode retry
741
- > timing via `{ delay }` instead.
742
-
743
- ## 10. `Mappers` methods return the unwrapped body
744
-
745
- `Mappers.getByTargetId` / `getByExternalId` / `create` / `update` changed their
746
- return type from `Promise<AxiosResponse<T>>` to `Promise<T>` — they now return the
747
- response **body** directly. Drop the `.data` access:
748
-
749
- ### Before (v1)
750
-
751
- ```ts
752
- const mapperResponse = await this.mappers.getByTargetId({ sync_unit, target });
753
- const resolvedId = mapperResponse.data.sync_mapper_record.external_ids[0];
754
- ```
755
-
756
- ### After (v2)
757
-
758
- ```ts
759
- const mapperResponse = await this.mappers.getByTargetId({ sync_unit, target });
760
- const resolvedId = mapperResponse.sync_mapper_record.external_ids[0];
761
- ```
762
-
763
- A **silent** change for code that reads `.data` (fails to type-check, or reads
764
- `undefined` if loosely typed). `Mappers` is now also exported from the package
765
- root, so you no longer need to deep-import it from `dist/mappers/mappers`.
766
-
767
- Three places must change **together**, or the failure is silent at runtime:
768
-
769
- 1. the read site (drop `.data`, above);
770
- 2. any hand-rolled structural param type that wrapped the body in `data?:`;
771
- 3. **mapper test doubles** — `jest.fn().mockResolvedValue({ data: {
772
- sync_mapper_record: ... } })` must become `mockResolvedValue({
773
- sync_mapper_record: ... })`, or the migrated source reads `undefined` and the
774
- test fails. These test files often import nothing from the SDK, so a
775
- specifier-based search misses them — grep for `sync_mapper_record`.
776
-
777
- > `mappers` lives on `LoadingAdapter` only (§5). Extraction-phase code that used
778
- > `adapter.mappers` must construct its own instance —
779
- > `new Mappers({ event: adapter.event })` (hoist it out of per-item loops).
780
-
781
- ## 11. Deleted legacy modules
782
-
783
- Everything under the v1 `deprecated/` tree is gone:
784
-
785
- | Removed | Replacement |
786
- |---------|-------------|
787
- | `Adapter`, `createAdapter` | `ExtractionAdapter` / `LoadingAdapter` + `processExtractionTask` / `processLoadingTask` |
788
- | `DemoExtractor` | — (reference implementation only) |
789
- | `HTTPClient`, `defaultResponse` | your own axios client (§9) |
790
- | deprecated `Uploader` | repos (`initializeRepos` / `getRepo` / `push`) |
791
-
792
- (The SDK has an internal `Uploader` class, but it was never part of the public API
793
- in either version — the *public* v1 `Uploader` was the deprecated one.)
794
-
795
- ## 12. Deleted deprecated enum members, types & enums
796
-
797
- The old/new duplicate enum members were collapsed; only the modern names remain.
798
- **The string values of the surviving members are byte-identical to v1**, so nothing
799
- changes on the wire — only the TypeScript member names.
800
-
801
- > ⚠️ The **deleted** `EventType` / `ExtractorEventType` members carried
802
- > *different, older* string values than their replacements (e.g. v1
803
- > `ExtractionDataStart = 'EXTRACTION_DATA_START'` vs the surviving
804
- > `StartExtractingData = 'START_EXTRACTING_DATA'`). The modern members already
805
- > existed in v1 with the modern values, so survivors are wire-compatible — but a
806
- > deleted member and its replacement did **not** share a value.
807
-
808
- **`EventType` (incoming):**
809
-
810
- | Deleted (v1 deprecated) | Use instead |
811
- |--------------------------|-------------|
812
- | `ExtractionExternalSyncUnitsStart` | `StartExtractingExternalSyncUnits` |
813
- | `ExtractionMetadataStart` | `StartExtractingMetadata` |
814
- | `ExtractionDataStart` | `StartExtractingData` |
815
- | `ExtractionDataContinue` | `ContinueExtractingData` |
816
- | `ExtractionDataDelete` | `StartDeletingExtractorState` |
817
- | `ExtractionAttachmentsStart` | `StartExtractingAttachments` |
818
- | `ExtractionAttachmentsContinue` | `ContinueExtractingAttachments` |
819
- | `ExtractionAttachmentsDelete` | `StartDeletingExtractorAttachmentsState` |
820
-
821
- **`ExtractorEventType` (outgoing):** the `Extraction*`-prefixed members
822
- (`ExtractionDataDone`, `ExtractionDataDelay`, `ExtractionAttachmentsProgress`, …)
823
- are deleted; use the `*Extraction*` members (`DataExtractionDone`,
824
- `DataExtractionDelayed`, `AttachmentExtractionProgress`, …). In practice you'll
825
- rarely reference `ExtractorEventType` at all in v2 — see §4.
826
-
827
- **`LoaderEventType`:** the duplicate members `DataLoadingDelay` and
828
- `AttachmentsLoading*` (plural) are deleted; use `DataLoadingDelayed` and
829
- `AttachmentLoading*` (singular). These duplicates shared their survivors' string
830
- value, so removing them is a pure source-name change.
831
-
832
- > **No more incoming-event-type translation.** v1 shipped an `event-type-translation`
833
- > module mapping legacy platform strings onto the modern enum members (and
834
- > translating outgoing types). v2 removed it entirely and passes incoming
835
- > `payload.event_type` through untouched. Any connector importing
836
- > `translateIncomingEventType` / `translateOutgoingEventType` /
837
- > `translateExtractorEventType` / `translateLoaderEventType` from the SDK will fail
838
- > to compile — drop them; the platform sends modern strings.
839
-
840
- ### Removed `UnknownEventType` members
841
-
842
- `UnknownEventType = 'UNKNOWN_EVENT_TYPE'` was declared on three enums in v1
843
- (`EventType`, `ExtractorEventType`, `LoaderEventType`). All three copies are
844
- **removed** in v2; the SDK's "unrecognized event" sentinel is now an internal,
845
- un-exported constant with the same string value. If you matched on any enum
846
- member, compare against the raw `'UNKNOWN_EVENT_TYPE'` string instead (the wire
847
- value is unchanged).
848
-
849
- ### Removed deprecated types & enums
850
-
851
- These were `@deprecated` in v1 and are **deleted from the public API** in v2. None
852
- are referenced by the modern worker contract; each row gives the replacement (or
853
- "no replacement" where the concept is gone):
854
-
855
- | Removed | Replacement |
856
- |---------|-------------|
857
- | `ExtractionMode` (enum) | `SyncMode` (adds `LOADING` alongside `INITIAL`/`INCREMENTAL`) |
858
- | `EventContextIn` (interface) | `EventContext` (the single, current event-context type) |
859
- | `EventContextOut` (interface) | `EventContext` |
860
- | `DomainObjectState` (interface) | — (no replacement; was an unused per-object state shape) |
861
- | `ErrorLevel` (enum) | — (logger uses its own internal log level) |
862
- | `LogRecord` (interface) | — (unused) |
863
- | `AdapterUpdateParams` (interface) | — (unused) |
864
- | `AdapterState<T>` (type alias) | your connector `State` for `adapter.state`; `adapter.sdkState` for SDK fields (§8) |
865
-
866
- Also removed from `EventData`: the deprecated `external_sync_units` field (§7) and
867
- the deprecated `progress` field (a no-op since v1 — the backend computes progress).
868
- The `artifacts` field on `EventData` is **kept** — it is how the SDK attaches
869
- uploaded repo artifacts (including external sync units) to the emitted event.
870
-
871
- ### Worker-thread IPC types no longer on the root barrel
872
-
873
- The worker↔main-thread plumbing types are no longer exported from the package root.
874
- They were never part of the connector-authoring surface — they describe the SDK's
875
- internal `parentPort` message protocol — but v1's barrel re-exported them, so a
876
- connector (usually a test simulating the protocol) could import them from
877
- `@devrev/ts-adaas`. Now removed from the root: `WorkerEvent`,
878
- `WorkerMessageSubject` (enums), `WorkerMessage`, `WorkerMessageEmitted`,
879
- `WorkerMessageExit`, `WorkerMessageLog`, `WorkerMessageFailed`, `WorkerData`,
880
- `GetWorkerPathInterface`. The declarations still exist internally (deep-importable
881
- via `@devrev/airsync-sdk/dist/types/workers`), but treat them as SDK-internal — a
882
- worker-protocol test should assert on the `TaskResult` your task returns (§4), not
883
- on raw IPC messages. (`WorkerMessageLog` also dropped its `isSdkLog` field.)
884
-
885
- > **Emitted logs dropped the `is_sdk_log` field (observability only).** v1 tagged
886
- > every log line with `is_sdk_log: true | false` (SDK vs connector origin), driven
887
- > by an `AsyncLocalStorage` log-context layer that v2 removed. v2's log JSON no
888
- > longer contains `is_sdk_log`. **No connector code changes** — not a compile or
889
- > runtime break — but any platform dashboard, monitor, or saved query filtering
890
- > logs on `@is_sdk_log` silently stops matching. Update those filters (drop the
891
- > facet, or distinguish origin another way) after connectors upgrade.
892
-
893
- ## 13. Deep-import paths that moved or broke
894
-
895
- The compiled `dist/` mirrors `src/` 1:1, so a deep import works only if the source
896
- file still lives at the same relative path. Status of the paths real connectors
897
- used:
898
-
899
- | Deep import | Status in v2 |
900
- |-------------|--------------|
901
- | `dist/http/axios-client` (`axiosClient`) | ❌ **broken** — file renamed to `http/client` and made internal. Bring your own axios (§9). |
902
- | `dist/state/state` (`State`, `createAdapterState`) | ❌ **both symbols removed** — the module now exports `BaseState`, `ExtractionState`/`createExtractionState`, `LoadingState`/`createLoadingState`. Map sync `new State(...)` → sync `new ExtractionState(...)`/`new LoadingState(...)`; async `createAdapterState(...)` → the async `create*State(...)` factory (§5). |
903
- | `dist/state/state.interfaces` (`ToDevRev`) | ⚠️ still resolves, but `ToDevRev` is SDK-internal now (§8) — **drop** the import, don't repoint it |
904
- | `dist/mappers/mappers.interface` (singular) | ❌ **broken** — file renamed to `mappers.interfaces` (plural). The four `*Params` interfaces and the two `SyncMapperRecord*` enums are on the root barrel (prefer root); the four `*Response` interfaces, `SyncMapperRecord`, `SyncMapperRecordExternalVersion`, `UpdateSyncMapperRecordParams`, and `MappersFactoryInterface` are **not** — repoint those to `dist/mappers/mappers.interfaces`. |
905
- | `dist/logger/logger` (`serializeAxiosError`) | ⚠️ resolves, symbol kept internally — see the conditional rule in §9 (keep the deep import for object use; root `serializeError` for string use) |
906
- | `dist/repo/repo.interfaces` (`Item`) | ✅ resolves — but `Item` is now on the root barrel, prefer the root import |
907
- | `dist/types/loading` (`ItemTypeToLoad`) | ✅ resolves — also now on the root barrel |
908
- | `dist/mappers/mappers` (`Mappers`) | ✅ resolves — also now on the root barrel (and note the return-type change, §10) |
909
- | `dist/types/extraction` (`InitialSyncScope`) | ✅ resolves — also on the root barrel |
910
-
911
- **Recommended:** replace all deep `dist/**` imports with root imports. If a symbol
912
- you need isn't on the root barrel, request it rather than deep-importing.
913
-
914
- ## 14. Edge regression: very old in-flight attachment syncs
915
-
916
- In v1 the SDK migrated the legacy `string[]` form of the processed-attachments
917
- dedup list (`lastProcessedAttachmentsIdsList`) to the current `{ id, parent_id }[]`
918
- form on read. v2 removed that conversion.
919
-
920
- The `string[]` form only exists in state written by SDK **< 1.15.2**. If an
921
- attachment-extraction phase started on a pre-1.15.2 SDK and is **still mid-flight**
922
- when the connector upgrades to v2, the v2 dedup check (`it.id === …`) won't match
923
- the bare-string entries, so attachments already downloaded in that sync get
924
- re-uploaded once. New syncs — and any sync started on ≥ 1.15.2 — are unaffected;
925
- the platform deduplicates downstream, so the only cost is the wasted
926
- re-download/upload on that one continuation.
927
-
928
- If this matters for your deployment, drain in-flight attachment syncs before
929
- upgrading.
930
-
931
- ## 15. `spawn`'s deprecated `workerPath` option removed
932
-
933
- `SpawnFactoryInterface.workerPath` was `@deprecated` in v1 and is **removed** in v2.
934
- Point `spawn` at your worker directory with `baseWorkerPath: __dirname` — the SDK
935
- resolves the per-event worker file from there (`workerPathOverrides` still works
936
- for custom paths).
937
-
938
- ### Before (v1)
939
-
940
- ```ts
941
- spawn({ event, initialState, workerPath: __dirname + '/workers/data-extraction' });
942
- ```
943
-
944
- ### After (v2)
945
-
946
- ```ts
947
- spawn({ event, initialState, baseWorkerPath: __dirname });
948
- ```
949
-
950
- > **Indirected `workerPath`** (a variable fed by a `switch`/dispatcher helper like
951
- > `getWorkerPerLoadingPhase(event)`): replace with `baseWorkerPath: __dirname`
952
- > **and delete the now-dead dispatcher function**, its locals, and any imports it
953
- > orphaned (`EventType`, …) — otherwise `noUnusedLocals`/lint fails.
954
- > `workerPathOverrides` is unrelated and kept.
955
-
956
- ## 16. Jest mocks & test assertions
957
-
958
- Tests are first-class migration targets — a source-only migration passes `tsc` but
959
- fails the test gate. SDK-module mocks hardcode the v1 shape.
960
-
961
- **Mechanical changes per test file:**
962
-
963
- - The module specifier in `jest.mock('...')` / `jest.requireActual('...')` /
964
- `moduleNameMapper` was already renamed in §1. In mock factories: `processTask` →
965
- `processExtractionTask` / `processLoadingTask` (matching the worker under test,
966
- §3), including the `X as jest.Mock` capture.
967
- - Drop `WorkerAdapter: {}` and `axiosClient: {}` keys from mock factories; remove
968
- vestigial `emit: jest.fn()` from mock adapters.
969
- - `AirdropEvent` → `AirSyncEvent` in fixtures/annotations (§2). Keep
970
- `jest.requireActual` only for enums/constants still referenced.
971
- - Propagate every state-field rename and `AdapterState` → your-State-type
972
- replacement into fixtures (§8) — fixtures are the most-missed target.
973
-
974
- **Semantic changes:**
975
-
976
- - `expect(adapter.emit).toHaveBeenCalledWith(EventType.X)` becomes an assertion on
977
- the awaited **return** of the captured task/onTimeout function:
978
-
979
- ```ts
980
- // v1
981
- await mockProcessTask.mock.calls[0][0].task({ adapter });
982
- expect(adapter.emit).toHaveBeenCalledWith(ExtractorEventType.DataExtractionDone);
983
- // v2
984
- const result = await mockProcessTask.mock.calls[0][0].task({ adapter });
985
- expect(result).toEqual({ status: 'success' });
986
- ```
987
-
988
- The same rewrite applies to tests that invoke a **helper directly** (the helper
989
- now returns a `TaskResult` per §4) — these lack the `mockProcessTask` capture
990
- shape and are easy to miss.
991
- - Expected delay objects use `delaySeconds`, not `delay`.
992
- - Pass-through loaders (§6): assert the task returns the
993
- `loadItemTypes`/`streamAttachments` mock's value.
994
- - Source that now does `new Mappers({ event })` (§10) needs a `Mappers` mock in the
995
- factory; source that moved to `import axios from 'axios'` (§9) needs
996
- `jest.mock('axios')` (plus `jest.mock('axios-retry')` if you built a retry
997
- client) instead of mocking the SDK's axios.
998
- - Unwrap mapper test doubles (§10). Hand-constructed adapters in integration tests
999
- follow §5.
1000
-
1001
- ---
1002
-
1003
- ## Migration checklist
1004
-
1005
- 1. `npm uninstall @devrev/ts-adaas && npm install @devrev/airsync-sdk@beta`; replace the import specifier everywhere — imports, `jest.mock`/`jest.requireActual`, `moduleNameMapper` (§1).
1006
- 2. Rename `AirdropEvent` → `AirSyncEvent`, `AirdropMessage` → `AirSyncMessage`. Drop any `CustomAirdropEvent` cast that only added `user_id`/`dev_oid`/`source_id`/`service_account_id` (§2).
1007
- 3. Split workers: extraction files use `processExtractionTask`, loading files use `processLoadingTask` (§3). Rewrite `ProcessTaskInterface<State>`/`TaskAdapterInterface<State>` annotations to take the adapter type (§3–4).
1008
- 4. Convert **every** `adapter.emit(...)` into a returned `TaskResult`; bubble outcomes up from helpers and class methods; `delay` → `delaySeconds` (§4).
1009
- 5. Replace `WorkerAdapter<T>` annotations with `ExtractionAdapter<T>` / `LoadingAdapter<T>`, and `new WorkerAdapter(...)`/`new State(...)`/`createAdapterState(...)` constructions per §5. Move any `mappers`/`reports`/`processedFiles` access into the loading path.
1010
- 6. Pass the `TaskResult` straight through from `loadItemTypes` / `loadAttachments` / `streamAttachments`; a connector that augmented `reports` pushes onto `adapter.reports` before returning (§6).
1011
- 7. ESU workers: push external sync units to the `EXTERNAL_SYNC_UNITS` repo; remove any `external_sync_units` (and `progress`) from emit data — those fields are gone from `EventData` (§7, §12).
1012
- 8. Remove SDK-owned fields from your `State` interface and `getInitialState`; apply the cursor decision rule for `lastSyncStarted`/`lastSuccessfulSyncStarted` — rename a connector-declared cursor, repoint an SDK-supplied read to `event_context.extract_from` (§8).
1013
- 9. Replace `axios` / `axiosClient` / `formatAxiosError` SDK imports with your own axios client + `serializeError`; apply the conditional `serializeAxiosError` rule (§9).
1014
- 10. Drop `.data` from `adapter.mappers.*` result reads — source, hand-rolled types, and test doubles together (§10).
1015
- 11. Remove usage of deleted legacy modules and event-type-translation helpers (§11, §12).
1016
- 12. Replace deleted enum members with their modern names; drop any use of the removed deprecated types (`ExtractionMode`, `EventContextIn`/`Out`, `DomainObjectState`, `ErrorLevel`, `LogRecord`, `AdapterUpdateParams`, `AdapterState` — replace an `AdapterState<S>` annotation with your own State type, don't just delete it) (§12).
1017
- 13. Replace deep `dist/**` imports with root imports; drop the now-internal `ToDevRev` import (§13).
1018
- 14. Drop each `onTimeout` that only emitted the phase-appropriate event — the SDK default covers it (progress for resumable phases, a timeout error for ESU / metadata / state-deletion). Keep one only when its v1 body did real work (cleanup, a custom error message), preserving the body (§4).
1019
- 15. Replace `spawn({ workerPath })` with `spawn({ baseWorkerPath: __dirname })`; delete any dead worker-path dispatcher (§15).
1020
- 16. Migrate jest mocks and assertions — emit-called assertions become return-value assertions (§16).
1021
-
1022
- ## A note on the early betas
1023
-
1024
- The betas `2.0.0-beta.0` through `2.0.0-beta.3` still re-exported `axios` /
1025
- `axiosClient`; they were removed in `2.0.0-beta.4` (and stay removed in GA — see §9).
1026
- If you migrated a connector against any beta before `beta.4` and imported either
1027
- symbol from `@devrev/airsync-sdk`, it will fail to compile once you move to
1028
- `beta.4`/GA — apply §9.