@devrev/airsync-sdk 2.0.0-beta.6 → 2.0.0-beta.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +8 -12
- package/dist/attachments-streaming/attachments-streaming-pool.d.ts.map +1 -1
- package/dist/attachments-streaming/attachments-streaming-pool.js +8 -9
- package/dist/common/constants.d.ts +0 -7
- package/dist/common/constants.d.ts.map +1 -1
- package/dist/common/constants.js +1 -28
- package/dist/common/helpers.d.ts.map +1 -1
- package/dist/common/helpers.js +1 -1
- package/dist/http/errors.d.ts +8 -0
- package/dist/http/errors.d.ts.map +1 -0
- package/dist/http/errors.js +31 -0
- package/dist/index.d.ts +18 -13
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +14 -11
- package/dist/logger/logger.d.ts.map +1 -1
- package/dist/logger/logger.interfaces.d.ts +2 -2
- package/dist/logger/logger.interfaces.d.ts.map +1 -1
- package/dist/mappers/mappers.d.ts +9 -3
- package/dist/mappers/mappers.d.ts.map +1 -1
- package/dist/mappers/mappers.interfaces.d.ts +0 -2
- package/dist/mappers/mappers.interfaces.d.ts.map +1 -1
- package/dist/mappers/mappers.js +33 -13
- package/dist/multithreading/adapters/base-adapter.d.ts +16 -35
- package/dist/multithreading/adapters/base-adapter.d.ts.map +1 -1
- package/dist/multithreading/adapters/base-adapter.js +89 -23
- package/dist/multithreading/adapters/extraction-adapter.d.ts +24 -20
- package/dist/multithreading/adapters/extraction-adapter.d.ts.map +1 -1
- package/dist/multithreading/adapters/extraction-adapter.js +175 -122
- package/dist/multithreading/adapters/loading-adapter.d.ts +13 -23
- package/dist/multithreading/adapters/loading-adapter.d.ts.map +1 -1
- package/dist/multithreading/adapters/loading-adapter.js +176 -301
- package/dist/multithreading/emit.d.ts.map +1 -1
- package/dist/multithreading/process-task.d.ts.map +1 -1
- package/dist/multithreading/process-task.js +24 -6
- package/dist/multithreading/spawn/spawn.d.ts +6 -1
- package/dist/multithreading/spawn/spawn.d.ts.map +1 -1
- package/dist/multithreading/spawn/spawn.helpers.d.ts.map +1 -1
- package/dist/multithreading/spawn/spawn.helpers.js +17 -52
- package/dist/multithreading/spawn/spawn.js +30 -26
- package/dist/multithreading/worker.js +15 -1
- package/dist/repo/repo.d.ts +4 -5
- package/dist/repo/repo.d.ts.map +1 -1
- package/dist/repo/repo.interfaces.d.ts +5 -4
- package/dist/repo/repo.interfaces.d.ts.map +1 -1
- package/dist/repo/repo.js +13 -15
- package/dist/state/base-state.d.ts +10 -5
- package/dist/state/base-state.d.ts.map +1 -1
- package/dist/state/base-state.js +39 -36
- package/dist/state/extraction-state.d.ts.map +1 -1
- package/dist/state/extraction-state.js +39 -4
- package/dist/state/state.interfaces.d.ts +13 -7
- package/dist/state/state.interfaces.d.ts.map +1 -1
- package/dist/state/state.interfaces.js +6 -8
- package/dist/state/time-value-resolver.d.ts.map +1 -1
- package/dist/state/time-value-resolver.js +3 -2
- package/dist/testing/mock-event.d.ts.map +1 -1
- package/dist/testing/mock-event.js +0 -8
- package/dist/testing/mock-server.d.ts +2 -0
- package/dist/testing/mock-server.d.ts.map +1 -1
- package/dist/testing/mock-server.interfaces.d.ts +6 -0
- package/dist/testing/mock-server.interfaces.d.ts.map +1 -1
- package/dist/testing/mock-server.js +37 -6
- package/dist/testing/test-adapters.d.ts +22 -0
- package/dist/testing/test-adapters.d.ts.map +1 -0
- package/dist/testing/test-adapters.js +32 -0
- package/dist/types/common.d.ts +4 -1
- package/dist/types/common.d.ts.map +1 -1
- package/dist/types/common.js +1 -1
- package/dist/types/errors.d.ts +13 -7
- package/dist/types/errors.d.ts.map +1 -1
- package/dist/types/errors.js +24 -0
- package/dist/types/extraction.d.ts +60 -44
- package/dist/types/extraction.d.ts.map +1 -1
- package/dist/types/extraction.js +10 -1
- package/dist/types/loading.d.ts +11 -30
- package/dist/types/loading.d.ts.map +1 -1
- package/dist/types/workers.d.ts +27 -23
- package/dist/types/workers.d.ts.map +1 -1
- package/dist/uploader/uploader.d.ts +1 -2
- package/dist/uploader/uploader.d.ts.map +1 -1
- package/dist/uploader/uploader.helpers.d.ts +3 -0
- package/dist/uploader/uploader.helpers.d.ts.map +1 -1
- package/dist/uploader/uploader.helpers.js +16 -0
- package/dist/uploader/uploader.interfaces.d.ts +4 -29
- package/dist/uploader/uploader.interfaces.d.ts.map +1 -1
- package/dist/uploader/uploader.js +4 -23
- package/docs/MIGRATION.md +665 -0
- package/package.json +24 -10
- package/MIGRATION.md +0 -1028
package/MIGRATION.md
DELETED
|
@@ -1,1028 +0,0 @@
|
|
|
1
|
-
# Migrating connectors from v1 (`@devrev/ts-adaas`) to v2 (`@devrev/airsync-sdk`)
|
|
2
|
-
|
|
3
|
-
Covers every connector-facing breaking change between `@devrev/ts-adaas` v1.x
|
|
4
|
-
and `@devrev/airsync-sdk` v2.0.0, with before/after examples from real
|
|
5
|
-
connectors. Self-contained: the numbered sections, applied **in order** (see
|
|
6
|
-
execution plan), are the complete migration — executable top to bottom without
|
|
7
|
-
any other reference. It ships in the npm package, so after installing v2 the
|
|
8
|
-
version-matched copy is at `node_modules/@devrev/airsync-sdk/MIGRATION.md`.
|
|
9
|
-
|
|
10
|
-
**The wire protocol is unchanged.** Event payloads, API routes, headers, the
|
|
11
|
-
artifact/upload flow, and every surviving event-type **string value** are
|
|
12
|
-
byte-for-byte identical to v1. Only the connector-facing **TypeScript API**
|
|
13
|
-
changed.
|
|
14
|
-
|
|
15
|
-
> **Why a hard break with no deprecation shims?** v2 is a single bundled-pain
|
|
16
|
-
> major: the package was renamed, so there are no existing consumers to keep
|
|
17
|
-
> backward-compatible. Everything changed at once in one guide.
|
|
18
|
-
|
|
19
|
-
---
|
|
20
|
-
|
|
21
|
-
## TL;DR — what breaks
|
|
22
|
-
|
|
23
|
-
| # | Change | Who it affects |
|
|
24
|
-
|---|--------|----------------|
|
|
25
|
-
| 1 | Package renamed `@devrev/ts-adaas` → `@devrev/airsync-sdk` | every import |
|
|
26
|
-
| 2 | `AirdropEvent` → `AirSyncEvent`, `AirdropMessage` → `AirSyncMessage` | type annotations (all connectors) |
|
|
27
|
-
| 3 | `processTask` → `processExtractionTask` / `processLoadingTask` | every worker file |
|
|
28
|
-
| 4 | `adapter.emit(...)` is gone — tasks **return** a `TaskResult` instead | every worker file |
|
|
29
|
-
| 5 | `WorkerAdapter` **class** removed → `ExtractionAdapter` / `LoadingAdapter` | helper signatures |
|
|
30
|
-
| 6 | `loadItemTypes` / `loadAttachments` / `streamAttachments` now **return** a `TaskResult` | loading + attachment workers |
|
|
31
|
-
| 7 | `EventData.external_sync_units` **removed** — external sync units must be pushed to a repo | ESU workers |
|
|
32
|
-
| 8 | `adapter.state` is connector-state only; SDK fields readable via `adapter.sdkState`. `lastSyncStarted` / `lastSuccessfulSyncStarted` **removed** | connectors reading/writing SDK bookkeeping |
|
|
33
|
-
| 9 | `axios`, `axiosClient`, `formatAxiosError`, `serializeAxiosError`, `HTTPResponse` no longer exported | anyone importing the SDK's HTTP/axios surface |
|
|
34
|
-
| 10 | `Mappers` methods return the unwrapped body (`Promise<T>`), not `Promise<AxiosResponse<T>>` | loaders calling `adapter.mappers.*` |
|
|
35
|
-
| 11 | Deprecated v1 modules deleted (`Adapter`, `createAdapter`, `DemoExtractor`, `HTTPClient`, `defaultResponse`, deprecated `Uploader`) | only legacy code |
|
|
36
|
-
| 12 | Deprecated event-type enum **members** deleted; deprecated types/enums (`ExtractionMode`, `EventContextIn`/`Out`, `DomainObjectState`, `ErrorLevel`, `LogRecord`, `AdapterUpdateParams`, `AdapterState`) **removed**; internal worker-IPC types (`WorkerEvent`, `WorkerMessageSubject`, `WorkerMessage*`, `WorkerData`, `GetWorkerPathInterface`) off the root barrel | only if you used the old members/types or the internal IPC types |
|
|
37
|
-
| 13 | Several deep `dist/**` import paths moved/removed | deep-importers |
|
|
38
|
-
| 14 | Legacy `string[]` attachment-dedup migration dropped | in-flight attachment syncs started on SDK **< 1.15.2** |
|
|
39
|
-
| 15 | `spawn`'s deprecated `workerPath` option **removed** — use `baseWorkerPath: __dirname` | connectors still passing `workerPath` |
|
|
40
|
-
| 16 | Jest mocks of the SDK hardcode the v1 shape; emit-called assertions become return-value assertions | every test suite mocking the SDK |
|
|
41
|
-
|
|
42
|
-
(Also: `EventData.progress` was removed — a no-op since v1; the backend computes
|
|
43
|
-
progress. Covered with the other `EventData` removals in §12.)
|
|
44
|
-
|
|
45
|
-
(Also: emitted logs no longer carry the `is_sdk_log` field — an
|
|
46
|
-
observability-only change that breaks no connector code; see §12.)
|
|
47
|
-
|
|
48
|
-
What does **not** change: `spawn(...)` and its surviving options (`initialState`,
|
|
49
|
-
`initialDomainMapping`, `workerPathOverrides`, `baseWorkerPath`,
|
|
50
|
-
`options.batchSize`, `timeout`, `isLocalDevelopment`, …) — the one exception is
|
|
51
|
-
the long-deprecated `workerPath`, removed (§15); default worker paths
|
|
52
|
-
(`/workers/data-extraction`, etc.); repos (`initializeRepos`, `getRepo`,
|
|
53
|
-
`push`); the normalization interfaces (`NormalizedItem`, `NormalizedAttachment`,
|
|
54
|
-
`RepoInterface`, `ExternalSyncUnit`); `installInitialDomainMapping`;
|
|
55
|
-
`createMockEvent` / `MockServer`; HTTP retry behavior;
|
|
56
|
-
`event_context.extract_from` / `extract_to`; and every surviving event-type
|
|
57
|
-
string value on the wire.
|
|
58
|
-
|
|
59
|
-
### Execution plan (for migrating a whole connector in one pass)
|
|
60
|
-
|
|
61
|
-
Apply sections **in numbered order** — later ones assume earlier ones ran (e.g.
|
|
62
|
-
§3–§6 rewrite emit sites that §5's adapter types must already annotate). For each
|
|
63
|
-
section: search the whole connector — source **and** tests — apply every hit,
|
|
64
|
-
then move on. Verify at the end, not per-section (a half-applied section never
|
|
65
|
-
compiles). Two search rules:
|
|
66
|
-
|
|
67
|
-
- **After §1, the old specifier is gone.** Every later search must target the new
|
|
68
|
-
`@devrev/airsync-sdk` specifier or a bare symbol name (`\bWorkerAdapter\b`,
|
|
69
|
-
`\baxiosClient\b`, `\.emit\(`, `lastSuccessfulSyncStarted`, …) — searching
|
|
70
|
-
`@devrev/ts-adaas` after §1 finds nothing and makes a section look like a false
|
|
71
|
-
no-op.
|
|
72
|
-
- **A section with zero hits is a verified no-op** — confirm and move on; many
|
|
73
|
-
connectors already use `baseWorkerPath`, push ESUs to a repo, and have no deep
|
|
74
|
-
imports.
|
|
75
|
-
- **Don't narrate the migration in code.** The `// v1` / `// v2` labels here are
|
|
76
|
-
reading aids — never copy them into the connector. Apply each edit cleanly, no
|
|
77
|
-
comment explaining what changed. Add a comment only where the resulting logic
|
|
78
|
-
is genuinely non-obvious independent of this migration.
|
|
79
|
-
|
|
80
|
-
Final verification, from the connector's package dir: `npm install` (regenerates
|
|
81
|
-
the lockfile), `npx tsc --noEmit`, `npm run lint` (unused imports/locals orphaned
|
|
82
|
-
by rewrites fail here — clean them up), `npm run build`, and the repo's test
|
|
83
|
-
command if tests exist. A leftover `adapter.emit`, a removed symbol, a missing
|
|
84
|
-
`delaySeconds`, a destructured `{ reports }`, or an un-renamed test fixture is
|
|
85
|
-
the usual failure cause.
|
|
86
|
-
|
|
87
|
-
---
|
|
88
|
-
|
|
89
|
-
## 1. Package rename
|
|
90
|
-
|
|
91
|
-
```bash
|
|
92
|
-
npm uninstall @devrev/ts-adaas
|
|
93
|
-
npm install @devrev/airsync-sdk@beta
|
|
94
|
-
```
|
|
95
|
-
|
|
96
|
-
> Until GA ships, install via the **`beta` dist-tag** — it always resolves to the
|
|
97
|
-
> newest published 2.x beta. A plain `npm install @devrev/airsync-sdk` may resolve
|
|
98
|
-
> an older beta, and `@2.0.0` does not exist yet and 404s. Once GA is published,
|
|
99
|
-
> plain install (`@latest`) is correct.
|
|
100
|
-
|
|
101
|
-
> **Start from the latest stable v1.** This guide describes the delta from the
|
|
102
|
-
> latest stable `@devrev/ts-adaas` (1.x). If your connector is on an older 1.x
|
|
103
|
-
> that skipped an intra-v1 migration, do that first so the examples match your
|
|
104
|
-
> code. Check **https://github.com/devrev/adaas-sdk/releases** for any release
|
|
105
|
-
> between your version and latest whose notes call out a migration or breaking
|
|
106
|
-
> change (the last such were **1.18.0**, **1.17.0**, **1.16.0**; **1.19.0 →
|
|
107
|
-
> 1.20.0** were migration-free). Bring the connector to a green latest-stable v1
|
|
108
|
-
> build before starting.
|
|
109
|
-
|
|
110
|
-
Then global-replace the import specifier **everywhere the string appears**: every
|
|
111
|
-
import, every `jest.mock('...')` / `jest.requireActual('...')` first argument, and
|
|
112
|
-
any jest `moduleNameMapper`. This also rewrites deep-import path prefixes
|
|
113
|
-
(`@devrev/ts-adaas/dist/x` → `@devrev/airsync-sdk/dist/x`); §13 handles the paths
|
|
114
|
-
that moved. Do not rename any symbols in this step.
|
|
115
|
-
|
|
116
|
-
```ts
|
|
117
|
-
// v1
|
|
118
|
-
import { spawn, EventType } from '@devrev/ts-adaas';
|
|
119
|
-
// v2
|
|
120
|
-
import { spawn, EventType } from '@devrev/airsync-sdk';
|
|
121
|
-
```
|
|
122
|
-
|
|
123
|
-
> **Deep imports** like `@devrev/ts-adaas/dist/...` are fragile — several paths
|
|
124
|
-
> moved or were removed in v2 (§13). Prefer root imports; the v2 barrel now exports
|
|
125
|
-
> several symbols that previously required a deep import (`Mappers`, `Item`,
|
|
126
|
-
> `ItemTypeToLoad`).
|
|
127
|
-
|
|
128
|
-
## 2. Type renames: `AirdropEvent` → `AirSyncEvent`
|
|
129
|
-
|
|
130
|
-
Hard rename, no compatibility alias. The payload **shape** is identical — only the
|
|
131
|
-
type name changed.
|
|
132
|
-
|
|
133
|
-
| v1 | v2 |
|
|
134
|
-
|----|----|
|
|
135
|
-
| `AirdropEvent` | `AirSyncEvent` |
|
|
136
|
-
| `AirdropMessage` | `AirSyncMessage` |
|
|
137
|
-
|
|
138
|
-
E.g. `(events: AirdropEvent[])` → `(events: AirSyncEvent[])`.
|
|
139
|
-
|
|
140
|
-
No other public type was renamed — `ConnectionData`, `EventContext`, `EventData`,
|
|
141
|
-
`ExtractorEvent`, and `ExternalSyncUnit` keep their v1 names. Platform-owned
|
|
142
|
-
strings (`/internal/airdrop.*` routes, the `'ADaaS'` external system type, the
|
|
143
|
-
`adaas_library_version` metadata key, `airdrop_*` mapping enum values) are
|
|
144
|
-
intentionally unchanged.
|
|
145
|
-
|
|
146
|
-
### `AirSyncEvent.context` gained identity fields
|
|
147
|
-
|
|
148
|
-
`AirSyncEvent.context` now declares the identity fields the platform already
|
|
149
|
-
sends, in addition to `secrets`, `snap_in_id`, `snap_in_version_id`:
|
|
150
|
-
|
|
151
|
-
```ts
|
|
152
|
-
// v2 — AirSyncEvent.context
|
|
153
|
-
context: {
|
|
154
|
-
secrets: { service_account_token: string };
|
|
155
|
-
snap_in_version_id: string;
|
|
156
|
-
snap_in_id: string;
|
|
157
|
-
user_id: string; // new
|
|
158
|
-
dev_oid: string; // new
|
|
159
|
-
source_id: string; // new
|
|
160
|
-
service_account_id: string; // new
|
|
161
|
-
};
|
|
162
|
-
```
|
|
163
|
-
|
|
164
|
-
If you previously extended the event to read these (e.g. a hand-rolled
|
|
165
|
-
`CustomAirdropEvent` that added `user_id`), drop the extension and read
|
|
166
|
-
`adapter.event.context.user_id` directly. (`snap_in_id` was already in v1; only
|
|
167
|
-
the four fields above are new.)
|
|
168
|
-
|
|
169
|
-
> Note: these four fields live on the **top-level** `AirSyncEvent.context`, not on
|
|
170
|
-
> the `EventContext` inside `payload.event_context` (a different, unchanged
|
|
171
|
-
> object).
|
|
172
|
-
|
|
173
|
-
## 3 + 4. The new worker contract: **return a `TaskResult`** instead of emitting
|
|
174
|
-
|
|
175
|
-
The core change of v2. In v1 the connector decided *which event* to emit and
|
|
176
|
-
called `adapter.emit(...)`. In v2 it only reports *how the phase ended* by
|
|
177
|
-
**returning** a `TaskResult`; the SDK maps it to the correct platform event for
|
|
178
|
-
the current phase and emits it exactly once.
|
|
179
|
-
|
|
180
|
-
```ts
|
|
181
|
-
// the exact union (exported from @devrev/airsync-sdk)
|
|
182
|
-
export type TaskResult =
|
|
183
|
-
| { status: 'success' }
|
|
184
|
-
| { status: 'progress' }
|
|
185
|
-
| { status: 'delay'; delaySeconds: number } // note: delaySeconds, not delay
|
|
186
|
-
| { status: 'error'; error: ErrorRecord }; // ErrorRecord = { message: string }
|
|
187
|
-
```
|
|
188
|
-
|
|
189
|
-
`processTask` is split into two typed entry points — pick the one matching the
|
|
190
|
-
worker's phase. Phase is per-file: workers under `extraction/` (or whose body uses
|
|
191
|
-
`initializeRepos`/`getRepo`/`streamAttachments`) are extraction; workers under
|
|
192
|
-
`loading/` (or using `loadItemTypes`/`loadAttachments`/`mappers`) are loading.
|
|
193
|
-
|
|
194
|
-
### Before (v1)
|
|
195
|
-
|
|
196
|
-
```ts
|
|
197
|
-
import { processTask, ExtractorEventType } from '@devrev/ts-adaas';
|
|
198
|
-
|
|
199
|
-
processTask({
|
|
200
|
-
task: async ({ adapter }) => {
|
|
201
|
-
// ... extract ...
|
|
202
|
-
await adapter.emit(ExtractorEventType.DataExtractionDone);
|
|
203
|
-
},
|
|
204
|
-
onTimeout: async ({ adapter }) => {
|
|
205
|
-
await adapter.postState();
|
|
206
|
-
await adapter.emit(ExtractorEventType.DataExtractionProgress, { progress: 50 });
|
|
207
|
-
},
|
|
208
|
-
});
|
|
209
|
-
```
|
|
210
|
-
|
|
211
|
-
### After (v2)
|
|
212
|
-
|
|
213
|
-
```ts
|
|
214
|
-
import { processExtractionTask } from '@devrev/airsync-sdk';
|
|
215
|
-
|
|
216
|
-
processExtractionTask({
|
|
217
|
-
task: async ({ adapter }) => {
|
|
218
|
-
// ... extract ...
|
|
219
|
-
return { status: 'success' };
|
|
220
|
-
},
|
|
221
|
-
// onTimeout can be omitted entirely for resumable phases:
|
|
222
|
-
// the SDK emits a progress (continuation) result by default.
|
|
223
|
-
});
|
|
224
|
-
```
|
|
225
|
-
|
|
226
|
-
Loading workers use `processLoadingTask` the same way.
|
|
227
|
-
|
|
228
|
-
> `adapter.emit(...)` is now **`protected`** — calling `adapter.emit(...)` or
|
|
229
|
-
> `this.adapter.emit(...)` is a hard **compile error** in v2, not a deprecation.
|
|
230
|
-
> Every emit site must be converted.
|
|
231
|
-
|
|
232
|
-
### `emit()` → `return` translation table
|
|
233
|
-
|
|
234
|
-
| v1 emit call | v2 return |
|
|
235
|
-
|--------------|-----------|
|
|
236
|
-
| `await adapter.emit(XxxDone)` | `return { status: 'success' }` |
|
|
237
|
-
| `await adapter.emit(XxxProgress, { progress })` | `return { status: 'progress' }` |
|
|
238
|
-
| `await adapter.emit(XxxDelayed, { delay })` | `return { status: 'delay', delaySeconds: delay }` |
|
|
239
|
-
| `await adapter.emit(XxxError, { error })` | `return { status: 'error', error }` |
|
|
240
|
-
| `await adapter.emit(ExternalSyncUnitExtractionDone, { external_sync_units })` | push to repo + `return { status: 'success' }` — see §7 |
|
|
241
|
-
|
|
242
|
-
`progress` (`{ progress: n }`) carried no semantic value in v1 beyond "not done
|
|
243
|
-
yet" and is dropped; the platform tracks progress itself.
|
|
244
|
-
|
|
245
|
-
> **A `Done` emit that carried a non-fatal error summary** (v1
|
|
246
|
-
> `emit(XxxDone, { error })` — "finished, but here's what went wrong") maps to
|
|
247
|
-
> `return { status: 'success' }` plus surfacing the error some other way (a report
|
|
248
|
-
> entry or `console.warn`) — **not** to `{ status: 'error' }`, which would emit
|
|
249
|
-
> `*Error` and flip a successful phase to failed.
|
|
250
|
-
|
|
251
|
-
> **`ProcessTaskInterface` / `TaskAdapterInterface` changed their generic.** Both
|
|
252
|
-
> survive, but the type parameter now names the **adapter**, not the connector
|
|
253
|
-
> state: v1 `ProcessTaskInterface<State>` / `TaskAdapterInterface<State>` → v2
|
|
254
|
-
> `ProcessTaskInterface<ExtractionAdapter<State>>` (or `LoadingAdapter<State>`),
|
|
255
|
-
> same for `TaskAdapterInterface`. A bare `<State>` compiles in loosely-typed
|
|
256
|
-
> spots but silently types `adapter` as the connector state. `onTimeout` is now
|
|
257
|
-
> optional, and both callbacks return `Promise<TaskResult>` not `Promise<void>`.
|
|
258
|
-
|
|
259
|
-
### Status → emitted event, per phase
|
|
260
|
-
|
|
261
|
-
The SDK picks the platform event from the **current phase** and the returned
|
|
262
|
-
status:
|
|
263
|
-
|
|
264
|
-
| status | Resumable phases — data/attachment **extraction**, data/attachment **loading** | Non-resumable — external sync units, metadata, state deletion |
|
|
265
|
-
|--------|------------------------------------------------------------------------------|---------------------------------------------------------------|
|
|
266
|
-
| `'success'` | `*Done` | `*Done` |
|
|
267
|
-
| `'progress'` | `*Progress` (continuation) | **`*Error`** (illegal for these phases; emitted with a generated message) |
|
|
268
|
-
| `'delay'` | `*Delayed` (with `delaySeconds`) | **`*Error`** (illegal) |
|
|
269
|
-
| `'error'` | `*Error` (with the error record) | `*Error` |
|
|
270
|
-
|
|
271
|
-
### Emits buried inside helper functions
|
|
272
|
-
|
|
273
|
-
A common v1 pattern: emit deep inside a helper, return a boolean telling the
|
|
274
|
-
caller to stop. Since only the **task's return value** reaches the SDK in v2, the
|
|
275
|
-
helper must **bubble the outcome up**.
|
|
276
|
-
|
|
277
|
-
A clean way (used by the migrated google-drive connector): store the terminal
|
|
278
|
-
result on a field and return it once the loop unwinds:
|
|
279
|
-
|
|
280
|
-
```ts
|
|
281
|
-
// v1 — helper emits, returns false to abort
|
|
282
|
-
private async handlePermissionDeniedError(error: unknown) {
|
|
283
|
-
await this.adapter.emit(ExtractorEventType.DataExtractionError, {
|
|
284
|
-
error: { message: '...' },
|
|
285
|
-
});
|
|
286
|
-
}
|
|
287
|
-
```
|
|
288
|
-
|
|
289
|
-
```ts
|
|
290
|
-
// v2 — helper records the result; the task returns it
|
|
291
|
-
private result: TaskResult | undefined;
|
|
292
|
-
|
|
293
|
-
private handlePermissionDeniedError(error: unknown) {
|
|
294
|
-
this.result = { status: 'error', error: { message: '...' } };
|
|
295
|
-
}
|
|
296
|
-
|
|
297
|
-
// ... and where the loop ends:
|
|
298
|
-
private finalResult(): TaskResult {
|
|
299
|
-
// a stored error/delay set by a helper, else progress (continuation)
|
|
300
|
-
return this.result ?? { status: 'progress' };
|
|
301
|
-
}
|
|
302
|
-
```
|
|
303
|
-
|
|
304
|
-
For a simple "stop iterating" signal, return the `TaskResult` directly up the call
|
|
305
|
-
chain:
|
|
306
|
-
|
|
307
|
-
```ts
|
|
308
|
-
// v2
|
|
309
|
-
async function extractList(adapter: ExtractionAdapter<State>): Promise<TaskResult | null> {
|
|
310
|
-
if (rateLimited) return { status: 'delay', delaySeconds: retryAfter };
|
|
311
|
-
// ...
|
|
312
|
-
return null; // keep going
|
|
313
|
-
}
|
|
314
|
-
|
|
315
|
-
processExtractionTask<State>({
|
|
316
|
-
task: async ({ adapter }) => {
|
|
317
|
-
for (const itemType of itemTypes) {
|
|
318
|
-
const stop = await extractList(adapter);
|
|
319
|
-
if (stop) return stop;
|
|
320
|
-
}
|
|
321
|
-
return { status: 'success' };
|
|
322
|
-
},
|
|
323
|
-
});
|
|
324
|
-
```
|
|
325
|
-
|
|
326
|
-
### Timeout handling
|
|
327
|
-
|
|
328
|
-
Checking `adapter.isTimeout` in your extraction loop still works as in v1 — but
|
|
329
|
-
instead of emitting progress and exiting, **return** progress:
|
|
330
|
-
|
|
331
|
-
```ts
|
|
332
|
-
if (adapter.isTimeout) {
|
|
333
|
-
return { status: 'progress' }; // platform sends CONTINUE_* next
|
|
334
|
-
}
|
|
335
|
-
```
|
|
336
|
-
|
|
337
|
-
Two behaviors matter when migrating:
|
|
338
|
-
|
|
339
|
-
- **The timeout outcome always wins.** Once the soft timeout fires, the SDK emits
|
|
340
|
-
the `onTimeout` result (or its default) and **ignores whatever the task
|
|
341
|
-
returned** — a phase that ran out of time must hand off for continuation, not
|
|
342
|
-
report itself complete.
|
|
343
|
-
- **Omit `onTimeout` unless it does real work.** Its default is phase-aware:
|
|
344
|
-
`progress` (continuation) for resumable phases, a timeout **error** for
|
|
345
|
-
non-resumable ones (external sync units, metadata, state deletion) where
|
|
346
|
-
continuation is impossible. So a v1 `onTimeout` that only emitted the
|
|
347
|
-
phase-appropriate event (optionally after `postState()`) is now redundant —
|
|
348
|
-
**delete it**. Keep an explicit `onTimeout` only when its v1 body did real work
|
|
349
|
-
that must survive a timeout (cancelling rate limiting, clearing upload
|
|
350
|
-
bookkeeping, a custom error message) — preserve that body, replacing the
|
|
351
|
-
trailing emit with a returned `TaskResult`:
|
|
352
|
-
|
|
353
|
-
```ts
|
|
354
|
-
onTimeout: async () => ({
|
|
355
|
-
status: 'error',
|
|
356
|
-
error: { message: 'Custom timeout message.' },
|
|
357
|
-
}),
|
|
358
|
-
```
|
|
359
|
-
|
|
360
|
-
> Do **not** call `process.exit()` yourself in v2 — the SDK owns the single worker
|
|
361
|
-
> exit after it emits your `TaskResult`. Likewise drop `adapter.postState()` inside
|
|
362
|
-
> `onTimeout`; the SDK persists state around the timeout emit.
|
|
363
|
-
|
|
364
|
-
## 5. `WorkerAdapter` → `ExtractionAdapter` / `LoadingAdapter`
|
|
365
|
-
|
|
366
|
-
The `WorkerAdapter` **class** is gone. Replace the type annotation on your helpers
|
|
367
|
-
with the mode-specific adapter:
|
|
368
|
-
|
|
369
|
-
```ts
|
|
370
|
-
// v1
|
|
371
|
-
async function extractList(adapter: WorkerAdapter<State>) { ... }
|
|
372
|
-
// v2
|
|
373
|
-
import { ExtractionAdapter } from '@devrev/airsync-sdk';
|
|
374
|
-
async function extractList(adapter: ExtractionAdapter<State>) { ... }
|
|
375
|
-
```
|
|
376
|
-
|
|
377
|
-
| Surface | Lives on |
|
|
378
|
-
|---------|----------|
|
|
379
|
-
| `initializeRepos`, `getRepo`, `streamAttachments`, `shouldExtract`, `artifacts` | `ExtractionAdapter` |
|
|
380
|
-
| `loadItemTypes`, `loadAttachments`, `mappers`, `reports`, `processedFiles` | `LoadingAdapter` |
|
|
381
|
-
| `event`, `state`, `sdkState`, `postState`, `isTimeout`, `extractionScope` | both (shared `BaseAdapter`) |
|
|
382
|
-
|
|
383
|
-
> Only the **class** was removed. The **types** `WorkerAdapterInterface` and
|
|
384
|
-
> `WorkerAdapterOptions` still exist and are still exported — don't blindly rename
|
|
385
|
-
> every `WorkerAdapter` token; replace the `WorkerAdapter<T>` annotations and
|
|
386
|
-
> `new WorkerAdapter(...)` constructions only.
|
|
387
|
-
|
|
388
|
-
### Hand-constructed adapters (integration tests)
|
|
389
|
-
|
|
390
|
-
Some connectors construct the adapter directly in integration tests
|
|
391
|
-
(`new WorkerAdapter({ event, adapterState })`). The same construction works with
|
|
392
|
-
the phase-specific classes — but `adapterState` must be a v2 state class, which
|
|
393
|
-
replaced the v1 `State` class / `createAdapterState` factory (both removed; §13
|
|
394
|
-
for the import path):
|
|
395
|
-
|
|
396
|
-
```ts
|
|
397
|
-
// v1
|
|
398
|
-
const adapterState = new State({ event, initialState });
|
|
399
|
-
const adapter = new WorkerAdapter({ event, adapterState });
|
|
400
|
-
|
|
401
|
-
// v2 — same shape, phase-specific classes
|
|
402
|
-
import { LoadingState } from '@devrev/airsync-sdk/dist/state/state';
|
|
403
|
-
const adapterState = new LoadingState({ event, initialState });
|
|
404
|
-
const adapter = new LoadingAdapter({ event, adapterState });
|
|
405
|
-
```
|
|
406
|
-
|
|
407
|
-
Sync `new State(...)` maps to sync `new ExtractionState(...)` /
|
|
408
|
-
`new LoadingState(...)` (same `{ event, initialState, initialDomainMapping?,
|
|
409
|
-
options? }` params). The async `createAdapterState(...)` (fetched persisted state)
|
|
410
|
-
maps to the async `createExtractionState(...)` / `createLoadingState(...)`
|
|
411
|
-
factories at the same import path. A test that poked SDK fields via
|
|
412
|
-
`adapterState.state = { fromDevRev: ... }` must move them onto the `sdkState`
|
|
413
|
-
envelope (§8).
|
|
414
|
-
|
|
415
|
-
> **`mappers` / `reports` / `processedFiles` moved to `LoadingAdapter` only.** In
|
|
416
|
-
> v1 they were on the single `WorkerAdapter`, reachable in any phase. In v2 they
|
|
417
|
-
> are not on `ExtractionAdapter`. Code that touched `adapter.mappers` (etc.)
|
|
418
|
-
> during an extraction phase must move to the loading path.
|
|
419
|
-
|
|
420
|
-
## 6. Loading & attachment methods return a `TaskResult`
|
|
421
|
-
|
|
422
|
-
`loadItemTypes`, `loadAttachments`, and `streamAttachments` no longer emit or exit
|
|
423
|
-
mid-flight — they **return** a `TaskResult` you pass straight through. Rate limits
|
|
424
|
-
(→ `delay`), timeouts (→ `progress`), errors (→ `error`), completion (→ `success`)
|
|
425
|
-
are all encoded in the result; `reports` / `processed_files` (loading) and
|
|
426
|
-
artifacts (extraction) are attached to the emitted event automatically.
|
|
427
|
-
|
|
428
|
-
### Loading — before (v1)
|
|
429
|
-
|
|
430
|
-
```ts
|
|
431
|
-
import { LoaderEventType, processTask } from '@devrev/ts-adaas';
|
|
432
|
-
|
|
433
|
-
processTask({
|
|
434
|
-
task: async ({ adapter }) => {
|
|
435
|
-
await adapter.loadItemTypes({ itemTypesToLoad });
|
|
436
|
-
await adapter.emit(LoaderEventType.DataLoadingDone);
|
|
437
|
-
},
|
|
438
|
-
onTimeout: async ({ adapter }) => {
|
|
439
|
-
await adapter.postState();
|
|
440
|
-
await adapter.emit(LoaderEventType.DataLoadingProgress);
|
|
441
|
-
},
|
|
442
|
-
});
|
|
443
|
-
```
|
|
444
|
-
|
|
445
|
-
### Loading — after (v2)
|
|
446
|
-
|
|
447
|
-
```ts
|
|
448
|
-
import { processLoadingTask } from '@devrev/airsync-sdk';
|
|
449
|
-
|
|
450
|
-
processLoadingTask({
|
|
451
|
-
task: async ({ adapter }) => {
|
|
452
|
-
return adapter.loadItemTypes({ itemTypesToLoad });
|
|
453
|
-
},
|
|
454
|
-
});
|
|
455
|
-
```
|
|
456
|
-
|
|
457
|
-
### Attachment streaming — before (v1)
|
|
458
|
-
|
|
459
|
-
```ts
|
|
460
|
-
const response = await adapter.streamAttachments({ stream: getFileStream, batchSize: 50 });
|
|
461
|
-
if (response?.delay) {
|
|
462
|
-
await adapter.emit(ExtractorEventType.AttachmentExtractionDelayed, { delay: response.delay });
|
|
463
|
-
} else if (response?.error) {
|
|
464
|
-
await adapter.emit(ExtractorEventType.AttachmentExtractionError, { error: response.error });
|
|
465
|
-
} else {
|
|
466
|
-
await adapter.emit(ExtractorEventType.AttachmentExtractionDone);
|
|
467
|
-
}
|
|
468
|
-
```
|
|
469
|
-
|
|
470
|
-
### Attachment streaming — after (v2)
|
|
471
|
-
|
|
472
|
-
```ts
|
|
473
|
-
return adapter.streamAttachments({ stream: getFileStream, batchSize: 50 });
|
|
474
|
-
```
|
|
475
|
-
|
|
476
|
-
> **Never destructure the returned `TaskResult`.** The union has **no** `reports` /
|
|
477
|
-
> `processed_files` members, so v1 code like
|
|
478
|
-
> `const { reports, processed_files } = await adapter.loadAttachments(...)` — or
|
|
479
|
-
> pushing a synthetic report into the returned array — is a compile error. A
|
|
480
|
-
> connector that augmented the reports pushes onto the **live getter before**
|
|
481
|
-
> returning instead:
|
|
482
|
-
>
|
|
483
|
-
> ```ts
|
|
484
|
-
> adapter.reports.push(buildNotesReport(...));
|
|
485
|
-
> return await adapter.loadAttachments({ create });
|
|
486
|
-
> ```
|
|
487
|
-
>
|
|
488
|
-
> A defensive outer try/catch with bespoke rate-limit handling can stay:
|
|
489
|
-
> `return await adapter.loadItemTypes(...)` in the `try`, map escaped throws to
|
|
490
|
-
> `{ status: 'delay' | 'error' }` in the `catch`.
|
|
491
|
-
|
|
492
|
-
Custom attachment processors (reducer/iterator) keep the same call signatures;
|
|
493
|
-
only their `adapter` parameter type changes from `WorkerAdapter<C>` to
|
|
494
|
-
`ExtractionAdapter<C>` (§5). The `getAttachmentStream` function you implement still
|
|
495
|
-
returns `{ httpStream }` / `{ delay }` / `{ error }` — but see §9 for the
|
|
496
|
-
`httpStream` type change.
|
|
497
|
-
|
|
498
|
-
## 7. External sync units go through a repo
|
|
499
|
-
|
|
500
|
-
In v1 the SDK accepted `external_sync_units` in emit data and uploaded them
|
|
501
|
-
internally. With emit gone, push them to the `EXTERNAL_SYNC_UNITS` repo yourself.
|
|
502
|
-
The `EventData.external_sync_units` field — deprecated in v1 — is **removed
|
|
503
|
-
entirely** in v2: there is no inline ESU path, and any code still referencing
|
|
504
|
-
`external_sync_units` in emit data is now a compile error. External sync units
|
|
505
|
-
leave the worker only as repo artifacts.
|
|
506
|
-
|
|
507
|
-
### Before (v1)
|
|
508
|
-
|
|
509
|
-
```ts
|
|
510
|
-
await adapter.emit(ExtractorEventType.ExternalSyncUnitExtractionDone, {
|
|
511
|
-
external_sync_units: externalSyncUnits,
|
|
512
|
-
});
|
|
513
|
-
```
|
|
514
|
-
|
|
515
|
-
### After (v2)
|
|
516
|
-
|
|
517
|
-
```ts
|
|
518
|
-
import { AirSyncDefaultItemTypes, processExtractionTask } from '@devrev/airsync-sdk';
|
|
519
|
-
|
|
520
|
-
adapter.initializeRepos([
|
|
521
|
-
{
|
|
522
|
-
itemType: AirSyncDefaultItemTypes.EXTERNAL_SYNC_UNITS,
|
|
523
|
-
// mirror the batching the v1 SDK used internally for ESUs
|
|
524
|
-
overridenOptions: { batchSize: 25000, skipConfirmation: true },
|
|
525
|
-
},
|
|
526
|
-
]);
|
|
527
|
-
await adapter.getRepo(AirSyncDefaultItemTypes.EXTERNAL_SYNC_UNITS)?.push(externalSyncUnits);
|
|
528
|
-
|
|
529
|
-
return { status: 'success' };
|
|
530
|
-
```
|
|
531
|
-
|
|
532
|
-
The repo is uploaded automatically before the `Done` event is emitted.
|
|
533
|
-
|
|
534
|
-
> If your connector already pushed ESUs to a repo in v1 (some do), this section is
|
|
535
|
-
> a no-op — just convert the trailing emit to `return`.
|
|
536
|
-
|
|
537
|
-
## 8. State split: `adapter.state` vs `adapter.sdkState`
|
|
538
|
-
|
|
539
|
-
In v1, connector state and SDK bookkeeping lived in one flat blob and
|
|
540
|
-
`adapter.state` exposed both. In v2:
|
|
541
|
-
|
|
542
|
-
- **`adapter.state` is connector state only** — exactly the shape of the
|
|
543
|
-
`initialState` you pass to `spawn`. Getter and setter both survive, so in-place
|
|
544
|
-
reads/writes work unchanged:
|
|
545
|
-
|
|
546
|
-
```ts
|
|
547
|
-
// works identically in v1 and v2
|
|
548
|
-
adapter.state[itemType].cursor = nextCursor;
|
|
549
|
-
adapter.state[itemType].complete = true;
|
|
550
|
-
```
|
|
551
|
-
|
|
552
|
-
- **SDK bookkeeping** (`workersOldest`/`workersNewest`, `pendingWorkers*`,
|
|
553
|
-
`toDevRev`, `fromDevRev`, `snapInVersionId`) moved to **`adapter.sdkState`**
|
|
554
|
-
(read-only getter).
|
|
555
|
-
- On disk, state is persisted as a `{ connectorState, sdkState }` envelope. The SDK
|
|
556
|
-
**migrates a v1 flat blob automatically on first read** (recognized SDK keys
|
|
557
|
-
split into `sdkState`, the rest into `connectorState`), so in-flight syncs
|
|
558
|
-
survive the upgrade.
|
|
559
|
-
|
|
560
|
-
> **`lastSyncStarted` / `lastSuccessfulSyncStarted` are gone.** Both were
|
|
561
|
-
> `@deprecated` in v1 and are **removed from `SdkState` in v2** — the SDK no longer
|
|
562
|
-
> sets, reads, or declares them. They were wall-clock sync-start timestamps; the
|
|
563
|
-
> SDK now resolves the incremental window from extraction-data boundaries instead
|
|
564
|
-
> (`workersOldest`/`workersNewest` and the resolved `extract_from`/`extract_to`).
|
|
565
|
-
> The two old key names are still recognized as SDK-owned by the v1-blob migration,
|
|
566
|
-
> so they route into `sdkState` (where they sit unused) rather than leaking into
|
|
567
|
-
> your connector state.
|
|
568
|
-
|
|
569
|
-
### If your v1 connector mixed SDK fields into its own `State` interface
|
|
570
|
-
|
|
571
|
-
Some connectors declared SDK-owned fields (e.g. `toDevRev`,
|
|
572
|
-
`lastSuccessfulSyncStarted`) on their hand-written `State` type and seeded them in
|
|
573
|
-
`getInitialState()`. Remove those — `toDevRev`/`fromDevRev`/`workers*` are
|
|
574
|
-
SDK-managed now, and `lastSyncStarted`/`lastSuccessfulSyncStarted` no longer exist
|
|
575
|
-
at all:
|
|
576
|
-
|
|
577
|
-
```ts
|
|
578
|
-
// v1 — State interface mixing SDK fields (remove these)
|
|
579
|
-
export interface State {
|
|
580
|
-
// ... your fields ...
|
|
581
|
-
lastSyncStarted?: string; // removed in v2 — no replacement field
|
|
582
|
-
lastSuccessfulSyncStarted?: string; // removed in v2 — read workersNewest / extract_to instead
|
|
583
|
-
toDevRev?: ToDevRev; // also drop the dist/state/state.interfaces import
|
|
584
|
-
}
|
|
585
|
-
```
|
|
586
|
-
|
|
587
|
-
### Cursor decision rule: rename vs repoint
|
|
588
|
-
|
|
589
|
-
For `lastSyncStarted` / `lastSuccessfulSyncStarted` specifically, what to do
|
|
590
|
-
depends on **whether the field is declared in your own `State` interface /
|
|
591
|
-
`getInitialState`** — getting this backwards compiles but silently breaks
|
|
592
|
-
incremental sync:
|
|
593
|
-
|
|
594
|
-
- **Not declared** (only accessed loosely as `adapter.state.<field>`, absent from
|
|
595
|
-
your interface): it was SDK-supplied via the removed v1 `AdapterState<T>`, and
|
|
596
|
-
the access is now a hard type error. **Delete** a bare write; **repoint** a read
|
|
597
|
-
to `adapter.event.payload.event_context.extract_from` (the resolved window start
|
|
598
|
-
— usually already destructured nearby). There is no field to preserve, so do not
|
|
599
|
-
rename.
|
|
600
|
-
- **Declared** (your connector writes and reads it as its own cross-sync cursor,
|
|
601
|
-
often consuming `extract_from` separately): **rename** it off the reserved SDK
|
|
602
|
-
key (e.g. `lastSuccessfulSyncStarted` → `lastSuccessfulWindowStart`) throughout
|
|
603
|
-
the interface, `getInitialState`, all read/write sites, and test fixtures. The
|
|
604
|
-
rename is mandatory because the v1-blob auto-migration routes the reserved key
|
|
605
|
-
into `sdkState` — a connector-owned cursor left on that name would be stripped
|
|
606
|
-
from your state. Substituting `extract_from` here would destroy the cursor
|
|
607
|
-
mechanism.
|
|
608
|
-
|
|
609
|
-
### Reading the incremental-sync window
|
|
610
|
-
|
|
611
|
-
In v1 you read `lastSuccessfulSyncStarted` to decide an incremental cursor. That
|
|
612
|
-
field is gone (above). Read the window the SDK resolves for you instead:
|
|
613
|
-
|
|
614
|
-
```ts
|
|
615
|
-
// preferred — the resolved window for this invocation, on the event context
|
|
616
|
-
const { extract_from, extract_to } = adapter.event.payload.event_context;
|
|
617
|
-
|
|
618
|
-
// the persisted cross-sync high-water mark (committed at end of each cycle)
|
|
619
|
-
const lastExtractedTo = adapter.sdkState.workersNewest;
|
|
620
|
-
```
|
|
621
|
-
|
|
622
|
-
`extract_from`/`extract_to` are the per-invocation window the SDK computes from the
|
|
623
|
-
platform's `extraction_start_time`/`extraction_end_time`;
|
|
624
|
-
`workersOldest`/`workersNewest` are the persisted boundaries committed at the end
|
|
625
|
-
of a completed cycle — the true replacement for the old "last successful sync"
|
|
626
|
-
resume point.
|
|
627
|
-
|
|
628
|
-
> **`AdapterState<ConnectorState>` is removed.** The deprecated flat
|
|
629
|
-
> `ConnectorState & SdkState` alias no longer exists. If you annotated anything
|
|
630
|
-
> `AdapterState<S>`, drop the import — use your own connector `State` type for
|
|
631
|
-
> `adapter.state`, and read SDK fields from `adapter.sdkState` (or the event
|
|
632
|
-
> context). The v2 on-disk shape is the `AdapterStateEnvelope`
|
|
633
|
-
> (`{ connectorState, sdkState }`), which the SDK manages for you.
|
|
634
|
-
|
|
635
|
-
> **Edge regression (old in-flight incremental syncs):** the SDK used to fall back
|
|
636
|
-
> to `lastSuccessfulSyncStarted` when resolving a `WORKERS_NEWEST` window on state
|
|
637
|
-
> predating `workersNewest` (SDK **< 1.17.1**). With the field removed, that
|
|
638
|
-
> fallback is gone: such a window now resolves to the unbounded epoch, so an
|
|
639
|
-
> incremental sync still mid-flight across the upgrade re-extracts from the
|
|
640
|
-
> beginning **once**. The next completed cycle commits `workersNewest` and the sync
|
|
641
|
-
> self-heals; the platform deduplicates downstream. New syncs and any sync whose
|
|
642
|
-
> first full cycle completed on ≥ 1.17.1 are unaffected. Drain in-flight
|
|
643
|
-
> incremental syncs before upgrading if the one-time re-extract matters.
|
|
644
|
-
|
|
645
|
-
> **Edge case:** if your v1 connector state had a top-level key literally named
|
|
646
|
-
> `connectorState` or `sdkState`, the auto-migration mis-reads it (the envelope is
|
|
647
|
-
> detected by those key names). Rename such a key **before** upgrading.
|
|
648
|
-
|
|
649
|
-
## 9. HTTP / axios surface removed from the SDK
|
|
650
|
-
|
|
651
|
-
The SDK no longer re-exports any axios surface from its public entry point:
|
|
652
|
-
|
|
653
|
-
| v1 export | v2 |
|
|
654
|
-
|-----------|----|
|
|
655
|
-
| `axios` (raw instance) | **removed** — import `axios` in your connector |
|
|
656
|
-
| `axiosClient` (retry-wrapped instance) | **removed from the public API** (still exists internally, but not exported and not at the old deep path) |
|
|
657
|
-
| `formatAxiosError` | **removed** (deleted from source) |
|
|
658
|
-
| `serializeAxiosError` | **removed from the public API** — use `serializeError` |
|
|
659
|
-
| `HTTPResponse` | **removed** |
|
|
660
|
-
|
|
661
|
-
`axios` and `axios-retry` are still runtime **dependencies** of the SDK (so
|
|
662
|
-
they're available transitively), but you must construct your own client:
|
|
663
|
-
|
|
664
|
-
### Before (v1)
|
|
665
|
-
|
|
666
|
-
```ts
|
|
667
|
-
import { axios, axiosClient } from '@devrev/ts-adaas';
|
|
668
|
-
|
|
669
|
-
const res = await axiosClient.get(url, { responseType: 'stream' });
|
|
670
|
-
```
|
|
671
|
-
|
|
672
|
-
### After (v2)
|
|
673
|
-
|
|
674
|
-
```ts
|
|
675
|
-
import axios from 'axios';
|
|
676
|
-
import axiosRetry from 'axios-retry';
|
|
677
|
-
|
|
678
|
-
const axiosClient = axios.create();
|
|
679
|
-
axiosRetry(axiosClient, { retries: 5, retryDelay: axiosRetry.exponentialDelay });
|
|
680
|
-
|
|
681
|
-
const res = await axiosClient.get(url, { responseType: 'stream' });
|
|
682
|
-
```
|
|
683
|
-
|
|
684
|
-
Add `axios` and `axios-retry` to your `package.json` **dependencies** if missing —
|
|
685
|
-
`axios-retry` in particular is usually not yet a direct dependency. The minimal
|
|
686
|
-
client above does not replicate three behaviors of v1's public `axiosClient`:
|
|
687
|
-
(1) its retry condition excluded 429s (letting your rate-limit handling see them);
|
|
688
|
-
(2) it stripped the `Authorization` header from logged errors after max retries;
|
|
689
|
-
(3) it used a **1000 ms** exponential-backoff base
|
|
690
|
-
(`exponentialDelay(retryCount, error, 1000)` → ~2s, 4s, 8s, 16s, 32s), whereas
|
|
691
|
-
`axios-retry`'s bare `exponentialDelay` defaults to a 100 ms factor (~0.2s … 3.2s)
|
|
692
|
-
— 10× more aggressive. To match v1's backoff, pass the base explicitly:
|
|
693
|
-
|
|
694
|
-
```ts
|
|
695
|
-
axiosRetry(axiosClient, {
|
|
696
|
-
retries: 5,
|
|
697
|
-
retryDelay: (retryCount, error) =>
|
|
698
|
-
axiosRetry.exponentialDelay(retryCount, error, 1000),
|
|
699
|
-
});
|
|
700
|
-
```
|
|
701
|
-
|
|
702
|
-
Replicate any of these only if your connector relied on the SDK client's behavior.
|
|
703
|
-
|
|
704
|
-
> If you previously deep-imported `@devrev/ts-adaas/dist/http/axios-client`, that
|
|
705
|
-
> path **no longer exists** (the file was renamed to `http/client` and is
|
|
706
|
-
> internal). Bring your own axios instance as above.
|
|
707
|
-
|
|
708
|
-
For error logging, replace `formatAxiosError` / `serializeAxiosError` with the
|
|
709
|
-
exported `serializeError`:
|
|
710
|
-
|
|
711
|
-
```ts
|
|
712
|
-
import { serializeError } from '@devrev/airsync-sdk';
|
|
713
|
-
console.error(serializeError(error));
|
|
714
|
-
```
|
|
715
|
-
|
|
716
|
-
> **`serializeAxiosError` is conditional on how you used the result.**
|
|
717
|
-
> `serializeError` returns a **string**; `serializeAxiosError` returned an
|
|
718
|
-
> **object**. Where the result was used as a string (logging, concatenation), swap
|
|
719
|
-
> to `serializeError`. Where it was **spread or property-accessed as an object**
|
|
720
|
-
> (`{ ...serializeAxiosError(e) }`, `serializeAxiosError(e).message`), swapping
|
|
721
|
-
> produces a TS2698 "spread types" error — instead keep the function via its deep
|
|
722
|
-
> path (it still exists internally and still returns the object):
|
|
723
|
-
> `import { serializeAxiosError } from '@devrev/airsync-sdk/dist/logger/logger';`
|
|
724
|
-
|
|
725
|
-
### `httpStream` type changed
|
|
726
|
-
|
|
727
|
-
The connector-implemented attachment-stream function returns
|
|
728
|
-
`ExternalSystemAttachmentStreamingResponse`, whose `httpStream` field changed from
|
|
729
|
-
axios's `AxiosResponse` to the new public `HttpStreamResponse`
|
|
730
|
-
(`{ data: any; headers: Record<string, any> }`). An axios stream response still
|
|
731
|
-
satisfies it structurally, but if you annotated the stream with `AxiosResponse`
|
|
732
|
-
imported *from the SDK*, switch to `HttpStreamResponse` (or import `AxiosResponse`
|
|
733
|
-
from `axios` directly).
|
|
734
|
-
|
|
735
|
-
> **`error.statusCode` dropped from the streaming response.** v1.20 briefly widened
|
|
736
|
-
> `ExternalSystemAttachmentStreamingResponse.error` to
|
|
737
|
-
> `ErrorRecord & { statusCode?: number }`; v2 narrows it back to a plain
|
|
738
|
-
> `ErrorRecord`. The SDK never read `statusCode`, so this only affects a
|
|
739
|
-
> `getAttachmentStream` that returned `{ error: { message, statusCode } }` — drop
|
|
740
|
-
> the extra property (TS excess-property checking will flag it). Encode retry
|
|
741
|
-
> timing via `{ delay }` instead.
|
|
742
|
-
|
|
743
|
-
## 10. `Mappers` methods return the unwrapped body
|
|
744
|
-
|
|
745
|
-
`Mappers.getByTargetId` / `getByExternalId` / `create` / `update` changed their
|
|
746
|
-
return type from `Promise<AxiosResponse<T>>` to `Promise<T>` — they now return the
|
|
747
|
-
response **body** directly. Drop the `.data` access:
|
|
748
|
-
|
|
749
|
-
### Before (v1)
|
|
750
|
-
|
|
751
|
-
```ts
|
|
752
|
-
const mapperResponse = await this.mappers.getByTargetId({ sync_unit, target });
|
|
753
|
-
const resolvedId = mapperResponse.data.sync_mapper_record.external_ids[0];
|
|
754
|
-
```
|
|
755
|
-
|
|
756
|
-
### After (v2)
|
|
757
|
-
|
|
758
|
-
```ts
|
|
759
|
-
const mapperResponse = await this.mappers.getByTargetId({ sync_unit, target });
|
|
760
|
-
const resolvedId = mapperResponse.sync_mapper_record.external_ids[0];
|
|
761
|
-
```
|
|
762
|
-
|
|
763
|
-
A **silent** change for code that reads `.data` (fails to type-check, or reads
|
|
764
|
-
`undefined` if loosely typed). `Mappers` is now also exported from the package
|
|
765
|
-
root, so you no longer need to deep-import it from `dist/mappers/mappers`.
|
|
766
|
-
|
|
767
|
-
Three places must change **together**, or the failure is silent at runtime:
|
|
768
|
-
|
|
769
|
-
1. the read site (drop `.data`, above);
|
|
770
|
-
2. any hand-rolled structural param type that wrapped the body in `data?:`;
|
|
771
|
-
3. **mapper test doubles** — `jest.fn().mockResolvedValue({ data: {
|
|
772
|
-
sync_mapper_record: ... } })` must become `mockResolvedValue({
|
|
773
|
-
sync_mapper_record: ... })`, or the migrated source reads `undefined` and the
|
|
774
|
-
test fails. These test files often import nothing from the SDK, so a
|
|
775
|
-
specifier-based search misses them — grep for `sync_mapper_record`.
|
|
776
|
-
|
|
777
|
-
> `mappers` lives on `LoadingAdapter` only (§5). Extraction-phase code that used
|
|
778
|
-
> `adapter.mappers` must construct its own instance —
|
|
779
|
-
> `new Mappers({ event: adapter.event })` (hoist it out of per-item loops).
|
|
780
|
-
|
|
781
|
-
## 11. Deleted legacy modules
|
|
782
|
-
|
|
783
|
-
Everything under the v1 `deprecated/` tree is gone:
|
|
784
|
-
|
|
785
|
-
| Removed | Replacement |
|
|
786
|
-
|---------|-------------|
|
|
787
|
-
| `Adapter`, `createAdapter` | `ExtractionAdapter` / `LoadingAdapter` + `processExtractionTask` / `processLoadingTask` |
|
|
788
|
-
| `DemoExtractor` | — (reference implementation only) |
|
|
789
|
-
| `HTTPClient`, `defaultResponse` | your own axios client (§9) |
|
|
790
|
-
| deprecated `Uploader` | repos (`initializeRepos` / `getRepo` / `push`) |
|
|
791
|
-
|
|
792
|
-
(The SDK has an internal `Uploader` class, but it was never part of the public API
|
|
793
|
-
in either version — the *public* v1 `Uploader` was the deprecated one.)
|
|
794
|
-
|
|
795
|
-
## 12. Deleted deprecated enum members, types & enums
|
|
796
|
-
|
|
797
|
-
The old/new duplicate enum members were collapsed; only the modern names remain.
|
|
798
|
-
**The string values of the surviving members are byte-identical to v1**, so nothing
|
|
799
|
-
changes on the wire — only the TypeScript member names.
|
|
800
|
-
|
|
801
|
-
> ⚠️ The **deleted** `EventType` / `ExtractorEventType` members carried
|
|
802
|
-
> *different, older* string values than their replacements (e.g. v1
|
|
803
|
-
> `ExtractionDataStart = 'EXTRACTION_DATA_START'` vs the surviving
|
|
804
|
-
> `StartExtractingData = 'START_EXTRACTING_DATA'`). The modern members already
|
|
805
|
-
> existed in v1 with the modern values, so survivors are wire-compatible — but a
|
|
806
|
-
> deleted member and its replacement did **not** share a value.
|
|
807
|
-
|
|
808
|
-
**`EventType` (incoming):**
|
|
809
|
-
|
|
810
|
-
| Deleted (v1 deprecated) | Use instead |
|
|
811
|
-
|--------------------------|-------------|
|
|
812
|
-
| `ExtractionExternalSyncUnitsStart` | `StartExtractingExternalSyncUnits` |
|
|
813
|
-
| `ExtractionMetadataStart` | `StartExtractingMetadata` |
|
|
814
|
-
| `ExtractionDataStart` | `StartExtractingData` |
|
|
815
|
-
| `ExtractionDataContinue` | `ContinueExtractingData` |
|
|
816
|
-
| `ExtractionDataDelete` | `StartDeletingExtractorState` |
|
|
817
|
-
| `ExtractionAttachmentsStart` | `StartExtractingAttachments` |
|
|
818
|
-
| `ExtractionAttachmentsContinue` | `ContinueExtractingAttachments` |
|
|
819
|
-
| `ExtractionAttachmentsDelete` | `StartDeletingExtractorAttachmentsState` |
|
|
820
|
-
|
|
821
|
-
**`ExtractorEventType` (outgoing):** the `Extraction*`-prefixed members
|
|
822
|
-
(`ExtractionDataDone`, `ExtractionDataDelay`, `ExtractionAttachmentsProgress`, …)
|
|
823
|
-
are deleted; use the `*Extraction*` members (`DataExtractionDone`,
|
|
824
|
-
`DataExtractionDelayed`, `AttachmentExtractionProgress`, …). In practice you'll
|
|
825
|
-
rarely reference `ExtractorEventType` at all in v2 — see §4.
|
|
826
|
-
|
|
827
|
-
**`LoaderEventType`:** the duplicate members `DataLoadingDelay` and
|
|
828
|
-
`AttachmentsLoading*` (plural) are deleted; use `DataLoadingDelayed` and
|
|
829
|
-
`AttachmentLoading*` (singular). These duplicates shared their survivors' string
|
|
830
|
-
value, so removing them is a pure source-name change.
|
|
831
|
-
|
|
832
|
-
> **No more incoming-event-type translation.** v1 shipped an `event-type-translation`
|
|
833
|
-
> module mapping legacy platform strings onto the modern enum members (and
|
|
834
|
-
> translating outgoing types). v2 removed it entirely and passes incoming
|
|
835
|
-
> `payload.event_type` through untouched. Any connector importing
|
|
836
|
-
> `translateIncomingEventType` / `translateOutgoingEventType` /
|
|
837
|
-
> `translateExtractorEventType` / `translateLoaderEventType` from the SDK will fail
|
|
838
|
-
> to compile — drop them; the platform sends modern strings.
|
|
839
|
-
|
|
840
|
-
### Removed `UnknownEventType` members
|
|
841
|
-
|
|
842
|
-
`UnknownEventType = 'UNKNOWN_EVENT_TYPE'` was declared on three enums in v1
|
|
843
|
-
(`EventType`, `ExtractorEventType`, `LoaderEventType`). All three copies are
|
|
844
|
-
**removed** in v2; the SDK's "unrecognized event" sentinel is now an internal,
|
|
845
|
-
un-exported constant with the same string value. If you matched on any enum
|
|
846
|
-
member, compare against the raw `'UNKNOWN_EVENT_TYPE'` string instead (the wire
|
|
847
|
-
value is unchanged).
|
|
848
|
-
|
|
849
|
-
### Removed deprecated types & enums
|
|
850
|
-
|
|
851
|
-
These were `@deprecated` in v1 and are **deleted from the public API** in v2. None
|
|
852
|
-
are referenced by the modern worker contract; each row gives the replacement (or
|
|
853
|
-
"no replacement" where the concept is gone):
|
|
854
|
-
|
|
855
|
-
| Removed | Replacement |
|
|
856
|
-
|---------|-------------|
|
|
857
|
-
| `ExtractionMode` (enum) | `SyncMode` (adds `LOADING` alongside `INITIAL`/`INCREMENTAL`) |
|
|
858
|
-
| `EventContextIn` (interface) | `EventContext` (the single, current event-context type) |
|
|
859
|
-
| `EventContextOut` (interface) | `EventContext` |
|
|
860
|
-
| `DomainObjectState` (interface) | — (no replacement; was an unused per-object state shape) |
|
|
861
|
-
| `ErrorLevel` (enum) | — (logger uses its own internal log level) |
|
|
862
|
-
| `LogRecord` (interface) | — (unused) |
|
|
863
|
-
| `AdapterUpdateParams` (interface) | — (unused) |
|
|
864
|
-
| `AdapterState<T>` (type alias) | your connector `State` for `adapter.state`; `adapter.sdkState` for SDK fields (§8) |
|
|
865
|
-
|
|
866
|
-
Also removed from `EventData`: the deprecated `external_sync_units` field (§7) and
|
|
867
|
-
the deprecated `progress` field (a no-op since v1 — the backend computes progress).
|
|
868
|
-
The `artifacts` field on `EventData` is **kept** — it is how the SDK attaches
|
|
869
|
-
uploaded repo artifacts (including external sync units) to the emitted event.
|
|
870
|
-
|
|
871
|
-
### Worker-thread IPC types no longer on the root barrel
|
|
872
|
-
|
|
873
|
-
The worker↔main-thread plumbing types are no longer exported from the package root.
|
|
874
|
-
They were never part of the connector-authoring surface — they describe the SDK's
|
|
875
|
-
internal `parentPort` message protocol — but v1's barrel re-exported them, so a
|
|
876
|
-
connector (usually a test simulating the protocol) could import them from
|
|
877
|
-
`@devrev/ts-adaas`. Now removed from the root: `WorkerEvent`,
|
|
878
|
-
`WorkerMessageSubject` (enums), `WorkerMessage`, `WorkerMessageEmitted`,
|
|
879
|
-
`WorkerMessageExit`, `WorkerMessageLog`, `WorkerMessageFailed`, `WorkerData`,
|
|
880
|
-
`GetWorkerPathInterface`. The declarations still exist internally (deep-importable
|
|
881
|
-
via `@devrev/airsync-sdk/dist/types/workers`), but treat them as SDK-internal — a
|
|
882
|
-
worker-protocol test should assert on the `TaskResult` your task returns (§4), not
|
|
883
|
-
on raw IPC messages. (`WorkerMessageLog` also dropped its `isSdkLog` field.)
|
|
884
|
-
|
|
885
|
-
> **Emitted logs dropped the `is_sdk_log` field (observability only).** v1 tagged
|
|
886
|
-
> every log line with `is_sdk_log: true | false` (SDK vs connector origin), driven
|
|
887
|
-
> by an `AsyncLocalStorage` log-context layer that v2 removed. v2's log JSON no
|
|
888
|
-
> longer contains `is_sdk_log`. **No connector code changes** — not a compile or
|
|
889
|
-
> runtime break — but any platform dashboard, monitor, or saved query filtering
|
|
890
|
-
> logs on `@is_sdk_log` silently stops matching. Update those filters (drop the
|
|
891
|
-
> facet, or distinguish origin another way) after connectors upgrade.
|
|
892
|
-
|
|
893
|
-
## 13. Deep-import paths that moved or broke
|
|
894
|
-
|
|
895
|
-
The compiled `dist/` mirrors `src/` 1:1, so a deep import works only if the source
|
|
896
|
-
file still lives at the same relative path. Status of the paths real connectors
|
|
897
|
-
used:
|
|
898
|
-
|
|
899
|
-
| Deep import | Status in v2 |
|
|
900
|
-
|-------------|--------------|
|
|
901
|
-
| `dist/http/axios-client` (`axiosClient`) | ❌ **broken** — file renamed to `http/client` and made internal. Bring your own axios (§9). |
|
|
902
|
-
| `dist/state/state` (`State`, `createAdapterState`) | ❌ **both symbols removed** — the module now exports `BaseState`, `ExtractionState`/`createExtractionState`, `LoadingState`/`createLoadingState`. Map sync `new State(...)` → sync `new ExtractionState(...)`/`new LoadingState(...)`; async `createAdapterState(...)` → the async `create*State(...)` factory (§5). |
|
|
903
|
-
| `dist/state/state.interfaces` (`ToDevRev`) | ⚠️ still resolves, but `ToDevRev` is SDK-internal now (§8) — **drop** the import, don't repoint it |
|
|
904
|
-
| `dist/mappers/mappers.interface` (singular) | ❌ **broken** — file renamed to `mappers.interfaces` (plural). The four `*Params` interfaces and the two `SyncMapperRecord*` enums are on the root barrel (prefer root); the four `*Response` interfaces, `SyncMapperRecord`, `SyncMapperRecordExternalVersion`, `UpdateSyncMapperRecordParams`, and `MappersFactoryInterface` are **not** — repoint those to `dist/mappers/mappers.interfaces`. |
|
|
905
|
-
| `dist/logger/logger` (`serializeAxiosError`) | ⚠️ resolves, symbol kept internally — see the conditional rule in §9 (keep the deep import for object use; root `serializeError` for string use) |
|
|
906
|
-
| `dist/repo/repo.interfaces` (`Item`) | ✅ resolves — but `Item` is now on the root barrel, prefer the root import |
|
|
907
|
-
| `dist/types/loading` (`ItemTypeToLoad`) | ✅ resolves — also now on the root barrel |
|
|
908
|
-
| `dist/mappers/mappers` (`Mappers`) | ✅ resolves — also now on the root barrel (and note the return-type change, §10) |
|
|
909
|
-
| `dist/types/extraction` (`InitialSyncScope`) | ✅ resolves — also on the root barrel |
|
|
910
|
-
|
|
911
|
-
**Recommended:** replace all deep `dist/**` imports with root imports. If a symbol
|
|
912
|
-
you need isn't on the root barrel, request it rather than deep-importing.
|
|
913
|
-
|
|
914
|
-
## 14. Edge regression: very old in-flight attachment syncs
|
|
915
|
-
|
|
916
|
-
In v1 the SDK migrated the legacy `string[]` form of the processed-attachments
|
|
917
|
-
dedup list (`lastProcessedAttachmentsIdsList`) to the current `{ id, parent_id }[]`
|
|
918
|
-
form on read. v2 removed that conversion.
|
|
919
|
-
|
|
920
|
-
The `string[]` form only exists in state written by SDK **< 1.15.2**. If an
|
|
921
|
-
attachment-extraction phase started on a pre-1.15.2 SDK and is **still mid-flight**
|
|
922
|
-
when the connector upgrades to v2, the v2 dedup check (`it.id === …`) won't match
|
|
923
|
-
the bare-string entries, so attachments already downloaded in that sync get
|
|
924
|
-
re-uploaded once. New syncs — and any sync started on ≥ 1.15.2 — are unaffected;
|
|
925
|
-
the platform deduplicates downstream, so the only cost is the wasted
|
|
926
|
-
re-download/upload on that one continuation.
|
|
927
|
-
|
|
928
|
-
If this matters for your deployment, drain in-flight attachment syncs before
|
|
929
|
-
upgrading.
|
|
930
|
-
|
|
931
|
-
## 15. `spawn`'s deprecated `workerPath` option removed
|
|
932
|
-
|
|
933
|
-
`SpawnFactoryInterface.workerPath` was `@deprecated` in v1 and is **removed** in v2.
|
|
934
|
-
Point `spawn` at your worker directory with `baseWorkerPath: __dirname` — the SDK
|
|
935
|
-
resolves the per-event worker file from there (`workerPathOverrides` still works
|
|
936
|
-
for custom paths).
|
|
937
|
-
|
|
938
|
-
### Before (v1)
|
|
939
|
-
|
|
940
|
-
```ts
|
|
941
|
-
spawn({ event, initialState, workerPath: __dirname + '/workers/data-extraction' });
|
|
942
|
-
```
|
|
943
|
-
|
|
944
|
-
### After (v2)
|
|
945
|
-
|
|
946
|
-
```ts
|
|
947
|
-
spawn({ event, initialState, baseWorkerPath: __dirname });
|
|
948
|
-
```
|
|
949
|
-
|
|
950
|
-
> **Indirected `workerPath`** (a variable fed by a `switch`/dispatcher helper like
|
|
951
|
-
> `getWorkerPerLoadingPhase(event)`): replace with `baseWorkerPath: __dirname`
|
|
952
|
-
> **and delete the now-dead dispatcher function**, its locals, and any imports it
|
|
953
|
-
> orphaned (`EventType`, …) — otherwise `noUnusedLocals`/lint fails.
|
|
954
|
-
> `workerPathOverrides` is unrelated and kept.
|
|
955
|
-
|
|
956
|
-
## 16. Jest mocks & test assertions
|
|
957
|
-
|
|
958
|
-
Tests are first-class migration targets — a source-only migration passes `tsc` but
|
|
959
|
-
fails the test gate. SDK-module mocks hardcode the v1 shape.
|
|
960
|
-
|
|
961
|
-
**Mechanical changes per test file:**
|
|
962
|
-
|
|
963
|
-
- The module specifier in `jest.mock('...')` / `jest.requireActual('...')` /
|
|
964
|
-
`moduleNameMapper` was already renamed in §1. In mock factories: `processTask` →
|
|
965
|
-
`processExtractionTask` / `processLoadingTask` (matching the worker under test,
|
|
966
|
-
§3), including the `X as jest.Mock` capture.
|
|
967
|
-
- Drop `WorkerAdapter: {}` and `axiosClient: {}` keys from mock factories; remove
|
|
968
|
-
vestigial `emit: jest.fn()` from mock adapters.
|
|
969
|
-
- `AirdropEvent` → `AirSyncEvent` in fixtures/annotations (§2). Keep
|
|
970
|
-
`jest.requireActual` only for enums/constants still referenced.
|
|
971
|
-
- Propagate every state-field rename and `AdapterState` → your-State-type
|
|
972
|
-
replacement into fixtures (§8) — fixtures are the most-missed target.
|
|
973
|
-
|
|
974
|
-
**Semantic changes:**
|
|
975
|
-
|
|
976
|
-
- `expect(adapter.emit).toHaveBeenCalledWith(EventType.X)` becomes an assertion on
|
|
977
|
-
the awaited **return** of the captured task/onTimeout function:
|
|
978
|
-
|
|
979
|
-
```ts
|
|
980
|
-
// v1
|
|
981
|
-
await mockProcessTask.mock.calls[0][0].task({ adapter });
|
|
982
|
-
expect(adapter.emit).toHaveBeenCalledWith(ExtractorEventType.DataExtractionDone);
|
|
983
|
-
// v2
|
|
984
|
-
const result = await mockProcessTask.mock.calls[0][0].task({ adapter });
|
|
985
|
-
expect(result).toEqual({ status: 'success' });
|
|
986
|
-
```
|
|
987
|
-
|
|
988
|
-
The same rewrite applies to tests that invoke a **helper directly** (the helper
|
|
989
|
-
now returns a `TaskResult` per §4) — these lack the `mockProcessTask` capture
|
|
990
|
-
shape and are easy to miss.
|
|
991
|
-
- Expected delay objects use `delaySeconds`, not `delay`.
|
|
992
|
-
- Pass-through loaders (§6): assert the task returns the
|
|
993
|
-
`loadItemTypes`/`streamAttachments` mock's value.
|
|
994
|
-
- Source that now does `new Mappers({ event })` (§10) needs a `Mappers` mock in the
|
|
995
|
-
factory; source that moved to `import axios from 'axios'` (§9) needs
|
|
996
|
-
`jest.mock('axios')` (plus `jest.mock('axios-retry')` if you built a retry
|
|
997
|
-
client) instead of mocking the SDK's axios.
|
|
998
|
-
- Unwrap mapper test doubles (§10). Hand-constructed adapters in integration tests
|
|
999
|
-
follow §5.
|
|
1000
|
-
|
|
1001
|
-
---
|
|
1002
|
-
|
|
1003
|
-
## Migration checklist
|
|
1004
|
-
|
|
1005
|
-
1. `npm uninstall @devrev/ts-adaas && npm install @devrev/airsync-sdk@beta`; replace the import specifier everywhere — imports, `jest.mock`/`jest.requireActual`, `moduleNameMapper` (§1).
|
|
1006
|
-
2. Rename `AirdropEvent` → `AirSyncEvent`, `AirdropMessage` → `AirSyncMessage`. Drop any `CustomAirdropEvent` cast that only added `user_id`/`dev_oid`/`source_id`/`service_account_id` (§2).
|
|
1007
|
-
3. Split workers: extraction files use `processExtractionTask`, loading files use `processLoadingTask` (§3). Rewrite `ProcessTaskInterface<State>`/`TaskAdapterInterface<State>` annotations to take the adapter type (§3–4).
|
|
1008
|
-
4. Convert **every** `adapter.emit(...)` into a returned `TaskResult`; bubble outcomes up from helpers and class methods; `delay` → `delaySeconds` (§4).
|
|
1009
|
-
5. Replace `WorkerAdapter<T>` annotations with `ExtractionAdapter<T>` / `LoadingAdapter<T>`, and `new WorkerAdapter(...)`/`new State(...)`/`createAdapterState(...)` constructions per §5. Move any `mappers`/`reports`/`processedFiles` access into the loading path.
|
|
1010
|
-
6. Pass the `TaskResult` straight through from `loadItemTypes` / `loadAttachments` / `streamAttachments`; a connector that augmented `reports` pushes onto `adapter.reports` before returning (§6).
|
|
1011
|
-
7. ESU workers: push external sync units to the `EXTERNAL_SYNC_UNITS` repo; remove any `external_sync_units` (and `progress`) from emit data — those fields are gone from `EventData` (§7, §12).
|
|
1012
|
-
8. Remove SDK-owned fields from your `State` interface and `getInitialState`; apply the cursor decision rule for `lastSyncStarted`/`lastSuccessfulSyncStarted` — rename a connector-declared cursor, repoint an SDK-supplied read to `event_context.extract_from` (§8).
|
|
1013
|
-
9. Replace `axios` / `axiosClient` / `formatAxiosError` SDK imports with your own axios client + `serializeError`; apply the conditional `serializeAxiosError` rule (§9).
|
|
1014
|
-
10. Drop `.data` from `adapter.mappers.*` result reads — source, hand-rolled types, and test doubles together (§10).
|
|
1015
|
-
11. Remove usage of deleted legacy modules and event-type-translation helpers (§11, §12).
|
|
1016
|
-
12. Replace deleted enum members with their modern names; drop any use of the removed deprecated types (`ExtractionMode`, `EventContextIn`/`Out`, `DomainObjectState`, `ErrorLevel`, `LogRecord`, `AdapterUpdateParams`, `AdapterState` — replace an `AdapterState<S>` annotation with your own State type, don't just delete it) (§12).
|
|
1017
|
-
13. Replace deep `dist/**` imports with root imports; drop the now-internal `ToDevRev` import (§13).
|
|
1018
|
-
14. Drop each `onTimeout` that only emitted the phase-appropriate event — the SDK default covers it (progress for resumable phases, a timeout error for ESU / metadata / state-deletion). Keep one only when its v1 body did real work (cleanup, a custom error message), preserving the body (§4).
|
|
1019
|
-
15. Replace `spawn({ workerPath })` with `spawn({ baseWorkerPath: __dirname })`; delete any dead worker-path dispatcher (§15).
|
|
1020
|
-
16. Migrate jest mocks and assertions — emit-called assertions become return-value assertions (§16).
|
|
1021
|
-
|
|
1022
|
-
## A note on the early betas
|
|
1023
|
-
|
|
1024
|
-
The betas `2.0.0-beta.0` through `2.0.0-beta.3` still re-exported `axios` /
|
|
1025
|
-
`axiosClient`; they were removed in `2.0.0-beta.4` (and stay removed in GA — see §9).
|
|
1026
|
-
If you migrated a connector against any beta before `beta.4` and imported either
|
|
1027
|
-
symbol from `@devrev/airsync-sdk`, it will fail to compile once you move to
|
|
1028
|
-
`beta.4`/GA — apply §9.
|