@bernierllc/contentful-sync-service 1.4.4 → 1.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -156,13 +156,13 @@ new ContentfulSyncService(config: SyncServiceConfig)
156
156
  - `syncStateStorage` (SyncStateStorage) - Sync state storage implementation
157
157
  - `webhookSecret` (string, optional) - Webhook validation secret
158
158
  - `syncInterval` (number, optional) - Auto-sync interval in ms (default: 300000)
159
- - `neverhubEnabled` (boolean, optional) - Enable NeverHub integration
159
+ - `neverhubEnabled` (boolean, optional) - **Deprecated since 1.5.0, removed in 2.0.0.** The service registers with NeverHub by default. `false` still opts out during 1.x. Setting the option at all logs a one-time warning; remove it from your config.
160
160
 
161
161
  #### Methods
162
162
 
163
163
  ##### `initialize(): Promise<void>`
164
164
 
165
- Initialize the service and optionally connect to NeverHub.
165
+ Initialize the service. NeverHub registration already started in the constructor, so this does not wait on it.
166
166
 
167
167
  ```typescript
168
168
  await service.initialize();
@@ -170,7 +170,7 @@ await service.initialize();
170
170
 
171
171
  ##### `initialSync(managementToken: string): Promise<SyncResult>`
172
172
 
173
- Perform initial full export from Contentful.
173
+ Perform initial full export from Contentful. The CDA sync cursor is taken before the export starts and stored only after the export is saved, so anything edited while the export runs is picked up again by the next `incrementalSync()`.
174
174
 
175
175
  ```typescript
176
176
  const result = await service.initialSync('cma-token');
@@ -179,16 +179,16 @@ console.log(result.stats.entriesSynced); // Number of entries synced
179
179
 
180
180
  ##### `incrementalSync(): Promise<SyncResult>`
181
181
 
182
- Perform incremental sync using the CDA Sync API.
182
+ Perform incremental sync using the CDA Sync API. Applies the response's `entries`, `assets`, `deletedEntries` and `deletedAssets`, then stores the new `nextSyncToken`. If Contentful returns no token, nothing is written and the old token is kept. Delta items are stored as the CDA delivers them: their `sys` carries `revision`, not the CMA `version` that the export path stores, so a storage adapter must not rely on `sys.version` for synced items. The CDA client also resolves links between items in the same delta response (the linked entry or asset is embedded in place of its `Link` object), so delta items can differ in shape from exported ones.
183
183
 
184
184
  ```typescript
185
185
  const result = await service.incrementalSync();
186
186
  console.log(result.stats.itemsDeleted); // Number of items deleted
187
187
  ```
188
188
 
189
- ##### `processWebhook(body: string, headers: Record<string, string>): Promise<void>`
189
+ ##### `processWebhook(body: string, headers: Record<string, string>): Promise<WebhookApplyResult>`
190
190
 
191
- Process a webhook event from Contentful.
191
+ Verify a Contentful webhook, then run `incrementalSync()` for it (see `applyWebhookEvent` below). It returns `{ topic, id, outcome }`, where `outcome` is `'synced'` or `'ignored'`. A failed sync throws, so answer 500 and Contentful retries the delivery.
192
192
 
193
193
  ```typescript
194
194
  app.post('/webhook', async (req, res) => {
@@ -200,6 +200,21 @@ app.post('/webhook', async (req, res) => {
200
200
  });
201
201
  ```
202
202
 
203
+ ##### `applyWebhookEvent(topic: string, payload: ContentfulWebhookPayload): Promise<WebhookApplyResult>`
204
+
205
+ Handle an already-verified event. Use it when your own receiver checks signatures. The webhook body is never written to storage: Contentful delivers webhooks unordered, with retries and duplicates, while the CDA sync API is ordered and authoritative. So a delivery change triggers `incrementalSync()`, and storage only ever reflects what the sync returns.
206
+
207
+ | Topic | Effect |
208
+ |-------|--------|
209
+ | `Entry`/`Asset` `.publish`, `.unpublish`, `.archive`, `.delete` | Runs `incrementalSync()` (`synced`) |
210
+ | Draft topics (`create`, `save`, `auto_save`, `unarchive`), `ContentType.*`, events for another space or environment | Nothing (`ignored`) |
211
+
212
+ Syncs never overlap: while one runs, every new request waits for one follow-up sync that starts after it, so at most one sync runs and one more is queued at any time, and none races on the sync token. `startAutoSync()` uses the same path.
213
+
214
+ The CDA can lag the webhook by a few seconds. A change the follow-up sync does not see yet is picked up by the next webhook or auto-sync, so keep `startAutoSync()` running.
215
+
216
+ Both `ContentManagement.Entry.publish` (the `X-Contentful-Topic` header) and `Entry.publish` work.
217
+
203
218
  ##### `startAutoSync(): void`
204
219
 
205
220
  Start automatic incremental sync at the configured interval.
@@ -294,18 +309,16 @@ app.listen(3000);
294
309
 
295
310
  ### NeverHub Integration
296
311
 
312
+ Unless `neverhubEnabled: false` is set (deprecated), the constructor registers the service with NeverHub (type `service`, with the package name, version and dependencies read from `package.json` at runtime). Registration runs in the background and never blocks syncing. When no NeverHub is reachable, `register()` does nothing. When NeverHub is reachable but rejects the registration, the service logs a `NeverHub registration failed` warning and keeps working.
313
+
297
314
  ```typescript
298
315
  const service = new ContentfulSyncService({
299
316
  spaceId: 'my-space',
300
317
  environmentId: 'master',
301
318
  accessToken: 'cda-token',
302
319
  contentStorage: new InMemoryContentStorage(),
303
- syncStateStorage: new InMemorySyncStateStorage(),
304
- neverhubEnabled: true // Enable NeverHub
320
+ syncStateStorage: new InMemorySyncStateStorage()
305
321
  });
306
-
307
- await service.initialize();
308
- // NeverHub registration happens automatically if available
309
322
  ```
310
323
 
311
324
  ## TypeScript Support
@@ -324,13 +337,39 @@ import type {
324
337
 
325
338
  ## Error Handling
326
339
 
340
+ `initialSync()` and `incrementalSync()` never throw. They return `{ success: false, error }`, where `error` is the underlying message. They also log the failure as a `ContentfulSyncServiceError` with its `cause` chain (`errorChain` in the log metadata), so the original Contentful or storage error is never lost.
341
+
327
342
  ```typescript
328
- const result = await service.initialSync('cma-token');
343
+ const result = await service.incrementalSync();
329
344
 
330
345
  if (!result.success) {
331
346
  console.error('Sync failed:', result.error);
332
- } else {
333
- console.log('Sync succeeded:', result.stats);
347
+ }
348
+ ```
349
+
350
+ `processWebhook()` and `applyWebhookEvent()` throw a `ContentfulSyncServiceError`:
351
+
352
+ | Code | When |
353
+ |------|------|
354
+ | `INVALID_INPUT` | No `webhookSecret` was configured, the signature or payload is invalid, or the topic or `sys.id` is missing |
355
+ | `OPERATION_FAILED` | The sync for the event failed (the sync's `ContentfulSyncServiceError` in `cause`) |
356
+
357
+ Sync failures are logged with these codes:
358
+
359
+ | Code | When |
360
+ |------|------|
361
+ | `INVALID_INPUT` | `incrementalSync()` ran before `initialSync()` stored a sync token |
362
+ | `OPERATION_FAILED` | Export, CDA sync or storage failed (original error in `cause`), or Contentful returned no `nextSyncToken` |
363
+
364
+ ```typescript
365
+ import { ContentfulSyncServiceError } from '@bernierllc/contentful-sync-service';
366
+
367
+ try {
368
+ await service.processWebhook(body, headers);
369
+ } catch (error) {
370
+ if (error instanceof ContentfulSyncServiceError && error.code === 'INVALID_INPUT') {
371
+ res.sendStatus(400);
372
+ }
334
373
  }
335
374
  ```
336
375
 
@@ -0,0 +1,75 @@
1
+ import type { ContentfulWebhookPayload } from '@bernierllc/contentful-types';
2
+ import type { SyncServiceConfig, SyncResult, WebhookApplyResult } from './types';
3
+ /**
4
+ * ContentfulSyncService
5
+ *
6
+ * Syncs Contentful content to local storage for ML training pipelines.
7
+ * Performs initial full export, then maintains sync via CDA Sync API + webhook processing.
8
+ */
9
+ export declare class ContentfulSyncService {
10
+ private readonly config;
11
+ private readonly exportAdapter;
12
+ private readonly cdaClient;
13
+ private readonly webhookHandler?;
14
+ private readonly logger;
15
+ private readonly neverhub;
16
+ private syncIntervalId?;
17
+ private syncRun;
18
+ private syncFollowUp;
19
+ constructor(config: SyncServiceConfig);
20
+ /**
21
+ * Register with NeverHub. register() no-ops when NeverHub is unreachable,
22
+ * so no detect() guard is needed.
23
+ */
24
+ private registerWithNeverHub;
25
+ /**
26
+ * Initialize the service
27
+ */
28
+ initialize(): Promise<void>;
29
+ /**
30
+ * Perform initial full sync
31
+ *
32
+ * @param managementToken - Contentful Management API token for export
33
+ */
34
+ initialSync(managementToken: string): Promise<SyncResult>;
35
+ /**
36
+ * Perform incremental sync using CDA Sync API. A call made while a sync is running waits for
37
+ * one follow-up run, so every caller gets a sync that started after its call.
38
+ */
39
+ incrementalSync(): Promise<SyncResult>;
40
+ private syncOnce;
41
+ private runIncrementalSync;
42
+ /**
43
+ * Verify a webhook and apply it to storage (see applyWebhookEvent)
44
+ *
45
+ * @param body - Webhook request body
46
+ * @param headers - Webhook request headers
47
+ */
48
+ processWebhook(body: string, headers: Record<string, string>): Promise<WebhookApplyResult>;
49
+ /**
50
+ * Bring storage up to date for a verified webhook event. A publish, unpublish, archive or
51
+ * delete of an Entry or Asset in this space and environment runs `incrementalSync()`; the
52
+ * payload itself is never written, because the CDA sync is ordered and Contentful's webhook
53
+ * deliveries are not (retries and duplicates can arrive late). Draft topics (create, save,
54
+ * auto_save, unarchive), ContentType topics and events for another space or environment run
55
+ * nothing and come back as `ignored`.
56
+ *
57
+ * @param topic - `x-contentful-topic`, e.g. `ContentManagement.Entry.publish` or `Entry.publish`
58
+ * @param payload - Webhook body, already verified by the caller
59
+ * @throws ContentfulSyncServiceError `INVALID_INPUT` for a malformed topic or payload,
60
+ * `OPERATION_FAILED` (sync error in `cause`) when the sync fails, so the webhook is retried
61
+ */
62
+ applyWebhookEvent(topic: string, payload: ContentfulWebhookPayload): Promise<WebhookApplyResult>;
63
+ /**
64
+ * Start automatic sync at configured interval
65
+ */
66
+ startAutoSync(): void;
67
+ /**
68
+ * Stop automatic sync
69
+ */
70
+ stopAutoSync(): void;
71
+ /**
72
+ * Shutdown the service
73
+ */
74
+ shutdown(): Promise<void>;
75
+ }
@@ -0,0 +1,393 @@
1
+ "use strict";
2
+ /*
3
+ Copyright (c) 2025 Bernier LLC
4
+
5
+ This file is licensed to the client under a limited-use license.
6
+ The client may use and modify this code *only within the scope of the project it was delivered for*.
7
+ Redistribution or use in other products or commercial offerings is not permitted without written consent from Bernier LLC.
8
+ */
9
+ Object.defineProperty(exports, "__esModule", { value: true });
10
+ exports.ContentfulSyncService = void 0;
11
+ const contentful_export_adapter_1 = require("@bernierllc/contentful-export-adapter");
12
+ const contentful_cda_client_1 = require("@bernierllc/contentful-cda-client");
13
+ const contentful_webhook_handler_1 = require("@bernierllc/contentful-webhook-handler");
14
+ const logger_1 = require("@bernierllc/logger");
15
+ const neverhub_adapter_1 = require("@bernierllc/neverhub-adapter");
16
+ const errors_1 = require("./errors");
17
+ // Read at runtime: Lerna bumps package.json, so a literal version drifts.
18
+ // eslint-disable-next-line @typescript-eslint/no-require-imports -- JSON read at runtime, outside rootDir
19
+ const pkg = require('../package.json');
20
+ let warnedNeverhubEnabled = false;
21
+ function toServiceError(error, message, context) {
22
+ if (error instanceof errors_1.ContentfulSyncServiceError) {
23
+ return error;
24
+ }
25
+ return new errors_1.ContentfulSyncServiceError(message, {
26
+ code: 'OPERATION_FAILED',
27
+ cause: error instanceof Error ? error : new Error(String(error)),
28
+ context
29
+ });
30
+ }
31
+ /**
32
+ * ContentfulSyncService
33
+ *
34
+ * Syncs Contentful content to local storage for ML training pipelines.
35
+ * Performs initial full export, then maintains sync via CDA Sync API + webhook processing.
36
+ */
37
+ class ContentfulSyncService {
38
+ config;
39
+ exportAdapter;
40
+ cdaClient;
41
+ webhookHandler;
42
+ logger;
43
+ neverhub = new neverhub_adapter_1.NeverHubAdapter();
44
+ syncIntervalId;
45
+ syncRun = null;
46
+ syncFollowUp = null;
47
+ constructor(config) {
48
+ this.config = {
49
+ syncInterval: 300000, // 5 minutes default
50
+ ...config
51
+ };
52
+ this.exportAdapter = new contentful_export_adapter_1.ContentfulExportAdapter();
53
+ this.cdaClient = new contentful_cda_client_1.ContentfulCDAClient({
54
+ space: config.spaceId,
55
+ accessToken: config.accessToken,
56
+ environment: config.environmentId
57
+ });
58
+ if (config.webhookSecret) {
59
+ this.webhookHandler = new contentful_webhook_handler_1.ContentfulWebhookHandler({ webhookSecret: config.webhookSecret });
60
+ }
61
+ this.logger = (0, logger_1.createLogger)({
62
+ context: {
63
+ service: 'ContentfulSyncService',
64
+ version: pkg.version,
65
+ environment: process.env.NODE_ENV || 'development'
66
+ }
67
+ });
68
+ if (config.neverhubEnabled !== undefined && !warnedNeverhubEnabled) {
69
+ warnedNeverhubEnabled = true;
70
+ console.warn('[@bernierllc/contentful-sync-service] neverhubEnabled is deprecated: ' +
71
+ 'the service registers with NeverHub by default, which no-ops when NeverHub is unreachable. ' +
72
+ 'neverhubEnabled: false still opts out until 2.0.0, when the option is deleted.');
73
+ }
74
+ // Explicit opt-out is honoured through the 1.x deprecation window; removed in 2.0.0.
75
+ if (config.neverhubEnabled !== false) {
76
+ // Not awaited: registration can retry for tens of seconds, and NeverHub is optional.
77
+ void this.registerWithNeverHub();
78
+ }
79
+ }
80
+ /**
81
+ * Register with NeverHub. register() no-ops when NeverHub is unreachable,
82
+ * so no detect() guard is needed.
83
+ */
84
+ async registerWithNeverHub() {
85
+ try {
86
+ await this.neverhub.register({
87
+ type: 'service',
88
+ name: pkg.name,
89
+ version: pkg.version,
90
+ dependencies: Object.keys(pkg.dependencies ?? {}),
91
+ capabilities: [
92
+ { type: 'data', name: 'sync', version: pkg.version },
93
+ { type: 'webhook', name: 'webhook-processing', version: pkg.version }
94
+ ]
95
+ });
96
+ // register() resolves on timeouts and non-2xx responses, so check the outcome.
97
+ if (this.neverhub.isAvailable() && !this.neverhub.isRegistered()) {
98
+ this.logger.warn('NeverHub registration failed', { error: 'registration not accepted' });
99
+ }
100
+ }
101
+ catch (error) {
102
+ // NeverHub is optional: sync keeps running without it.
103
+ this.logger.warn('NeverHub registration failed', {
104
+ error: error instanceof Error ? error.message : String(error)
105
+ });
106
+ }
107
+ }
108
+ /**
109
+ * Initialize the service
110
+ */
111
+ async initialize() {
112
+ // NeverHub registration starts in the constructor, so callers that skip initialize() still register.
113
+ this.logger.info('ContentfulSyncService initialized');
114
+ }
115
+ /**
116
+ * Perform initial full sync
117
+ *
118
+ * @param managementToken - Contentful Management API token for export
119
+ */
120
+ async initialSync(managementToken) {
121
+ const startTime = Date.now();
122
+ this.logger.info('Starting initial full sync', { spaceId: this.config.spaceId });
123
+ try {
124
+ // Take the CDA cursor before the export: changes made while the export runs are then replayed
125
+ // by the next incrementalSync() (writes are keyed by id, so replay is idempotent) instead of lost.
126
+ const syncResult = await this.cdaClient.initialSync();
127
+ if (!syncResult.nextSyncToken) {
128
+ throw new errors_1.ContentfulSyncServiceError('Contentful initial sync returned no nextSyncToken', {
129
+ code: 'OPERATION_FAILED',
130
+ context: { spaceId: this.config.spaceId }
131
+ });
132
+ }
133
+ const syncToken = syncResult.nextSyncToken;
134
+ // Clear existing content
135
+ await this.config.contentStorage.clearSpace(this.config.spaceId);
136
+ await this.config.syncStateStorage.clearSyncToken(this.config.spaceId);
137
+ // Export space
138
+ const exportResult = await this.exportAdapter.exportSpace({
139
+ spaceId: this.config.spaceId,
140
+ managementToken,
141
+ downloadAssets: true
142
+ });
143
+ // Store content types
144
+ if (exportResult.contentTypes && exportResult.contentTypes.length > 0) {
145
+ // eslint-disable-next-line @typescript-eslint/no-explicit-any
146
+ await this.config.contentStorage.storeContentTypes(exportResult.contentTypes);
147
+ this.logger.info('Stored content types', { count: exportResult.contentTypes.length });
148
+ }
149
+ // Store entries
150
+ if (exportResult.entries && exportResult.entries.length > 0) {
151
+ // eslint-disable-next-line @typescript-eslint/no-explicit-any
152
+ await this.config.contentStorage.storeEntries(exportResult.entries);
153
+ this.logger.info('Stored entries', { count: exportResult.entries.length });
154
+ }
155
+ // Store assets
156
+ if (exportResult.assets && exportResult.assets.length > 0) {
157
+ // eslint-disable-next-line @typescript-eslint/no-explicit-any
158
+ await this.config.contentStorage.storeAssets(exportResult.assets);
159
+ this.logger.info('Stored assets', { count: exportResult.assets.length });
160
+ }
161
+ await this.config.syncStateStorage.setSyncToken(this.config.spaceId, syncToken);
162
+ const duration = Date.now() - startTime;
163
+ const stats = {
164
+ entriesSynced: exportResult.entries?.length || 0,
165
+ assetsSynced: exportResult.assets?.length || 0,
166
+ itemsDeleted: 0,
167
+ duration,
168
+ timestamp: new Date()
169
+ };
170
+ this.logger.info('Initial sync completed', stats);
171
+ return {
172
+ success: true,
173
+ stats
174
+ };
175
+ }
176
+ catch (error) {
177
+ const duration = Date.now() - startTime;
178
+ const errorMessage = error instanceof Error ? error.message : String(error);
179
+ const failure = toServiceError(error, 'Initial sync failed', { spaceId: this.config.spaceId });
180
+ this.logger.error('Initial sync failed', failure, {
181
+ spaceId: this.config.spaceId,
182
+ errorChain: (0, logger_1.getErrorChainEntries)(failure)
183
+ });
184
+ return {
185
+ success: false,
186
+ stats: {
187
+ entriesSynced: 0,
188
+ assetsSynced: 0,
189
+ itemsDeleted: 0,
190
+ duration,
191
+ timestamp: new Date()
192
+ },
193
+ error: errorMessage
194
+ };
195
+ }
196
+ }
197
+ /**
198
+ * Perform incremental sync using CDA Sync API. A call made while a sync is running waits for
199
+ * one follow-up run, so every caller gets a sync that started after its call.
200
+ */
201
+ async incrementalSync() {
202
+ return (await this.syncOnce()).result;
203
+ }
204
+ // Single-flight: overlapping syncs would race on the sync token and could write an older delta last.
205
+ syncOnce() {
206
+ if (!this.syncRun) {
207
+ this.syncRun = this.runIncrementalSync().finally(() => {
208
+ this.syncRun = null;
209
+ });
210
+ return this.syncRun;
211
+ }
212
+ // The running sync may have fetched before the caller's change landed, so queue one more run after it.
213
+ this.syncFollowUp ??= this.syncRun.then(() => {
214
+ this.syncFollowUp = null;
215
+ return this.syncOnce();
216
+ });
217
+ return this.syncFollowUp;
218
+ }
219
+ async runIncrementalSync() {
220
+ const startTime = Date.now();
221
+ this.logger.info('Starting incremental sync', { spaceId: this.config.spaceId });
222
+ try {
223
+ const syncToken = await this.config.syncStateStorage.getSyncToken(this.config.spaceId);
224
+ if (!syncToken) {
225
+ throw new errors_1.ContentfulSyncServiceError('No sync token found - run initialSync first', {
226
+ code: 'INVALID_INPUT',
227
+ context: { spaceId: this.config.spaceId }
228
+ });
229
+ }
230
+ const syncResult = await this.cdaClient.deltaSync(syncToken);
231
+ // Check before writing anything: without a new token the old one stays valid for the retry.
232
+ if (!syncResult.nextSyncToken) {
233
+ throw new errors_1.ContentfulSyncServiceError('Contentful delta sync returned no nextSyncToken', {
234
+ code: 'OPERATION_FAILED',
235
+ context: { spaceId: this.config.spaceId }
236
+ });
237
+ }
238
+ // shortcut: CDA sync items carry sys.revision, not the CMA sys.version that ContentStorage's types declare;
239
+ // they are stored as delivered. Reconciling the storage contract is tracked separately.
240
+ for (const entry of syncResult.entries) {
241
+ await this.config.contentStorage.updateEntry(entry);
242
+ }
243
+ for (const asset of syncResult.assets) {
244
+ await this.config.contentStorage.updateAsset(asset);
245
+ }
246
+ for (const deleted of syncResult.deletedEntries) {
247
+ await this.config.contentStorage.deleteEntry(deleted.sys.id);
248
+ }
249
+ for (const deleted of syncResult.deletedAssets) {
250
+ await this.config.contentStorage.deleteAsset(deleted.sys.id);
251
+ }
252
+ const entriesSynced = syncResult.entries.length;
253
+ const assetsSynced = syncResult.assets.length;
254
+ const itemsDeleted = syncResult.deletedEntries.length + syncResult.deletedAssets.length;
255
+ await this.config.syncStateStorage.setSyncToken(this.config.spaceId, syncResult.nextSyncToken);
256
+ const duration = Date.now() - startTime;
257
+ const stats = {
258
+ entriesSynced,
259
+ assetsSynced,
260
+ itemsDeleted,
261
+ duration,
262
+ timestamp: new Date()
263
+ };
264
+ this.logger.info('Incremental sync completed', stats);
265
+ return { result: { success: true, stats } };
266
+ }
267
+ catch (error) {
268
+ const duration = Date.now() - startTime;
269
+ const errorMessage = error instanceof Error ? error.message : String(error);
270
+ const failure = toServiceError(error, 'Incremental sync failed', { spaceId: this.config.spaceId });
271
+ this.logger.error('Incremental sync failed', failure, {
272
+ spaceId: this.config.spaceId,
273
+ errorChain: (0, logger_1.getErrorChainEntries)(failure)
274
+ });
275
+ return {
276
+ result: {
277
+ success: false,
278
+ stats: {
279
+ entriesSynced: 0,
280
+ assetsSynced: 0,
281
+ itemsDeleted: 0,
282
+ duration,
283
+ timestamp: new Date()
284
+ },
285
+ error: errorMessage
286
+ },
287
+ failure
288
+ };
289
+ }
290
+ }
291
+ /**
292
+ * Verify a webhook and apply it to storage (see applyWebhookEvent)
293
+ *
294
+ * @param body - Webhook request body
295
+ * @param headers - Webhook request headers
296
+ */
297
+ async processWebhook(body, headers) {
298
+ if (!this.webhookHandler) {
299
+ throw new errors_1.ContentfulSyncServiceError('Webhook handler not configured - provide webhookSecret in config', {
300
+ code: 'INVALID_INPUT'
301
+ });
302
+ }
303
+ this.logger.info('Processing webhook event');
304
+ const result = this.webhookHandler.parseWebhook(body, headers);
305
+ if (!result.valid || !result.payload) {
306
+ throw new errors_1.ContentfulSyncServiceError('Invalid webhook signature or payload', {
307
+ code: 'INVALID_INPUT',
308
+ context: { topic: result.topic }
309
+ });
310
+ }
311
+ return this.applyWebhookEvent(result.topic ?? '', result.payload);
312
+ }
313
+ /**
314
+ * Bring storage up to date for a verified webhook event. A publish, unpublish, archive or
315
+ * delete of an Entry or Asset in this space and environment runs `incrementalSync()`; the
316
+ * payload itself is never written, because the CDA sync is ordered and Contentful's webhook
317
+ * deliveries are not (retries and duplicates can arrive late). Draft topics (create, save,
318
+ * auto_save, unarchive), ContentType topics and events for another space or environment run
319
+ * nothing and come back as `ignored`.
320
+ *
321
+ * @param topic - `x-contentful-topic`, e.g. `ContentManagement.Entry.publish` or `Entry.publish`
322
+ * @param payload - Webhook body, already verified by the caller
323
+ * @throws ContentfulSyncServiceError `INVALID_INPUT` for a malformed topic or payload,
324
+ * `OPERATION_FAILED` (sync error in `cause`) when the sync fails, so the webhook is retried
325
+ */
326
+ async applyWebhookEvent(topic, payload) {
327
+ // Contentful sends `ContentManagement.<Resource>.<action>`; the short `<Resource>.<action>` form is accepted too.
328
+ const [resource, action] = topic.split('.').slice(-2);
329
+ const id = payload?.sys?.id;
330
+ if (!resource || !action || !id) {
331
+ throw new errors_1.ContentfulSyncServiceError('Webhook topic or payload is malformed', {
332
+ code: 'INVALID_INPUT',
333
+ context: { topic, id }
334
+ });
335
+ }
336
+ const ignored = (reason) => {
337
+ this.logger.info('Webhook event ignored', { topic, id, reason });
338
+ return { topic, id, outcome: 'ignored' };
339
+ };
340
+ // Optional chaining: the payload comes off the wire, so the declared type is not a guarantee.
341
+ if (payload.sys.space?.sys.id !== this.config.spaceId || payload.sys.environment?.sys.id !== this.config.environmentId) {
342
+ return ignored('other space or environment');
343
+ }
344
+ if (resource !== 'Entry' && resource !== 'Asset') {
345
+ return ignored('resource not synced');
346
+ }
347
+ if (!['publish', 'unpublish', 'archive', 'delete'].includes(action)) {
348
+ return ignored('not a delivery change');
349
+ }
350
+ const { failure } = await this.syncOnce();
351
+ if (failure) {
352
+ throw new errors_1.ContentfulSyncServiceError('Sync for webhook event failed', {
353
+ code: 'OPERATION_FAILED',
354
+ cause: failure,
355
+ context: { topic, id }
356
+ });
357
+ }
358
+ this.logger.info('Webhook event synced', { topic, id });
359
+ return { topic, id, outcome: 'synced' };
360
+ }
361
+ /**
362
+ * Start automatic sync at configured interval
363
+ */
364
+ startAutoSync() {
365
+ if (this.syncIntervalId) {
366
+ this.logger.warn('Auto-sync already running');
367
+ return;
368
+ }
369
+ this.logger.info('Starting auto-sync', { interval: this.config.syncInterval });
370
+ this.syncIntervalId = setInterval(async () => {
371
+ await this.incrementalSync();
372
+ }, this.config.syncInterval);
373
+ }
374
+ /**
375
+ * Stop automatic sync
376
+ */
377
+ stopAutoSync() {
378
+ if (this.syncIntervalId) {
379
+ clearInterval(this.syncIntervalId);
380
+ this.syncIntervalId = undefined;
381
+ this.logger.info('Auto-sync stopped');
382
+ }
383
+ }
384
+ /**
385
+ * Shutdown the service
386
+ */
387
+ async shutdown() {
388
+ this.logger.info('Shutting down ContentfulSyncService');
389
+ this.stopAutoSync();
390
+ this.logger.info('ContentfulSyncService shutdown complete');
391
+ }
392
+ }
393
+ exports.ContentfulSyncService = ContentfulSyncService;
@@ -0,0 +1,11 @@
1
+ export type ContentfulSyncServiceErrorCode = 'CONTENTFUL_SYNC_SERVICE_ERROR' | 'INVALID_INPUT' | 'OPERATION_FAILED';
2
+ export interface ContentfulSyncServiceErrorOptions {
3
+ cause?: Error;
4
+ code?: ContentfulSyncServiceErrorCode;
5
+ context?: Record<string, unknown>;
6
+ }
7
+ export declare class ContentfulSyncServiceError extends Error {
8
+ readonly code: ContentfulSyncServiceErrorCode;
9
+ readonly context?: Record<string, unknown>;
10
+ constructor(message: string, options?: ContentfulSyncServiceErrorOptions);
11
+ }
package/dist/errors.js ADDED
@@ -0,0 +1,24 @@
1
+ "use strict";
2
+ /*
3
+ Copyright (c) 2025 Bernier LLC
4
+
5
+ This file is licensed to the client under a limited-use license.
6
+ The client may use and modify this code *only within the scope of the project it was delivered for*.
7
+ Redistribution or use in other products or commercial offerings is not permitted without written consent from Bernier LLC.
8
+ */
9
+ Object.defineProperty(exports, "__esModule", { value: true });
10
+ exports.ContentfulSyncServiceError = void 0;
11
+ class ContentfulSyncServiceError extends Error {
12
+ code;
13
+ context;
14
+ constructor(message, options) {
15
+ super(message, options?.cause ? { cause: options.cause } : undefined);
16
+ this.name = 'ContentfulSyncServiceError';
17
+ this.code = options?.code ?? 'CONTENTFUL_SYNC_SERVICE_ERROR';
18
+ if (options?.context !== undefined) {
19
+ this.context = options.context;
20
+ }
21
+ Error.captureStackTrace?.(this, this.constructor);
22
+ }
23
+ }
24
+ exports.ContentfulSyncServiceError = ContentfulSyncServiceError;
@@ -0,0 +1,37 @@
1
+ import type { ContentfulEntry as Entry, ContentfulAsset as Asset, ContentfulContentType as ContentType } from '@bernierllc/contentful-types';
2
+ import type { ContentStorage, SyncStateStorage } from './types';
3
+ /**
4
+ * In-memory implementation of ContentStorage for testing
5
+ */
6
+ export declare class InMemoryContentStorage implements ContentStorage {
7
+ private contentTypes;
8
+ private entries;
9
+ private assets;
10
+ storeContentTypes(contentTypes: ContentType[]): Promise<void>;
11
+ storeEntries(entries: Entry[]): Promise<void>;
12
+ storeAssets(assets: Asset[]): Promise<void>;
13
+ updateEntry(entry: Entry): Promise<void>;
14
+ updateAsset(asset: Asset): Promise<void>;
15
+ deleteEntry(entryId: string): Promise<void>;
16
+ deleteAsset(assetId: string): Promise<void>;
17
+ getEntry(entryId: string): Promise<Entry | null>;
18
+ getAsset(assetId: string): Promise<Asset | null>;
19
+ clearSpace(_spaceId: string): Promise<void>;
20
+ getContentTypeCount(): number;
21
+ getEntryCount(): number;
22
+ getAssetCount(): number;
23
+ getAllEntries(): Entry[];
24
+ getAllAssets(): Asset[];
25
+ getAllContentTypes(): ContentType[];
26
+ }
27
+ /**
28
+ * In-memory implementation of SyncStateStorage for testing
29
+ */
30
+ export declare class InMemorySyncStateStorage implements SyncStateStorage {
31
+ private syncTokens;
32
+ getSyncToken(spaceId: string): Promise<string | null>;
33
+ setSyncToken(spaceId: string, token: string): Promise<void>;
34
+ clearSyncToken(spaceId: string): Promise<void>;
35
+ getAllTokens(): Map<string, string>;
36
+ clear(): void;
37
+ }
@@ -0,0 +1,99 @@
1
+ "use strict";
2
+ /*
3
+ Copyright (c) 2025 Bernier LLC
4
+
5
+ This file is licensed to the client under a limited-use license.
6
+ The client may use and modify this code *only within the scope of the project it was delivered for*.
7
+ Redistribution or use in other products or commercial offerings is not permitted without written consent from Bernier LLC.
8
+ */
9
+ Object.defineProperty(exports, "__esModule", { value: true });
10
+ exports.InMemorySyncStateStorage = exports.InMemoryContentStorage = void 0;
11
+ /**
12
+ * In-memory implementation of ContentStorage for testing
13
+ */
14
+ class InMemoryContentStorage {
15
+ contentTypes = new Map();
16
+ entries = new Map();
17
+ assets = new Map();
18
+ async storeContentTypes(contentTypes) {
19
+ for (const contentType of contentTypes) {
20
+ this.contentTypes.set(contentType.sys.id, contentType);
21
+ }
22
+ }
23
+ async storeEntries(entries) {
24
+ for (const entry of entries) {
25
+ this.entries.set(entry.sys.id, entry);
26
+ }
27
+ }
28
+ async storeAssets(assets) {
29
+ for (const asset of assets) {
30
+ this.assets.set(asset.sys.id, asset);
31
+ }
32
+ }
33
+ async updateEntry(entry) {
34
+ this.entries.set(entry.sys.id, entry);
35
+ }
36
+ async updateAsset(asset) {
37
+ this.assets.set(asset.sys.id, asset);
38
+ }
39
+ async deleteEntry(entryId) {
40
+ this.entries.delete(entryId);
41
+ }
42
+ async deleteAsset(assetId) {
43
+ this.assets.delete(assetId);
44
+ }
45
+ async getEntry(entryId) {
46
+ return this.entries.get(entryId) || null;
47
+ }
48
+ async getAsset(assetId) {
49
+ return this.assets.get(assetId) || null;
50
+ }
51
+ async clearSpace(_spaceId) {
52
+ this.contentTypes.clear();
53
+ this.entries.clear();
54
+ this.assets.clear();
55
+ }
56
+ // Test helper methods
57
+ getContentTypeCount() {
58
+ return this.contentTypes.size;
59
+ }
60
+ getEntryCount() {
61
+ return this.entries.size;
62
+ }
63
+ getAssetCount() {
64
+ return this.assets.size;
65
+ }
66
+ getAllEntries() {
67
+ return Array.from(this.entries.values());
68
+ }
69
+ getAllAssets() {
70
+ return Array.from(this.assets.values());
71
+ }
72
+ getAllContentTypes() {
73
+ return Array.from(this.contentTypes.values());
74
+ }
75
+ }
76
+ exports.InMemoryContentStorage = InMemoryContentStorage;
77
+ /**
78
+ * In-memory implementation of SyncStateStorage for testing
79
+ */
80
+ class InMemorySyncStateStorage {
81
+ syncTokens = new Map();
82
+ async getSyncToken(spaceId) {
83
+ return this.syncTokens.get(spaceId) || null;
84
+ }
85
+ async setSyncToken(spaceId, token) {
86
+ this.syncTokens.set(spaceId, token);
87
+ }
88
+ async clearSyncToken(spaceId) {
89
+ this.syncTokens.delete(spaceId);
90
+ }
91
+ // Test helper methods
92
+ getAllTokens() {
93
+ return new Map(this.syncTokens);
94
+ }
95
+ clear() {
96
+ this.syncTokens.clear();
97
+ }
98
+ }
99
+ exports.InMemorySyncStateStorage = InMemorySyncStateStorage;
@@ -0,0 +1,4 @@
1
+ export { ContentfulSyncService } from './contentful-sync-service';
2
+ export { ContentfulSyncServiceError, type ContentfulSyncServiceErrorCode, type ContentfulSyncServiceErrorOptions } from './errors';
3
+ export { InMemoryContentStorage, InMemorySyncStateStorage } from './in-memory-storage';
4
+ export type { ContentStorage, SyncStateStorage, SyncServiceConfig, SyncStats, WebhookApplyResult, SyncResult } from './types';
package/dist/index.js ADDED
@@ -0,0 +1,17 @@
1
+ "use strict";
2
+ /*
3
+ Copyright (c) 2025 Bernier LLC
4
+
5
+ This file is licensed to the client under a limited-use license.
6
+ The client may use and modify this code *only within the scope of the project it was delivered for*.
7
+ Redistribution or use in other products or commercial offerings is not permitted without written consent from Bernier LLC.
8
+ */
9
+ Object.defineProperty(exports, "__esModule", { value: true });
10
+ exports.InMemorySyncStateStorage = exports.InMemoryContentStorage = exports.ContentfulSyncServiceError = exports.ContentfulSyncService = void 0;
11
+ var contentful_sync_service_1 = require("./contentful-sync-service");
12
+ Object.defineProperty(exports, "ContentfulSyncService", { enumerable: true, get: function () { return contentful_sync_service_1.ContentfulSyncService; } });
13
+ var errors_1 = require("./errors");
14
+ Object.defineProperty(exports, "ContentfulSyncServiceError", { enumerable: true, get: function () { return errors_1.ContentfulSyncServiceError; } });
15
+ var in_memory_storage_1 = require("./in-memory-storage");
16
+ Object.defineProperty(exports, "InMemoryContentStorage", { enumerable: true, get: function () { return in_memory_storage_1.InMemoryContentStorage; } });
17
+ Object.defineProperty(exports, "InMemorySyncStateStorage", { enumerable: true, get: function () { return in_memory_storage_1.InMemorySyncStateStorage; } });
@@ -0,0 +1,156 @@
1
+ import type { ContentfulEntry as Entry, ContentfulAsset as Asset, ContentfulContentType as ContentType } from '@bernierllc/contentful-types';
2
+ /**
3
+ * Storage interface for synced content
4
+ */
5
+ export interface ContentStorage {
6
+ /**
7
+ * Store content types
8
+ */
9
+ storeContentTypes(contentTypes: ContentType[]): Promise<void>;
10
+ /**
11
+ * Store entries
12
+ */
13
+ storeEntries(entries: Entry[]): Promise<void>;
14
+ /**
15
+ * Store assets
16
+ */
17
+ storeAssets(assets: Asset[]): Promise<void>;
18
+ /**
19
+ * Update a single entry
20
+ */
21
+ updateEntry(entry: Entry): Promise<void>;
22
+ /**
23
+ * Update a single asset
24
+ */
25
+ updateAsset(asset: Asset): Promise<void>;
26
+ /**
27
+ * Delete an entry by ID
28
+ */
29
+ deleteEntry(entryId: string): Promise<void>;
30
+ /**
31
+ * Delete an asset by ID
32
+ */
33
+ deleteAsset(assetId: string): Promise<void>;
34
+ /**
35
+ * Get an entry by ID
36
+ */
37
+ getEntry(entryId: string): Promise<Entry | null>;
38
+ /**
39
+ * Get an asset by ID
40
+ */
41
+ getAsset(assetId: string): Promise<Asset | null>;
42
+ /**
43
+ * Clear all content for a space
44
+ */
45
+ clearSpace(spaceId: string): Promise<void>;
46
+ }
47
+ /**
48
+ * Storage interface for sync state (tokens)
49
+ */
50
+ export interface SyncStateStorage {
51
+ /**
52
+ * Get the sync token for a space
53
+ */
54
+ getSyncToken(spaceId: string): Promise<string | null>;
55
+ /**
56
+ * Set the sync token for a space
57
+ */
58
+ setSyncToken(spaceId: string, token: string): Promise<void>;
59
+ /**
60
+ * Clear sync token for a space
61
+ */
62
+ clearSyncToken(spaceId: string): Promise<void>;
63
+ }
64
+ /**
65
+ * Configuration for ContentfulSyncService
66
+ */
67
+ export interface SyncServiceConfig {
68
+ /**
69
+ * Contentful space ID
70
+ */
71
+ spaceId: string;
72
+ /**
73
+ * Contentful environment ID
74
+ */
75
+ environmentId: string;
76
+ /**
77
+ * Contentful CDA access token
78
+ */
79
+ accessToken: string;
80
+ /**
81
+ * Content storage implementation
82
+ */
83
+ contentStorage: ContentStorage;
84
+ /**
85
+ * Sync state storage implementation
86
+ */
87
+ syncStateStorage: SyncStateStorage;
88
+ /**
89
+ * Webhook secret for validation (optional)
90
+ */
91
+ webhookSecret?: string;
92
+ /**
93
+ * Sync interval in milliseconds (default: 300000 = 5 minutes)
94
+ */
95
+ syncInterval?: number;
96
+ /**
97
+ * The service registers with NeverHub by default; register() no-ops when NeverHub is unreachable.
98
+ * `false` still opts out during 1.x.
99
+ *
100
+ * @deprecated since 1.5.0; removed in 2.0.0; drop the option, NeverHub registration is automatic.
101
+ */
102
+ neverhubEnabled?: boolean;
103
+ }
104
+ /**
105
+ * Sync statistics
106
+ */
107
+ export interface SyncStats {
108
+ /**
109
+ * Total entries synced
110
+ */
111
+ entriesSynced: number;
112
+ /**
113
+ * Total assets synced
114
+ */
115
+ assetsSynced: number;
116
+ /**
117
+ * Total items deleted
118
+ */
119
+ itemsDeleted: number;
120
+ /**
121
+ * Sync duration in milliseconds
122
+ */
123
+ duration: number;
124
+ /**
125
+ * Sync timestamp
126
+ */
127
+ timestamp: Date;
128
+ }
129
+ /**
130
+ * Sync result
131
+ */
132
+ export interface SyncResult {
133
+ /**
134
+ * Whether the sync was successful
135
+ */
136
+ success: boolean;
137
+ /**
138
+ * Sync statistics
139
+ */
140
+ stats: SyncStats;
141
+ /**
142
+ * Error message if sync failed
143
+ */
144
+ error?: string;
145
+ }
146
+ /**
147
+ * What applyWebhookEvent() did with one webhook event
148
+ */
149
+ export interface WebhookApplyResult {
150
+ /** Topic as received */
151
+ topic: string;
152
+ /** Entry or asset id */
153
+ id: string;
154
+ /** `synced` (an incremental sync ran and succeeded) or `ignored` (no delivery change for this space and environment) */
155
+ outcome: 'synced' | 'ignored';
156
+ }
package/dist/types.js ADDED
@@ -0,0 +1,9 @@
1
+ "use strict";
2
+ /*
3
+ Copyright (c) 2025 Bernier LLC
4
+
5
+ This file is licensed to the client under a limited-use license.
6
+ The client may use and modify this code *only within the scope of the project it was delivered for*.
7
+ Redistribution or use in other products or commercial offerings is not permitted without written consent from Bernier LLC.
8
+ */
9
+ Object.defineProperty(exports, "__esModule", { value: true });
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@bernierllc/contentful-sync-service",
3
- "version": "1.4.4",
3
+ "version": "1.6.0",
4
4
  "description": "Full export + incremental webhook sync to local storage for Contentful content",
5
5
  "keywords": [
6
6
  "contentful",
@@ -20,13 +20,12 @@
20
20
  "README.md"
21
21
  ],
22
22
  "dependencies": {
23
+ "@bernierllc/contentful-cda-client": "^1.5.3",
24
+ "@bernierllc/contentful-export-adapter": "^1.5.3",
23
25
  "@bernierllc/contentful-types": "^1.5.0",
24
- "@bernierllc/contentful-export-adapter": "^1.5.2",
25
- "@bernierllc/logger": "^2.0.0",
26
- "@bernierllc/message-queue": "^1.5.2",
27
- "@bernierllc/neverhub-adapter": "^0.4.1",
28
- "@bernierllc/contentful-webhook-handler": "^1.3.2",
29
- "@bernierllc/contentful-cda-client": "^1.5.2"
26
+ "@bernierllc/contentful-webhook-handler": "^1.3.3",
27
+ "@bernierllc/logger": "^2.0.1",
28
+ "@bernierllc/neverhub-adapter": "^0.4.1"
30
29
  },
31
30
  "devDependencies": {
32
31
  "@types/jest": "^29.5.14",