@commercetools-frontend-extensions/operations 2.0.1 → 3.1.0

package/CHANGELOG.md

@@ -1,5 +1,25 @@
 # @commercetools-frontend-extensions/operations
 
+## 3.1.0
+
+### Minor Changes
+
+- [#1669](https://github.com/commercetools/merchant-center-operations/pull/1669) [`a4a9c65`](https://github.com/commercetools/merchant-center-operations/commit/a4a9c6513e9a0b6da7d0681f044d7640ad8f0132) Thanks [@yassinejebli](https://github.com/yassinejebli)! - feat(useFileUpload): add `total` to `validationProgress` for tracking validation progress
+
+  The `validationProgress` object returned by `useFileUpload` now includes a `total` field representing the total number of unique resources in the CSV file (counted by unique keys). This allows consumers to display progress like "Validating X of Y resources" during the job-based validation flow.
+
+- [#1669](https://github.com/commercetools/merchant-center-operations/pull/1669) [`a4a9c65`](https://github.com/commercetools/merchant-center-operations/commit/a4a9c6513e9a0b6da7d0681f044d7640ad8f0132) Thanks [@yassinejebli](https://github.com/yassinejebli)! - feat(import): server side pagination for new import flow errors
+
+### Patch Changes
+
+- [#1669](https://github.com/commercetools/merchant-center-operations/pull/1669) [`a4a9c65`](https://github.com/commercetools/merchant-center-operations/commit/a4a9c6513e9a0b6da7d0681f044d7640ad8f0132) Thanks [@yassinejebli](https://github.com/yassinejebli)! - feat: add `total` for tracking validation progress
+
+## 3.0.0
+
+### Major Changes
+
+- [#1646](https://github.com/commercetools/merchant-center-operations/pull/1646) [`c1c381d`](https://github.com/commercetools/merchant-center-operations/commit/c1c381da4aa5c47c6fcc63940f164defe08170bd) Thanks [@yassinejebli](https://github.com/yassinejebli)! - feat: introduce the new Import job flow
+
 ## 2.0.1
 
 ### Patch Changes

package/README.md

@@ -2,75 +2,97 @@
 
 Shared functionality for import/export operations across multiple frontend applications and extensions.
 
+## Installation
+
+```bash
+pnpm add @commercetools-frontend-extensions/operations
+```
+
 ## Hooks
 
-### `useImportContainerUpload`
+### `useFileUpload`
 
-Handles file upload to import containers with automatic cleanup on errors.
+A unified hook for file uploads that handles both the old flow (import container) and new flow (file import job).
 
 ```typescript
-const { upload, abort, isUploading, progress } = useImportContainerUpload({
-  projectKey: string
+const { upload, isUploading, progress, validationProgress } = useFileUpload({
+  projectKey: string,
+  useJobBasedFlow?: boolean,
+  pollingInterval?: number,
+  maxPollingAttempts?: number
 })
 ```
 
 **Parameters:**
 - `projectKey` (required): The commercetools project key
+- `useJobBasedFlow` (optional): Whether to use the job-based flow. Default: `false`
+- `pollingInterval` (optional): Polling interval in ms for job-based flow. Default: `5000`
+- `maxPollingAttempts` (optional): Maximum polling attempts. Default: `120`
 
 **Returns:**
-- `upload`: Function to start file upload
-- `abort`: Function to cancel ongoing upload
-- `isUploading`: Boolean indicating upload state
-- `progress`: Upload progress (0-100)
+- `upload` - Function to start the upload
+- `isUploading` - Whether upload is in progress
+- `progress` - Upload progress (0-100)
+- `validationProgress` - `{ processed: number, total: number, isValidating: boolean }` (job-based flow only)
+  - `processed`: Number of resources validated so far (from backend)
+  - `total`: Total number of unique resources in the file (counted by unique keys in the CSV)
+  - `isValidating`: Whether validation is in progress
 
 **Usage:**
 ```typescript
-const projectKey = useApplicationContext((context) => context.project?.key)
-const { upload, isUploading, progress } = useImportContainerUpload({ 
-  projectKey: projectKey! 
+import { useFileUpload } from '@commercetools-frontend-extensions/operations'
+
+const { upload, isUploading, progress, validationProgress } = useFileUpload({ 
+  projectKey,
+  useJobBasedFlow: isFeatureFlagEnabled
 })
 
+const abortController = new AbortController()
+
 await upload({
   file: File,
   resourceType: 'product' | 'category' | ...,
   settings?: {
     format?: 'CSV' | 'JSON',
     decimalSeparator?: '.' | ',',
-    resourceType?: string,  // example: 'product-draft' for products
+    resourceType?: 'product-draft' | 'category' | ...,
     options?: {
-      publishAllChanges?: boolean,      // product-specific
-      unpublishAllChanges?: boolean     // product-specific
+      publishAllChanges?: boolean,
+      unpublishAllChanges?: boolean
     }
   },
-  onSuccess: (response, containerKey) => { /* ... */ },
+  abortSignal: abortController.signal,
+  onSuccess: (result) => {
+    // result.containerKey - Import container key
+    // result.summary - { total, valid, invalid, fields, ignoredFields, results }
+    // result.jobId - Job ID (job-based flow only)
+    // result.job - Full job object (job-based flow only)
+  },
   onError: (error) => { /* ... */ },
-  onProgress: (progress) => { /* ... */ }
+  onProgress: (progress) => { /* 0-100 */ },
+  onValidationProgress: (job) => { /* job-based flow only */ }
 })
+
+// To cancel:
+abortController.abort()
 ```
 
 ---
 
 ### `useFetchImportOperations`
 
-Fetches import operations for a specific container with optional polling.
+Fetches import operations for a container with optional polling.
 
 ```typescript
-const { data, error, isLoading, refetch, lastFetchTime } = useFetchImportOperations({
-  projectKey: string,
-  importContainerKey: string,
-  queryParams?: ImportOperationQueryParams,
+const { data, error, isLoading, refetch } = useFetchImportOperations({
+  projectKey,
+  importContainerKey,
+  queryParams?: { limit, offset, ... },
   pollingInterval?: number,
   shouldContinuePolling?: (data) => boolean
 })
 ```
 
-**Parameters:**
-- `projectKey` (required): The commercetools project key
-- `importContainerKey` (required): The import container key
-- `queryParams`: Optional query parameters (limit, offset, etc.)
-- `pollingInterval`: Polling interval in milliseconds
-- `shouldContinuePolling`: Function to determine if polling should continue
-
 ---
 
 ### `useFetchImportSummaries`
@@ -78,41 +100,29 @@ const { data, error, isLoading, refetch, lastFetchTime } = useFetchImportOperati
 Fetches import summaries (containers) with optional polling.
 
 ```typescript
-const { data, error, isLoading, refetch, lastFetchTime } = useFetchImportSummaries({
-  projectKey: string,
-  queryParams?: ImportContainerQueryParams,
+const { data, error, isLoading, refetch } = useFetchImportSummaries({
+  projectKey,
+  queryParams?: { limit, offset, states, ... },
   pollingInterval?: number,
   shouldContinuePolling?: (data) => boolean
 })
 ```
 
-**Parameters:**
-- `projectKey` (required): The commercetools project key
-- `queryParams`: Optional query parameters (limit, offset, states, etc.)
-- `pollingInterval`: Polling interval in milliseconds
-- `shouldContinuePolling`: Function to determine if polling should continue
-
 ---
 
 ### `useFetchImportContainerDetails`
 
-Fetches details for a specific import container with optional polling.
+Fetches details for a specific import container.
 
 ```typescript
-const { data, error, isLoading, refetch, lastFetchTime } = useFetchImportContainerDetails({
-  projectKey: string,
-  importContainerKey: string,
+const { data, error, isLoading, refetch } = useFetchImportContainerDetails({
+  projectKey,
+  importContainerKey,
   pollingInterval?: number,
   shouldContinuePolling?: (data) => boolean
 })
 ```
 
-**Parameters:**
-- `projectKey` (required): The commercetools project key
-- `importContainerKey` (required): The import container key
-- `pollingInterval`: Polling interval in milliseconds
-- `shouldContinuePolling`: Function to determine if polling should continue
-
 ---
 
 ### `useFetchExportOperations`
@@ -120,50 +130,107 @@ const { data, error, isLoading, refetch, lastFetchTime } = useFetchImportContain
 Fetches export operations with optional polling.
 
 ```typescript
-const { data, error, isLoading, refetch, lastFetchTime } = useFetchExportOperations({
-  projectKey: string,
-  queryParams?: ExportOperationQueryParams,
+const { data, error, isLoading, refetch } = useFetchExportOperations({
+  projectKey,
+  queryParams?: { limit, offset, resourceTypes, ... },
   pollingInterval?: number,
   shouldContinuePolling?: (data) => boolean
 })
 ```
 
-**Parameters:**
-- `projectKey` (required): The commercetools project key
-- `queryParams`: Optional query parameters (limit, offset, resourceTypes, etc.)
-- `pollingInterval`: Polling interval in milliseconds
-- `shouldContinuePolling`: Function to determine if polling should continue
-
 ---
 
-## Notes
+### `useFetchFileImportJob`
+
+Polls a File Import Job for status updates.
 
-- **All hooks require `projectKey`** to be passed explicitly (retrieved from `useApplicationContext`)
-- **Polling is opt-in** via `pollingInterval` parameter
-- **Standard return structure**: All fetch hooks return `{ data, error, isLoading, refetch, lastFetchTime }`
-- **Automatic cleanup**: `useImportContainerUpload` automatically cleans up containers on upload errors
-- **File upload limits**: The package exports backend-recommended limits enforced in frontend validation:
-  - `MAX_FILE_SIZE_MB` (35 MB)
-  - `MAX_ROW_COUNT` (80,000)
-- **Internal use only**: This package also exports additional components, utilities, types, and API functions that are intended for internal use within the operations repository (operations app and frontend extensions). Only the hooks documented above are considered stable public API for external consumption.
+```typescript
+const { data, error, isLoading, refetch } = useFetchFileImportJob({
+  projectKey,
+  importContainerKey,
+  jobId,
+  pollingInterval?: number,
+  shouldContinuePolling?: (job) => boolean
+})
+```
+
+**Job states:** `queued` → `processing` → `validated` → `initialising` → `ready`
 
 ---
 
-## Common patterns
+### `useFetchFileImportJobRecords`
 
-### Polling example
+Fetches records (errors or valid entries) from a File Import Job with pagination support. Keeps previous data visible while loading new pages.
+
+```typescript
+const { data, error, isLoading, refetch } = useFetchFileImportJobRecords({
+  projectKey,
+  importContainerKey,
+  jobId,
+  limit?: number,
+  offset?: number,
+  isValid?: boolean,
+  skip?: boolean
+})
+```
+
+**Parameters:**
+- `projectKey`: The commercetools project key
+- `importContainerKey`: The import container key
+- `jobId`: The file import job ID
+- `limit` (optional): Number of records to fetch per page
+- `offset` (optional): Offset for pagination
+- `isValid` (optional): Filter by valid (`true`) or invalid (`false`) records
+- `skip` (optional): Skip fetching (useful for conditional fetching)
+
+**Returns:**
+- `data` - `{ results, total, limit, offset, count }` or `null`
+- `error` - Error object if fetch failed
+- `isLoading` - Whether fetch is in progress
+- `refetch` - Function to manually trigger a refetch
 
+**Usage example (paginated error table):**
 ```typescript
-const POLL_INTERVAL = 5000 // 5 seconds
+const pagination = usePaginationState()
+const offset = (pagination.page.value - 1) * pagination.perPage.value
 
+const { data, isLoading } = useFetchFileImportJobRecords({
+  projectKey,
+  importContainerKey: containerKey,
+  jobId,
+  offset,
+  limit: pagination.perPage.value,
+  isValid: false, // Fetch only invalid records (errors)
+})
+
+```
+
+---
+
+## Helper Functions
+
+```typescript
+import {
+  isImportJobQueued,
+  isImportJobProcessing,
+  isImportJobValidated,
+  isImportJobInitializing,
+  isImportJobReady,
+  shouldContinuePollingForImportValidation
+} from '@commercetools-frontend-extensions/operations'
+```
+
+---
+
+## Polling Example
+
+```typescript
 const { data, isLoading } = useFetchImportOperations({
   projectKey,
   importContainerKey,
-  pollingInterval: POLL_INTERVAL,
+  pollingInterval: 5000,
   shouldContinuePolling: (data) => {
-    // Stop polling when all operations are completed
     return data?.results?.some(op => op.state === 'processing') ?? false
   }
 })
 ```
-