ti2 1.0.122 → 1.0.124

package/README.md

@@ -65,6 +65,35 @@ Plugins are the connectores to other systems and/or features you intent to use;
 |searchItineraries|||||||||✓|
 |queryAllotment|||||||||✓|
 
+### Product catalog refresh contract
+
+`POST /products/{appKey}/{userId}/{hint}/search` returns a canonical object with a
+`products` array. Use `cacheOnly: true` to read Ti2's current catalog without
+calling the integration; `cacheFound` distinguishes a cache miss from a cached
+empty catalog.
+
+A forced request is a full-catalog refresh by default. `fullSyncTrigger` may be
+`scheduled`, `manual`, or `organic`; an omitted trigger on a forced request is
+treated as manual. Manual refreshes require a non-blank `admissionOverrideReason`.
+Scheduled and manual refreshes reject `searchInput`, `optionId`, `productId`,
+`productName`, or `lastUpdatedFrom` selectors. An empty `searchInput` or
+`searchInput: "*"` represents the full catalog. Scheduler-owned requests may
+also carry `fullSyncStartedAt` and `fullSyncAdmissionToken`.
+
+Unscoped forced responses include `catalogRefreshOutcome`, `cacheUpdated`,
+`cachePreserved`, and `cachedProductCount`. Consumers that update a downstream
+catalog should use these fields instead of assuming that HTTP 200 or the returned
+`products.length` means Ti2 replaced its cache. Empty or partial plugin results
+can preserve an existing catalog. If a concurrent refresh remains in progress
+past the lock wait, Ti2 serves the retained catalog with
+`catalogRefreshOutcome: "refresh_in_progress_cache_served"`. This outcome is
+non-terminal: background-job `success` means the HTTP call completed, not that
+the catalog was written.
+
+The product cache refresh interval defaults to seven days when no TTR is set.
+Any explicit `ttlForProducts` or plugin `cacheSettings.bookingsProductSearch.ttr`
+value is honored, including `86400` for a one-day interval.
+
 ## Contributing
 
 Contributions are welcome and ecouraged.
@@ -94,4 +123,3 @@ TL;DR Here's what the license entails:
 7. Any modifications of this code base MUST be distributed with the same license, GPLv3.
 8. This software is provided without warranty.
 9. The software author or license can not be held liable for any damages inflicted by the software.
-

package/__tests__/plugin-capabilities.js

@@ -0,0 +1,42 @@
+/* globals describe expect it */
+
+const createApp = require('../index');
+
+class SelectivePlugin {
+  constructor({ cache, events, name }) {
+    this.cache = cache;
+    this.events = events;
+    this.name = name;
+  }
+}
+
+describe('host-owned plugin capabilities', () => {
+  it('assigns configured capabilities when a plugin ignores unknown constructor fields', async () => {
+    const app = await createApp({
+      pluginCapabilities: {
+        selective: { weeklyProductCatalogSync: true },
+      },
+      plugins: { selective: SelectivePlugin },
+      startServer: false,
+    });
+
+    expect(app.plugins[0].capabilities).toEqual({
+      weeklyProductCatalogSync: true,
+    });
+  });
+
+  it('assigns configured capabilities to pre-instantiated plugins', async () => {
+    const plugin = { name: 'instantiated' };
+    const app = await createApp({
+      pluginCapabilities: {
+        instantiated: { weeklyProductCatalogSync: true },
+      },
+      pluginsInstantiated: [plugin],
+      startServer: false,
+    });
+
+    expect(app.plugins[0].capabilities).toEqual({
+      weeklyProductCatalogSync: true,
+    });
+  });
+});

package/api.yml

@@ -275,6 +275,11 @@ components:
     appMethods:
       type: object
       properties:
+        capabilities:
+          type: object
+          properties:
+            weeklyProductCatalogSync:
+              type: boolean
         methods:
           type: array
           items:
@@ -426,15 +431,56 @@ components:
     bookingsProductQuery:
       type: object
       additionalProperties: true
-      description: Additional properties may be included
+      description: Product search or full-catalog refresh request. Scheduled and manual force refreshes must not include selectors.
       properties:
+        searchInput:
+          type: string
+          description: Free-text product selector. An empty value or "*" means the full catalog.
+        optionId:
+          description: Option ID selector
+          oneOf:
+            - type: string
+            - type: array
+              items:
+                type: string
+        productId:
+          type: string
+          description: Product ID selector
         productName:
           type: string
-          required: false
           description: Product Name wildcard match
           example: '*bus*'
+        lastUpdatedFrom:
+          description: Return products updated since this value
+        forceRefresh:
+          type: boolean
+          default: false
+          description: Call the plugin directly. Full-catalog refreshes require the app's weeklyProductCatalogSync capability and default to a manual trigger when fullSyncTrigger is omitted. Manual refreshes require admissionOverrideReason.
+        cacheOnly:
+          type: boolean
+          default: false
+          description: Return the current Ti2 cache without calling the plugin. cacheFound distinguishes a miss from a cached empty catalog.
+        backgroundJob:
+          type: boolean
+          default: false
+          description: Execute the request through the Ti2 worker.
+        fullSyncTrigger:
+          type: string
+          description: Catalog refresh owner. Forced requests default to manual when omitted. Scheduled and manual triggers reject selector fields.
+          enum:
+            - organic
+            - scheduled
+            - manual
+        admissionOverrideReason:
+          type: string
+          description: Required non-blank audit reason for a manual refresh override.
+        fullSyncStartedAt:
+          type: number
+          description: Start time supplied by the full-sync admission owner.
+        fullSyncAdmissionToken:
+          type: string
+          description: Opaque token supplied by the full-sync admission owner.
         omitServiceCodes:
-          required: false
           description: Service codes to omit from the product catalog. When the integration has productSearchOmitServiceCodes, that list is used instead.
           example: ['AC', 'SM']
           oneOf:
@@ -443,25 +489,42 @@ components:
               items:
                 type: string
         credentials:
-          required: false
           $ref: '#/components/schemas/integrationCredentials'
     bookingsProductSearchReturn:
       type: object
       additionalProperties: true
-      description: Additional properties may be included
+      required:
+        - products
+      description: Canonical product search response. Additional properties may be included.
       properties:
         products:
           type: array
-          required: false
-          description: Product Search Results
+          description: Effective Ti2 catalog; preserved refreshes return the retained cached products.
           items: 
             type: object
-        accommodation:
-          type: array
-          required: false,
-          decription: Accommodation search results
-          items:
-            type: object
+        cacheFound:
+          type: boolean
+          description: For cacheOnly requests, whether Ti2 had a cached catalog (including an intentionally cached empty catalog).
+        catalogRefreshOutcome:
+          type: string
+          description: Outcome of an unscoped forceRefresh catalog operation
+          enum:
+            - cache_updated
+            - empty_cache_saved
+            - empty_result_preserved_cache
+            - partial_result_preserved_cache
+            - partial_result_not_cached
+            - refresh_in_progress_cache_served
+        cacheUpdated:
+          type: boolean
+          description: Whether this forceRefresh wrote the returned catalog to Ti2 cache
+        cachePreserved:
+          type: boolean
+          description: Whether this forceRefresh retained a prior catalog instead of replacing it
+        cachedProductCount:
+          type: integer
+          nullable: true
+          description: Product count in the Ti2 cache after the forceRefresh decision
     pricingItem:
       type: object
       additionalProperties: true

package/controllers/README.md

@@ -1,6 +1,6 @@
 # Bookings Controller Caching and Locking Strategy
 
-The `$bookingsProductSearch` function in `controllers/bookings.js` implements a sophisticated caching and locking strategy to optimize product searches, manage stale data, and prevent redundant operations. This document outlines its core flow and the locking mechanisms involved.
+The `$bookingsProductSearch` function in `controllers/bookings.js` implements a caching and locking strategy to optimize product searches, manage stale data, and prevent redundant plugin calls. This document outlines its core flow.
 
 ## Core Flow of `$bookingsProductSearch`
 
@@ -9,63 +9,53 @@ The `$bookingsProductSearch` function in `controllers/bookings.js` implements a
     *   Determines the specific plugin function to call for product search (e.g., `searchProducts` or `searchProductsForItinerary`).
     *   Injects the integration's configured `productSearchOmitServiceCodes` into the plugin payload so every caller for that token omits the same service codes. When the setting is empty, the request body is unchanged.
     *   Calculates a `cacheKey` based on `userId`, `hint`, and a static `operationId` (`bookingsProductSearch`). Changing the omit setting does not change the key; clear that integration's product-search cache and resync.
-    *   Defines two distinct lock keys derived from this `cacheKey`:
+    *   Defines a lock key derived from this `cacheKey`:
         *   `pluginExecutionLockKey` (resolves to `${cacheKey}:lock`): Used to serialize direct calls to the plugin.
-        *   `jobQueueLockKey` (resolves to `${cacheKey}:jobLock`): Used to prevent multiple submissions of background refresh jobs.
+        *   `${pluginExecutionLockKey}:outcome`: Short-lived refresh metadata used by concurrent waiters.
     *   Fetches the current cache content (`initialActualCacheContent`) and its `lastUpdated` timestamp.
     *   Determines if the cache is stale (`isStaleByTTR`) based on the Time-To-Refresh (`ttr`) value from the token or plugin settings.
     *   Checks for a `doNotCallPluginForProducts` flag (from token or plugin settings) and whether a `pluginExecutionLockKey` is currently active.
 
 2.  **Request Handling Logic (Simplified Order):**
 
-    *   **Condition 1: `doNotCallPluginForProducts` is true AND NOT `forceRefresh`**:
+    *   **Condition 1: `cacheOnly` is true**:
+        *   Returns the current cache without calling the plugin. `cacheFound` distinguishes a cache miss from an intentionally cached empty catalog.
+
+    *   **Condition 2: `doNotCallPluginForProducts` is true AND NOT `forceRefresh`**:
         *   If this flag is set and the request is not a forced refresh, the system serves data directly from the cache if available.
         *   If the cache is empty, it returns an empty product list.
-        *   No plugin calls are made, and no background jobs are queued in this path.
+        *   No plugin calls are made in this path.
 
-    *   **Condition 2: `forceRefresh` is true**:
-        *   The system attempts to fetch fresh data directly from the plugin.
+    *   **Condition 3: `forceRefresh` is true**:
+        *   This is the catalog refresh owner. The system fetches fresh data from the plugin.
+        *   Ti2 rejects the request unless the host declared `weeklyProductCatalogSync: true` for the plugin. Manual audit reasons may override timing admission, but not catalog-completeness eligibility.
+        *   An omitted `fullSyncTrigger` is treated as `manual`, and manual refreshes require a non-blank `admissionOverrideReason`. Weekly scheduler requests must explicitly send `fullSyncTrigger: scheduled`.
+        *   Unless `fullSyncTrigger: organic` is explicit, a force refresh is a scheduled or manual full-catalog request and rejects `searchInput`, `optionId`, `productId`, `productName`, and `lastUpdatedFrom` selectors. `searchInput: "*"` is unscoped.
         *   The `fetchFromPluginAndCache` helper function is invoked. This function:
             *   Sets the `pluginExecutionLockKey` before calling the plugin to prevent other concurrent direct calls.
             *   Calls the plugin's product search method.
             *   If the plugin returns valid products, these are saved to the cache (both `cacheKey` for data and `${cacheKey}:lastUpdated` for timestamp).
+            *   If the plugin returns empty or partial products and a non-empty cache already exists, the existing cache is kept and returned.
+            *   Concurrent waiters return the same effective catalog and refresh outcome. If the wait expires while the leader is still active, the waiter returns the retained cache with `refresh_in_progress_cache_served`.
             *   Drops the `pluginExecutionLockKey` after completion.
-        *   The (potentially filtered) results from the plugin are returned to the client.
+        *   The response includes `catalogRefreshOutcome`, `cacheUpdated`, `cachePreserved`, and `cachedProductCount`. `catalogRefreshOutcome` is authoritative; background-job `success` only means the HTTP request completed and does not prove a terminal catalog write.
 
-    *   **Condition 3: Cache Exists AND NOT `forceRefresh`**:
-        *   **Sub-condition 3a: Cache is fresh OR `pluginExecutionLockKey` is active**:
-            *   If the cache is not stale according to its TTR, or if a direct plugin execution (like a `forceRefresh` or initial population) is already in progress (indicated by an active `pluginExecutionLockKey`), the system serves data from the existing cache.
-        *   **Sub-condition 3b: Cache is stale AND no `pluginExecutionLockKey` is active**:
-            *   The system checks for an active `jobQueueLockKey`.
-            *   If `jobQueueLockKey` IS active: It implies that another request very recently detected the stale cache and has already queued a background refresh job. The current request serves the stale data from the cache without queueing another job.
-            *   If `jobQueueLockKey` is NOT active: This request is the first (or among the first) to find the stale cache without an ongoing direct refresh or a recently queued job. It will:
-                1.  Set the `jobQueueLockKey` with a short TTL (e.g., 60 seconds).
-                2.  Queue a background job using `addJob`. This job will eventually call the plugin to refresh the data and then use `$updateProductSearchCache` to update the cache.
-                3.  Serve the stale data from the cache to the current client.
+    *   **Condition 4: Cache Exists AND NOT `forceRefresh`**:
+        *   Organic search never starts a catalog refresh.
+        *   If the cache is fresh, or a plugin fetch is already running, serve the cache.
+        *   If the cache is stale, serve it while the scheduler separately owns the next `forceRefresh` catalog sync.
 
-    *   **Condition 4: No Cache Content AND NOT `forceRefresh` AND NOT `doNotCallPluginForProducts`**:
-        *   If there's no existing cache content and none of the preceding conditions (like `forceRefresh` or `doNotCallPluginForProducts`) were met, the system needs to populate the cache.
+    *   **Condition 5: No Cache Content AND NOT `forceRefresh` AND NOT `doNotCallPluginForProducts`**:
+        *   Cache miss still fetches from the plugin so search can return products.
         *   It calls `fetchFromPluginAndCache` (which sets `pluginExecutionLockKey`, calls the plugin, caches results, and drops the lock) to get initial data.
         *   The (potentially filtered) results are returned to the client.
 
 ## Locking Mechanisms Explained
 
-The system uses two types of locks, both based on the primary `cacheKey`, to manage concurrency and prevent redundant operations:
-
 1.  **`pluginExecutionLockKey` (derived from `${cacheKey}:lock`)**:
-    *   **Purpose**: To prevent multiple simultaneous *direct calls* to the external plugin for the same product search parameters. This is crucial during `forceRefresh` scenarios or when the cache is being populated for the first time by concurrent requests.
+    *   **Purpose**: To prevent multiple simultaneous *direct calls* to the external plugin for the same product search parameters. This is used during `forceRefresh` and first-time cache population.
     *   **Behavior**:
         *   This lock is set by the `fetchFromPluginAndCache` helper function immediately before it makes an actual call to the plugin's `searchProducts` (or equivalent) method.
         *   It is configured with a TTL (e.g., 120 seconds) to ensure it automatically expires if the process holding the lock crashes or fails to release it.
         *   The lock is explicitly dropped by `fetchFromPluginAndCache` after the plugin call completes (whether successfully or with an error).
-        *   Other parts of the main logic (e.g., in Condition 3a) check for the presence of this lock (`hasPluginExecutionLock`). If active, it signals that a direct plugin data fetch is already in progress, prompting the current request to, for example, serve stale data or wait, rather than initiating another direct plugin call.
-
-2.  **`jobQueueLockKey` (derived from `${cacheKey}:jobLock`)**:
-    *   **Purpose**: To prevent the submission of multiple identical *background refresh jobs* by nearly simultaneous requests when the cache is found to be stale and no direct plugin execution (covered by `pluginExecutionLockKey`) is active.
-    *   **Behavior**:
-        *   This lock is checked specifically when the cache is determined to be effectively stale (`isEffectivelyStale`) and no `pluginExecutionLockKey` is currently active (Condition 3b).
-        *   If the `jobQueueLockKey` is NOT found in the cache, the current request assumes responsibility for queuing the refresh job. It sets this lock with a short TTL (e.g., 60 seconds) *before* calling `addJob`.
-        *   If the `jobQueueLockKey` IS found, it indicates that another request has very recently detected the stale state and has already taken action to queue the refresh job. The current request will then proceed to serve stale data without attempting to queue another job.
-        *   This mechanism ensures that even if numerous requests detect stale data at virtually the same moment, only one of them will succeed in setting the `jobQueueLockKey` and thereby be responsible for queueing the single background refresh task.
-
-These two locks work in tandem: `pluginExecutionLockKey` manages contention for direct, immediate plugin interactions, while `jobQueueLockKey` manages contention for initiating background refresh tasks when stale data is being served. This dual-lock strategy helps maintain system performance and avoids overwhelming external plugin services or the background job queue.
+        *   Other parts of the main logic (e.g., in Condition 3) check for the presence of this lock (`hasPluginExecutionLock`). If active, it signals that a direct plugin data fetch is already in progress, prompting the current request to serve cache rather than initiating another direct plugin call.