ti2 1.0.118 → 1.0.120

package/.windsurfrules

@@ -0,0 +1,9 @@
+This is a Node.js project using Sequelize for database operations and Express.js for the web server.
+The API is defined in api.yml using OpenAPI 3.0.2
+The database is defined in models/index.js
+The models are defined in models/
+to run tests you have to run them on the ti2 named docker container, and consider this codebase is part of node_modules/ti2 path for example : 
+`` $ docker exec ti2 bash -c "cd /ti2 && npx jest lib/__tests__/callback.js --forceExit"``
+When writing / editing tests, make use of test/utils.js and review their implementation to understand how to mock the environment.
+Do not make any changes until you 95% confident of the changes you want to make, ask follow up questions until you have that confidence.
+when sending jobs to the background worker using the queue, asssume a worker is always runing to execute it but is on a different instance thread, so to review / wait for jobs we have to use the wokrer/queue.js listJobs fn to check for it

package/CLAUDE.md

@@ -0,0 +1,166 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+Ti2 (Tourism Information Interchange) is an open-source integration framework for the tourism industry. It provides standardized functions for bookings, content, and rates management through a plugin-based architecture.
+
+## Critical Development Rules
+
+1. **Docker Container Execution**: ALL commands must be run inside the `ti2` Docker container
+   - The codebase is located at `/ti2` path inside the container
+   - This project is part of `node_modules/ti2` in the container environment
+   - **NEVER run npm, node, or npx commands on the host machine**
+   - Always use: `docker exec ti2 bash -c "cd /ti2 && <command>"`
+   - Example for npm install: `docker exec ti2 bash -c "cd /ti2 && npm install <package>"`
+
+2. **Testing Rules**:
+   - Tests MUST be run inside the Docker container
+   - Example: `docker exec ti2 bash -c "cd /ti2 && npx jest lib/__tests__/callback.js --forceExit"`
+   - Use `test/utils.js` for mocking the environment
+   - Review test utility implementations before writing/editing tests
+
+3. **Background Jobs**:
+   - Jobs sent to the queue run on a separate worker thread/instance
+   - Use `worker/queue.js` `listJobs` function to check job status
+   - Workers are always assumed to be running
+
+4. **Code Confidence**: Do not make changes until 95% confident. Ask follow-up questions if uncertain.
+
+## Public Repo Privacy
+
+- `ti2` is open source. Before opening or updating public PRs, review summaries, inline comments, release notes, or commit messages, scrub internal TourConnect service names and private implementation details.
+- Do not mention internal services such as `pyfilematch`, `filematch`, `tcoutlook`, private frontend names, private trace URLs, private customer identifiers, or internal deployment architecture in public GitHub text.
+- Use generic wording such as `upstream caller`, `internal catalog caller`, `private frontend`, `caller-side timeout`, or `internal workflow` when context is needed.
+- After editing public GitHub text, search the PR body, issue comments, review comments, and review summaries for internal service names before handoff.
+
+## Common Development Commands
+
+```bash
+# Run tests (inside Docker container)
+docker exec ti2 bash -c "cd /ti2 && npm test"
+docker exec ti2 bash -c "cd /ti2 && npx jest [test-file] --forceExit"
+
+# Run specific test file
+docker exec ti2 bash -c "cd /ti2 && npx jest controllers/__tests__/bookings.js --forceExit"
+
+# Database migrations
+docker exec ti2 bash -c "cd /ti2 && npm run sequelize db:migrate"
+
+# Generate documentation
+docker exec ti2 bash -c "cd /ti2 && npm run doc"
+```
+
+## Architecture
+
+### Core Components
+
+1. **Main Entry Point** (`index.js`):
+   - Initializes Express server on port 10010 (default)
+   - Sets up Swagger documentation at `/api-docs`
+   - Handles plugin instantiation and middleware
+   - Implements caching mechanism
+   - Manages background job processing via `worker/queue.js`
+
+2. **API Definition** (`api.yml`):
+   - OpenAPI 3.0.0 specification
+   - Defines all endpoints and schemas
+   - JWT Bearer authentication
+
+3. **Plugin System**:
+   - Two types: Integration Plugins (connect systems) and App Plugins (value-added tools)
+   - Plugins are instantiated with cache, axios, events, and configuration
+   - Plugin schemas are merged with main API schema
+   - Each plugin gets its own routes under `/apps/{pluginName}`
+
+4. **Controllers** (`controllers/`):
+   - `app.js`: Core application controller
+   - `bookings.js`: Booking operations (searchProducts, createBooking, etc.)
+   - `admin.js`: Administrative functions
+   - `user.js`: User management
+   - `ping.js`: Health check endpoints
+   - `allotment.js`: Allotment management
+
+5. **Database Models** (`models/`):
+   - Sequelize ORM with MySQL
+   - Key models: Integration, User, UserAppKey, CronJobs, ApiCronJobs
+   - Migrations in `migrations/` directory
+
+6. **Worker System** (`worker/`):
+   - Bull queue for background job processing
+   - Redis-backed job queue
+   - Supports plugin jobs, API jobs, and callback jobs
+   - Multi-worker support with throng
+
+7. **Authentication** (`auth/authHandler.js`):
+   - JWT-based authentication
+   - Bearer token scheme
+
+### Key Features
+
+- **Caching**: Redis-based caching with configurable TTL
+- **Event System**: EventEmitter2 for plugin communication
+- **Background Jobs**: Async job processing with Bull/Redis
+- **Cron Jobs**: Scheduled task execution
+- **Multi-tenancy**: Support for multiple integrations/plugins
+
+### Plugin Development
+
+Plugins must implement standard methods:
+- `validateToken`: Token validation
+- `tokenTemplate`: Token configuration template
+- Booking methods: `searchProducts`, `searchAvailability`, `createBooking`, `cancelBooking`
+- Content methods: `getProducts`, `getProduct`, `createProduct`, `updateProduct`
+
+### Testing Strategy
+
+- Jest testing framework
+- Test fixtures in `__fixtures__/`
+- Mock plugins in `__mocks__/`
+- Use `test/utils.js` for creating test environments
+- Tests organized by controller in `controllers/__tests__/`
+
+## Database
+
+- Sequelize ORM with MySQL/MariaDB
+- Connection managed through `models/db.js`
+- Migrations handled via sequelize-cli
+- Models auto-loaded from `models/` directory
+
+## Environment Variables
+
+Key variables:
+- `PORT`: Server port (default: 10010)
+- `REDIS_URL`: Redis connection (default: redis://redis:6379)
+- `jwtSecret`: JWT signing secret
+- `adminKey`: Admin authentication key
+- `WEB_CONCURRENCY`: Number of workers
+- `MAX_JOBS_PER_WORKER`: Job concurrency per worker
+- `SSL_INSECURE_ALLOWED_DOMAINS`: Pipe-separated list of domains where SSL certificate verification is skipped (e.g., `localhost|staging.example.com|192.168.1.100`)
+
+Plugin-specific variables follow pattern: `ti2_{pluginName}_{setting}`
+
+### SSL Configuration
+
+The `SSL_INSECURE_ALLOWED_DOMAINS` environment variable allows you to specify domains where SSL certificate verification should be skipped. This is useful for:
+- Development environments with self-signed certificates
+- Testing against staging servers with invalid certificates
+- Local development with HTTPS
+
+Example usage:
+```bash
+# Skip SSL verification for specific domains
+export SSL_INSECURE_ALLOWED_DOMAINS="localhost|staging.api.com|192.168.1.100"
+
+# Multiple domains separated by pipe
+export SSL_INSECURE_ALLOWED_DOMAINS="dev.example.com|test.example.com"
+```
+
+**Important Implementation Notes**:
+- SSL configuration is ONLY applied to plugin axios instances (for external API calls) and worker axios instances
+- Internal communication (callbacks, per-request axios) do NOT use SSL configuration  
+- SSL configuration uses a per-request interceptor approach for better isolation
+- The configuration checks domains dynamically on each request
+
+**Warning**: Only use this in development/testing environments. Never skip SSL verification in production for security reasons.

package/controllers/__tests__/bookings-searchProducts.js

@@ -198,6 +198,26 @@ describe('user: bookings controller - searchProducts', () => {
         });
         emitSpy.mockRestore();
       });
+
+      it('includes raw scope only on legacy cache-save events', async () => {
+        const emitSpy = jest.spyOn(travelgatePlugin.events, 'emit');
+        await doApiPost({
+          url: `/products/${testAppName}/${testUserId}/${testHint}/search`,
+          token: userToken,
+          payload: { forceRefresh: true },
+        });
+        const cacheSavePayload = emitSpy.mock.calls
+          .filter(call => call[0] === 'bookingsProductSearch:cache:save')
+          .map(call => call[1])
+          .find(payload => payload.action === 'cache_saved');
+
+        expect(cacheSavePayload).toBeTruthy();
+        expect(cacheSavePayload.userId).toBe(testUserId);
+        expect(cacheSavePayload.hint).toBe(testHint);
+        expect(cacheSavePayload.userIdHash).toBeTruthy();
+        expect(cacheSavePayload.hintHash).toBeTruthy();
+        emitSpy.mockRestore();
+      });
     });
     describe('doNotCallPluginForProducts flag', () => {
       const doNotCallHint = 'hint_for_doNotCallPluginForProducts';
@@ -852,6 +872,85 @@ describe('Bookings Product Search Lock Mechanism (Job Queuing on Stale Cache)',
     expect(addJob).not.toHaveBeenCalled();
   });
 
+  it('does not save a cold optionId hydration as the full product catalog', async () => {
+    await clearProductSearchCache();
+
+    const scopedProducts = [{
+      productId: 'scoped-product',
+      productName: 'Scoped Product',
+      options: [{
+        optionId: 'scoped-option',
+        optionName: 'Scoped Option',
+      }],
+    }];
+    const fullCatalog = [{
+      productId: 'catalog-product',
+      productName: 'Catalog Product',
+      options: [{
+        optionId: 'catalog-option',
+        optionName: 'Catalog Option',
+      }],
+    }];
+    const emitSpy = jest.spyOn(lockTestPlugin.events, 'emit');
+    lockTestPlugin.searchProducts
+      .mockResolvedValueOnce({ products: scopedProducts })
+      .mockResolvedValueOnce({ products: fullCatalog });
+
+    const scopedResult = await doApiPost({
+      url: `/products/${testAppName}/${testUserId}/${ttrTestHint}/search`,
+      token: userToken,
+      payload: { optionId: ['scoped-option'] },
+    });
+    const catalogResult = await makeProductSearchRequest();
+
+    expect(scopedResult.products).toEqual(scopedProducts);
+    expect(catalogResult.products).toEqual(fullCatalog);
+    expect(lockTestPlugin.searchProducts).toHaveBeenCalledTimes(2);
+    expect(lockTestPlugin.searchProducts.mock.calls[0][0].payload.optionId).toEqual([
+      'scoped-option',
+    ]);
+    expect(lockTestPlugin.searchProducts.mock.calls[1][0].payload.optionId).toBeUndefined();
+    expect(emitSpy).toHaveBeenCalledWith(
+      'bookingsProductSearch:cache:decision',
+      expect.objectContaining({
+        action: 'scoped_result_not_cached',
+        returnedProductCount: scopedProducts.length,
+        returnedOptionCount: 1,
+      }),
+    );
+    emitSpy.mockRestore();
+  });
+
+  it('queues an unscoped refresh when an optionId request finds stale catalog data', async () => {
+    await clearProductSearchCache();
+    const cachedProducts = [{
+      productId: 'cached-product',
+      productName: 'Cached Product',
+      options: [{
+        optionId: 'cached-option',
+        optionName: 'Cached Option',
+      }],
+    }];
+    lockTestPlugin.searchProducts.mockResolvedValueOnce({ products: cachedProducts });
+    await makeProductSearchRequest();
+    lockTestPlugin.searchProducts.mockClear();
+    addJob.mockClear();
+    await new Promise(resolve => {
+      setTimeout(resolve, 1500);
+    });
+
+    const result = await doApiPost({
+      url: `/products/${testAppName}/${testUserId}/${ttrTestHint}/search`,
+      token: userToken,
+      payload: { optionId: ['cached-option'] },
+    });
+
+    expect(result.products).toEqual(cachedProducts);
+    expect(lockTestPlugin.searchProducts).not.toHaveBeenCalled();
+    expect(addJob).toHaveBeenCalledTimes(1);
+    expect(addJob.mock.calls[0][0].payload.payload).toEqual({ forceRefresh: false });
+  });
+
   it('concurrent forceRefresh requests should wait for one plugin fetch', async () => {
     await clearProductSearchCache();
 

package/controllers/bookings.js

@@ -52,8 +52,29 @@ const sleep = ms => new Promise(resolve => setTimeout(resolve, ms));
 
 const productSearchCacheKey = ({ userId, hint }) => hash({ userId, hint, operationId: productSearchOperationId });
 
+const productSearchSelectorFields = [
+  'searchInput',
+  'optionId',
+  'productId',
+  'productName',
+  'lastUpdatedFrom',
+  'skipAccommodation',
+];
+
+const hasProductSearchSelector = payload => productSearchSelectorFields.some(field => {
+  const value = R.path([field], payload);
+  if (Array.isArray(value)) return value.length > 0;
+  if (typeof value === 'string') return Boolean(value.trim());
+  return value !== undefined && value !== null && value !== false;
+});
+
 const productCount = value => R.pathOr([], ['products'], value).length;
 
+const optionCount = value => R.pathOr([], ['products'], value).reduce(
+  (count, product) => count + R.pathOr([], ['options'], product).length,
+  0,
+);
+
 const hasProductCache = cacheContent => Boolean(cacheContent && cacheContent.products);
 
 const safeHash = value => (value === undefined || value === null ? undefined : hash(String(value)));
@@ -76,7 +97,11 @@ const emitProductSearchCacheDecision = ({ app, cacheKey, userId, hint, requestId
   };
   app.events.emit(productSearchCacheDecisionEvent, payload);
   if (legacyProductSearchCacheEvents[action]) {
-    app.events.emit(legacyProductSearchCacheEvents[action], payload);
+    app.events.emit(legacyProductSearchCacheEvents[action], {
+      ...payload,
+      userId,
+      hint,
+    });
   }
 };
 
@@ -287,7 +312,11 @@ const $bookingsProductSearch = plugins => async ({
 
   // Payload for the plugin function (func) - pass forceRefresh so plugins can trigger background cache rebuilds
   const payloadForPlugin = { ...R.omit(['forceRefresh'], originalRequestBody), forceRefresh };
-  const payloadForBackgroundJob = R.omit(['credentials'], payloadForPlugin);
+  const isScopedSearch = hasProductSearchSelector(originalRequestBody);
+  const payloadForBackgroundJob = R.omit(
+    ['credentials', ...productSearchSelectorFields],
+    payloadForPlugin,
+  );
 
   const { app, token } = await getAppAndToken({ plugins, appKey, userId, hint });
   assert(userId, 'userId is required');
@@ -355,6 +384,26 @@ const $bookingsProductSearch = plugins => async ({
 
   // Helper function to call the plugin, save cache, and return results
   const fetchFromPluginAndCache = async (reason = 'cache_miss') => {
+    if (isScopedSearch) {
+      const pluginStartedAt = Date.now();
+      const pluginResults = await func({
+        axios,
+        token,
+        payload: payloadForPlugin,
+        typeDefsAndQueries,
+        requestId,
+        userId,
+        hint,
+      });
+      emitDecision('scoped_result_not_cached', {
+        reason,
+        pluginElapsedMs: Date.now() - pluginStartedAt,
+        returnedProductCount: productCount(pluginResults),
+        returnedOptionCount: optionCount(pluginResults),
+      });
+      return pluginResults || { products: [] };
+    }
+
     const lockOwnerToken = crypto.randomBytes(16).toString('hex');
     const lockStartedAt = Date.now();
     const lockAcquired = await acquirePluginExecutionLock(lockOwnerToken);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ti2",
-  "version": "1.0.118",
+  "version": "1.0.120",
   "description": "Tourist Industry Exchange (TI2)",
   "main": "index.js",
   "scripts": {