@clidey/dory 0.1.0 → 0.3.0

package/docs/configuration/installation-setup/installation.mdx

@@ -1,127 +0,0 @@
----
-title: 'Installation & Setup Guide'
-description: 'Steps to clone, install dependencies, run the application, and prepare the database.'
----
-
-## Installation & Setup Guide
-
-### Overview
-This guide covers the installation and setup of the Genie application backend and frontend components essential for onboarding, data synchronization, and product variant management.
-
-Genie integrates tightly with Shopify stores to synchronize product, inventory, bundle, and purchase order data. It handles onboarding workflows, monitors product variant counts, and manages webhooks for real-time data sync.
-
-### Components Involved
-- **Onboarding Service**: Manages user onboarding progress, data synchronization, and notification workflows.
-- **Shopify Webhooks Service**: Registers and manages Shopify webhook subscriptions for data updates.
-- **Database Views and Migrations**: Defines database schemas, especially for inventory data and product variant tracking.
-- **Frontend Modals and UI Components**: Provide user feedback and guides, such as store size warnings when variant count is large.
-
----
-
-## Backend Installation & Setup
-
-### 1. Database Setup and Migrations
-
-Genie uses a PostgreSQL database with TypeORM. Core migrations include:
-
-- **Initial Database Schema**: Creates tables for users, teams, products, product variants, purchase orders, and related entities. [Initialise Migration](apps/api/src/migrations/db/1688040345407-Initialise.ts)
-- **Inventory Item View**: Creates a SQL view consolidating product and variant data for inventory management, including sales and stock metrics. This view gets updated with new columns like `isNewProduct` for enhanced data insights. [Inventory View Migration](apps/api/src/migrations/db/1692960155848-add-is-new-product-to-inventory-item-view.ts)
-- **Product Variant Location & Shopify Inventory Item ID**: Adds location-based inventory tracking and links variants to Shopify InventoryItem IDs. [Variant Location Migration](apps/api/src/migrations/db/1701958615964-add-product-variant-location-and-shopify-inventory-item-id.ts)
-
-Run migrations after database setup to ensure schema is current.
-
-### 2. Onboarding Service Setup
-
-The `OnboardingService` handles setup progress and sync workflows:
-
-- Initializes onboarding progress record for a new team.
-- Verifies product variant count does not exceed limits and disables teams if necessary.
-- Manages multi-step onboarding including insight settings, grading, suppliers, purchase order import, bundle mapping, and final completion.
-- Tracks progress and triggers background data sync jobs using BullMQ queues (`QUEUE.ONBOARDING`).
-
-Key files and location:
-- Service: `apps/api/src/setup/services/onboarding.service.ts`
-- Controller: `apps/api/src/setup/controllers/onboarding.controller.ts`
-- Module: `apps/api/src/setup/setup.module.ts`
-
-**Important:** On module initialization, the onboarding service resumes incomplete onboarding flows for all teams to ensure no data sync or onboarding step is left hanging after a server restart.
-
-### 3. Shopify Webhooks
-
-Webhook registration is essential for real-time syncing of Shopify events such as:
-- Products and variants create, update, delete
-- Locations activate, create, update, delete
-- Inventory levels update and disconnect
-- Collections create, update, delete
-- Orders create, update, cancel, delete
-- Shop updates
-
-The `WebhooksInsightsSyncService` manages webhook registration and event processing pipelines. It includes:
-- Registration and unregistration of webhooks for active/inactive teams
-- Processing batched Shopify events through a set of ordered pipes for syncing data to database
-- Retry mechanism for failed webhook registrations
-
-Files of interest:
-- `apps/api/src/pim/services/webhooks-insights-sync.service.ts`
-
-### 4. Billing & Subscription
-
-The `MantleService` integrates billing, subscription state handling, and plan assignments:
-- Identifies customers in the billing system and assigns plans
-- Handles trial expirations, subscription cancellations, activations
-- Reacts to install/reinstall/uninstall Shopify app webhooks
-
-File:
-- `apps/api/src/billing/services/mantle.service.ts`
-
----
-
-## Frontend Setup
-
-### 1. Store Size Warning Modal
-
-To alert users whose Shopify store has a very high number of product variants (exceeding `MAX_ALLOWED_PRODUCT_VARIANTS_COUNT`), Genie shows a modal prompting a discovery call before proceeding with onboarding.
-
-- The modal displays the exact variant count fetched by querying the API.
-- Includes an embedded HubSpot calendar iframe to book time with the Genie team.
-
-Relevant file:
-- `apps/web/src/components/pages/Onboarding/store-too-large-modal.tsx`
-
-### 2. Sync Product Variants Manually
-
-Users can manually refresh Shopify store sync from settings:
-
-- The "Shopify Store Data" card lets users trigger manual sync to refresh product variants.
-- Sync status and feedback notifications are provided.
-
-Relevant file:
-- `apps/web/src/components/pages/settings/general-settings.tsx`
-
----
-
-## Running Genie Locally
-
-1. **Set up all environment variables** as per your environment. Refer to `Environment Variables & Configuration` documentation for required keys.
-2. **Initialize and migrate the database** using TypeORM migrations.
-3. **Start backend services**, ensuring BullMQ queues and Redis are operational for job processing.
-4. **Start the frontend web server** using Next.js; the onboarding modals and syncing features will be accessible.
-
----
-
-## Summary
-This installation and setup guide provides technical details for initializing Genie’s backend services, database, webhooks integration with Shopify, billing system connectivity, and frontend user experience components necessary for a seamless onboarding and store data synchronization process.
-
-Developers are encouraged to familiarize themselves with the onboarding service flows and webhook synchronization pipelines to understand the end-to-end data flow from Shopify to Genie’s internal representations and UI triggers.
-
-For deeper understanding, review:
-- [Onboarding Service Implementation](apps/api/src/setup/services/onboarding.service.ts)
-- [Webhook Synchronization Service](apps/api/src/pim/services/webhooks-insights-sync.service.ts)
-- [General Settings Sync Component](apps/web/src/components/pages/settings/general-settings.tsx)
-- [Store Too Large Modal](apps/web/src/components/pages/Onboarding/store-too-large-modal.tsx)
-
-Regularly check migration files to keep the schema updated and refer to the workflow queues for processing status.
-
----
-
-Please refer to the respective internal modules and service implementations for detailed operational logic and error handling behaviors.

package/docs/integrations/analytics-monitoring/error-tracking.mdx

@@ -1,199 +0,0 @@
----
-title: 'Error Tracking with Sentry'
-description: 'Sentry integration details including error filters and module usages.'
----
-
-## Error Tracking with Sentry
-
-### Purpose and Scope
-
-The Sentry integration within the Genie App is designed to capture, track, and monitor runtime errors and performance issues across the entire application stack. This includes capturing errors in the backend API (NestJS), frontend web application (Next.js), and edge middleware/runtime environments. The primary role of the Sentry integration is to provide comprehensive error reporting and observability, enabling developers to diagnose, trace, and resolve issues promptly.
-
-This integration handles exception capturing, context enrichment, transaction tracing, profiling, and custom event tracking. It supports error reporting from different runtime contexts such as server-side, client-side, and edge environments.
-
-### Technical Overview
-
-The Sentry service is integrated at multiple layers of the Genie App:
-
-- **Backend (API)**: Sentry is initialized using `@sentry/nestjs` with advanced configurations including performance tracing, CPU profiling, and custom sampling strategies. The setup is found in `apps/api/src/sentry-integration/instrument.ts`.
-- **Frontend (Web)**: Sentry initialization is split into client, server, and edge configurations using the `@sentry/nextjs` package, each tailored for their execution context (`sentry.client.config.ts`, `sentry.server.config.ts`, `sentry.edge.config.ts`).
-- **Custom Exception Handling**: A helper `captureException` function enriches error events with additional contextual data such as user details, team info, and breadcrumbs specific to internally defined `GenieException` class.
-
-#### Architectural Context
-
-```mermaid
-flowchart TD
-  subgraph Backend API Layer
-    APISentryInit[Sentry Init]:::backend
-    TrackingService[TrackingService]:::backend
-    ExceptionHandler[captureException Helper]:::backend
-  end
-
-  subgraph Frontend Web Layer
-    ClientSentryInit[Sentry Init Next.js Client]:::frontend
-    ServerSentryInit[Sentry Init Next.js Server]:::frontend
-    EdgeSentryInit[Sentry Init Next.js Edge]:::frontend
-    OnErrorHandler[Global onError Handler]:::frontend
-    TestPage[Error Trigger Page]:::frontend
-  end
-
-  subgraph External Services
-    SentryService[Sentry Platform]:::external
-    SegmentService[Segment Analytics Service]:::backend
-    HubspotService[Hubspot Analytics Service]:::backend
-  end
-
-  APISentryInit -->|captures errors| SentryService
-  ExceptionHandler -->|enhanced capture| SentryService
-  TrackingService -->|captures exceptions using captureException| ExceptionHandler
-
-  ClientSentryInit -->|captures client-side errors| SentryService
-  ServerSentryInit -->|captures server-side errors| SentryService
-  EdgeSentryInit -->|captures edge errors| SentryService
-  OnErrorHandler -->|captures errors on UI| ClientSentryInit
-  TestPage -->|explicitly capture test error| ClientSentryInit
-
-  classDef backend fill:#d0ebff,stroke:#339af0,color:#1c7ed6,stroke-width:1px;
-  classDef frontend fill:#d3f9d8,stroke:#40c057,color:#2b8a3e,stroke-width:1px;
-  classDef external fill:#ffe3e3,stroke:#fa5252,color:#c92a2a,stroke-width:1px;
-
-  click APISentryInit "apps/api/src/sentry-integration/instrument.ts"
-  click ExceptionHandler "apps/api/src/commons/error-handlers/captureException.ts"
-  click TrackingService "apps/api/src/tracking/services/tracking.service.ts"
-  click ClientSentryInit "apps/web/sentry.client.config.ts"
-  click ServerSentryInit "apps/web/sentry.server.config.ts"
-  click EdgeSentryInit "apps/web/sentry.edge.config.ts"
-  click OnErrorHandler "apps/web/src/utils/on-error.ts"
-  click TestPage "apps/web/src/pages/test-error.tsx"
-  click SegmentService "apps/api/src/tracking/services/segment.service.ts"
-  click HubspotService "apps/api/src/tracking/services/tracking.service.ts"
-```
-
-### Core Concepts and Responsibilities
-
-- **Sentry Initialization Per Context**: Sentry is initialized separately for server, client, and edge environments, ensuring tailored configurations for each runtime.
-- **Error Enrichment**: Errors thrown as `GenieException` carry rich context. These exceptions are parsed to add breadcrumbs recursively, and user/team contextual data is embedded before sending to Sentry.
-- **Performance and Profiling**: On the API backend, the Sentry SDK uses node processing and CPU profiling integrations, with finely controlled sampling based on request URL and environment.
-- **Event Tracking Integration**: The tracking system (`TrackingService`) leverages `captureException` to log error events internally, integrating Sentry error tracking with business events.
-- **Interactive Error Capture**: Frontend includes manual error capturing on demand (e.g., in `test-error.tsx`) for validation and debugging.
-
-### Code Structure
-
-- **API Layer:**
-  - `apps/api/src/sentry-integration/instrument.ts`: Sets up Sentry SDK with environment-based sampling and profiling.
-  - `apps/api/src/commons/error-handlers/captureException.ts`: Provides a utility function to capture exceptions with enhanced context and breadcrumbs.
-  - `apps/api/src/tracking/services/tracking.service.ts`: Integrates error capture into event tracking flows.
-
-- **Web Frontend:**
-  - `apps/web/sentry.client.config.ts`: Initializes Sentry for client-side errors and session replay with specific integrations.
-  - `apps/web/sentry.server.config.ts`: Configures server-side Sentry initialization for Next.js pages.
-  - `apps/web/sentry.edge.config.ts`: Configures Sentry for edge runtime features.
-  - `apps/web/src/utils/on-error.ts`: Central error handler capturing errors and sending to Sentry while managing authentication errors.
-  - `apps/web/src/pages/test-error.tsx`: Example page to trigger and capture errors for testing Sentry integration.
-
-#### Example: Enhanced Exception Capture (from `captureException.ts`)
-```ts
-export function captureException({
-  error,
-  context,
-  logger,
-  genieContext,
-}: {
-  error: unknown;
-  context?: ExecutionContext;
-  logger?: Logger;
-  genieContext?: GenieExecutionContext;
-}) {
-  (logger ?? console).error(getErrorMessages(error).join("\n\n"));
-
-  Sentry.withScope((scope: Sentry.Scope) => {
-    if (error instanceof GenieException) {
-      addGenieBreadcrumb(scope, error.cause);
-    }
-
-    const transaction = context?.switchToHttp().getRequest();
-    if (transaction) {
-      scope.addEventProcessor((event) =>
-        addRequestDataToEvent(event, transaction, { include: { ip: true, request: ["method", "url", "headers"] } })
-      );
-    }
-
-    if (genieContext?.user) {
-      scope.setContext("user", genieContext.user);
-    }
-    if (genieContext?.team) {
-      scope.setContext("team", genieContext.team);
-    }
-
-    Sentry.captureException(error, error instanceof GenieException ? { extra: error.context } : undefined);
-  });
-}
-```
-
-### Environment & Configuration
-
-| Environment Variable       | Purpose                                    | Used In                                         |
-|----------------------------|--------------------------------------------|------------------------------------------------|
-| `SENTRY_DSN`               | DSN for backend Sentry error reporting    | `apps/api/src/sentry-integration/instrument.ts`|
-| `API_ENV`                  | API environment (e.g., production, staging) | Controls sampling and profiling                  |
-| `NEXT_PUBLIC_SENTRY_DSN`   | DSN for frontend (client, server, edge)   | `apps/web/sentry.*.config.ts`                    |
-
-#### Sampling Strategies (Backend)
-- Transactions and CPU profiling sampling are customized based on URL filtering and environment (production uses 0.5 and 0.3 rates respectively)
-- Certain webhook URLs (`/shopify/webhooks`) and endpoints ending with space are ignored from sampling.
-
-#### Frontend Sampling
-- Traces sample rate is set to 1 for client, server, and edge in non-production setups.
-- Client-side config enables session replay at 100% on error and samples sessions at 10%.
-
-### Usage Examples
-
-#### Triggering a Manual Error Capture on Frontend
-```tsx
-import * as Sentry from "@sentry/nextjs";
-
-const triggerError = () => {
-  try {
-    throw new Error("This is a test error!");
-  } catch (error) {
-    Sentry.captureException(error);
-    console.error("Error captured by Sentry:", error);
-  }
-};
-```
-Use this pattern on internal test pages to verify Sentry capturing behavior.
-
-#### Tracking Errors in Backend Service
-Within the `TrackingService`, any error during event processing is captured:
-```ts
-try {
-  // event tracking logic
-} catch (error) {
-  captureException({ error, logger: this.logger });
-}
-```
-
-### Testing, Monitoring, and Debugging
-
-- Logs are available via NestJS `Logger` instances and consolidated with Sentry events.
-- The `captureException` helper prints error details to logger or console in addition to sending to Sentry.
-- The frontend global error handler enriches Axios errors with HTTP request and response data in Sentry.
-- The profiling integrations on the backend allow monitoring of CPU and transaction duration for performance tuning.
-
-### Integration Details
-
-- The Sentry service is integrated closely with the custom error type `GenieException`, allowing detailed breadcrumbs and context to be sent.
-- Error capture is spread across backend services, frontend global error handlers, and manual triggers for comprehensive coverage.
-- This integration supports multi-environment setup via environment variables, allowing differentiation between dev, staging, and production.
-
----
-
-For further exploration, see the following implementation files:
-- Backend Sentry init: `apps/api/src/sentry-integration/instrument.ts`
-- Backend exception capture: `apps/api/src/commons/error-handlers/captureException.ts`
-- Tracking service usage: `apps/api/src/tracking/services/tracking.service.ts`
-- Frontend client config: `apps/web/sentry.client.config.ts`
-- Frontend server config: `apps/web/sentry.server.config.ts`
-- Frontend edge config: `apps/web/sentry.edge.config.ts`
-- Frontend error handler: `apps/web/src/utils/on-error.ts`
-- Test error page: `apps/web/src/pages/test-error.tsx`

package/docs/integrations/analytics-monitoring/notifications.mdx

@@ -1,189 +0,0 @@
----
-title: 'Slack Notifications'
-description: 'Slack service integrations for alerts and application notifications.'
----
-
-# Slack Notifications
-
-## Purpose and Scope
-
-The `SlackService` is a core notification component within the Genie application ecosystem. It facilitates sending varied notifications directly to designated Slack channels, primarily concerning user reports, onboarding progress, feature suggestions, system alerts, and purchase order activities. This service abstracts Slack integration details and provides specialized message formats to ensure consistent communication flow.
-
-It serves as the bridge between the Genie backend and Slack, supporting actionable alerts and update notifications critical to team coordination and monitoring.
-
-This service operates within the backend API layer of the application and depends on Slack's Web API via the `@slack/web-api` client.
-
-## Technical Overview
-
-`SlackService` is implemented as an injectable singleton service in a NestJS environment, leveraging `@nestjs/common` for dependency injection and lifecycle management.
-
-- Uses Slack's WebClient initialized with an application token (`SLACK_APP_TOKEN`).
-- Interacts with Slack channels to post messages ensuring the app is a member before posting.
-- Retrieves application user ID dynamically to verify channel membership.
-- Wraps operations with custom exceptions (`GenieException`) to provide contextual error handling.
-- Retrieves environment configuration values dynamically via a `ConfigService`.
-
-### Integration Points
-
-- ConfigService: Supplies Slack tokens and channel IDs.
-- GenieException: Provides domain-specific error wrapping.
-- Slack Web API: Underlying external dependency for communications.
-- Util functions: `getTeamLabel` and `getUserLabel` generate formatted labels in notifications.
-
-### Key Functional Roles
-
-- Joining channels automatically if not a member.
-- Sending generalized text notifications.
-- Sending specialized, structured notifications for:
-  - New user report requests
-  - Date range feature suggestions
-  - Daily data sync performance summaries
-  - Product bundles import requests and onboarding
-  - Purchase order import requests
-  - Sales sync health check updates
-
-## Core Concepts and Responsibilities
-
-### Design Assumptions and Decisions
-
-- Slack authorization is managed using a single app token stored in configuration.
-- The service proactively joins channels to avoid permission errors when sending messages.
-- Custom error handling enhances observability by including contextual metadata.
-- Notification content follows Slack's markdown formatting conventions for readability.
-- Separate system alert and user request channels are used, identified via dedicated env variables.
-
-### Ownership and Maintenance
-
-- Maintained by the backend team managing integrations and notifications.
-- Errors and failures are surfaced through `GenieException` for centralized logging.
-
-### Non-Functional Requirements
-
-- Reliability: Ensures app joins channels before messaging.
-- Scalability: Modular methods allow adding more notification types easily.
-- Security: The app token is securely managed and scoped.
-
-## Code Structure
-
-### Location
-
-- Primary source: `apps/api/src/slack/services/slack.service.ts`
-
-### Overview of Components
-
-- **SlackService Class**: Main class encapsulating all Slack notification logic.
-- **Methods:**
-  - `joinChannel(channelId)`: Joins a Slack channel.
-  - `isAppMemberOfChannel(channelId)`: Checks membership status.
-  - `getAppUserId()`: Retrieves the Slack app user ID.
-  - `sendNotification(channel, content)`: Core method to send messages.
-  - Specialized notification methods for different use cases, e.g., `sendNewReportsNotification`, `sendDailyDataSyncPerformanceNotification`, etc.
-
-### Example Snippet
-
-```ts
-async sendNotification(channel: string, content: string): Promise<void> {
-  const isMember = await this.isAppMemberOfChannel(channel);
-  if (!isMember) {
-    await this.joinChannel(channel);
-  }
-
-  try {
-    await this.webClient.chat.postMessage({
-      channel,
-      text: content,
-    });
-  } catch (error) {
-    throw new GenieException(`Failed to send a message`, { error, channel });
-  }
-}
-```
-
-## API and Interfaces
-
-The `SlackService` exposes primarily asynchronous methods designed to send notifications:
-
-| Method                                  | Parameters                                             | Description                                                       |
-|-----------------------------------------|--------------------------------------------------------|-------------------------------------------------------------------|
-| `sendNotification(channel, content)`   | `channel: string`, `content: string`                   | Sends a raw text notification to specified Slack channel.         |
-| `sendNewReportsNotification({ userEmail, content })` | `{ userEmail: string; content: string }`                | Sends a new user report request notification.                     |
-| `sendDateRangeFeatureSuggestionNotification({ team, userName, userSurname, userEmail, suggestion })` | Issues a date range feature suggestion notification.             |
-| `sendDailyDataSyncPerformanceNotification(summary: string)` | Summary string of data sync duration/performance.             |
-| `sendProductBundlesImportRequestNotification({ team, user, description })` | Notification for team-requested product bundle imports.         |
-| `sendOnboardingProductBundlesNotification({ team, user, products })` | Notifies about onboarding with bundles.                         |
-| `sendImportPORequestNotification({ team, user, fileUrls })` | Purchase order import requests with associated files.          |
-| `sendSalesSyncHealthCheckNotification(summary)`             | Health check results summary for team sales sync accuracy.     |
-
-### Failure Modes
-
-- Throws `GenieException` wrapping Slack Web API errors with channel context.
-- Throws an error if fetching app user ID fails during membership checks.
-
-### Authentication & Authorization
-
-- Slack app token is injected from environment via `ConfigService` (`SLACK_APP_TOKEN`).
-- This token scopes API access and membership checks.
-
-### Sample Notification Template
-
-New User Report Example:
-
-```
-*New User Report request*
-
-User Email: <user@example.com>
-
-Content: Example report content
-```
-
-## Environment & Configuration
-
-### Required Environment Variables
-
-| Variable                            | Description                                      |
-|-----------------------------------|------------------------------------------------|
-| `SLACK_APP_TOKEN`                 | OAuth token for Slack Web API authentication.  |
-| `SlACK_APP_USER_REQUESTS_CHANNEL_ID`  | Slack channel ID for user requests notifications. |
-| `SLACK_APP_SYSTEM_ALERTS_CHANNEL_ID` | Slack channel ID for system alert notifications.  |
-| `API_ENV`                        | Current environment label (e.g., production).  |
-
-### Notes
-
-- Channel IDs are stored as config keys and injected at runtime.
-- Slack app token must have necessary permission scopes for reading channel info, joining channels, and posting messages.
-
-## Testing, Monitoring, and Deployment
-
-- Error handling throws domain exceptions that are logged by the system's centralized logger.
-- No explicit unit testing or mocks shown here, but methods are async and can be unit tested by mocking `WebClient`.
-- The structured exception handling facilitates diagnosing Slack API interaction failures.
-
-## Troubleshooting & Known Issues
-
-- If messaging fails due to channel membership, the service attempts to join the channel automatically.
-- Errors when joining or messaging channels raise `GenieException` with detailed error data.
-
-## Mermaid Diagram
-
-```mermaid
-flowchart TD
-  subgraph Backend API
-    SLACK["SlackService"]:::service
-    CONF["ConfigService"]:::service
-    GENIEEXC["GenieException"]:::exception
-  end
-
-  subgraph External Services
-    SLACKAPI["Slack Web API"]:::external
-  end
-
-  CONF -->|provides tokens and channel IDs| SLACK
-  SLACK -->|uses| SLACKAPI
-  SLACK -->|raises errors via| GENIEEXC
-
-  classDef service fill:#e3f2fd,stroke:#2196f3,stroke-width:2px,color:#0d47a1;
-  classDef external fill:#ede7f6,stroke:#5e35b1,stroke-width:2px,color:#311b92;
-  classDef exception fill:#ffebee,stroke:#f44336,stroke-width:2px,color:#b71c1c;
-
-  click SLACK "apps/api/src/slack/services/slack.service.ts"
-  click CONF "apps/api/src/config"