adserver-dashboard 1.0.0

package/PCM/README.md

@@ -0,0 +1,558 @@
+# DOOH AdServer API
+
+A Digital Out-of-Home (DOOH) AdServer microservice built with Node.js and Express.js for managing programmatic advertising deals, line items, and creative assets.
+
+## Overview
+
+The DOOH AdServer is a RESTful API that manages programmatic advertising campaigns for digital out-of-home displays. It provides comprehensive endpoints for creating and managing deals with various transaction types, cost models, and creative formats, complete with automatic metric calculations, robust validation, and comprehensive audit trails.
+
+## Features
+
+### Deal Management
+- **Campaign Modes**: PROGRAMMATIC and DIRECT modes for different campaign types
+- **Multiple Deal Types**: GUARANTEED, PREFERRED_DEAL, PRIVATE_AUCTION, EVERGREEN_PMP (Programmatic), DIRECT (Direct mode)
+- **Cost Models**: CPD (Cost Per Day), CPM (Cost Per Mille), CPS (Cost Per Second - non-guaranteed only)
+- **Auto-calculated Metrics**: Deal-level metrics (delivery goal, net cost, dates, timezone) automatically aggregated from line items
+- **Soft Delete (Archiving)**: Deals and line items are archived instead of deleted, preserving data integrity
+- **Dual ID Support**: Access resources by internal UUID or external identifier
+- **Conflict Detection**: Prevents duplicate deals based on source + external_id
+- **Auto-generated Deal IDs**: Format {PREFIX}-I-{10-digit-sequence} with prefixes GD, PD, PA, EG, DIR
+
+### Line Item Management
+- **Creative Types**: DISPLAY, VIDEO, AUDIO
+- **Schedule Types**: ALL_TIME, ON_SCHEDULE with day/time preferences
+- **Impression Types**: SPOT, AUDIENCE
+- **Inventory Management**: Associate multiple inventory locations per line item
+- **Creative Rotation**: EVEN, OPTIMIZED, MANUAL, SEQUENTIAL rotation types
+
+### Creative Asset Management
+- **Multiple Formats**: Support for video, display, and audio creatives
+- **Inventory Targeting**: Associate creatives with specific inventory IDs
+- **Metadata Support**: Flexible JSONB metadata storage
+
+### API Features
+- **Bulk Import/Sync**: Atomic insertion of complete deal structures with nested entities
+- **Pagination**: All list endpoints support page-based pagination
+- **Comprehensive Filtering**: Filter deals by search, mode, dealType, status, currency, brand, advertiser, seller, publisher, source, and date ranges
+- **Request Tracking**: Every API request logged with unique request ID
+- **Comprehensive Validation**: Joi schema validation for all payloads
+- **Case-insensitive**: Accepts both camelCase and snake_case payloads
+- **Nested Request/Response**: Mode-specific fields grouped under `direct` and `programmatic` objects
+- **Swagger Documentation**: Interactive API documentation at `/api-docs`
+
+### Testing & Monitoring
+- **Health Check**: Service health status and uptime monitoring
+- **127 Total Tests**: 62 unit tests + 65 comprehensive tests with 100% pass rate
+- **Test Categories**: Deal types, auction types, cost type restrictions, campaign modes, schedules, CRUD operations, date validation, timezone consistency, aggregation, bulk import, validation
+- **Automated Test Endpoints**: Execute and view test results via API
+
+## Technology Stack
+
+- **Runtime**: Node.js 20 with ES Modules
+- **Framework**: Express.js 5
+- **Database**: PostgreSQL 12+ (via Sequelize ORM 6)
+- **Validation**: Joi
+- **Documentation**: Swagger/OpenAPI 3.0
+- **Key Features**: UUID primary keys, JSONB metadata, ENUM types, CASCADE deletes
+
+## Getting Started
+
+### Prerequisites
+
+- Node.js 20 or higher
+- PostgreSQL 12 or higher
+
+### Installation
+
+1. Clone the repository:
+```bash
+git clone <repository-url>
+cd dooh-adserver
+```
+
+2. Install dependencies:
+```bash
+npm install
+```
+
+3. Set up environment variables:
+```bash
+# PostgreSQL database connection
+DATABASE_URL=postgresql://user:password@localhost:5432/dooh_adserver
+
+# Optional: Server port (default: 5000)
+PORT=5000
+```
+
+4. Initialize the database:
+```bash
+# Database tables are automatically created on first run
+npm start
+```
+
+### Running the Server
+
+**Development mode** (with auto-reload):
+```bash
+npm run dev
+```
+
+**Production mode**:
+```bash
+npm start
+```
+
+The API will be available at `http://localhost:5000/api/v1/rest`
+
+## API Documentation
+
+### Base URL
+```
+http://localhost:5000/api/v1/rest
+```
+
+### Interactive Documentation
+Access the Swagger UI at:
+```
+http://localhost:5000/api-docs
+```
+
+### Download API Specification
+Download the complete OpenAPI 3.0 specification as JSON:
+```
+http://localhost:5000/api-docs.json
+```
+
+Or via curl:
+```bash
+curl -O http://localhost:5000/api-docs.json
+```
+
+### Core Endpoints
+
+#### Deals
+- `GET /deals` - List all deals (paginated with filters)
+  - Query params: `search`, `source`, `mode`, `dealType`, `status`, `currency`, `brand`, `advertiser`, `seller`, `publisher`, `startDate`, `endDate`, `page`, `limit`
+- `POST /deals` - Create a new deal
+- `GET /deals/{dealId}` - Get deal by ID
+- `PUT /deals/{dealId}` - Update deal
+- `DELETE /deals/{dealId}` - Archive deal (soft delete)
+- `POST /deals/sync` - Bulk import/sync deals with nested entities
+
+#### Line Items
+- `GET /deals/{dealId}/line-items` - List line items for a deal
+- `POST /deals/{dealId}/line-items` - Create line item
+- `GET /deals/{dealId}/line-items/{lineItemId}` - Get line item by ID
+- `PUT /deals/{dealId}/line-items/{lineItemId}` - Update line item
+- `DELETE /deals/{dealId}/line-items/{lineItemId}` - Archive line item (soft delete)
+
+#### Creatives
+- `GET /deals/{dealId}/line-items/{lineItemId}/creatives` - List creatives
+- `POST /deals/{dealId}/line-items/{lineItemId}/creatives` - Create creative
+- `GET /deals/{dealId}/line-items/{lineItemId}/creatives/{creativeId}` - Get creative
+- `PUT /deals/{dealId}/line-items/{lineItemId}/creatives/{creativeId}` - Update creative
+- `DELETE /deals/{dealId}/line-items/{lineItemId}/creatives/{creativeId}` - Delete creative
+
+#### System & Testing
+- `GET /health` - Service health check
+- `GET /test-results` - List unit test cases (13 tests)
+- `POST /test-results/run` - Execute unit test suite
+- `GET /comprehensive-tests` - List comprehensive tests (33 tests)
+- `POST /comprehensive-tests/run` - Execute comprehensive test suite
+
+### Example Requests
+
+#### Create a Deal
+```bash
+curl -X POST http://localhost:5000/api/v1/rest/deals \
+  -H "Content-Type: application/json" \
+  -d '{
+    "source": "Hivestack",
+    "externalId": "deal-001",
+    "dealId": "GD-00001-12345",
+    "name": "Tokyo Metro Campaign",
+    "dealType": "GUARANTEED",
+    "transactionType": "SPOT",
+    "auctionType": 3,
+    "currency": "JPY",
+    "costType": "CPD",
+    "status": "LIVE"
+  }'
+```
+
+#### Create a Line Item
+```bash
+curl -X POST http://localhost:5000/api/v1/rest/deals/GD-00001-12345/line-items \
+  -H "Content-Type: application/json" \
+  -d '{
+    "externalId": "li-001",
+    "name": "Morning Rush Hour",
+    "dealType": "GUARANTEED",
+    "auctionType": 3,
+    "creativeType": "VIDEO",
+    "duration": 15,
+    "impressionType": "SPOT",
+    "timezoneId": "Asia/Tokyo",
+    "startDate": "2025-10-01",
+    "endDate": "2025-10-15",
+    "thresholdCountPerDay": 50,
+    "deliveryGoal": 750,
+    "currency": "JPY",
+    "costType": "CPD",
+    "netCost": 3750,
+    "schedule": [
+      {
+        "type": "WEEKDAY",
+        "daysOfWeek": [1, 2, 3, 4, 5],
+        "hours": [{ "start": 7, "end": 9 }],
+        "validity": { "startDate": "2025-01-01", "endDate": "2025-12-31" }
+      }
+    ]
+  }'
+```
+
+#### Bulk Import with Sync
+```bash
+curl -X POST http://localhost:5000/api/v1/rest/deals/sync \
+  -H "Content-Type: application/json" \
+  -d '[{
+    "deal": {
+      "source": "Hivestack",
+      "external_id": "deal-bulk-001",
+      "deal_id": "GD-00002-56789",
+      "name": "Bulk Import Campaign",
+      "deal_type": "GUARANTEED",
+      "transaction_type": "SPOT",
+      "auction_type": 3,
+      "currency": "JPY",
+      "cost_type": "CPD",
+      "status": "LIVE"
+    },
+    "line_items": [{
+      "line_item": {
+        "external_id": "li-bulk-001",
+        "name": "Bulk Line Item",
+        "deal_type": "GUARANTEED",
+        "auction_type": 3,
+        "creative_type": "VIDEO",
+        "duration": 15,
+        "impression_type": "SPOT",
+        "timezone_id": "Asia/Tokyo",
+        "start_date": "2025-10-01",
+        "end_date": "2025-10-15",
+        "delivery_goal": 1000,
+        "currency": "JPY",
+        "cost_type": "CPD",
+        "net_cost": 5000,
+        "schedule": [
+          {
+            "type": "DEFAULT",
+            "hours": [{ "start": 0, "end": 23 }],
+            "validity": { "startDate": "2025-01-01", "endDate": "2025-12-31" }
+          }
+        ]
+      },
+      "inventories": [],
+      "creatives": []
+    }]
+  }]'
+```
+
+### Bulk Sync/Import Behavior
+
+The `/deals/sync` endpoint provides intelligent upsert functionality for bulk operations:
+
+#### Upsert Logic (Match & Update or Create)
+
+- **Deals**: Matches by `(source + externalId)` unique constraint
+  - If match found → Updates existing deal
+  - If no match → Creates new deal
+
+- **Line Items**: Matches by `(source + externalId)` unique constraint
+  - If match found → Updates existing line item
+  - If no match → Creates new line item
+
+- **Creatives**: Matches by `(creativeId + dealId)` unique constraint
+  - If match found → Updates existing creative
+  - If no match → Creates new creative
+
+- **Inventories**: Uses standard upsert by primary key
+
+#### Primary Key Protection (Oct 2025 Fix)
+
+The sync endpoint automatically protects database primary keys:
+
+- If `id` field is present in sync payload, it is **excluded from update operations**
+- This prevents primary key mutation which would break foreign key relationships
+- Primary keys remain stable across sync operations
+- Safe to include `id` in sync payloads - used for creation, ignored for updates
+
+#### Transaction Safety
+
+- Each sync item is processed in an atomic database transaction
+- If any operation within an item fails, the entire item is rolled back
+- Partial success: Some items may succeed while others fail
+- Response includes detailed success/failure status for each item
+
+## Business Logic
+
+### Auto-calculated Deal Metrics
+
+Deal-level metrics are automatically calculated from associated **active (non-archived)** line items:
+
+- **startDate**: Earliest line item start date
+- **endDate**: Latest line item end date
+- **deliveryGoal**: Sum of line item delivery goals
+- **netCost**: Sum of line item net costs
+- **netCostPerDay**: Sum of line item net cost per day
+- **impressions**: Sum of line item impressions
+- **timezoneId**: First non-null timezone from line items
+
+These fields are **read-only** at the deal level and recalculated automatically when line items are created, updated, or deleted.
+
+### Soft Delete (Archiving)
+
+The API implements soft delete for data integrity:
+
+- **Line Items**: DELETE operations set `status = 'ARCHIVED'` instead of removing the record
+- **Deals**: DELETE operations set `status = 'ARCHIVED'`
+- **Archived line items** are excluded from deal metric calculations
+- **Creatives**: Hard delete (no status field per specification)
+
+### Validation Rules
+
+- **Deal Types**: GUARANTEED, PREFERRED_DEAL, PRIVATE_AUCTION, EVERGREEN_PMP (Programmatic) and DIRECT (Direct mode)
+- **Schedule Types**: Only ALL_TIME and ON_SCHEDULE supported
+- **Date Ranges**: startDate must be ≤ endDate
+- **Timezone Consistency**: All line items in a deal should use the same timezone
+- **Required Fields**: Validated per endpoint specification
+- **Unique Constraints**: source + externalId must be unique for deals
+
+### Response Format
+
+All API responses follow this structure:
+
+```json
+{
+  "requestId": "req-abc123",
+  "status": 200,
+  "result": {
+    "data": { /* resource data */ }
+  }
+}
+```
+
+Error responses include:
+```json
+{
+  "code": "VALIDATION_ERROR",
+  "message": "Invalid request payload",
+  "errors": [
+    {
+      "field": "dealType",
+      "message": "dealType must be one of [GUARANTEED, PREFERRED_DEAL, PRIVATE_AUCTION, EVERGREEN_PMP]"
+    }
+  ]
+}
+```
+
+## Testing
+
+### Run Comprehensive Test Suite
+
+Execute all tests via API:
+```bash
+curl -X POST http://localhost:5000/api/v1/rest/comprehensive-tests/run \
+  -H "Content-Type: application/json" \
+  -d '{"cleanup": true}'
+```
+
+### Test Categories
+
+**Unit Tests (62 tests):**
+- Deal types, auction type rules (IAB OpenRTB), cost type restrictions
+- Campaign modes (PROGRAMMATIC/DIRECT), business rule validation
+- Schedule types, date validation, field constraints
+
+**Comprehensive Tests (65 tests):**
+- DEAL_TYPES - All supported deal types + negative tests
+- SCHEDULE_TYPES - ALL_TIME, ON_SCHEDULE validation + negative tests
+- LINE_ITEM_CRUD - Create, Read, Update, Delete operations
+- DATE_VALIDATION - Valid/invalid date ranges
+- TIMEZONE_CONSISTENCY - Timezone consistency validation
+- AGGREGATION - Auto-calculated deal metrics
+- BULK_IMPORT - Valid and invalid bulk scenarios
+- VALIDATION - Missing fields, auto-calculated field rejection
+
+### View Test Results
+
+List all test cases:
+```bash
+curl http://localhost:5000/api/v1/rest/comprehensive-tests
+```
+
+Current test status: **127/127 tests passing (100% success rate)**
+
+## Database Schema
+
+### Core Tables
+
+- **deals**: Deal master records with UUID primary keys
+- **line_items**: Line item details with deal_id foreign key
+- **line_item_inventories**: JSONB array of inventory locations
+- **line_item_creatives**: Creative assets per line item
+- **transaction_history**: Audit log of all API requests
+
+### Key Design Choices
+
+- UUID primary keys for all tables
+- `deal_id` as unique business identifier
+- Foreign keys with CASCADE delete
+- JSONB fields for flexible metadata
+- ENUM types for constrained values
+- Composite uniqueness on `source + external_id`
+
+## Campaign Approval Workflow (DIRECT Mode)
+
+The API includes a multi-approver approval workflow for DIRECT mode campaigns with OTP authentication.
+
+### Workflow Overview
+
+1. **AdOps triggers approval**: `POST /deals/{dealId}/approval/request`
+2. **System behavior**:
+   - If `deal.approvalEmails` is populated → Full approval workflow with emails to approvers
+   - If `deal.approvalEmails` is empty/missing → Auto-approves immediately
+3. **Approvers receive emails** with encrypted approval URLs
+4. **Public approval portal** (no auth required):
+   - View campaign: `GET /api/v1/public/approval/campaign?token=<encrypted>`
+   - View line items: `GET /api/v1/public/approval/line-items?token=<encrypted>`
+   - View inventories: `GET /api/v1/public/approval/inventories?token=<encrypted>`
+5. **OTP verification**: Approvers must verify via OTP before submitting decision
+6. **Decision submission**: Approve or reject with optional reason
+7. **Status cascade**: Decision cascades to all line items automatically
+
+### Approval Criteria
+
+- **ALL approvers must approve** for campaign to become APPROVED
+- **ANY rejection** immediately sets status to REJECTED
+- Line item status changes cascade automatically (archived items skipped)
+
+### Campaign Reopen
+
+DIRECT campaigns can be reopened from: APPROVED, LIVE, COMPLETED, EXPIRED, REJECTED, ARCHIVED
+
+```bash
+POST /deals/{dealId}/reopen
+{ "reason": "Need to update inventory" }
+```
+
+Both deal and line items cascade to REOPENED status.
+
+## Environment Configuration
+
+Copy `.env.example` to `.env` and configure the required values.
+
+### Required Variables
+
+| Variable | Description | Example |
+|----------|-------------|---------|
+| `DATABASE_URL` | PostgreSQL connection string | `postgresql://user:pass@host:5432/db` |
+| `APPROVAL_SECRET_KEY` | Secret for encrypting tokens (32+ chars) | `your-secret-key-min-32-chars` |
+
+### Server Configuration
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `PORT` | `5000` | Server port |
+
+### Approval Workflow Configuration
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `APPROVAL_TOKEN_EXPIRY_DAYS` | `30` | Token expiry in days |
+| `APPROVAL_BASE_URL` | `http://localhost:5000/approve` | Base URL for approval links |
+| `OTP_EXPIRY_MINUTES` | `10` | OTP code expiry |
+| `MAX_OTP_ATTEMPTS` | `5` | Max OTP verification attempts |
+| `MAX_OTP_REQUESTS` | `10` | Max OTP requests per token |
+| `OTP_COOLDOWN_SECONDS` | `60` | Cooldown between OTP requests |
+| `DEV_OTP_CODE` | - | Development only: Predictable OTP |
+
+### Email Notification Configuration
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `MAIL_ENABLED` | `false` | Enable email sending |
+| `NOTIFICATION_URL` | - | External notification service URL |
+| `MAIL_FROM` | `notification@movingwalls.com` | Sender email address |
+| `NOTIFICATION_TIMEOUT_MS` | `10000` | Request timeout (ms) |
+
+### Email Templates
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `TEMPLATE_APPROVAL_REQUEST` | `lmx_influence_rfa.tpl` | Approval request email |
+| `TEMPLATE_OTP` | `lmx_EmailVerificationCode.tpl` | OTP verification email |
+| `TEMPLATE_DECISION_APPROVED` | `lmx_influence_public_approved.tpl` | Approved notification |
+| `TEMPLATE_DECISION_REJECTED` | `lmx_influence_public_rejected.tpl` | Rejected notification |
+
+### Email Branding
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `LAYOUT_COMPANY_NAME` | `Influence` | Company name in emails |
+| `LAYOUT_ADDRESS` | - | Company address in footer |
+| `LAYOUT_PHONE_NUMBER` | - | Company phone number |
+| `LAYOUT_EMAIL` | - | Company contact email |
+| `DIRECT_APPROVAL_LINK` | - | Base URL for approval links in emails |
+
+## Deployment
+
+### Production Deployment
+
+The API is ready for deployment on platforms like:
+- **Replit Deployments**: Configured for autoscale or VM deployment
+- **Heroku**: Include Procfile with `node index.js`
+- **AWS/GCP/Azure**: Deploy as container or serverless function
+- **Docker**: Create Dockerfile with Node.js 20 base image
+
+### Performance Considerations
+
+- Use connection pooling for PostgreSQL (already configured in Sequelize)
+- Enable CORS for frontend integration
+- Add rate limiting for production (not included in current version)
+- Consider read replicas for high-traffic scenarios
+- Enable query logging for debugging (set `logging: console.log` in Sequelize config)
+
+## API Versioning
+
+Current version: **v1.0.0**
+
+All endpoints are prefixed with `/api/v1/rest` to support future API versions.
+
+## Error Handling
+
+The API returns standard HTTP status codes:
+
+- **200**: Success
+- **201**: Created
+- **400**: Bad Request (validation error)
+- **404**: Not Found
+- **409**: Conflict (duplicate resource)
+- **500**: Internal Server Error
+
+All errors include a consistent error structure with code, message, and detailed field-level errors when applicable.
+
+## License
+
+This project is proprietary software. All rights reserved.
+
+## Support
+
+For API support or questions:
+- Review the Swagger documentation at `/api-docs`
+- Check the comprehensive test suite for usage examples
+- Contact the development team for technical assistance
+
+---
+
+**Built with ❤️ for programmatic DOOH advertising**