soajs.controller 4.2.41 → 4.2.43

package/docs/data-flow.md

@@ -0,0 +1,138 @@
+# Data Flow
+
+Database interactions and data flow in the SOAJS Controller Gateway.
+
+## Database Overview
+
+```
+┌─────────────────────────────────────────────────────────────────────────────────┐
+│                                 DATA FLOW                                       │
+└─────────────────────────────────────────────────────────────────────────────────┘
+
+┌─────────────────────────────────────────────────────────────────────────────────┐
+│                              MONGODB DATABASES                                  │
+│                                                                                 │
+│   ┌─────────────────┐    ┌─────────────────┐    ┌─────────────────────────────┐ │
+│   │    Core DB      │    │    Meta DB      │    │    Throttle DB (Optional)   │ │
+│   ├─────────────────┤    ├─────────────────┤    ├─────────────────────────────┤ │
+│   │ • Environment   │    │ • Tenant-       │    │ • Rate limit counters       │ │
+│   │ • Tenants       │    │   specific      │    │ • Request windows           │ │
+│   │ • Applications  │    │   databases     │    │ • IP-based tracking         │ │
+│   │ • Keys          │    │ • Custom data   │    │                             │ │
+│   │ • Packages      │    │                 │    │                             │ │
+│   │ • Services      │    │                 │    │                             │ │
+│   │ • OAuth tokens  │    │                 │    │                             │ │
+│   └────────┬────────┘    └────────┬────────┘    └──────────────┬──────────────┘ │
+│            │                      │                            │                │
+└────────────┼──────────────────────┼────────────────────────────┼────────────────┘
+             │                      │                            │
+             ▼                      ▼                            ▼
+    ┌─────────────────────────────────────────────────────────────────────────────┐
+    │                         SOAJS CONTROLLER                                    │
+    │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  ┌──────────────────┐ │
+    │  │   Registry   │  │   Provision  │  │    URAC      │  │     Traffic      │ │
+    │  │    Module    │  │    Module    │  │   Driver     │  │     Module       │ │
+    │  └──────────────┘  └──────────────┘  └──────────────┘  └──────────────────┘ │
+    └─────────────────────────────────────────────────────────────────────────────┘
+```
+
+## Core Database Collections
+
+The Core DB contains the central configuration for the SOAJS platform.
+
+### Environment Registry
+
+**Collection:** `environment`
+
+Stores environment-specific configuration including services, hosts, and settings.
+
+### Tenants
+
+**Collection:** `tenants`
+
+Stores tenant information, applications, and keys.
+
+### Products & Packages
+
+**Collection:** `products`
+
+Stores product definitions and package ACLs.
+
+### OAuth Tokens
+
+**Collection:** `oauth_tokens`
+
+Stores OAuth2 access and refresh tokens.
+
+## Module Interactions
+
+### Registry Module
+
+**Source:** `soajs.core.modules/soajs.core/registry`
+
+- Loads environment configuration from Core DB
+- Caches registry data in memory
+- Provides service definitions and hosts
+- Manages custom gateway settings
+
+### Provision Module
+
+**Source:** `soajs.core.modules/soajs.core/provision`
+
+- Loads tenant/key data
+- Validates external keys
+- Retrieves package ACLs
+- Caches provision data with TTL
+
+### URAC Driver
+
+**Source:** `soajs.urac.driver`
+
+- Loads user profiles
+- Resolves user groups
+- Manages user-specific ACLs
+
+### Traffic Module
+
+**File:** `mw/traffic/index.js`
+
+- Tracks request counts
+- Enforces rate limits
+- Supports memory or MongoDB storage
+
+## Data Caching
+
+The gateway implements multi-level caching:
+
+### Registry Cache
+
+- Cached in memory
+- Refreshed via maintenance endpoint
+- TTL-based expiration
+
+### Provision Cache
+
+- Per-key caching with TTL
+- Configurable via `_TTL` field
+- Refreshed on cache miss
+
+### Token Cache
+
+- OAuth tokens validated against DB
+- Short-lived cache for performance
+
+## Throttle Database
+
+When using MongoDB for rate limiting (`traffic.model: "mongo"`):
+
+**Collection:** `throttle`
+
+```javascript
+{
+  "_id": ObjectId("..."),
+  "key": "192.168.1.1:oauth:/token",  // IP:service:api
+  "count": 15,                         // Request count
+  "timestamp": ISODate("..."),         // Window start
+  "window": 60000                      // Window size (ms)
+}
+```

package/docs/deployment.md

@@ -0,0 +1,205 @@
+# Deployment Architecture
+
+High availability deployment patterns for the SOAJS Controller Gateway.
+
+## Architecture Overview
+
+```
+┌─────────────────────────────────────────────────────────────────────────────────┐
+│                          DEPLOYMENT ARCHITECTURE                                │
+└─────────────────────────────────────────────────────────────────────────────────┘
+
+                        ┌─────────────────────────────┐
+                        │       Load Balancer         │
+                        │    (External/Cloud LB)      │
+                        └─────────────┬───────────────┘
+                                      │
+              ┌───────────────────────┼───────────────────────┐
+              │                       │                       │
+              ▼                       ▼                       ▼
+    ┌─────────────────────┐ ┌─────────────────────┐ ┌─────────────────────┐
+    │  Controller Node 1  │ │  Controller Node 2  │ │  Controller Node N  │
+    │  ┌───────────────┐  │ │  ┌───────────────┐  │ │  ┌───────────────┐  │
+    │  │  Main: 4000   │  │ │  │  Main: 4000   │  │ │  │  Main: 4000   │  │
+    │  │  Maint: 5000  │  │ │  │  Maint: 5000  │  │ │  │  Maint: 5000  │  │
+    │  └───────────────┘  │ │  └───────────────┘  │ │  └───────────────┘  │
+    └─────────┬───────────┘ └─────────┬───────────┘ └─────────┬───────────┘
+              │                       │                       │
+              └───────────────────────┼───────────────────────┘
+                                      │
+                                      ▼
+    ┌─────────────────────────────────────────────────────────────────────────────┐
+    │                         SHARED INFRASTRUCTURE                               │
+    │                                                                             │
+    │   ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────────────────┐ │
+    │   │  MongoDB Cluster│  │ Kubernetes/Swarm│  │     Microservices           │ │
+    │   │  (Core DB)      │  │ (Service Mesh)  │  │     (Backend Services)      │ │
+    │   └─────────────────┘  └─────────────────┘  └─────────────────────────────┘ │
+    │                                                                             │
+    └─────────────────────────────────────────────────────────────────────────────┘
+```
+
+## Deployment Modes
+
+### Standalone Mode
+
+Single instance deployment for development/testing.
+
+```bash
+# Environment variables
+export SOAJS_ENV=dev
+export SOAJS_PROFILE=/path/to/profile.js
+
+# Start
+node .
+```
+
+### Cluster Mode (Custom)
+
+Multiple instances with external load balancer.
+
+```bash
+# Each node
+export SOAJS_ENV=prod
+export SOAJS_PROFILE=/path/to/profile.js
+
+# Start multiple instances behind LB
+node .
+```
+
+### Kubernetes Mode
+
+Deploy as Kubernetes Deployment/Service.
+
+```yaml
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: soajs-controller
+  labels:
+    soajs.service.name: controller
+    soajs.service.version: "1"
+spec:
+  replicas: 3
+  selector:
+    matchLabels:
+      soajs.service.name: controller
+  template:
+    metadata:
+      labels:
+        soajs.service.name: controller
+        soajs.service.version: "1"
+        soajs.env.code: prod
+    spec:
+      containers:
+      - name: controller
+        image: soajsorg/gateway:latest
+        ports:
+        - containerPort: 4000
+          name: main
+        - containerPort: 5000
+          name: maintenance
+        env:
+        - name: SOAJS_ENV
+          value: "prod"
+        - name: SOAJS_DEPLOY_HA
+          value: "kubernetes"
+        - name: SOAJS_PROFILE
+          value: "/opt/soajs/profiles/profile.js"
+        livenessProbe:
+          httpGet:
+            path: /heartbeat
+            port: 5000
+          initialDelaySeconds: 30
+          periodSeconds: 10
+        readinessProbe:
+          httpGet:
+            path: /heartbeat
+            port: 5000
+          initialDelaySeconds: 5
+          periodSeconds: 5
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: soajs-controller
+spec:
+  selector:
+    soajs.service.name: controller
+  ports:
+  - port: 4000
+    targetPort: 4000
+    name: main
+  - port: 5000
+    targetPort: 5000
+    name: maintenance
+```
+
+### Docker Swarm Mode
+
+Deploy as Docker Swarm service.
+
+```bash
+docker service create \
+  --name soajs-controller \
+  --replicas 3 \
+  --publish 4000:4000 \
+  --publish 5000:5000 \
+  --env SOAJS_ENV=prod \
+  --env SOAJS_DEPLOY_HA=swarm \
+  --env SOAJS_PROFILE=/opt/soajs/profiles/profile.js \
+  --mount type=bind,source=/opt/soajs/profiles,target=/opt/soajs/profiles \
+  soajsorg/gateway:latest
+```
+
+## Port Configuration
+
+| Port | Purpose | Protocol |
+|------|---------|----------|
+| 4000 | Main gateway | HTTP |
+| 5000 | Maintenance | HTTP |
+
+## Health Checks
+
+### Liveness Check
+
+```bash
+curl http://localhost:5000/heartbeat
+```
+
+Returns `200 OK` if the gateway is running.
+
+### Readiness Check
+
+```bash
+curl http://localhost:5000/awarenessStat
+```
+
+Returns service awareness state.
+
+## Scaling Considerations
+
+### Horizontal Scaling
+
+- All instances are stateless
+- Share MongoDB for state
+- Load balance with sticky sessions not required
+
+### Resource Recommendations
+
+| Environment | CPU | Memory |
+|-------------|-----|--------|
+| Development | 0.5 | 512MB |
+| Staging | 1 | 1GB |
+| Production | 2+ | 2GB+ |
+
+## Environment Variables
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `SOAJS_ENV` | Environment code | `"dev"` |
+| `SOAJS_SOLO` | Solo mode (no DB) | - |
+| `SOAJS_DEPLOY_HA` | HA mode | - |
+| `SOAJS_PROFILE` | Profile path | `"./profiles/single.js"` |
+| `SOAJS_MAX_BODY_SIZE` | Max body size | `"10mb"` |
+| `SOAJS_SENSITIVE_ENVS` | Sensitive envs | `["dashboard"]` |

package/docs/external-integrations.md

@@ -0,0 +1,159 @@
+# External Integrations
+
+The SOAJS Controller Gateway integrates with external services and infrastructure platforms.
+
+## Integration Overview
+
+```
+┌─────────────────────────────────────────────────────────────────────────────────┐
+│                          EXTERNAL INTEGRATIONS                                  │
+└─────────────────────────────────────────────────────────────────────────────────┘
+
+┌─────────────────────────────────────────────────────────────────────────────────┐
+│                                                                                 │
+│  ┌──────────────────────┐    ┌──────────────────────┐    ┌────────────────────┐ │
+│  │    OAuth Service     │    │    URAC Service      │    │   SOA Monitor      │ │
+│  ├──────────────────────┤    ├──────────────────────┤    ├────────────────────┤ │
+│  │                      │    │                      │    │                    │ │
+│  │  Endpoints:          │    │  Endpoints:          │    │  Endpoints:        │ │
+│  │  • /token            │    │  • /user/getprofile  │    │  • /monitor/item   │ │
+│  │  • /authorization    │    │  • /user/last/seen   │    │                    │ │
+│  │                      │    │                      │    │                    │ │
+│  │  Functions:          │    │  Functions:          │    │  Functions:        │ │
+│  │  • Token validation  │    │  • User profiles     │    │  • Request metrics │ │
+│  │  • Token refresh     │    │  • Group resolution  │    │  • Response times  │ │
+│  │  • Authorization     │    │  • Activity tracking │    │  • Error tracking  │ │
+│  │                      │    │  • ACL by user       │    │                    │ │
+│  └──────────────────────┘    └──────────────────────┘    └────────────────────┘ │
+│                                                                                 │
+│  ┌──────────────────────┐    ┌──────────────────────┐                           │
+│  │   Kubernetes API     │    │   Docker Swarm API   │                           │
+│  ├──────────────────────┤    ├──────────────────────┤                           │
+│  │                      │    │                      │                           │
+│  │  Functions:          │    │  Functions:          │                           │
+│  │  • Service discovery │    │  • Service discovery │                           │
+│  │  • Pod listing       │    │  • Task listing      │                           │
+│  │  • Health checks     │    │  • Health checks     │                           │
+│  │  • Label selectors   │    │  • Label selectors   │                           │
+│  │                      │    │                      │                           │
+│  └──────────────────────┘    └──────────────────────┘                           │
+│                                                                                 │
+└─────────────────────────────────────────────────────────────────────────────────┘
+```
+
+## OAuth Service Integration
+
+The gateway integrates with the SOAJS OAuth service for authentication.
+
+### Token Validation
+
+**Middleware:** `mw/oauth/index.js`
+
+- Validates access tokens against OAuth service
+- Supports both OAuth2 password grant and JWT
+- Handles token refresh
+
+### Endpoints Used
+
+| Endpoint | Method | Purpose |
+|----------|--------|---------|
+| `/token` | POST | Exchange credentials for tokens |
+| `/authorization` | GET | Validate token |
+
+### Configuration
+
+OAuth is configured via the registry:
+
+```javascript
+{
+  "serviceConfig": {
+    "oauth": {
+      "grants": ["password", "refresh_token"],
+      "tokenLife": 7200,
+      "refreshTokenLife": 1209600
+    }
+  }
+}
+```
+
+## URAC Service Integration
+
+The gateway integrates with URAC (User Registration and Access Control) for user management.
+
+### User Profile Loading
+
+**Middleware:** `mw/mt/urac.js`
+
+- Loads user profiles for authenticated requests
+- Resolves user groups and permissions
+- Applies user-specific ACL overrides
+
+### Activity Tracking (Last Seen)
+
+**Middleware:** `mw/lastSeen/index.js`
+
+- Notifies URAC of user activity
+- Configurable via custom registry
+
+### Endpoints Used
+
+| Endpoint | Method | Purpose |
+|----------|--------|---------|
+| `/user/getprofile` | GET | Get user profile |
+| `/user/last/seen` | POST | Update last seen timestamp |
+
+## SOA Monitor Integration
+
+Optional integration for request monitoring and metrics.
+
+### Request Monitoring
+
+**Middleware:** `mw/gotoService/redirectToService.js`
+
+- Sends request metrics to SOA Monitor
+- Configurable via custom registry blacklist/whitelist
+
+### Endpoints Used
+
+| Endpoint | Method | Purpose |
+|----------|--------|---------|
+| `/monitor/item` | POST | Submit request metrics |
+
+## Kubernetes API Integration
+
+**Module:** `modules/driver/kubernetes.js`
+
+Used when `SOAJS_DEPLOY_HA=kubernetes`.
+
+### Operations
+
+| Operation | API Endpoint | Purpose |
+|-----------|--------------|---------|
+| List Pods | `/api/v1/namespaces/{ns}/pods` | Service discovery |
+| Get Pod | `/api/v1/namespaces/{ns}/pods/{name}` | Health check |
+| List Services | `/api/v1/namespaces/{ns}/services` | Service info |
+
+### Authentication
+
+- Uses in-cluster service account token
+- Token path: `/var/run/secrets/kubernetes.io/serviceaccount/token`
+- CA path: `/var/run/secrets/kubernetes.io/serviceaccount/ca.crt`
+
+## Docker Swarm API Integration
+
+**Module:** `modules/driver/swarm.js`
+
+Used when `SOAJS_DEPLOY_HA=swarm`.
+
+### Operations
+
+| Operation | API Endpoint | Purpose |
+|-----------|--------------|---------|
+| List Tasks | `/tasks` | Service discovery |
+| List Services | `/services` | Service info |
+| Inspect Node | `/nodes/{id}` | Node info |
+
+### Authentication
+
+- Connects to Docker socket
+- Default: `/var/run/docker.sock`