@akson/cortex-analytics 0.8.0

package/README.md

@@ -0,0 +1,870 @@
+# @akson/cortex-analytics
+
+Command-line interface for unified analytics management across Google Tag Manager, Google Ads, PostHog, and Google Search Console platforms.
+
+## User Stories
+
+### Unified Analytics Management Stories
+
+**As a Marketing Operations Manager**, I want to manage all analytics platforms from a single CLI, so that I can streamline operations without switching between multiple tools.
+
+**As a DevOps Engineer**, I want to integrate analytics management into CI/CD pipelines, so that I can automate tracking deployment alongside application releases.
+
+**As a Digital Marketing Director**, I want to perform health checks across all analytics platforms, so that I can ensure data integrity and tracking reliability.
+
+**As a Data Engineer**, I want to export data from multiple analytics platforms simultaneously, so that I can create unified reporting dashboards.
+
+### Platform Health Monitoring Stories
+
+**As a Site Reliability Engineer**, I want automated health checks for all analytics platforms, so that I can detect issues before they impact data collection.
+
+**As a Marketing Technology Manager**, I want real-time status monitoring of GTM, Google Ads, PostHog, and GSC, so that I can ensure continuous data flow.
+
+**As a Quality Assurance Manager**, I want comprehensive validation reports across platforms, so that I can verify tracking implementation accuracy.
+
+**As a Marketing Analyst**, I want to monitor data freshness and completeness, so that I can ensure reports reflect current performance.
+
+### Campaign Management Stories
+
+**As a Performance Marketing Manager**, I want to launch campaigns across multiple platforms from one interface, so that I can coordinate multi-channel campaigns efficiently.
+
+**As a Campaign Manager**, I want to update tracking parameters across platforms simultaneously, so that I can maintain attribution consistency during campaign launches.
+
+**As a Growth Marketing Manager**, I want automated campaign optimization suggestions from all platforms, so that I can improve performance through unified insights.
+
+**As a Marketing Operations Specialist**, I want bulk campaign configuration updates, so that I can implement changes at scale across client accounts.
+
+### Data Integration & Analysis Stories
+
+**As a Business Intelligence Analyst**, I want to correlate data across GTM, Google Ads, PostHog, and GSC, so that I can understand complete customer journeys.
+
+**As a Marketing Analyst**, I want unified conversion attribution across platforms, so that I can measure true marketing effectiveness.
+
+**As a Data Scientist**, I want programmatic access to all analytics data, so that I can build predictive models and advanced analytics.
+
+**As a Marketing Director**, I want executive dashboards combining all platform data, so that I can understand overall marketing performance.
+
+### Automation & Workflow Stories
+
+**As a Marketing Automation Engineer**, I want to create workflows that respond to changes across platforms, so that I can implement dynamic optimization strategies.
+
+**As a Marketing Technology Lead**, I want scheduled reports combining insights from all platforms, so that I can provide regular stakeholder updates.
+
+**As a Performance Marketing Director**, I want automated alerting when metrics deviate across platforms, so that I can respond to issues proactively.
+
+**As a Digital Marketing Manager**, I want template-based campaign deployment across platforms, so that I can standardize campaign implementation.
+
+### Multi-Client Management Stories
+
+**As an Agency Owner**, I want to manage analytics for multiple client accounts from one interface, so that I can scale operations efficiently.
+
+**As a Marketing Consultant**, I want comparative analytics across client accounts, so that I can identify best practices and optimization opportunities.
+
+**As an Account Manager**, I want client-specific dashboards and reports, so that I can provide tailored insights without manual data manipulation.
+
+**As a Marketing Services Director**, I want bulk operations across multiple client accounts, so that I can implement changes at agency scale.
+
+### Troubleshooting & Support Stories
+
+**As a Technical Support Manager**, I want diagnostic tools for analytics implementation issues, so that I can resolve client problems quickly.
+
+**As a Marketing Operations Analyst**, I want detailed error reporting across platforms, so that I can identify and fix tracking issues systematically.
+
+**As a Customer Success Manager**, I want health reports for client analytics setups, so that I can proactively address implementation issues.
+
+**As a Quality Assurance Engineer**, I want validation tools for tracking implementations, so that I can ensure client setups meet standards.
+
+## Features
+
+- **Unified Platform Management**: Control GTM, Google Ads, PostHog, and GSC from one CLI
+- **Health Monitoring**: Comprehensive health checks across all platforms
+- **Automated Reporting**: Generate unified reports combining data from all sources
+- **Campaign Coordination**: Launch and manage campaigns across multiple platforms
+- **Data Validation**: Verify tracking accuracy and data quality
+- **Real-time Monitoring**: Live status updates and alerting
+- **Multi-Client Support**: Manage multiple accounts and clients efficiently
+- **CI/CD Integration**: Automation-friendly for DevOps workflows
+
+## Installation
+
+```bash
+npm install -g @akson/cortex-analytics
+```
+
+## Quick Start
+
+```bash
+# Initialize configuration
+akson-analytics init
+
+# Run health check across all platforms
+akson-analytics health
+
+# Generate unified performance report
+akson-analytics report --period last-30-days
+
+# Monitor real-time status
+akson-analytics monitor --watch
+
+# Deploy campaign tracking
+akson-analytics campaign deploy --config campaign-config.json
+```
+
+## GTM Setup Guide
+
+### Prerequisites
+
+1. **Service Account Setup**:
+
+   ```json
+   {
+     "type": "service_account",
+     "project_id": "your-project",
+     "private_key_id": "...",
+     "private_key": "...",
+     "client_email": "gtm-automation@your-project.iam.gserviceaccount.com"
+   }
+   ```
+
+2. **GTM Permissions**: Grant Editor access to GTM container
+
+3. **Configuration File** (`gtm.config.yaml`):
+
+   ```yaml
+   container_id: "GTM-T8WRBMWV"
+   workspace_id: 40 # Default workspace
+   google_ads_customer_id: "659644670"
+
+   # Tags with firing triggers
+   tags:
+     - name: "GA4 Configuration"
+       type: "googtag"
+       firing_triggers:
+         - "All Pages"
+       parameters:
+         - key: "tagId"
+           value: "G-PTZF5JDTMH"
+
+     - name: "Google Ads - Form Submission"
+       type: "awct"
+       firing_triggers:
+         - "Form Submission - Lead"
+       parameters:
+         - key: "conversionId"
+           value: "8847935674"
+         - key: "conversionLabel"
+           value: "JIHLCN-r-IwbEP7BxboC"
+
+   # Required triggers
+   triggers:
+     - name: "All Pages"
+       type: "pageview"
+     - name: "Form Submission - Lead"
+       type: "customEvent"
+       custom_event_filter:
+         - type: "equals"
+           parameter:
+             - key: "arg0"
+               value: "{{_event}}"
+             - key: "arg1"
+               value: "lead_form_submitted"
+   ```
+
+### Quick Setup Commands
+
+```bash
+# 1. Initialize GTM workspace
+npx @akson/cortex-analytics gtm init --config gtm.config.yaml
+
+# 2. Deploy configuration (creates all resources)
+npx @akson/cortex-analytics gtm apply --config gtm.config.yaml --yes
+
+# 3. Verify deployment
+npx @akson/cortex-analytics gtm verify
+
+# 4. Publish to production (optional)
+npx @akson/cortex-analytics gtm version-create --name "Production Deploy v1.0"
+npx @akson/cortex-analytics gtm publish --version-id 123
+```
+
+### Firing Triggers Management
+
+The CLI automatically handles firing trigger assignments:
+
+```bash
+# Fix missing firing triggers on existing tags
+node scripts/gtm-sync/fix-all-firing-triggers.js
+
+# Complete configuration sync with verification
+node scripts/gtm-sync/enhanced-gtm-sync.js
+
+# Test firing triggers API functionality
+node scripts/tests/test-firing-triggers-api.js
+```
+
+**Key Features**:
+
+- ✅ Automatic trigger name → ID resolution
+- ✅ Idempotent operations (safe to re-run)
+- ✅ State verification and compliance checking
+- ✅ Comprehensive error handling
+
+### Common Issues & Solutions
+
+**Missing Firing Triggers**:
+
+```bash
+# Diagnosis
+npx @akson/cortex-analytics gtm query --type tags
+
+# Fix
+node scripts/gtm-sync/fix-all-firing-triggers.js
+```
+
+**Configuration Validation Errors**:
+
+```bash
+# Validate YAML syntax and references
+npx @akson/cortex-analytics gtm validate --config gtm.config.yaml
+```
+
+**API Authentication Issues**:
+
+```bash
+# Test authentication
+npx @akson/cortex-analytics gtm query --type workspaces
+```
+
+## Commands
+
+### Configuration
+
+#### `init`
+
+Initialize CLI configuration and authentication.
+
+```bash
+# Interactive setup
+akson-analytics init
+
+# Initialize with config file
+akson-analytics init --config analytics-config.json
+
+# Setup for specific platforms
+akson-analytics init --platforms gtm,google-ads,posthog
+```
+
+#### `config`
+
+Manage configuration settings.
+
+```bash
+# Show current configuration
+akson-analytics config show
+
+# Set configuration value
+akson-analytics config set gtm.container-id GTM-XXXXXXX
+
+# List all available settings
+akson-analytics config list
+
+# Validate configuration
+akson-analytics config validate
+```
+
+### Health Monitoring
+
+#### `health`
+
+Perform comprehensive health checks across platforms.
+
+```bash
+# Check all platforms
+akson-analytics health
+
+# Check specific platforms
+akson-analytics health --platforms gtm,posthog
+
+# Detailed health report
+akson-analytics health --verbose
+
+# Output as JSON
+akson-analytics health --format json
+
+# Check with specific thresholds
+akson-analytics health --thresholds thresholds.json
+```
+
+#### `monitor`
+
+Real-time monitoring with live updates.
+
+```bash
+# Start monitoring dashboard
+akson-analytics monitor
+
+# Monitor with refresh interval
+akson-analytics monitor --interval 30s
+
+# Watch for specific issues
+akson-analytics monitor --alerts-only
+
+# Monitor specific metrics
+akson-analytics monitor --metrics "gtm.tags,ads.conversions"
+
+# Save monitoring data
+akson-analytics monitor --output monitoring-data.json
+```
+
+### Reporting
+
+#### `report`
+
+Generate unified analytics reports.
+
+```bash
+# Generate standard performance report
+akson-analytics report
+
+# Report for specific time period
+akson-analytics report --period last-7-days
+akson-analytics report --from 2025-01-01 --to 2025-01-31
+
+# Platform-specific reports
+akson-analytics report --platforms gtm,google-ads
+
+# Custom report formats
+akson-analytics report --format pdf
+akson-analytics report --template executive-summary
+
+# Automated report scheduling
+akson-analytics report --schedule daily --email team@company.com
+```
+
+#### `dashboard`
+
+Interactive analytics dashboard.
+
+```bash
+# Launch interactive dashboard
+akson-analytics dashboard
+
+# Dashboard for specific platforms
+akson-analytics dashboard --platforms gtm,posthog
+
+# Client-specific dashboard
+akson-analytics dashboard --client client-123
+
+# Export dashboard data
+akson-analytics dashboard --export dashboard-data.json
+```
+
+### Campaign Management
+
+#### `campaign`
+
+Manage campaigns across platforms.
+
+```bash
+# List all campaigns
+akson-analytics campaign list
+
+# Deploy new campaign configuration
+akson-analytics campaign deploy --config new-campaign.json
+
+# Update existing campaigns
+akson-analytics campaign update --campaign-id "campaign-123" --budget 1000
+
+# Campaign performance analysis
+akson-analytics campaign analyze --campaign-id "campaign-123"
+
+# Bulk campaign operations
+akson-analytics campaign bulk-update --file campaigns-update.csv
+```
+
+#### `conversion`
+
+Manage conversion tracking across platforms.
+
+```bash
+# List conversion actions
+akson-analytics conversion list
+
+# Create new conversion tracking
+akson-analytics conversion create --name "Email Signup" --value 10
+
+# Upload conversion data
+akson-analytics conversion upload --file conversions.csv
+
+# Validate conversion tracking
+akson-analytics conversion validate
+```
+
+### Data Management
+
+#### `export`
+
+Export data from multiple platforms.
+
+```bash
+# Export all platform data
+akson-analytics export --period last-30-days
+
+# Export specific platform data
+akson-analytics export --platform gtm --output gtm-data.json
+
+# Export with custom queries
+akson-analytics export --query-file custom-queries.json
+
+# Scheduled exports
+akson-analytics export --schedule daily --destination s3://bucket/path
+```
+
+#### `sync`
+
+Synchronize data between platforms.
+
+```bash
+# Sync conversion data between platforms
+akson-analytics sync conversions
+
+# Sync audience data
+akson-analytics sync audiences --from posthog --to google-ads
+
+# Sync campaign configurations
+akson-analytics sync campaigns --template master-template.json
+
+# Real-time sync monitoring
+akson-analytics sync monitor --watch
+```
+
+### Validation & Testing
+
+#### `validate`
+
+Validate tracking implementations.
+
+```bash
+# Validate all platform configurations
+akson-analytics validate
+
+# Validate specific implementations
+akson-analytics validate --type gtm-tags
+
+# Validate against schema
+akson-analytics validate --schema validation-schema.json
+
+# Generate validation report
+akson-analytics validate --report validation-report.html
+```
+
+#### `test`
+
+Test analytics implementations.
+
+```bash
+# Run test suite across platforms
+akson-analytics test
+
+# Test specific scenarios
+akson-analytics test --scenario conversion-tracking
+
+# Load testing for high-volume events
+akson-analytics test --load --events 1000
+
+# Test with sample data
+akson-analytics test --sample-data test-events.json
+```
+
+## Configuration Files
+
+### Main Configuration (`analytics-config.json`)
+
+```json
+{
+  "platforms": {
+    "gtm": {
+      "accountId": "6005256287",
+      "containerId": "226061083",
+      "serviceAccount": {
+        "keyFile": "path/to/gtm-service-account.json",
+        "email": "gtm-automation@project.iam.gserviceaccount.com"
+      }
+    },
+    "googleAds": {
+      "customerId": "659-644-670",
+      "developerToken": "your-developer-token",
+      "serviceAccount": {
+        "keyFile": "path/to/ads-service-account.json",
+        "email": "ads-automation@project.iam.gserviceaccount.com"
+      }
+    },
+    "postHog": {
+      "apiKey": "phc_Y8vb3DFiRumtKXyoKQsVO77XlE26AuDWc1iXaZc8rjC",
+      "projectId": "105629",
+      "host": "https://us.i.posthog.com"
+    },
+    "gsc": {
+      "siteUrl": "https://myarmy.swiss",
+      "serviceAccount": {
+        "keyFile": "path/to/gsc-service-account.json",
+        "email": "gsc-automation@project.iam.gserviceaccount.com"
+      }
+    }
+  },
+  "monitoring": {
+    "healthCheckInterval": "5m",
+    "alertThresholds": {
+      "conversionRate": 0.02,
+      "errorRate": 0.01,
+      "dataFreshness": "1h"
+    }
+  },
+  "reporting": {
+    "defaultPeriod": "last-30-days",
+    "timezone": "UTC",
+    "formats": ["json", "csv", "html"]
+  }
+}
+```
+
+### Campaign Configuration (`campaign-config.json`)
+
+```json
+{
+  "name": "Q1 2025 Growth Campaign",
+  "platforms": {
+    "gtm": {
+      "variables": [
+        {
+          "name": "Campaign ID",
+          "type": "constant",
+          "value": "q1-2025-growth"
+        }
+      ],
+      "tags": [
+        {
+          "name": "Campaign Conversion Tag",
+          "type": "gaawe",
+          "conversionId": "AW-659644670",
+          "conversionLabel": "JIHLCN-r-IwbEP7BxboC"
+        }
+      ]
+    },
+    "googleAds": {
+      "campaigns": [
+        {
+          "name": "Q1 Growth - Search",
+          "type": "SEARCH",
+          "budget": 1000,
+          "bidding": "TARGET_CPA",
+          "targetCpa": 50
+        }
+      ]
+    },
+    "postHog": {
+      "events": ["campaign_view", "campaign_conversion"],
+      "properties": {
+        "campaign_id": "q1-2025-growth",
+        "campaign_source": "google-ads"
+      }
+    }
+  }
+}
+```
+
+### Health Check Thresholds (`thresholds.json`)
+
+```json
+{
+  "gtm": {
+    "publishedVersionAge": "7d",
+    "tagErrorRate": 0.01,
+    "triggerFireRate": 0.95
+  },
+  "googleAds": {
+    "accountStatus": "ENABLED",
+    "campaignStatus": "ENABLED",
+    "conversionTrackingStatus": "ACTIVE",
+    "minimumConversions": 1
+  },
+  "postHog": {
+    "eventIngestionDelay": "5m",
+    "dailyEventVolume": 1000,
+    "apiResponseTime": "2s"
+  },
+  "gsc": {
+    "dataFreshness": "24h",
+    "crawlErrorCount": 5,
+    "indexingIssues": 3
+  }
+}
+```
+
+## Usage Examples
+
+### Daily Operations Workflow
+
+```bash
+# Morning routine: Check platform health
+akson-analytics health --verbose
+
+# Generate daily performance report
+akson-analytics report --period yesterday --format html --email team@company.com
+
+# Monitor campaign performance
+akson-analytics campaign analyze --active-only
+
+# Check for any tracking issues
+akson-analytics validate --quick-check
+```
+
+### Campaign Launch Workflow
+
+```bash
+# 1. Validate campaign configuration
+akson-analytics campaign validate --config new-campaign.json
+
+# 2. Deploy tracking infrastructure
+akson-analytics campaign deploy --config new-campaign.json --dry-run
+akson-analytics campaign deploy --config new-campaign.json
+
+# 3. Verify deployment
+akson-analytics validate --campaign new-campaign
+akson-analytics test --scenario new-campaign
+
+# 4. Start monitoring
+akson-analytics monitor --campaign new-campaign --alerts-only
+```
+
+### Multi-Client Agency Workflow
+
+```bash
+# Generate reports for all clients
+for client in $(akson-analytics config clients list); do
+  akson-analytics report --client $client --format pdf --output "reports/${client}-report.pdf"
+done
+
+# Health check across all client accounts
+akson-analytics health --all-clients --summary
+
+# Bulk update across clients
+akson-analytics campaign bulk-update --file client-updates.csv --all-clients
+```
+
+### Data Export and Analysis Workflow
+
+```bash
+# Export last month's data from all platforms
+akson-analytics export --period last-month --all-platforms --format csv
+
+# Generate unified conversion report
+akson-analytics report --type conversions --platforms gtm,google-ads,posthog
+
+# Sync audience data from PostHog to Google Ads
+akson-analytics sync audiences --from posthog --to google-ads --type high-value-users
+
+# Validate data consistency across platforms
+akson-analytics validate --type data-consistency --platforms all
+```
+
+### Continuous Integration Workflow
+
+```bash
+#!/bin/bash
+# CI/CD integration script
+
+# Pre-deployment validation
+akson-analytics validate --strict --config production-config.json
+
+if [ $? -eq 0 ]; then
+  # Deploy tracking changes
+  akson-analytics campaign deploy --config deployment-config.json
+
+  # Verify deployment
+  sleep 60  # Allow propagation time
+  akson-analytics test --scenario post-deployment
+
+  # Start monitoring
+  akson-analytics monitor --alerts-only --duration 1h &
+
+  echo "Analytics deployment completed successfully"
+else
+  echo "Analytics validation failed - aborting deployment"
+  exit 1
+fi
+```
+
+## Advanced Features
+
+### Custom Plugins
+
+Create custom analytics plugins:
+
+```typescript
+// plugins/custom-analytics.ts
+import { AnalyticsPlugin } from "@akson/cortex-analytics";
+
+export class CustomAnalyticsPlugin extends AnalyticsPlugin {
+  name = "custom-analytics";
+
+  async healthCheck() {
+    // Custom health check logic
+    return { status: "healthy", metrics: {} };
+  }
+
+  async exportData(options: ExportOptions) {
+    // Custom data export logic
+    return { data: [], format: "json" };
+  }
+}
+```
+
+### Webhook Integrations
+
+Set up webhook notifications:
+
+```bash
+# Configure webhook for health alerts
+akson-analytics config set webhooks.health.url https://api.slack.com/webhooks/xxx
+akson-analytics config set webhooks.health.events "error,warning"
+
+# Configure webhook for campaign events
+akson-analytics config set webhooks.campaigns.url https://api.teams.com/webhooks/yyy
+akson-analytics config set webhooks.campaigns.events "launch,complete,budget_exceeded"
+```
+
+### Custom Dashboards
+
+Create custom dashboard configurations:
+
+```json
+// dashboards/executive-dashboard.json
+{
+  "name": "Executive Dashboard",
+  "widgets": [
+    {
+      "type": "kpi",
+      "metric": "total_conversions",
+      "platforms": ["gtm", "google-ads", "posthog"]
+    },
+    {
+      "type": "trend",
+      "metric": "conversion_rate",
+      "period": "last-30-days",
+      "breakdown": "platform"
+    },
+    {
+      "type": "funnel",
+      "events": ["page_view", "form_start", "form_complete", "purchase"],
+      "platform": "posthog"
+    }
+  ]
+}
+```
+
+## API Integration
+
+Use programmatically in Node.js applications:
+
+```typescript
+import { AnalyticsManager } from "@akson/cortex-analytics";
+
+const analytics = new AnalyticsManager({
+  configFile: "analytics-config.json",
+});
+
+// Perform health check
+const health = await analytics.healthCheck();
+
+// Generate report
+const report = await analytics.generateReport({
+  period: "last-30-days",
+  platforms: ["gtm", "google-ads", "posthog"],
+  format: "json",
+});
+
+// Monitor in real-time
+analytics.monitor({
+  interval: 30000,
+  onAlert: (alert) => {
+    console.log("Alert:", alert);
+  },
+});
+```
+
+## Output Formats
+
+### JSON Output
+
+```json
+{
+  "timestamp": "2025-01-27T10:30:00Z",
+  "status": "healthy",
+  "platforms": {
+    "gtm": {
+      "status": "healthy",
+      "lastCheck": "2025-01-27T10:29:45Z",
+      "metrics": {
+        "tagsActive": 15,
+        "triggersActive": 8,
+        "publishedVersionAge": "2d"
+      }
+    }
+  }
+}
+```
+
+### HTML Report Output
+
+- Executive summary with key insights
+- Platform-specific performance charts
+- Trend analysis and recommendations
+- Interactive data visualizations
+- Printable format for stakeholder sharing
+
+### CSV Export Format
+
+```csv
+date,platform,metric,value,change_pct
+2025-01-27,gtm,tags_fired,1247,+12.3
+2025-01-27,google-ads,conversions,89,+8.7
+2025-01-27,posthog,events,15678,+15.2
+```
+
+## Error Handling
+
+The CLI provides comprehensive error handling:
+
+```bash
+# Verbose error output
+akson-analytics health --verbose
+
+# Debug mode for troubleshooting
+akson-analytics health --debug
+
+# Validate configuration before operations
+akson-analytics config validate
+
+# Test connections before main operations
+akson-analytics test connections
+```
+
+## Performance Optimization
+
+- **Parallel Processing**: Commands run platform operations in parallel
+- **Caching**: Intelligent caching reduces API calls and improves response times
+- **Rate Limiting**: Built-in rate limiting prevents API quota exhaustion
+- **Incremental Updates**: Only update changed configurations
+- **Batch Operations**: Bulk operations for improved efficiency
+
+## Requirements
+
+- Node.js ≥18.0.0
+- Valid API credentials for enabled platforms
+- Sufficient API quota for planned operations
+- Network access to platform APIs
+
+## License
+
+MIT