kanmi-perf-revenue 1.0.0 → 1.2.0

package/README.md

@@ -1,247 +1,421 @@
-# Performance Revenue Intelligence Engine
+# kanmi-perf-revenue
 
-An autonomous CLI tool that connects web performance data (Datadog RUM) with business outcomes (GA4) to quantify revenue impact and prioritize fixes.
+Revenue intelligence for web performance — Native Datadog RUM integration or bring your own data.
 
-**One question answered:** *"What is this performance issue costing the business?"*
+**No assumed coefficients.** No "0.5% CVR per 100ms" guesses. This tool measures the actual conversion rates at each performance bucket from YOUR session data.
+
+## Data Sources
+
+| Source | Integration | Features |
+|--------|-------------|----------|
+| **Datadog RUM + Product Analytics** | Native API | Full: frustration signals, funnels, retention, replay |
+| **CSV/JSON import** | Universal | Core analysis works with any backend |
+| **web-vitals.js + your backend** | Self-hosted | Collect → export → analyze |
+
+## Install
+
+```bash
+npm install kanmi-perf-revenue
+```
 
 ## Quick Start
 
+### CLI Usage
+
 ```bash
-# Install
-npm install -g @perf-revenue/engine
+# Run with mock data (demo)
+npx kanmi-perf-revenue --demo
 
-# Initialize configuration
-perf-revenue config init
+# Import from CSV/JSON file
+npx kanmi-perf-revenue --file sessions.csv --client "Acme Corp"
 
-# Add a client
-perf-revenue config add-client \
-  --name acme \
-  --display-name "Acme Corp" \
-  --ga4-project my-gcp-project \
-  --ga4-property 123456789 \
-  --datadog-app rum-app-id
+# Use Datadog RUM directly
+DD_API_KEY=xxx DD_APP_KEY=xxx npx kanmi-perf-revenue --start 2025-01-01 --end 2025-01-14
+```
 
-# Set credentials
-export ANTHROPIC_API_KEY=sk-ant-...
-export DATADOG_API_KEY=...
-export DATADOG_APP_KEY=...
+### Library Usage
 
-# Run autonomous analysis
-perf-revenue analyze --client acme -v
+```typescript
+import {
+  analyzeFromFile,
+  analyzeWithMockData,
+  analyzeWithDatadog,
+} from 'kanmi-perf-revenue';
+
+// From CSV/JSON
+const result = analyzeFromFile({
+  filePath: './sessions.csv',
+  clientName: 'Acme Corp',
+});
+
+// From Datadog RUM
+const result = await analyzeWithDatadog({
+  apiKey: process.env.DD_API_KEY,
+  appKey: process.env.DD_APP_KEY,
+  startDate: '2025-01-01',
+  endDate: '2025-01-14',
+});
+
+console.log(result.report);
+console.log(`Top opportunity: $${result.topOpportunity?.monthlyRevenue}/month`);
 ```
 
-## How It Works
+## What You Get
+
+### Empirical Conversion Curves
+
+Actual CVR measured at each performance bucket:
 
 ```
-┌─────────────────────────────────────────────────────┐
-│  perf-revenue analyze --client acme                 │
-└─────────────────────────────────────────────────────┘
-                         │
-                         ▼
-┌─────────────────────────────────────────────────────┐
-│  Claude Agent (Autonomous)                          │
-│  ├── Tool: BigQuery (fetch GA4 business data)       │
-│  ├── Tool: Datadog API (fetch RUM performance)      │
-│  ├── Tool: Analysis Engine (calculate impact)       │
-│  └── Tool: Report Generator (format output)         │
-└─────────────────────────────────────────────────────┘
-                         │
-                         ▼
-┌─────────────────────────────────────────────────────┐
-│  Executive Report (markdown)                        │
-│  • $47K/month revenue opportunity                   │
-│  • Prioritized by financial impact                  │
-│  • CFO-ready format                                 │
-└─────────────────────────────────────────────────────┘
+| LCP Range   | Sessions | CVR    | Revenue |
+|-------------|----------|--------|---------|
+| 0.0s-1.0s   | 1,852    | 3.67%  | $8K     |
+| 1.0s-1.5s   | 12,339   | 2.79%  | $40K    |
+| 2.0s-2.5s   | 13,684   | 2.22%  | $37K    |
+| 3.0s+       | 420      | 1.40%  | $500    |
 ```
 
-## Features
-
-- **Autonomous Analysis** - Claude fetches, analyzes, and reports without manual intervention
-- **Financial-First** - Revenue impact in dollars, not performance scores
-- **Conservative Estimates** - Underestimates by design, defensible assumptions
-- **Executive Output** - VP/Director/CFO-ready reports
+### Revenue Opportunity Analysis
 
-## CLI Commands
+Prioritized recommendations with dollar amounts:
 
-### `perf-revenue analyze`
+```
+Top Opportunity: LCP
+37% of sessions have LCP > 2.5s
+These sessions convert 18.5% worse than faster sessions
+Monthly revenue opportunity: $10K/month
+```
 
-Run autonomous revenue impact analysis.
+## Features
 
-```bash
-perf-revenue analyze \
-  --client <name>           # Client name (required)
-  --start <YYYY-MM-DD>      # Start date (default: 14 days ago)
-  --end <YYYY-MM-DD>        # End date (default: yesterday)
-  --output <path>           # Output file path
-  --verbose                 # Show agent activity
-  --max-iterations <n>      # Max agent iterations (default: 20)
+| Feature | Description |
+|---------|-------------|
+| **8 Performance Metrics** | LCP, FCP, INP, CLS, TTFB, TTI, Onload, Page Size |
+| **Datadog Product Analytics** | Frustration signals, user journeys, retention — no GA needed |
+| **Segmentation** | Device, page type, geography, traffic source |
+| **Statistical Significance** | Confidence intervals, p-values, effect sizes |
+| **Correlation Analysis** | Which metrics correlate most with conversions |
+| **Multi-Metric Interactions** | How LCP + INP together affect CVR |
+| **Seasonality Patterns** | Hour-of-day, day-of-week analysis |
+| **Funnel Analysis** | Where in the journey does performance matter most |
+| **ROI Calculator** | Engineering effort vs expected revenue gain |
+| **Forecasting** | 12-month revenue projections |
+| **Cohort Analysis** | LTV by performance cohort |
+| **A/B Test Integration** | Compare performance variants statistically |
+| **Attribution Models** | First-touch, last-touch, linear, position-based |
+| **Alerting** | Threshold breaches, regressions, anomalies |
+| **Export Formats** | JSON, CSV, Slack blocks, HTML email, webhooks |
+
+## Data Input Format
+
+### CSV/JSON
+
+Required columns:
+- `session_id` — Unique session identifier
+- `converted` or `has_purchase` — Boolean or 0/1
+
+Optional columns:
+- `lcp_ms`, `inp_ms`, `cls`, `fcp_ms`, `ttfb_ms` — Performance metrics
+- `order_value` or `revenue` — Purchase amount
+- `device` — mobile, desktop, tablet
+- `page_type` — pdp, plp, checkout, home, etc.
+- `country`, `traffic_source` — Segmentation
+- `timestamp` — For time-based analysis
+
+Example CSV:
+```csv
+session_id,lcp_ms,converted,order_value,device,page_type
+abc123,1200,1,89.99,mobile,pdp
+def456,3500,0,0,desktop,plp
+ghi789,800,1,150.00,mobile,checkout
 ```
 
-### `perf-revenue config`
+### Datadog RUM
 
-Manage configuration.
+The tool queries Datadog's RUM API for session-level data including Core Web Vitals, user actions, and custom events.
 
-```bash
-# Initialize config directory
-perf-revenue config init
+### Datadog Product Analytics
 
-# Show current configuration
-perf-revenue config show
+Full integration with Datadog Product Analytics — **no need for Google Analytics**. One SDK provides:
 
-# Add a new client
-perf-revenue config add-client \
-  --name <id>               # Client identifier
-  --display-name <name>     # Display name
-  --ga4-project <id>        # GCP project ID
-  --ga4-property <id>       # GA4 property ID
-  --ga4-key-file <path>     # Service account JSON (optional)
-  --datadog-app <id>        # Datadog RUM app ID
-  --datadog-site <site>     # Datadog site (default: datadoghq.com)
+- Core Web Vitals + conversion data in the same session
+- Frustration signals (rage clicks, dead clicks, error clicks)
+- User journey mapping and retention analysis
+- Funnel analysis with performance correlation
 
-# Remove a client
-perf-revenue config remove-client <name>
+```typescript
+import {
+  queryEnhancedSessions,
+  queryFunnel,
+  queryCohortRetention,
+  analyzeFrustrationImpact,
+} from 'kanmi-perf-revenue';
+
+// Query sessions with frustration signals
+const { sessions } = await queryEnhancedSessions({
+  apiKey: process.env.DD_API_KEY,
+  appKey: process.env.DD_APP_KEY,
+  startDate: '2025-01-01',
+  endDate: '2025-01-14',
+  includeFrustrationSignals: true,
+});
 
-# List clients
-perf-revenue config list-clients
+// Analyze frustration impact on conversions
+const impact = analyzeFrustrationImpact(sessions);
+console.log(`Frustrated users convert ${(impact.summary.conversionImpact * 100).toFixed(1)}% less`);
+
+// Query conversion funnel
+const funnel = await queryFunnel(
+  { apiKey: process.env.DD_API_KEY, appKey: process.env.DD_APP_KEY, startDate: '2025-01-01', endDate: '2025-01-14' },
+  [
+    { name: 'View Product', selector: 'view_product' },
+    { name: 'Add to Cart', selector: 'add_to_cart' },
+    { name: 'Checkout', selector: 'checkout_start' },
+    { name: 'Purchase', selector: 'purchase', isConversion: true },
+  ]
+);
+
+// Cohort retention analysis
+const retention = await queryCohortRetention(
+  { apiKey: process.env.DD_API_KEY, appKey: process.env.DD_APP_KEY, startDate: '2025-01-01', endDate: '2025-01-14' },
+  {
+    name: 'January Signups',
+    query: '@session.is_first_visit:true',
+    period: { start: '2025-01-01', end: '2025-01-31' },
+  }
+);
 ```
 
-### `perf-revenue validate`
+### Self-Hosted with web-vitals.js
+
+Don't use Datadog? Collect your own data with Google's [web-vitals](https://github.com/GoogleChrome/web-vitals) library and export to CSV.
+
+```html
+<!-- Add to your site -->
+<script type="module">
+import { onLCP, onINP, onCLS, onFCP, onTTFB } from 'https://unpkg.com/web-vitals@4/dist/web-vitals.js';
+
+const sessionId = crypto.randomUUID();
+const metrics = {};
+
+// Detect page type from URL
+function getPageType(path) {
+  if (path === '/') return 'home';
+  if (path.includes('/product/')) return 'pdp';
+  if (path.includes('/category/')) return 'plp';
+  if (path.includes('/cart') || path.includes('/checkout')) return 'checkout';
+  return 'other';
+}
+
+function sendToBackend(metric) {
+  metrics[metric.name.toLowerCase()] = metric.value;
+
+  // Send to your backend (adjust URL)
+  navigator.sendBeacon('/api/vitals', JSON.stringify({
+    session_id: sessionId,
+    ...metrics,
+    timestamp: new Date().toISOString(),
+    device: /Mobile/.test(navigator.userAgent) ? 'mobile' : 'desktop',
+    page_type: getPageType(location.pathname),
+  }));
+}
+
+onLCP(sendToBackend);
+onINP(sendToBackend);
+onCLS(sendToBackend);
+onFCP(sendToBackend);
+onTTFB(sendToBackend);
+
+// Track conversions - call this on purchase
+window.trackConversion = (value) => {
+  navigator.sendBeacon('/api/vitals', JSON.stringify({
+    session_id: sessionId,
+    converted: true,
+    order_value: value,
+  }));
+};
+</script>
+```
 
-Validate configuration and connectivity.
+Then export your backend data to CSV and analyze:
 
 ```bash
-perf-revenue validate              # Validate all
-perf-revenue validate -c <client>  # Validate specific client
-```
+# Export from your database
+psql -c "COPY (SELECT * FROM sessions) TO STDOUT CSV HEADER" > sessions.csv
 
-## Configuration
+# Analyze
+npx kanmi-perf-revenue --file sessions.csv --client "My Site"
+```
 
-Configuration is stored in `~/.perf-revenue/`:
+## CLI Reference
 
-```
-~/.perf-revenue/
-├── config.json     # Client configurations
-└── .env            # API keys (optional)
+```bash
+kanmi-perf-revenue [options]
+
+Data Sources:
+  --file PATH         Import from CSV or JSON file
+  --demo              Run with mock data (for testing)
+  --api-key KEY       Datadog API key (or set DD_API_KEY)
+  --app-key KEY       Datadog App key (or set DD_APP_KEY)
+
+Options:
+  --start DATE        Start date (YYYY-MM-DD). Default: 14 days ago
+  --end DATE          End date (YYYY-MM-DD). Default: today
+  --client NAME       Client name for report header
+  --help, -h          Show help
+
+History & Tracking:
+  --save              Save analysis to history
+  --tag TAG           Tag the entry (e.g., "baseline", "post-fix")
+  --compare           Compare to baseline
+  --history           Show analysis history
+  --trend             Show CVR trend over time
+  --metric METRIC     Metric for trend (lcp, inp, cls)
 ```
 
-### Environment Variables
+### Workflow Example
 
-| Variable | Required | Description |
-|----------|----------|-------------|
-| `ANTHROPIC_API_KEY` | Yes | Claude API key |
-| `DATADOG_API_KEY` | Yes | Datadog API key |
-| `DATADOG_APP_KEY` | Yes | Datadog application key |
-| `GOOGLE_APPLICATION_CREDENTIALS` | No | Path to GCP service account JSON |
+```bash
+# 1. Establish baseline
+kanmi-perf-revenue --file week1.csv --save --tag baseline
 
-## Conversion Lift Model
+# 2. After performance fix, compare
+kanmi-perf-revenue --file week2.csv --save --tag post-fix --compare
 
-Conservative estimates based on industry research:
+# 3. View trend over time
+kanmi-perf-revenue --trend --metric lcp
 
-| Improvement | Conversion Lift |
-|-------------|-----------------|
-| 100ms LCP improvement | +0.5% conversion rate |
-| 100ms INP improvement | +0.3% conversion rate |
-| 0.01 CLS improvement | +0.1% conversion rate |
+# 4. View history
+kanmi-perf-revenue --history
+```
 
-## Output Format
+## Advanced Usage
 
-Reports follow a strict executive-friendly structure:
+### Segmentation Analysis
 
-1. **Executive Summary** - Revenue impact in one paragraph
-2. **Revenue Impact Analysis** - Detailed breakdown per issue
-3. **Prioritized Fix Order** - Ranked by financial impact
-4. **What Not To Fix** - Issues below threshold
-5. **Decision Guidance** - Actionable recommendations
+```typescript
+import { analyzeBySegment, generateSegmentationMarkdown } from 'kanmi-perf-revenue';
 
-## Library Usage
+const report = analyzeBySegment(sessions, 'device');
+console.log(generateSegmentationMarkdown(report));
+// Shows CVR breakdown by mobile vs desktop vs tablet
+```
 
-Can also be used programmatically:
+### Statistical Testing
 
 ```typescript
 import {
-  runAnalysis,
-  createGA4Adapter,
-  createDatadogAdapter,
-} from '@perf-revenue/engine';
-
-// Create adapters
-const ga4 = createGA4Adapter({
-  projectId: 'my-project',
-  propertyId: '123456789',
-});
+  twoProportionZTest,
+  calculateConfidenceInterval,
+  calculateRequiredSampleSize,
+} from 'kanmi-perf-revenue';
+
+// Compare two groups
+const test = twoProportionZTest('fast', 150, 5000, 'slow', 80, 4000);
+console.log(`p-value: ${test.p_value}, significant: ${test.is_significant}`);
+
+// Sample size planning
+const required = calculateRequiredSampleSize(0.03, 0.1); // 3% baseline, detect 10% lift
+console.log(`Need ${required.sample_size_per_group} sessions per group`);
+```
 
-const datadog = createDatadogAdapter({
-  site: 'datadoghq.com',
-  applicationId: 'rum-app-id',
-});
+### ROI Calculation
 
-// Fetch and analyze
-const ga4Data = await fetchGA4Data(ga4);
-const datadogData = await fetchDatadogData(datadog);
-const result = runAnalysis(ga4Data, datadogData, siteTotals);
-```
+```typescript
+import { calculateROI, DEFAULT_COSTS } from 'kanmi-perf-revenue';
 
-## Data Sources
+const roi = calculateROI(
+  { metric: 'lcp', currentValue: 3500, targetValue: 2500, description: 'CDN optimization' },
+  sessions,
+  { ...DEFAULT_COSTS, engineeringHours: 40 }
+);
 
-### GA4 (BigQuery Export)
+console.log(`ROI: ${roi.roi.roiPercent}%, break-even: ${roi.roi.breakEvenDays} days`);
+console.log(`Recommendation: ${roi.roi.recommendation}`);
+```
 
-- Sessions, conversions, revenue by page type + device
-- Funnel drop-off analysis
-- Conversion path journeys
+### Alerting
 
-### Datadog RUM
+```typescript
+import { generateAlertReport, DEFAULT_ALERT_CONFIG } from 'kanmi-perf-revenue';
 
-- Core Web Vitals (LCP, INP, CLS) at p75
-- Performance band distribution
-- Frustration signals
+// Check current sessions against thresholds
+const alerts = generateAlertReport(sessions, DEFAULT_ALERT_CONFIG);
 
-## Correlation
+// Optionally compare to baseline for regression detection
+// const alerts = generateAlertReport(sessions, DEFAULT_ALERT_CONFIG, baselineSessions);
 
-Data is joined on correlation keys:
-- Date (YYYY-MM-DD)
-- Page type (checkout, pdp, plp, etc.)
-- Device category (mobile, desktop, tablet)
+if (alerts.summary.critical > 0) {
+  console.log('CRITICAL ALERTS:', alerts.alerts.filter(a => a.severity === 'critical'));
+}
+```
 
-Join is **probabilistic** (aggregate dimensions), not deterministic.
+### Export to Slack
 
-## Development
+```typescript
+import { exportToSlack } from 'kanmi-perf-revenue';
 
-```bash
-# Install dependencies
-npm install
+const slackMessage = exportToSlack(result, { clientName: 'Acme Corp' });
+// Send slackMessage.blocks to Slack API
+```
 
-# Build
-npm run build
+### Correlation Analysis
 
-# Type check
-npm run typecheck
+```typescript
+import { calculateMetricImportance, generateCorrelationMarkdown } from 'kanmi-perf-revenue';
 
-# Run CLI locally
-npm run cli -- analyze --client test -v
+const importance = calculateMetricImportance(sessions);
+console.log(generateCorrelationMarkdown(importance));
+// Shows which metrics correlate most strongly with conversions
 ```
 
-## Project Structure
+### Forecasting
+
+```typescript
+import { forecastRevenue, generateForecastMarkdown } from 'kanmi-perf-revenue';
 
+const forecast = forecastRevenue(sessions, {
+  metric: 'lcp',
+  currentValue: 3500,
+  targetValue: 2500,
+  implementationMonth: 2,
+});
+console.log(generateForecastMarkdown(forecast));
+// 12-month revenue projection with cumulative gains
 ```
-src/
-├── adapters/           # GA4 + Datadog data adapters
-│   ├── ga4-adapter.ts
-│   ├── datadog-adapter.ts
-│   └── correlation-spec.ts
-├── engine/             # Core analysis logic
-│   ├── orchestrator.ts
-│   ├── assisted-attribution.ts
-│   └── period-utils.ts
-├── schema/             # Type definitions
-├── cli/                # Autonomous CLI
-│   ├── index.ts        # Entry point
-│   ├── agent.ts        # Claude agent
-│   ├── config.ts       # Configuration
-│   └── tools/          # Agent tools
-└── prompts/            # System prompts
+
+### A/B Test Analysis
+
+```typescript
+import { analyzeABTest, generateABTestMarkdown } from 'kanmi-perf-revenue';
+
+// Sessions must have experiment_id and experiment_variant fields
+const abReport = analyzeABTest(sessions, 'perf-optimization-test');
+if (abReport) {
+  console.log(generateABTestMarkdown(abReport));
+  console.log(`Winner: ${abReport.winner?.name || 'No clear winner yet'}`);
+}
 ```
 
+## How It Works
+
+1. **Bucket sessions by performance** — Group sessions into performance ranges (e.g., 0-1s, 1-2s, 2-3s LCP)
+2. **Measure actual CVR per bucket** — Calculate conversion rate from your real data
+3. **Calculate opportunity** — If slow sessions performed like fast sessions, how many more conversions?
+4. **Prioritize by revenue impact** — Rank metrics by potential monthly revenue gain
+
+No industry benchmarks. No assumed coefficients. Just your data.
+
+## Why Empirical?
+
+Traditional performance-revenue models assume fixed relationships like "0.5% CVR per 100ms LCP improvement." These assumptions:
+
+- Come from other companies' data, not yours
+- May not apply to your industry, audience, or product
+- Create false precision
+
+This tool measures the actual relationship in YOUR data. If your fast sessions convert 3.5% and slow sessions convert 2.1%, that's a 67% relative difference — measured, not assumed.
+
 ## License
 
 MIT

package/dist/empirical/ab-testing.d.ts

@@ -0,0 +1,83 @@
+/**
+ * A/B Test Integration
+ *
+ * Compare performance variants and analyze their impact on conversions.
+ * Integrates with experiment data to determine if performance changes
+ * drove conversion improvements.
+ *
+ * @author Kanmi Obasa <i@kanmiobasa.com>
+ */
+import type { SessionData } from './datadog-session-query.js';
+export interface ABTestVariant {
+    /** Variant identifier */
+    variantId: string;
+    /** Variant name/description */
+    name: string;
+    /** Sessions in this variant */
+    sessions: number;
+    /** Conversions in this variant */
+    conversions: number;
+    /** Conversion rate */
+    cvr: number;
+    /** 95% confidence interval */
+    cvrCI: {
+        lower: number;
+        upper: number;
+    };
+    /** Average LCP */
+    avgLcp: number | null;
+    /** Average INP */
+    avgInp: number | null;
+    /** p75 LCP */
+    p75Lcp: number | null;
+}
+export interface ABTestComparison {
+    control: ABTestVariant;
+    treatment: ABTestVariant;
+    /** Absolute CVR difference */
+    absoluteDifference: number;
+    /** Relative CVR difference (treatment vs control) */
+    relativeDifference: number;
+    /** p-value from z-test */
+    pValue: number;
+    /** Is statistically significant at alpha=0.05? */
+    isSignificant: boolean;
+    /** Effect size (Cohen's h) */
+    effectSize: number;
+    /** Performance comparison */
+    performanceComparison: {
+        lcpDelta: number | null;
+        inpDelta: number | null;
+        /** Did performance improve? */
+        performanceImproved: boolean;
+    };
+    /** Recommendation */
+    recommendation: 'ship_treatment' | 'keep_control' | 'need_more_data' | 'inconclusive';
+    rationale: string;
+}
+export interface ABTestReport {
+    experimentId: string;
+    variants: ABTestVariant[];
+    comparisons: ABTestComparison[];
+    winner: ABTestVariant | null;
+    sampleSizeAnalysis: {
+        currentSamples: number;
+        requiredSamples: number;
+        isSufficient: boolean;
+        daysToSignificance: number | null;
+    };
+    insights: string[];
+}
+/**
+ * Analyze an A/B test from session data.
+ */
+export declare function analyzeABTest(sessions: SessionData[], experimentId?: string): ABTestReport | null;
+/**
+ * Create a synthetic A/B test by splitting sessions on performance.
+ */
+export declare function createPerformanceExperiment(sessions: SessionData[], metric: "lcp" | "inp" | "cls" | undefined, threshold: number): ABTestReport;
+/**
+ * Generate markdown report for A/B test.
+ */
+export declare function generateABTestMarkdown(report: ABTestReport): string;
+//# sourceMappingURL=ab-testing.d.ts.map