kanmi-perf-revenue 1.0.0 → 1.1.0

package/README.md

@@ -1,246 +1,256 @@
-# Performance Revenue Intelligence Engine
+# kanmi-perf-revenue
 
-An autonomous CLI tool that connects web performance data (Datadog RUM) with business outcomes (GA4) to quantify revenue impact and prioritize fixes.
+Empirical Performance Revenue Engine — Measure how Core Web Vitals impact conversions from your actual data.
 
-**One question answered:** *"What is this performance issue costing the business?"*
+**No assumed coefficients.** No "0.5% CVR per 100ms" guesses. This tool measures the actual conversion rates at each performance bucket from YOUR session data.
 
-## Quick Start
+## Install
 
 ```bash
-# Install
-npm install -g @perf-revenue/engine
-
-# Initialize configuration
-perf-revenue config init
-
-# Add a client
-perf-revenue config add-client \
-  --name acme \
-  --display-name "Acme Corp" \
-  --ga4-project my-gcp-project \
-  --ga4-property 123456789 \
-  --datadog-app rum-app-id
-
-# Set credentials
-export ANTHROPIC_API_KEY=sk-ant-...
-export DATADOG_API_KEY=...
-export DATADOG_APP_KEY=...
-
-# Run autonomous analysis
-perf-revenue analyze --client acme -v
+npm install kanmi-perf-revenue
 ```
 
-## How It Works
+## Quick Start
 
-```
-┌─────────────────────────────────────────────────────┐
-│  perf-revenue analyze --client acme                 │
-└─────────────────────────────────────────────────────┘
-                         │
-                         ▼
-┌─────────────────────────────────────────────────────┐
-│  Claude Agent (Autonomous)                          │
-│  ├── Tool: BigQuery (fetch GA4 business data)       │
-│  ├── Tool: Datadog API (fetch RUM performance)      │
-│  ├── Tool: Analysis Engine (calculate impact)       │
-│  └── Tool: Report Generator (format output)         │
-└─────────────────────────────────────────────────────┘
-                         │
-                         ▼
-┌─────────────────────────────────────────────────────┐
-│  Executive Report (markdown)                        │
-│  • $47K/month revenue opportunity                   │
-│  • Prioritized by financial impact                  │
-│  • CFO-ready format                                 │
-└─────────────────────────────────────────────────────┘
-```
+### CLI Usage
 
-## Features
+```bash
+# Run with mock data (demo)
+npx kanmi-perf-revenue --demo
 
-- **Autonomous Analysis** - Claude fetches, analyzes, and reports without manual intervention
-- **Financial-First** - Revenue impact in dollars, not performance scores
-- **Conservative Estimates** - Underestimates by design, defensible assumptions
-- **Executive Output** - VP/Director/CFO-ready reports
+# Import from CSV/JSON file
+npx kanmi-perf-revenue --file sessions.csv --client "Acme Corp"
 
-## CLI Commands
+# Use Datadog RUM directly
+DD_API_KEY=xxx DD_APP_KEY=xxx npx kanmi-perf-revenue --start 2025-01-01 --end 2025-01-14
+```
 
-### `perf-revenue analyze`
+### Library Usage
 
-Run autonomous revenue impact analysis.
+```typescript
+import {
+  analyzeFromFile,
+  analyzeWithMockData,
+  analyzeWithDatadog,
+} from 'kanmi-perf-revenue';
+
+// From CSV/JSON
+const result = analyzeFromFile({
+  filePath: './sessions.csv',
+  clientName: 'Acme Corp',
+});
 
-```bash
-perf-revenue analyze \
-  --client <name>           # Client name (required)
-  --start <YYYY-MM-DD>      # Start date (default: 14 days ago)
-  --end <YYYY-MM-DD>        # End date (default: yesterday)
-  --output <path>           # Output file path
-  --verbose                 # Show agent activity
-  --max-iterations <n>      # Max agent iterations (default: 20)
+// From Datadog RUM
+const result = await analyzeWithDatadog({
+  apiKey: process.env.DD_API_KEY,
+  appKey: process.env.DD_APP_KEY,
+  startDate: '2025-01-01',
+  endDate: '2025-01-14',
+});
+
+console.log(result.report);
+console.log(`Top opportunity: $${result.topOpportunity?.monthlyRevenue}/month`);
 ```
 
-### `perf-revenue config`
+## What You Get
 
-Manage configuration.
+### Empirical Conversion Curves
 
-```bash
-# Initialize config directory
-perf-revenue config init
-
-# Show current configuration
-perf-revenue config show
-
-# Add a new client
-perf-revenue config add-client \
-  --name <id>               # Client identifier
-  --display-name <name>     # Display name
-  --ga4-project <id>        # GCP project ID
-  --ga4-property <id>       # GA4 property ID
-  --ga4-key-file <path>     # Service account JSON (optional)
-  --datadog-app <id>        # Datadog RUM app ID
-  --datadog-site <site>     # Datadog site (default: datadoghq.com)
-
-# Remove a client
-perf-revenue config remove-client <name>
-
-# List clients
-perf-revenue config list-clients
+Actual CVR measured at each performance bucket:
+
+```
+| LCP Range   | Sessions | CVR    | Revenue |
+|-------------|----------|--------|---------|
+| 0.0s-1.0s   | 1,852    | 3.67%  | $8K     |
+| 1.0s-1.5s   | 12,339   | 2.79%  | $40K    |
+| 2.0s-2.5s   | 13,684   | 2.22%  | $37K    |
+| 3.0s+       | 420      | 1.40%  | $500    |
 ```
 
-### `perf-revenue validate`
+### Revenue Opportunity Analysis
 
-Validate configuration and connectivity.
+Prioritized recommendations with dollar amounts:
 
-```bash
-perf-revenue validate              # Validate all
-perf-revenue validate -c <client>  # Validate specific client
+```
+Top Opportunity: LCP
+37% of sessions have LCP > 2.5s
+These sessions convert 18.5% worse than faster sessions
+Monthly revenue opportunity: $10K/month
 ```
 
-## Configuration
-
-Configuration is stored in `~/.perf-revenue/`:
+## Features
 
+| Feature | Description |
+|---------|-------------|
+| **8 Performance Metrics** | LCP, FCP, INP, CLS, TTFB, TTI, Onload, Page Size |
+| **Segmentation** | Device, page type, geography, traffic source |
+| **Statistical Significance** | Confidence intervals, p-values, effect sizes |
+| **Correlation Analysis** | Which metrics correlate most with conversions |
+| **Multi-Metric Interactions** | How LCP + INP together affect CVR |
+| **Seasonality Patterns** | Hour-of-day, day-of-week analysis |
+| **Funnel Analysis** | Where in the journey does performance matter most |
+| **ROI Calculator** | Engineering effort vs expected revenue gain |
+| **Forecasting** | 12-month revenue projections |
+| **Cohort Analysis** | LTV by performance cohort |
+| **A/B Test Integration** | Compare performance variants statistically |
+| **Attribution Models** | First-touch, last-touch, linear, position-based |
+| **Alerting** | Threshold breaches, regressions, anomalies |
+| **Export Formats** | JSON, CSV, Slack blocks, HTML email, webhooks |
+
+## Data Input Format
+
+### CSV/JSON
+
+Required columns:
+- `session_id` — Unique session identifier
+- `converted` or `has_purchase` — Boolean or 0/1
+
+Optional columns:
+- `lcp_ms`, `inp_ms`, `cls`, `fcp_ms`, `ttfb_ms` — Performance metrics
+- `order_value` or `revenue` — Purchase amount
+- `device` — mobile, desktop, tablet
+- `page_type` — pdp, plp, checkout, home, etc.
+- `country`, `traffic_source` — Segmentation
+- `timestamp` — For time-based analysis
+
+Example CSV:
+```csv
+session_id,lcp_ms,converted,order_value,device,page_type
+abc123,1200,1,89.99,mobile,pdp
+def456,3500,0,0,desktop,plp
+ghi789,800,1,150.00,mobile,checkout
 ```
-~/.perf-revenue/
-├── config.json     # Client configurations
-└── .env            # API keys (optional)
+
+### Datadog RUM
+
+The tool queries Datadog's RUM API for session-level data including Core Web Vitals, user actions, and custom events.
+
+## CLI Reference
+
+```bash
+kanmi-perf-revenue [options]
+
+Data Sources:
+  --file PATH         Import from CSV or JSON file
+  --demo              Run with mock data (for testing)
+  --api-key KEY       Datadog API key (or set DD_API_KEY)
+  --app-key KEY       Datadog App key (or set DD_APP_KEY)
+
+Options:
+  --start DATE        Start date (YYYY-MM-DD). Default: 14 days ago
+  --end DATE          End date (YYYY-MM-DD). Default: today
+  --client NAME       Client name for report header
+  --help, -h          Show help
+
+History & Tracking:
+  --save              Save analysis to history
+  --tag TAG           Tag the entry (e.g., "baseline", "post-fix")
+  --compare           Compare to baseline
+  --history           Show analysis history
+  --trend             Show CVR trend over time
+  --metric METRIC     Metric for trend (lcp, inp, cls)
 ```
 
-### Environment Variables
+### Workflow Example
 
-| Variable | Required | Description |
-|----------|----------|-------------|
-| `ANTHROPIC_API_KEY` | Yes | Claude API key |
-| `DATADOG_API_KEY` | Yes | Datadog API key |
-| `DATADOG_APP_KEY` | Yes | Datadog application key |
-| `GOOGLE_APPLICATION_CREDENTIALS` | No | Path to GCP service account JSON |
+```bash
+# 1. Establish baseline
+kanmi-perf-revenue --file week1.csv --save --tag baseline
 
-## Conversion Lift Model
+# 2. After performance fix, compare
+kanmi-perf-revenue --file week2.csv --save --tag post-fix --compare
 
-Conservative estimates based on industry research:
+# 3. View trend over time
+kanmi-perf-revenue --trend --metric lcp
 
-| Improvement | Conversion Lift |
-|-------------|-----------------|
-| 100ms LCP improvement | +0.5% conversion rate |
-| 100ms INP improvement | +0.3% conversion rate |
-| 0.01 CLS improvement | +0.1% conversion rate |
+# 4. View history
+kanmi-perf-revenue --history
+```
 
-## Output Format
+## Advanced Usage
 
-Reports follow a strict executive-friendly structure:
+### Segmentation Analysis
 
-1. **Executive Summary** - Revenue impact in one paragraph
-2. **Revenue Impact Analysis** - Detailed breakdown per issue
-3. **Prioritized Fix Order** - Ranked by financial impact
-4. **What Not To Fix** - Issues below threshold
-5. **Decision Guidance** - Actionable recommendations
+```typescript
+import { analyzeBySegment, generateSegmentationMarkdown } from 'kanmi-perf-revenue';
 
-## Library Usage
+const report = analyzeBySegment(sessions, 'device');
+console.log(generateSegmentationMarkdown(report));
+// Shows CVR breakdown by mobile vs desktop vs tablet
+```
 
-Can also be used programmatically:
+### Statistical Testing
 
 ```typescript
 import {
-  runAnalysis,
-  createGA4Adapter,
-  createDatadogAdapter,
-} from '@perf-revenue/engine';
-
-// Create adapters
-const ga4 = createGA4Adapter({
-  projectId: 'my-project',
-  propertyId: '123456789',
-});
+  twoProportionZTest,
+  calculateConfidenceInterval,
+  calculateRequiredSampleSize,
+} from 'kanmi-perf-revenue';
+
+// Compare two groups
+const test = twoProportionZTest('fast', 150, 5000, 'slow', 80, 4000);
+console.log(`p-value: ${test.p_value}, significant: ${test.is_significant}`);
+
+// Sample size planning
+const required = calculateRequiredSampleSize(0.03, 0.1); // 3% baseline, detect 10% lift
+console.log(`Need ${required.sample_size_per_group} sessions per group`);
+```
 
-const datadog = createDatadogAdapter({
-  site: 'datadoghq.com',
-  applicationId: 'rum-app-id',
-});
+### ROI Calculation
 
-// Fetch and analyze
-const ga4Data = await fetchGA4Data(ga4);
-const datadogData = await fetchDatadogData(datadog);
-const result = runAnalysis(ga4Data, datadogData, siteTotals);
-```
+```typescript
+import { calculateROI, DEFAULT_COSTS } from 'kanmi-perf-revenue';
 
-## Data Sources
+const roi = calculateROI(
+  { metric: 'lcp', currentValue: 3500, targetValue: 2500, description: 'CDN optimization' },
+  sessions,
+  { ...DEFAULT_COSTS, engineeringHours: 40 }
+);
 
-### GA4 (BigQuery Export)
+console.log(`ROI: ${roi.roi.roiPercent}%, break-even: ${roi.roi.breakEvenDays} days`);
+console.log(`Recommendation: ${roi.roi.recommendation}`);
+```
 
-- Sessions, conversions, revenue by page type + device
-- Funnel drop-off analysis
-- Conversion path journeys
+### Alerting
 
-### Datadog RUM
+```typescript
+import { generateAlertReport, DEFAULT_ALERT_CONFIG } from 'kanmi-perf-revenue';
 
-- Core Web Vitals (LCP, INP, CLS) at p75
-- Performance band distribution
-- Frustration signals
+const alerts = generateAlertReport(sessions, DEFAULT_ALERT_CONFIG, baselineSessions);
 
-## Correlation
+if (alerts.summary.critical > 0) {
+  console.log('CRITICAL ALERTS:', alerts.alerts.filter(a => a.severity === 'critical'));
+}
+```
+
+### Export to Slack
 
-Data is joined on correlation keys:
-- Date (YYYY-MM-DD)
-- Page type (checkout, pdp, plp, etc.)
-- Device category (mobile, desktop, tablet)
+```typescript
+import { exportToSlack } from 'kanmi-perf-revenue';
 
-Join is **probabilistic** (aggregate dimensions), not deterministic.
+const slackMessage = exportToSlack(result, { clientName: 'Acme Corp' });
+// Send slackMessage.blocks to Slack API
+```
 
-## Development
+## How It Works
 
-```bash
-# Install dependencies
-npm install
+1. **Bucket sessions by performance** — Group sessions into performance ranges (e.g., 0-1s, 1-2s, 2-3s LCP)
+2. **Measure actual CVR per bucket** — Calculate conversion rate from your real data
+3. **Calculate opportunity** — If slow sessions performed like fast sessions, how many more conversions?
+4. **Prioritize by revenue impact** — Rank metrics by potential monthly revenue gain
 
-# Build
-npm run build
+No industry benchmarks. No assumed coefficients. Just your data.
 
-# Type check
-npm run typecheck
+## Why Empirical?
 
-# Run CLI locally
-npm run cli -- analyze --client test -v
-```
+Traditional performance-revenue models assume fixed relationships like "0.5% CVR per 100ms LCP improvement." These assumptions:
 
-## Project Structure
+- Come from other companies' data, not yours
+- May not apply to your industry, audience, or product
+- Create false precision
 
-```
-src/
-├── adapters/           # GA4 + Datadog data adapters
-│   ├── ga4-adapter.ts
-│   ├── datadog-adapter.ts
-│   └── correlation-spec.ts
-├── engine/             # Core analysis logic
-│   ├── orchestrator.ts
-│   ├── assisted-attribution.ts
-│   └── period-utils.ts
-├── schema/             # Type definitions
-├── cli/                # Autonomous CLI
-│   ├── index.ts        # Entry point
-│   ├── agent.ts        # Claude agent
-│   ├── config.ts       # Configuration
-│   └── tools/          # Agent tools
-└── prompts/            # System prompts
-```
+This tool measures the actual relationship in YOUR data. If your fast sessions convert 3.5% and slow sessions convert 2.1%, that's a 67% relative difference — measured, not assumed.
 
 ## License
 

package/dist/empirical/ab-testing.d.ts

@@ -0,0 +1,83 @@
+/**
+ * A/B Test Integration
+ *
+ * Compare performance variants and analyze their impact on conversions.
+ * Integrates with experiment data to determine if performance changes
+ * drove conversion improvements.
+ *
+ * @author Kanmi Obasa <i@kanmiobasa.com>
+ */
+import type { SessionData } from './datadog-session-query.js';
+export interface ABTestVariant {
+    /** Variant identifier */
+    variantId: string;
+    /** Variant name/description */
+    name: string;
+    /** Sessions in this variant */
+    sessions: number;
+    /** Conversions in this variant */
+    conversions: number;
+    /** Conversion rate */
+    cvr: number;
+    /** 95% confidence interval */
+    cvrCI: {
+        lower: number;
+        upper: number;
+    };
+    /** Average LCP */
+    avgLcp: number | null;
+    /** Average INP */
+    avgInp: number | null;
+    /** p75 LCP */
+    p75Lcp: number | null;
+}
+export interface ABTestComparison {
+    control: ABTestVariant;
+    treatment: ABTestVariant;
+    /** Absolute CVR difference */
+    absoluteDifference: number;
+    /** Relative CVR difference (treatment vs control) */
+    relativeDifference: number;
+    /** p-value from z-test */
+    pValue: number;
+    /** Is statistically significant at alpha=0.05? */
+    isSignificant: boolean;
+    /** Effect size (Cohen's h) */
+    effectSize: number;
+    /** Performance comparison */
+    performanceComparison: {
+        lcpDelta: number | null;
+        inpDelta: number | null;
+        /** Did performance improve? */
+        performanceImproved: boolean;
+    };
+    /** Recommendation */
+    recommendation: 'ship_treatment' | 'keep_control' | 'need_more_data' | 'inconclusive';
+    rationale: string;
+}
+export interface ABTestReport {
+    experimentId: string;
+    variants: ABTestVariant[];
+    comparisons: ABTestComparison[];
+    winner: ABTestVariant | null;
+    sampleSizeAnalysis: {
+        currentSamples: number;
+        requiredSamples: number;
+        isSufficient: boolean;
+        daysToSignificance: number | null;
+    };
+    insights: string[];
+}
+/**
+ * Analyze an A/B test from session data.
+ */
+export declare function analyzeABTest(sessions: SessionData[], experimentId?: string): ABTestReport | null;
+/**
+ * Create a synthetic A/B test by splitting sessions on performance.
+ */
+export declare function createPerformanceExperiment(sessions: SessionData[], metric: "lcp" | "inp" | "cls" | undefined, threshold: number): ABTestReport;
+/**
+ * Generate markdown report for A/B test.
+ */
+export declare function generateABTestMarkdown(report: ABTestReport): string;
+//# sourceMappingURL=ab-testing.d.ts.map

package/dist/empirical/ab-testing.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"ab-testing.d.ts","sourceRoot":"","sources":["../../src/empirical/ab-testing.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AAEH,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,4BAA4B,CAAC;AAY9D,MAAM,WAAW,aAAa;IAC5B,yBAAyB;IACzB,SAAS,EAAE,MAAM,CAAC;IAClB,+BAA+B;IAC/B,IAAI,EAAE,MAAM,CAAC;IACb,+BAA+B;IAC/B,QAAQ,EAAE,MAAM,CAAC;IACjB,kCAAkC;IAClC,WAAW,EAAE,MAAM,CAAC;IACpB,sBAAsB;IACtB,GAAG,EAAE,MAAM,CAAC;IACZ,8BAA8B;IAC9B,KAAK,EAAE;QAAE,KAAK,EAAE,MAAM,CAAC;QAAC,KAAK,EAAE,MAAM,CAAA;KAAE,CAAC;IACxC,kBAAkB;IAClB,MAAM,EAAE,MAAM,GAAG,IAAI,CAAC;IACtB,kBAAkB;IAClB,MAAM,EAAE,MAAM,GAAG,IAAI,CAAC;IACtB,cAAc;IACd,MAAM,EAAE,MAAM,GAAG,IAAI,CAAC;CACvB;AAED,MAAM,WAAW,gBAAgB;IAC/B,OAAO,EAAE,aAAa,CAAC;IACvB,SAAS,EAAE,aAAa,CAAC;IACzB,8BAA8B;IAC9B,kBAAkB,EAAE,MAAM,CAAC;IAC3B,qDAAqD;IACrD,kBAAkB,EAAE,MAAM,CAAC;IAC3B,0BAA0B;IAC1B,MAAM,EAAE,MAAM,CAAC;IACf,kDAAkD;IAClD,aAAa,EAAE,OAAO,CAAC;IACvB,8BAA8B;IAC9B,UAAU,EAAE,MAAM,CAAC;IACnB,6BAA6B;IAC7B,qBAAqB,EAAE;QACrB,QAAQ,EAAE,MAAM,GAAG,IAAI,CAAC;QACxB,QAAQ,EAAE,MAAM,GAAG,IAAI,CAAC;QACxB,+BAA+B;QAC/B,mBAAmB,EAAE,OAAO,CAAC;KAC9B,CAAC;IACF,qBAAqB;IACrB,cAAc,EAAE,gBAAgB,GAAG,cAAc,GAAG,gBAAgB,GAAG,cAAc,CAAC;IACtF,SAAS,EAAE,MAAM,CAAC;CACnB;AAED,MAAM,WAAW,YAAY;IAC3B,YAAY,EAAE,MAAM,CAAC;IACrB,QAAQ,EAAE,aAAa,EAAE,CAAC;IAC1B,WAAW,EAAE,gBAAgB,EAAE,CAAC;IAChC,MAAM,EAAE,aAAa,GAAG,IAAI,CAAC;IAC7B,kBAAkB,EAAE;QAClB,cAAc,EAAE,MAAM,CAAC;QACvB,eAAe,EAAE,MAAM,CAAC;QACxB,YAAY,EAAE,OAAO,CAAC;QACtB,kBAAkB,EAAE,MAAM,GAAG,IAAI,CAAC;KACnC,CAAC;IACF,QAAQ,EAAE,MAAM,EAAE,CAAC;CACpB;AAMD;;GAEG;AACH,wBAAgB,aAAa,CAC3B,QAAQ,EAAE,WAAW,EAAE,EACvB,YAAY,CAAC,EAAE,MAAM,GACpB,YAAY,GAAG,IAAI,CAyHrB;AAsED;;GAEG;AACH,wBAAgB,2BAA2B,CACzC,QAAQ,EAAE,WAAW,EAAE,EACvB,MAAM,EAAE,KAAK,GAAG,KAAK,GAAG,KAAK,YAAQ,EACrC,SAAS,EAAE,MAAM,GAChB,YAAY,CAwCd;AAMD;;GAEG;AACH,wBAAgB,sBAAsB,CAAC,MAAM,EAAE,YAAY,GAAG,MAAM,CA+EnE"}