enrich.sh 0.0.1

package/README.md

@@ -0,0 +1,276 @@
+# enrich.sh
+
+```
+┌─────────────────────────────────────────────────┐
+│                                                 │
+│  ███████╗███╗   ██╗██████╗ ██╗ ██████╗██╗  ██╗  │
+│  ██╔════╝████╗  ██║██╔══██╗██║██╔════╝██║  ██║  │
+│  █████╗  ██╔██╗ ██║██████╔╝██║██║     ███████║  │
+│  ██╔══╝  ██║╚██╗██║██╔══██╗██║██║     ██╔══██║  │
+│  ███████╗██║ ╚████║██║  ██║██║╚██████╗██║  ██║  │
+│  ╚══════╝╚═╝  ╚═══╝╚═╝  ╚═╝╚═╝ ╚═════╝╚═╝  ╚═╝  │
+│                                                 │
+│     Serverless Data Ingestion Pipeline          │
+│    Your events → Parquet on R2 → DuckDB         │
+│                                                 │
+└─────────────────────────────────────────────────┘
+```
+
+Official JavaScript SDK for **[Enrich.sh](https://get.enrich.sh)**
+
+---
+
+## Install
+
+```bash
+npm install enrich.sh
+```
+
+## Quick Start
+
+```
+  ┌──────────┐    track()     ┌──────────┐    flush     ┌──────────┐
+  │  Your    │ ─────────────► │  SDK     │ ──────────► │  Enrich  │
+  │  App     │   (buffered)   │  Buffer  │  (batched)  │  API     │
+  └──────────┘                └──────────┘             └────┬─────┘
+                                                            │
+                                                       ┌────▼─────┐
+                                                       │ R2       │
+                                                       │ Parquet  │
+                                                       └────┬─────┘
+                                                            │
+                                                       ┌────▼─────┐
+                                                       │ DuckDB   │
+                                                       │ Query    │
+                                                       └──────────┘
+```
+
+### 1 · Initialize
+
+```javascript
+import { Enrich } from 'enrich.sh';
+
+const enrich = new Enrich('sk_live_your_api_key');
+```
+
+### 2 · Track Events
+
+Events are buffered locally, then flushed in batches.
+
+```javascript
+enrich.track('page_views', {
+  url: window.location.href,
+  referrer: document.referrer,
+  user_id: 'usr_42',
+});
+```
+
+### 3 · Direct Ingestion
+
+Send a batch immediately — useful server-side.
+
+```javascript
+await enrich.ingest('purchases', [
+  { item: 'Laptop', price: 999, user_id: 'usr_42' },
+  { item: 'Mouse',  price: 25,  user_id: 'usr_77' },
+]);
+```
+
+### 4 · Query with DuckDB
+
+Get presigned URLs, pass them straight to DuckDB.
+
+```javascript
+const urls = await enrich.query('page_views', { days: 7 });
+
+// DuckDB (JS/WASM)
+await conn.query(`SELECT * FROM read_parquet(${JSON.stringify(urls)})`);
+```
+
+---
+
+## API Reference
+
+```
+┌───────────────────────────────────────────────────────────────┐
+│  METHOD              │  DESCRIPTION                          │
+├───────────────────────┼───────────────────────────────────────┤
+│  new Enrich(key, opt) │  Create client instance               │
+│  .track(id, event)    │  Buffer event for batched delivery    │
+│  .ingest(id, data)    │  Send immediately (returns Promise)   │
+│  .query(id, params)   │  Get signed Parquet URLs              │
+│  .queryDetailed(id,p) │  Get URLs + file metadata             │
+│  .flush([id])         │  Force-flush one or all buffers       │
+│  .beacon([id])        │  Keepalive flush (tab close safe)    │
+│  .destroy()           │  Flush + cleanup timers               │
+└───────────────────────┴───────────────────────────────────────┘
+```
+
+### `new Enrich(apiKey, options?)`
+
+| Option          | Default              | Description                       |
+| :-------------- | :------------------- | :-------------------------------- |
+| `baseUrl`       | `https://enrich.sh`  | API endpoint                      |
+| `batchSize`     | `100`                | Events buffered before auto-flush |
+| `flushInterval` | `5000`               | Max ms between flushes            |
+| `maxRetries`    | `2`                  | Retry attempts on flush failure   |
+
+### `.track(streamId, event)`
+
+Buffers a single event. Auto-flushes at `batchSize` or every `flushInterval` ms.
+
+### `.ingest(streamId, data)`
+
+Sends one event or an array immediately. Returns the API response.
+
+### `.query(streamId, params?)`
+
+Returns `string[]` — signed URLs for Parquet files.
+
+| Param   | Format       | Example        |
+| :------ | :----------- | :------------- |
+| `date`  | `YYYY-MM-DD` | `2026-02-11`   |
+| `start` | `YYYY-MM-DD` | `2026-02-01`   |
+| `end`   | `YYYY-MM-DD` | `2026-02-10`   |
+| `days`  | number       | `7`            |
+
+### `.queryDetailed(streamId, params?)`
+
+Same params as `.query()`. Returns full response:
+
+```json
+{
+  "urls": ["https://..."],
+  "files": [{ "key": "...", "url": "...", "size": 1024, "uploaded_at": "..." }],
+  "file_count": 3,
+  "expires_at": "2026-02-12T..."
+}
+```
+
+### `.flush(streamId?)`
+
+Force-send buffered events. Omit `streamId` to flush all.
+
+### `.beacon(streamId?)`
+
+Best-effort flush that survives page unload/tab close.
+Uses `fetch({ keepalive: true })` with proper `Authorization` headers.
+Fire-and-forget — does not return a Promise.
+
+### `.destroy()`
+
+Flush everything and clear timers. Call before process exit.
+
+---
+
+## Limits & Specs
+
+```
+┌───────────────────────────────────────────────────────────────┐
+│  SPEC                │  VALUE                                │
+├───────────────────────┼───────────────────────────────────────┤
+│  Max request size     │  1 MB                                │
+│  Max events / request │  ~10,000 (depending on event size)   │
+│  Signed URL TTL       │  24 hours                            │
+│  Output format        │  Apache Parquet                      │
+│  Auth                 │  Bearer token (sk_live_* / sk_test_*)│
+│  Transport            │  HTTPS                               │
+│  Runtime              │  Node 18+, Browsers, Edge Workers    │
+│  Dependencies         │  0                                   │
+│  Package size         │  < 3 KB                              │
+└───────────────────────┴───────────────────────────────────────┘
+```
+
+### Rate Limits by Plan
+
+```
+┌──────────────┬──────────────┬────────────┬────────────────────┐
+│  Plan        │  Events/mo   │  Streams   │  Retention         │
+├──────────────┼──────────────┼────────────┼────────────────────┤
+│  Test Key    │  100K        │  3         │  30 days           │
+│  Starter     │  5M          │  3         │  30 days           │
+│  Pro         │  100M        │  Unlimited │  90 days           │
+│  Scale       │  500M        │  Unlimited │  1 year            │
+│  Enterprise  │  Custom      │  Unlimited │  Custom            │
+└──────────────┴──────────────┴────────────┴────────────────────┘
+```
+
+---
+
+## Examples
+
+### Browser — Clickstream
+
+```javascript
+import { Enrich } from 'enrich.sh';
+
+const enrich = new Enrich('sk_live_xxx');
+
+document.addEventListener('click', (e) => {
+  enrich.track('clicks', {
+    tag: e.target.tagName,
+    id: e.target.id,
+    path: location.pathname,
+    ts: new Date().toISOString(),
+  });
+});
+
+// Flush before tab close (beacon survives page unload)
+window.addEventListener('beforeunload', () => enrich.beacon());
+```
+
+### Node.js — Server-side
+
+```javascript
+import { Enrich } from 'enrich.sh';
+
+const enrich = new Enrich('sk_live_xxx');
+
+app.post('/checkout', async (req, res) => {
+  await enrich.ingest('transactions', {
+    order_id: req.body.id,
+    amount: req.body.total,
+    currency: 'USD',
+  });
+  res.json({ ok: true });
+});
+
+// Cleanup on shutdown
+process.on('SIGTERM', () => enrich.destroy());
+```
+
+### DuckDB — Analytics
+
+```javascript
+import * as duckdb from '@duckdb/duckdb-wasm';
+import { Enrich } from 'enrich.sh';
+
+const enrich = new Enrich('sk_live_xxx');
+const urls = await enrich.query('clicks', { days: 30 });
+
+const conn = await db.connect();
+const result = await conn.query(`
+  SELECT path, COUNT(*) as hits
+  FROM read_parquet(${JSON.stringify(urls)})
+  GROUP BY path
+  ORDER BY hits DESC
+`);
+```
+
+---
+
+## Keys
+
+```
+  sk_test_*  →  Sandbox (100K events, free tier limits)
+  sk_live_*  →  Production (paid plan limits apply)
+```
+
+Test keys work identically to live keys but enforce free-tier caps.
+Get your keys at **[dashboard.enrich.sh](https://dashboard.enrich.sh)**
+
+---
+
+## License
+
+MIT — [Enrich.sh](https://get.enrich.sh)

package/package.json

@@ -0,0 +1,41 @@
+{
+    "name": "enrich.sh",
+    "version": "0.0.1",
+    "description": "Official SDK for Enrich.sh — Serverless Data Ingestion Pipeline. Ingest → Parquet → DuckDB.",
+    "keywords": [
+        "enrich",
+        "analytics",
+        "ingestion",
+        "data-pipeline",
+        "parquet",
+        "duckdb",
+        "clickstream",
+        "serverless"
+    ],
+    "homepage": "https://get.enrich.sh",
+    "bugs": {
+        "url": "https://github.com/enrich-sh/sdk-js/issues"
+    },
+    "repository": {
+        "type": "git",
+        "url": "git+https://github.com/enrich-sh/sdk-js.git"
+    },
+    "license": "MIT",
+    "author": "Enrich.sh",
+    "type": "module",
+    "main": "src/index.js",
+    "types": "src/index.d.ts",
+    "exports": {
+        ".": "./src/index.js"
+    },
+    "files": [
+        "src",
+        "README.md"
+    ],
+    "scripts": {
+        "test": "node test.js"
+    },
+    "engines": {
+        "node": ">=18"
+    }
+}

package/src/index.d.ts

@@ -0,0 +1,111 @@
+/**
+ * Enrich.sh SDK
+ * Official client for data ingestion and retrieval.
+ */
+
+export interface EnrichOptions {
+    /** API base URL. Default: "https://enrich.sh" */
+    baseUrl?: string;
+    /** Events buffered before auto-flush. Default: 100 */
+    batchSize?: number;
+    /** Max milliseconds between flushes. Default: 5000 */
+    flushInterval?: number;
+    /** Retry attempts on flush failure. Default: 2 */
+    maxRetries?: number;
+}
+
+export interface QueryParams {
+    /** Single day — YYYY-MM-DD */
+    date?: string;
+    /** Range start — YYYY-MM-DD */
+    start?: string;
+    /** Range end — YYYY-MM-DD */
+    end?: string;
+    /** Last N days */
+    days?: number | string;
+}
+
+export interface FileInfo {
+    key: string;
+    url: string;
+    size: number;
+    uploaded_at: string;
+}
+
+export interface QueryDetailedResponse {
+    success: boolean;
+    stream_id: string;
+    file_count: number;
+    urls: string[];
+    files: FileInfo[];
+    expires_at: string;
+}
+
+export interface IngestResponse {
+    accepted: number;
+    buffered: number;
+}
+
+export class Enrich {
+    readonly apiKey: string;
+    readonly baseUrl: string;
+    readonly batchSize: number;
+    readonly flushInterval: number;
+    readonly maxRetries: number;
+
+    /**
+     * Create an Enrich client.
+     * @param apiKey Your API key (sk_live_* or sk_test_*)
+     * @param options Configuration options
+     */
+    constructor(apiKey: string, options?: EnrichOptions);
+
+    /**
+     * Send events immediately.
+     * @param streamId Target stream identifier
+     * @param data Single event or array of events
+     */
+    ingest(streamId: string, data: Record<string, unknown> | Record<string, unknown>[]): Promise<IngestResponse>;
+
+    /**
+     * Buffer a single event for batched delivery.
+     * Auto-flushes at batchSize or after flushInterval ms.
+     * @param streamId Target stream
+     * @param event Event payload (any JSON-serializable object)
+     */
+    track(streamId: string, event: Record<string, unknown>): void;
+
+    /**
+     * Flush buffered events.
+     * @param streamId Flush one stream, or omit to flush all
+     */
+    flush(streamId?: string): Promise<void>;
+
+    /**
+     * Get presigned Parquet file URLs for DuckDB.
+     * @param streamId Stream to query
+     * @param params Date filters
+     * @returns Array of signed URLs
+     */
+    query(streamId: string, params?: QueryParams): Promise<string[]>;
+
+    /**
+     * Get full query response with file metadata.
+     * @param streamId Stream to query
+     * @param params Date filters
+     */
+    queryDetailed(streamId: string, params?: QueryParams): Promise<QueryDetailedResponse>;
+
+    /**
+     * Best-effort flush that survives page unload/tab close.
+     * Uses fetch({ keepalive: true }) with proper Authorization headers.
+     * Fire-and-forget — does not return a Promise.
+     * @param streamId Flush one stream, or omit to flush all
+     */
+    beacon(streamId?: string): void;
+
+    /**
+     * Flush all buffers and stop timers. Call before shutdown.
+     */
+    destroy(): Promise<void>;
+}

package/src/index.js

@@ -0,0 +1,256 @@
+/**
+ * Enrich.sh SDK
+ * Official client for data ingestion and retrieval.
+ *
+ * @version 1.1.0
+ * @license MIT
+ */
+
+export class Enrich {
+    /**
+     * @param {string} apiKey - Your API key (sk_live_* or sk_test_*)
+     * @param {Object} [options]
+     * @param {string} [options.baseUrl=https://enrich.sh] - API base URL
+     * @param {number} [options.batchSize=100] - Events buffered before auto-flush
+     * @param {number} [options.flushInterval=5000] - Max ms between flushes
+     * @param {number} [options.maxRetries=2] - Retry count on flush failure
+     */
+    constructor(apiKey, options = {}) {
+        if (!apiKey) {
+            throw new Error('Enrich: API key is required. Get one at dashboard.enrich.sh');
+        }
+        if (!apiKey.startsWith('sk_live_') && !apiKey.startsWith('sk_test_')) {
+            throw new Error('Enrich: Invalid key format. Keys start with sk_live_ or sk_test_');
+        }
+
+        this.apiKey = apiKey;
+        this.baseUrl = (options.baseUrl || 'https://enrich.sh').replace(/\/+$/, '');
+        this.batchSize = options.batchSize || 100;
+        this.flushInterval = options.flushInterval || 5000;
+        this.maxRetries = options.maxRetries ?? 2;
+
+        this._buffers = new Map();  // streamId -> event[]
+        this._timers = new Map();   // streamId -> timerId
+    }
+
+    // ── Ingestion ───────────────────────────────────────────────
+
+    /**
+     * Validate stream ID format (must match backend rules).
+     * @param {string} streamId
+     */
+    _validateStreamId(streamId) {
+        if (!streamId || typeof streamId !== 'string') {
+            throw new Error('Enrich: streamId is required and must be a string');
+        }
+        if (!/^[a-zA-Z0-9_-]+$/.test(streamId)) {
+            throw new Error('Enrich: streamId must be alphanumeric (a-z, 0-9, _, -)');
+        }
+    }
+
+    /**
+     * Send events immediately.
+     * @param {string} streamId - Target stream identifier
+     * @param {Object|Object[]} data - Event object or array of events
+     * @returns {Promise<Object>} API response
+     */
+    async ingest(streamId, data) {
+        this._validateStreamId(streamId);
+        const events = Array.isArray(data) ? data : [data];
+        if (events.length === 0) return { success: true, events_received: 0 };
+
+        const res = await fetch(`${this.baseUrl}/ingest`, {
+            method: 'POST',
+            headers: {
+                'Authorization': `Bearer ${this.apiKey}`,
+                'Content-Type': 'application/json',
+            },
+            body: JSON.stringify({
+                stream_id: streamId,
+                data: events,
+            }),
+        });
+
+        if (!res.ok) {
+            const body = await res.json().catch(() => ({}));
+            throw new Error(body.error || `Enrich ingest failed (${res.status})`);
+        }
+
+        return res.json();
+    }
+
+    /**
+     * Buffer a single event for automatic batched delivery.
+     * Flushes when buffer hits batchSize or after flushInterval ms.
+     * @param {string} streamId - Target stream
+     * @param {Object} event - Event payload
+     */
+    track(streamId, event) {
+        this._validateStreamId(streamId);
+        if (!this._buffers.has(streamId)) {
+            this._buffers.set(streamId, []);
+        }
+
+        this._buffers.get(streamId).push({
+            ...event,
+            _ts: Date.now(),
+        });
+
+        this._resetTimer(streamId);
+
+        if (this._buffers.get(streamId).length >= this.batchSize) {
+            this.flush(streamId);
+        }
+    }
+
+    /**
+     * Flush buffered events for one stream, or all streams.
+     * @param {string} [streamId] - Omit to flush everything
+     * @returns {Promise}
+     */
+    async flush(streamId) {
+        if (streamId) return this._flushOne(streamId);
+        return Promise.all(
+            Array.from(this._buffers.keys()).map((id) => this._flushOne(id)),
+        );
+    }
+
+    // ── Query ───────────────────────────────────────────────────
+
+    /**
+     * Get presigned URLs for stored Parquet files.
+     * Pass these directly to DuckDB's read_parquet().
+     *
+     * @param {string} streamId
+     * @param {Object} [params]
+     * @param {string} [params.date]  - Single day, YYYY-MM-DD
+     * @param {string} [params.start] - Range start, YYYY-MM-DD
+     * @param {string} [params.end]   - Range end,   YYYY-MM-DD
+     * @param {number} [params.days]  - Last N days
+     * @returns {Promise<string[]>} Array of signed URLs
+     */
+    async query(streamId, params = {}) {
+        const qs = new URLSearchParams({ stream_id: streamId, ...params });
+
+        const res = await fetch(`${this.baseUrl}/query-urls?${qs}`, {
+            headers: { 'Authorization': `Bearer ${this.apiKey}` },
+        });
+
+        if (!res.ok) {
+            const body = await res.json().catch(() => ({}));
+            throw new Error(body.error || `Enrich query failed (${res.status})`);
+        }
+
+        const data = await res.json();
+        return data.urls || [];
+    }
+
+    /**
+     * Get full query response including file metadata.
+     * @param {string} streamId
+     * @param {Object} [params] - Same as query()
+     * @returns {Promise<Object>} { urls, files, file_count, expires_at }
+     */
+    async queryDetailed(streamId, params = {}) {
+        const qs = new URLSearchParams({ stream_id: streamId, ...params });
+
+        const res = await fetch(`${this.baseUrl}/query-urls?${qs}`, {
+            headers: { 'Authorization': `Bearer ${this.apiKey}` },
+        });
+
+        if (!res.ok) {
+            const body = await res.json().catch(() => ({}));
+            throw new Error(body.error || `Enrich query failed (${res.status})`);
+        }
+
+        return res.json();
+    }
+
+    // ── Lifecycle ───────────────────────────────────────────────
+
+    /**
+     * Best-effort flush that survives page unload.
+     * Uses fetch({ keepalive: true }) — works like sendBeacon but
+     * supports Authorization headers (no token in URL).
+     * @param {string} [streamId] - Flush one stream, or omit for all
+     */
+    beacon(streamId) {
+        const ids = streamId
+            ? [streamId]
+            : Array.from(this._buffers.keys());
+
+        for (const id of ids) {
+            const buffer = this._buffers.get(id);
+            if (!buffer || buffer.length === 0) continue;
+
+            const events = buffer.splice(0, buffer.length);
+            this._clearTimer(id);
+
+            fetch(`${this.baseUrl}/ingest`, {
+                method: 'POST',
+                headers: {
+                    'Authorization': `Bearer ${this.apiKey}`,
+                    'Content-Type': 'application/json',
+                },
+                body: JSON.stringify({ stream_id: id, data: events }),
+                keepalive: true,
+            }).catch(() => { });
+        }
+    }
+
+    /**
+     * Flush all buffers and stop timers. Call before shutdown.
+     */
+    async destroy() {
+        try {
+            await this.flush();
+        } catch {
+            // Best effort — cleanup must still happen
+        }
+        for (const [id, timer] of this._timers) {
+            clearTimeout(timer);
+        }
+        this._timers.clear();
+        this._buffers.clear();
+    }
+
+    // ── Private ─────────────────────────────────────────────────
+
+    async _flushOne(streamId) {
+        const buffer = this._buffers.get(streamId);
+        if (!buffer || buffer.length === 0) return;
+
+        const events = buffer.splice(0, buffer.length);
+        this._clearTimer(streamId);
+
+        let lastErr;
+        for (let attempt = 0; attempt <= this.maxRetries; attempt++) {
+            try {
+                return await this.ingest(streamId, events);
+            } catch (err) {
+                lastErr = err;
+                if (attempt < this.maxRetries) {
+                    await new Promise((r) => setTimeout(r, 500 * (attempt + 1)));
+                }
+            }
+        }
+
+        console.error(`[enrich] flush failed for "${streamId}" after ${this.maxRetries + 1} attempts:`, lastErr);
+        throw lastErr;
+    }
+
+    _resetTimer(streamId) {
+        this._clearTimer(streamId);
+        const timer = setTimeout(() => this.flush(streamId), this.flushInterval);
+        if (typeof timer === 'object' && timer.unref) timer.unref();
+        this._timers.set(streamId, timer);
+    }
+
+    _clearTimer(streamId) {
+        const timer = this._timers.get(streamId);
+        if (timer) {
+            clearTimeout(timer);
+            this._timers.delete(streamId);
+        }
+    }
+}