@friggframework/core 2.0.0-next.11 → 2.0.0-next.110

package/reporting/README.md

@@ -0,0 +1,154 @@
+# Reporting
+
+Deployment-wide reports for `@friggframework/core`, modelled as **admin
+operations** (ADR-010) — a sibling of the Admin Script Runner (ADR-005) on the
+same primitives: one registry, the shared admin API key, sync/async execution,
+the isolated admin execution store, and EventBridge scheduling.
+
+A report is a definition (`ReportBase`) whose output is its payload. **Core
+ships built-in reports; adopters register their own** — both are discovered,
+listed, and executed through the same runner. This package provides `ReportBase`,
+the built-in reports, and `createReportCommands()`. The HTTP router, runner, and
+executor live in `@friggframework/admin-scripts`; the infrastructure (queue,
+executor Lambda, artifact bucket, scheduler) is generated by the devtools
+`AdminScriptBuilder`.
+
+## Registering reports
+
+```js
+// app definition — same shape as adminScripts
+const Definition = {
+  name: 'my-app',
+  integrations: [HubSpotIntegration, SalesforceIntegration],
+  reports: [ConnectedAccountsActivity],            // adopter-defined
+  admin: { includeBuiltinReports: true },          // + core built-ins (integrations, ...)
+};
+```
+
+A report extends `ReportBase` and reads only through the injected admin command
+bundle (`frigg` / `this.context.commands`), never a repository directly:
+
+```js
+const { ReportBase } = require('@friggframework/core');
+
+class ConnectedAccountsActivity extends ReportBase {
+  static Definition = {
+    name: 'connected-accounts-activity',
+    version: '1.0.0',
+    runModes: ['snapshot', 'recorded', 'live'],    // first is the default
+    inputSchema: { type: 'object', properties: {
+      windowDays: { type: 'integer', enum: [30, 60, 90], default: 30 } } },
+    output: { format: 'json' },                    // 'csv'|'pdf'|'zip' → artifact storage
+    schedule: { enabled: true, cron: 'cron(0 6 * * ? *)', mode: 'snapshot' }, // default mode only; activate via PUT
+  };
+
+  async execute(frigg, params) {
+    const since = daysAgo(params.windowDays ?? 30);
+    const byType = await frigg.credentials.countActiveByType({ since });
+    return { windowDays: params.windowDays ?? 30, byType };
+  }
+}
+```
+
+## Auth
+
+All report endpoints use the **shared admin API key** (ADR-005): the
+`x-frigg-admin-api-key` header, validated against `ADMIN_API_KEY`. The dedicated
+reporting key (`x-frigg-reporting-api-key` / `REPORTING_API_KEY`) is **retired**.
+
+## Run modes
+
+Each invocation picks a mode; persistence follows from it. All three run through
+the one runner and the isolated admin execution store (`AdminScriptExecution`,
+`type: 'REPORT'` — no user/integration FK, so a user-scoped query can never
+return a report record).
+
+| Mode | Persists | Use it for |
+| --- | --- | --- |
+| `live` | nothing — computes and returns inline | cheap, always-fresh queries |
+| `recorded` | an execution record (input + results + logs) | audit trail, async polling, expensive scans |
+| `snapshot` | a recorded run tagged into a named series | trends over time |
+
+Non-JSON `output.format` (`csv`/`pdf`/`zip`) is written to artifact storage (S3,
+private + signed URL) instead of inline; JSON stays inline.
+
+## Endpoints
+
+| Method | Path | Description |
+| --- | --- | --- |
+| `GET` | `/api/v2/reports` | List registered report definitions. |
+| `GET` | `/api/v2/reports/:name` | Report definition detail (run modes, input schema, schedule). |
+| `POST` | `/api/v2/reports/:name/run` | Run `{ mode, params }`. `live` → `200` inline; `recorded`/`snapshot` → `202 { executionId }` (queued). |
+| `GET` | `/api/v2/reports/:name/snapshots?from=&to=` | A report's snapshot series (trend); each point carries a signed `artifactUrl` when the run produced a non-JSON artifact. |
+| `GET` | `/api/v2/reports/executions/:id` | Fetch one report execution (guarded to `type: 'REPORT'`); a stored artifact is returned as a signed `results.artifactUrl`. |
+| `GET`/`PUT`/`DELETE` | `/api/v2/reports/:name/schedule` | Manage a report's recurring schedule. |
+| `GET` | `/api/v2/reports/integrations` | Deprecated back-compat for PR #607 — runs the built-in `integrations` report in `live` mode and returns its payload directly. Prefer `POST /:name/run`. |
+
+## Built-in: `integrations`
+
+Integrations by status and type, with per-type usage columns. Input query params
+(all strings): `status` (an `IntegrationStatus`), `type` (`config.type` slug),
+`userId`. Response shape (`schemaVersion: 1`):
+
+```jsonc
+{
+  "schemaVersion": 1,
+  "service": "frigg-core-api",
+  "generatedAt": "2026-06-29T20:45:05.142Z",
+  "filters": { "status": null, "type": null, "userId": null },
+  "metrics": {
+    "total": 12,
+    "byStatus": { "ENABLED": 9, "ERROR": 2, "NEEDS_CONFIG": 1, "PROCESSING": 0, "DISABLED": 0 },
+    "byType": [
+      {
+        "type": "hubspot",
+        "label": "HubSpot CRM",
+        "total": 5,
+        "byStatus": { "ENABLED": 4, "ERROR": 1, "NEEDS_CONFIG": 0, "PROCESSING": 0, "DISABLED": 0 },
+        "usage": { "records.synced": 4200, "webhooks.received": 118 }
+      }
+    ],
+    "typeLabels": { "hubspot": "HubSpot CRM" },
+    "integrations": [ /* lightweight per-integration rows */ ]
+  }
+}
+```
+
+### Field reference
+
+- `schemaVersion` — contract version; branch on it. Additive changes do **not** bump it.
+- `filters` — echoes the applied filters (nulls when omitted).
+- `metrics.total` — count of integrations matching the filters.
+- `metrics.byStatus` — counts keyed by `IntegrationStatus` value.
+- `metrics.byType[]` — per `type` breakdown: `{ type, label, total, byStatus, usage? }`.
+  - `type` — the `config.type` slug. Integrations with no type bucket as `"unknown"`.
+  - `usage` — **additive** (ADR-011): present only when the deployment has the usage
+    store configured. A map of canonical counter → total for that type (`0` when a
+    type has no data). Read only from the Frigg usage store; a usage-store read
+    failure never fails the structural report. Populated by the counters an
+    integration opts into via `Definition.usage` — see [`../telemetry/README.md`](../telemetry/README.md).
+  - `label` — human-readable name from the integration class's `Definition.display.label`,
+    falling back to the `type` slug when no registered class supplies one.
+- `metrics.typeLabels` — map of `type` slug → label, so callers label
+  `integrations[].type` rows without bloating each row.
+- `metrics.integrations[]` — lightweight per-integration rows (`id`, `type`, `status`,
+  `userId`, `version`, `moduleCount`, `errorCount`, `mappedRecordCount`, `createdAt`,
+  `updatedAt`). Resolve display names via `metrics.typeLabels`.
+
+The built-in report reads its data through the admin command bundle
+(`frigg.integrations.listForReport`, `frigg.integrationMappings.countByIntegrationIds`,
+`frigg.usage.getTotalsByDimension`) — the retired reporting repository triad's
+logic now lives in the canonical integration/mapping repositories, so all
+database adapters (PostgreSQL, MongoDB, DocumentDB) are served through one path.
+
+## Caveats
+
+- Report output is sensitive (integration types, counts, user IDs, versions).
+  Treat the admin API key as an admin secret.
+- The `integrations` back-compat alias is transitional; migrate to
+  `POST /api/v2/reports/integrations/run { "mode": "live" }`.
+- Labels only appear once the app registers the integration classes.
+- A `schedule` block in a report's Definition only supplies the default scheduled
+  **mode**. It does not create a recurring trigger on its own — activate the
+  schedule with `PUT /api/v2/reports/:name/schedule`; the DB schedule
+  (`ScriptSchedule`) is the single source of truth.

package/reporting/builtin-reports.js

@@ -0,0 +1,6 @@
+const { IntegrationsReport } = require('./reports/integrations-report');
+
+// Registered by the admin-scripts bootstrap when the app definition sets `admin.includeBuiltinReports`.
+const BUILTIN_REPORTS = [IntegrationsReport];
+
+module.exports = { BUILTIN_REPORTS };

package/reporting/index.js

@@ -0,0 +1,13 @@
+const { ReportBase } = require('./report-base');
+const { IntegrationsReport } = require('./reports/integrations-report');
+const { BUILTIN_REPORTS } = require('./builtin-reports');
+const {
+    createReportCommands,
+} = require('../application/commands/report-commands');
+
+module.exports = {
+    ReportBase,
+    IntegrationsReport,
+    BUILTIN_REPORTS,
+    createReportCommands,
+};

package/reporting/report-base.js

@@ -0,0 +1,49 @@
+/**
+ * ReportBase — a report is an admin operation whose output is its payload.
+ *
+ * A report Definition declares:
+ *   - runModes: allowed modes; the first is the default.
+ *       'live'     — compute and return inline; persist nothing.
+ *       'recorded' — persist an execution record (input + results + logs).
+ *       'snapshot' — a recorded run tagged as a point in a named series.
+ *   - output.format: 'json' returns inline; 'csv'|'pdf'|'zip' go to artifact storage.
+ *   - schedule: optional recurring run via EventBridge.
+ *
+ * execute(frigg, params): `frigg` is the admin command bundle
+ * (=== this.context.commands); do data reads through it, never a repository.
+ */
+class ReportBase {
+    static Definition = {
+        name: 'Report Name',
+        version: '0.0.0',
+        description: 'What this report computes',
+        source: 'USER_DEFINED', // 'BUILTIN' | 'USER_DEFINED'
+
+        runModes: ['live', 'recorded', 'snapshot'],
+        inputSchema: null,
+        outputSchema: null,
+        output: { format: 'json' },
+        schedule: { enabled: false, cron: null, mode: 'snapshot' },
+
+        config: {
+            timeout: 300000,
+        },
+
+        display: {
+            category: 'reporting',
+            icon: null,
+        },
+    };
+
+    constructor(params = {}) {
+        this.context = params.context || null;
+        this.executionId = params.executionId || null;
+        this.integrationFactory = params.integrationFactory || null;
+    }
+
+    async execute(frigg, params) {
+        throw new Error('ReportBase.execute() must be implemented by subclass');
+    }
+}
+
+module.exports = { ReportBase };

package/reporting/reports/integrations-report.js

@@ -0,0 +1,221 @@
+const { ReportBase } = require('../report-base');
+const { CANONICAL_COUNTERS } = require('../../telemetry/canonical-counters');
+const { loadAppDefinition } = require('../../handlers/app-definition-loader');
+
+const SCHEMA_VERSION = 1;
+const SERVICE = 'frigg-core-api';
+
+// Seeded so every known status appears in output even at count 0.
+const KNOWN_STATUSES = [
+    'IN_CREATION',
+    'ENABLED',
+    'ERROR',
+    'NEEDS_CONFIG',
+    'PROCESSING',
+    'IN_DELETION',
+    'DISABLED',
+];
+
+// IntegrationBase.Definition default — skip it so the slug is used instead.
+const PLACEHOLDER_DISPLAY_NAME = 'Integration Name';
+
+class IntegrationsReport extends ReportBase {
+    static Definition = {
+        name: 'integrations',
+        version: '1.0.0',
+        description: 'Integrations by status and type, with per-type usage counts',
+        source: 'BUILTIN',
+        runModes: ['live', 'recorded', 'snapshot'],
+        inputSchema: {
+            type: 'object',
+            additionalProperties: false,
+            properties: {
+                status: { type: 'string' },
+                type: { type: 'string' },
+                userId: { type: 'string' },
+            },
+        },
+        output: { format: 'json' },
+        schedule: { enabled: false, cron: null, mode: 'snapshot' },
+        display: { category: 'reporting', icon: null },
+    };
+
+    async execute(frigg, params = {}) {
+        const { status, type, userId } = this._validateQuery(params);
+        const typeLabels = buildTypeLabels();
+
+        const rows = await frigg.integrations.listForReport({ status, userId });
+
+        // type lives in config.type, a JSON path not portably groupable across DBs, so filter here not in the query.
+        const filtered =
+            type === undefined
+                ? rows
+                : rows.filter((row) => (row.type ?? 'unknown') === type);
+
+        const ids = filtered.map((row) => row.id);
+        const mappingCounts = ids.length
+            ? await frigg.integrationMappings.countByIntegrationIds(ids)
+            : new Map();
+
+        const integrations = filtered.map((row) => ({
+            id: row.id,
+            type: row.type ?? 'unknown',
+            status: row.status ?? null,
+            userId: row.userId ?? null,
+            version: row.version ?? null,
+            moduleCount: row.moduleCount ?? 0,
+            errorCount: row.errorCount ?? 0,
+            mappedRecordCount: mappingCounts.get(row.id) ?? 0,
+            createdAt: toIso(row.createdAt),
+            updatedAt: toIso(row.updatedAt),
+        }));
+
+        const byStatus = emptyStatusCounts();
+        const byTypeMap = new Map();
+        for (const integration of integrations) {
+            // sentinel so a missing status never becomes a literal "null" key
+            const statusKey = integration.status ?? 'UNKNOWN';
+            byStatus[statusKey] = (byStatus[statusKey] ?? 0) + 1;
+
+            if (!byTypeMap.has(integration.type)) {
+                byTypeMap.set(integration.type, {
+                    type: integration.type,
+                    label: typeLabels[integration.type] || integration.type,
+                    total: 0,
+                    byStatus: emptyStatusCounts(),
+                });
+            }
+            const bucket = byTypeMap.get(integration.type);
+            bucket.total += 1;
+            bucket.byStatus[statusKey] = (bucket.byStatus[statusKey] ?? 0) + 1;
+        }
+
+        await this._attachUsageColumns(byTypeMap, frigg);
+
+        return {
+            schemaVersion: SCHEMA_VERSION,
+            service: SERVICE,
+            generatedAt: new Date().toISOString(),
+            filters: {
+                status: status ?? null,
+                type: type ?? null,
+                userId: userId ?? null,
+            },
+            metrics: {
+                total: integrations.length,
+                byStatus,
+                byType: Array.from(byTypeMap.values()),
+                typeLabels: { ...typeLabels },
+                integrations,
+            },
+        };
+    }
+
+    async _attachUsageColumns(byTypeMap, frigg) {
+        const metrics = Object.keys(CANONICAL_COUNTERS);
+        try {
+            const totalsByMetric = await Promise.all(
+                metrics.map(async (metric) => {
+                    const totals = await frigg.usage.getTotalsByDimension({
+                        metric,
+                        groupBy: 'integrationType',
+                    });
+                    const map = new Map(
+                        (totals || []).map((t) => [t.integrationType, t.value])
+                    );
+                    return [metric, map];
+                })
+            );
+
+            for (const bucket of byTypeMap.values()) {
+                bucket.usage = {};
+                for (const [metric, map] of totalsByMetric) {
+                    bucket.usage[metric] = map.get(bucket.type) ?? 0;
+                }
+            }
+        } catch (error) {
+            console.warn(
+                `[Frigg][reporting] usage columns unavailable: ${
+                    error && error.message
+                }`
+            );
+        }
+    }
+
+    _validateQuery({ status, type, userId } = {}) {
+        for (const [key, value] of Object.entries({ status, type, userId })) {
+            if (
+                value !== undefined &&
+                value !== null &&
+                typeof value !== 'string'
+            ) {
+                throw invalidInput(
+                    `Invalid query parameter '${key}': expected a string`
+                );
+            }
+        }
+        const normalize = (value) => value || undefined;
+        const normalized = {
+            status: normalize(status),
+            type: normalize(type),
+            userId: normalize(userId),
+        };
+        if (normalized.status && !KNOWN_STATUSES.includes(normalized.status)) {
+            throw invalidInput(
+                `Invalid status '${
+                    normalized.status
+                }'. Expected one of: ${KNOWN_STATUSES.join(', ')}`
+            );
+        }
+        return normalized;
+    }
+}
+
+// INVALID_INPUT keeps the report protocol-agnostic; the runner/router map it to a 400.
+function invalidInput(message) {
+    const error = new Error(message);
+    error.code = 'INVALID_INPUT';
+    return error;
+}
+
+function emptyStatusCounts() {
+    return KNOWN_STATUSES.reduce((acc, status) => {
+        acc[status] = 0;
+        return acc;
+    }, {});
+}
+
+function toIso(value) {
+    if (!value) return null;
+    if (value instanceof Date) return value.toISOString();
+    // DocumentDB raw reads surface dates as { $date: ... }
+    if (typeof value === 'object' && value.$date) {
+        const date = new Date(value.$date);
+        return Number.isNaN(date.getTime()) ? null : date.toISOString();
+    }
+    return String(value);
+}
+
+function buildTypeLabels() {
+    try {
+        const { integrations = [] } = loadAppDefinition();
+        const labels = {};
+        for (const IntegrationClass of integrations) {
+            const def = IntegrationClass?.Definition;
+            if (!def?.name) continue;
+            const label = def.display?.label;
+            if (label && label !== PLACEHOLDER_DISPLAY_NAME) {
+                labels[def.name] = label;
+            }
+        }
+        return labels;
+    } catch (error) {
+        console.error(
+            'Reporting: failed to load integration labels:',
+            error.message
+        );
+        return {};
+    }
+}
+
+module.exports = { IntegrationsReport, SCHEMA_VERSION };