@adobe/spacecat-shared-rum-api-client 1.8.4 → 2.0.1

package/CHANGELOG.md

@@ -1,3 +1,23 @@
+# [@adobe/spacecat-shared-rum-api-client-v2.0.1](https://github.com/adobe/spacecat-shared/compare/@adobe/spacecat-shared-rum-api-client-v2.0.0...@adobe/spacecat-shared-rum-api-client-v2.0.1) (2024-06-06)
+
+
+### Bug Fixes
+
+* **deps:** update dependency @adobe/fetch to v4.1.4 ([#253](https://github.com/adobe/spacecat-shared/issues/253)) ([687d739](https://github.com/adobe/spacecat-shared/commit/687d73947f15344ed6f4d6e74f223aa838ec3d6a))
+
+# [@adobe/spacecat-shared-rum-api-client-v2.0.0](https://github.com/adobe/spacecat-shared/compare/@adobe/spacecat-shared-rum-api-client-v1.8.4...@adobe/spacecat-shared-rum-api-client-v2.0.0) (2024-06-06)
+
+
+### Features
+
+* client for rum bundler ([#252](https://github.com/adobe/spacecat-shared/issues/252)) ([854fc58](https://github.com/adobe/spacecat-shared/commit/854fc583d7184048d22c357cf11f349d3c1cfc9d))
+
+
+### BREAKING CHANGES
+
+* With this change RUMAPIClient will not be using
+helix-run-query as datasource anymore
+
 # [@adobe/spacecat-shared-rum-api-client-v1.8.4](https://github.com/adobe/spacecat-shared/compare/@adobe/spacecat-shared-rum-api-client-v1.8.3...@adobe/spacecat-shared-rum-api-client-v1.8.4) (2024-06-01)
 
 

package/README.md

@@ -12,7 +12,7 @@ npm install @adobe/spacecat-shared-rum-api-client
 
 ## Usage
 
-### Creating and instance from Helix UniversalContext
+#### Creating and instance from Helix UniversalContext
 
 ```js
 const context = {}; // Your AWS Lambda context object
@@ -20,66 +20,117 @@ const rumApiClient = RUMAPIClient.createFrom(context);
 
 ```
 
-### Constructor
-
-`RUMAPIClient` class needs RUM API domain key to be instantiated:
+#### From constructor
 
 ```js
-const domainKey = "your-domain-key";
-const rumApiClient = new RUMAPIClient(domainKey);
+const rumApiClient = new RUMAPIClient();
 ```
 
-### Creating a RUM Backlink
+### Running a query
 
 ```js
-const url = "https://example.com";
-const expiryInDays = 7;
-
-const backlink = await rumApiClient.createRUMBacklink(url, expiryInDays);
-console.log(`Backlink created: ${backlink}`)
+const opts = {
+  domain: 'www.aem.live',
+  domainkey: '<domain-key>',
+  granularity: 'hourly',
+  interval: 10
+}
+
+const result = await rumApiClient.query('cwv', opts);
+console.log(`Query result: ${result}`)
 ```
 
-### Creating a 404 Report Backlink
-
-```js
-const url = "https://example.com";
-const expiryInDays = 7;
-
-const backlink = await rumApiClient.create404Backlink(url, expiryInDays);
-console.log(`Backlink created: ${backlink}`)
+**Note**: all queries must be lowercase
+
+### Query Options: the 'opts' object
+
+| option      | required | default | remarks             |
+|-------------|----------|---------|---------------------|
+| domain      | yes      |         |                     |
+| domainkey   | yes      |         |                     |
+| interval    | no       | 7       | days in integer     |
+| granularity | no       | daily   | 'daily' or 'hourly' |
+
+## Available queries
+
+### cwv
+
+Calculates the CWV data for a given domain within the requested interval. It gets the 
+P75 values for LCP, CLS, INP, TTFB metrics, along with the number of data points available for
+each metric. Additionally, it provides grouping by URL and includes the count of page view data.
+
+An example response:
+
+```json
+[
+  {
+    "url": "https://www.aem.live/home",
+    "pageviews": 2620,
+    "lcp": 2099.699999988079,
+    "lcpCount": 9,
+    "cls": 0.020660136604802475,
+    "clsCount": 7,
+    "inp": 12,
+    "inpCount": 3,
+    "ttfb": 520.4500000476837,
+    "ttfbCount": 18
+  },
+  {
+    "url": "https://www.aem.live/developer/block-collection",
+    "pageviews": 2000,
+    "lcp": 512.1249999403954,
+    "lcpCount": 4,
+    "cls": 0.0005409526209424976,
+    "clsCount": 4,
+    "inp": 20,
+    "inpCount": 2,
+    "ttfb": 122.90000003576279,
+    "ttfbCount": 4
+  }
+]
 ```
+### 404
+
+Calculates the number of 404 errors for a specified domain within the requested interval. The results 
+are grouped by URL and the source of the 404 error. The output includes all the various sources that 
+direct traffic to the 404 page, as well as the total number of views originating from these sources.
+
+An example response:
+
+```json
+[
+  {
+    "url": "https://www.aem.live/developer/tutorial",
+    "views": 400,
+    "all_sources": [
+      "https://www.google.com",
+      "",
+      "https://www.instagram.com"
+    ],
+    "source_count": 3,
+    "top_source": "https://www.google.com"
+  },
+  {
+    "url": "https://www.aem.live/some-other-page",
+    "views": 300,
+    "all_sources": [
+      "https://www.bing.com",
+      ""
+    ],
+    "source_count": 2,
+    "top_source": ""
+  },
+  {
+    "url": "https://www.aem.live/developer/",
+    "views": 100,
+    "all_sources": [
+      ""
+    ],
+    "source_count": 1,
+    "top_source": ""
+  }
+]
 
-### Getting RUM Dashboard Data
-
-```js
-const url = "example.com";
-
-const rumData = await rumApiClient.getRUMDashboard({ url });
-console.log(`RUM data: ${rumData}`)
-```
-
-### Getting 404 checkpoints
-
-```js
-const url = "example.com";
-
-const backlink = await rumApiClient.get404Sources({ url });
-console.log(`404 Checkpoints: ${backlink}`)
-```
-
-### Getting Edge Delivery Services Domains
-
-```js
-const url = "all";
-
-const domains = await rumApiClient.getDomainList({}, url);
-console.log(`Backlink created: ${backlink}`)
-```
-
-## Testing
-Run the included tests with the following command:
-```
-npm test
 ```
 
 ## Linting

package/package.json

@@ -1,14 +1,15 @@
 {
   "name": "@adobe/spacecat-shared-rum-api-client",
-  "version": "1.8.4",
+  "version": "2.0.1",
   "description": "Shared modules of the Spacecat Services - Rum API client",
   "type": "module",
   "main": "src/index.js",
   "types": "src/index.d.ts",
   "scripts": {
-    "test": "c8 mocha",
+    "test": "c8 mocha 'test/**/*.test.js'",
     "lint": "eslint .",
-    "clean": "rm -rf package-lock.json node_modules"
+    "clean": "rm -rf package-lock.json node_modules",
+    "run": "node src/test.js"
   },
   "mocha": {
     "require": "test/setup-env.js",
@@ -30,11 +31,12 @@
     "access": "public"
   },
   "dependencies": {
-    "@adobe/fetch": "4.1.3",
+    "@adobe/fetch": "4.1.4",
     "@adobe/helix-shared-wrap": "2.0.2",
     "@adobe/helix-universal": "4.5.2",
     "@adobe/spacecat-shared-utils": "1.4.0",
-    "aws4": "1.13.0"
+    "aws4": "1.13.0",
+    "d3-array": "3.2.4"
   },
   "devDependencies": {
     "chai": "4.4.1",

package/src/common/aggregateFns.js

@@ -0,0 +1,28 @@
+/*
+ * Copyright 2024 Adobe. All rights reserved.
+ * This file is licensed to you under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under
+ * the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR REPRESENTATIONS
+ * OF ANY KIND, either express or implied. See the License for the specific language
+ * governing permissions and limitations under the License.
+ */
+
+/**
+ * Calculates the total page views by URL from an array of bundles.
+ * @param {Array<Object>} bundles - An array of RUM bundles (NOT Flat bundles).
+ * @returns {Object} An object where keys are URLs and values are the total page views for each URL.
+ */
+function pageviewsByUrl(bundles) {
+  return bundles.reduce((acc, cur) => {
+    if (!acc[cur.url]) acc[cur.url] = 0;
+    acc[cur.url] += cur.weight;
+    return acc;
+  }, {});
+}
+
+export {
+  pageviewsByUrl,
+};

package/src/common/constants.js

@@ -0,0 +1,16 @@
+/*
+ * Copyright 2024 Adobe. All rights reserved.
+ * This file is licensed to you under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under
+ * the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR REPRESENTATIONS
+ * OF ANY KIND, either express or implied. See the License for the specific language
+ * governing permissions and limitations under the License.
+ */
+
+export const GRANULARITY = {
+  HOURLY: 'HOURLY',
+  DAILY: 'DAILY',
+};

package/src/common/flat-bundle.js

@@ -0,0 +1,195 @@
+/*
+ * Copyright 2024 Adobe. All rights reserved.
+ * This file is licensed to you under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under
+ * the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR REPRESENTATIONS
+ * OF ANY KIND, either express or implied. See the License for the specific language
+ * governing permissions and limitations under the License.
+ */
+
+/**
+ * This utility class is designed to flatten checkpoints of RUM bundles served from the
+ * RUM bundler API. Its primary purpose is to simplify data juggling.
+ *
+ * RUM bundles are returned grouped by id. Often, there's a need to further group the bundles by
+ * another checkpoint such as URL, source, or target. By using this class, RUM bundles can be
+ * flattened for easier implementation of all required grouping operations.
+ *
+ * This class extends the standard Array class, to make all standard functions such as filter, map,
+ * and reduce available for use with this class as well.
+ *
+ * For instance, when the RUM bundler returns "bundles" in the following format:
+ *
+ * ```json
+ * [
+ *   {
+ *     "id": "BSX",
+ *     "time": "2024-05-26T05:00:02.706Z",
+ *     "url": "https://www.aem.live/developer/tutorial",
+ *     "weight": 100,
+ *     "events": [
+ *       {
+ *         "checkpoint": "navigate",
+ *         "target": "visible",
+ *         "source": "https://www.aem.live/docs/",
+ *         "timeDelta": 2706.199951171875
+ *       },
+ *       {
+ *         "checkpoint": "loadresource",
+ *         "target": 4,
+ *         "source": "https://www.aem.live/new-nav.plain.html",
+ *         "timeDelta": 2707.699951171875
+ *       },
+ *       {
+ *         "checkpoint": "play",
+ *         "source": "https://www.hlx.live/developer/videos/tutorial-step1.mp4",
+ *         "timeDelta": 12671.89990234375
+ *       },
+ *       {
+ *         "checkpoint": "viewmedia",
+ *         "target": "https://www.aem.live/developer/media_1c03ad909a87a4e318a33e780b93e4a1f8e7581a3.png",
+ *         "timeDelta": 43258.39990234375
+ *       }
+ *     ]
+ *   }
+ * ]
+ * ```
+ *
+ * After flattening, the FlatBundle appears as follows:
+ *
+ * ```json
+ * [
+ *   {
+ *     "id": "BSX",
+ *     "time": "2024-05-26T05:00:02.706Z",
+ *     "url": "https://www.aem.live/developer/tutorial",
+ *     "weight": 100,
+ *     "checkpoint": "navigate",
+ *     "target": "visible",
+ *     "source": "https://www.aem.live/docs/",
+ *     "timeDelta": 2706.199951171875
+ *   },
+ *   {
+ *     "id": "BSX",
+ *     "time": "2024-05-26T05:00:02.706Z",
+ *     "url": "https://www.aem.live/developer/tutorial",
+ *     "weight": 100,
+ *     "checkpoint": "loadresource",
+ *     "target": 4,
+ *     "source": "https://www.aem.live/new-nav.plain.html",
+ *     "timeDelta": 2707.699951171875
+ *   },
+ *   {
+ *     "id": "BSX",
+ *     "time": "2024-05-26T05:00:02.706Z",
+ *     "url": "https://www.aem.live/developer/tutorial",
+ *     "weight": 100,
+ *     "checkpoint": "play",
+ *     "source": "https://www.hlx.live/developer/videos/tutorial-step1.mp4",
+ *     "timeDelta": 12671.89990234375
+ *   },
+ *   {
+ *     "id": "BSX",
+ *     "time": "2024-05-26T05:00:02.706Z",
+ *     "url": "https://www.aem.live/developer/tutorial",
+ *     "weight": 100,
+ *     "checkpoint": "viewmedia",
+ *     "target": "https://www.aem.live/developer/media_1c03ad909a87a4e318a33e780b93e4a1f8e7581a3.png",
+ *     "timeDelta": 43258.39990234375
+ *   }
+ * ]
+ *
+ * ```
+ *
+ * @extends Array
+ */
+export class FlatBundle extends Array {
+  static fromArray(array) {
+    const flattened = array.flatMap((bundle) => {
+      const temp = { ...bundle };
+      delete temp.events;
+      return bundle.events.map((event) => ({
+        ...temp,
+        ...event,
+      }));
+    });
+
+    Object.setPrototypeOf(flattened, FlatBundle.prototype);
+    return flattened;
+  }
+
+  /**
+   * Groups FlatBundles by one or more keys. For example, this method can be used to
+   * group Flat Bundles by url and source as shown below:
+   *
+   * ```js
+   * FlatBundle.fromArray(bundles)
+   *     .groupBy('url', 'source');
+   * ```
+   * The output of this code returns a nested object with keys ("url", "source") and
+   * "items", nested to the depth of the number of keys used for grouping.
+   *
+   * An example response for grouping by url and source could be like:
+   *
+   * ```json
+   * [
+   *   {
+   *     "url": "some-url",
+   *     "items": [
+   *       { "source": "source1", "items": [] },
+   *       { "source": "source2", "items": [] }
+   *     ]
+   *   },
+   *   {
+   *     "url": "some-other-url",
+   *     "items": [
+   *       { "source": "source3", "items": [] }
+   *     ]
+   *   }
+   * ]
+   * ```
+   *
+   *
+   * @param {...string} keys - The keys to group by.
+   * @returns {Array} The grouped flat bundles.
+   */
+  groupBy(...keys) {
+    // Initialize the result as an empty array
+    const result = [];
+
+    // Create a map to hold references to the current levels
+    const map = new Map();
+
+    // Iterate over each item in the array
+    for (const item of this) {
+      let currentLevel = result;
+      let mapLevel = map;
+
+      // Iterate over each key to build the nested structure
+      for (const key of keys) {
+        const groupValue = item[key];
+
+        // Check if the group already exists
+        if (!mapLevel.has(groupValue)) {
+          // Create a new group if it doesn't exist
+          const newGroup = { [key]: groupValue, items: [] };
+          currentLevel.push(newGroup);
+          mapLevel.set(groupValue, { group: newGroup, nextMap: new Map() });
+        }
+
+        // Move to the next level
+        const { group, nextMap } = mapLevel.get(groupValue);
+        currentLevel = group.items;
+        mapLevel = nextMap;
+      }
+
+      // Add the item to the final level
+      currentLevel.push(item);
+    }
+
+    return result;
+  }
+}

package/src/common/rum-bundler-client.js

@@ -0,0 +1,92 @@
+/*
+ * Copyright 2024 Adobe. All rights reserved.
+ * This file is licensed to you under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under
+ * the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR REPRESENTATIONS
+ * OF ANY KIND, either express or implied. See the License for the specific language
+ * governing permissions and limitations under the License.
+ */
+/* eslint-disable no-await-in-loop */
+
+import { hasText } from '@adobe/spacecat-shared-utils';
+import { fetch } from '../utils.js';
+import { GRANULARITY } from './constants.js';
+
+const BASE_URL = 'https://rum.fastly-aem.page/bundles';
+const HOURS_IN_DAY = 24;
+const ONE_HOUR = 1000 * 60 * 60;
+const ONE_DAY = ONE_HOUR * HOURS_IN_DAY;
+
+const CHUNK_SIZE = 31;
+
+function filterBundles(checkpoints = []) {
+  return (bundle) => {
+    if (checkpoints.length > 0) {
+      const events = bundle.events.filter((event) => checkpoints.includes(event.checkpoint));
+      return {
+        ...bundle,
+        events,
+      };
+    }
+    return bundle;
+  };
+}
+
+function constructUrl(domain, date, granularity, domainkey) {
+  const year = date.getUTCFullYear();
+  const month = (date.getUTCMonth() + 1).toString().padStart(2, '0');
+  const day = date.getUTCDate().toString().padStart(2, '0');
+  const hour = granularity.toUpperCase() === GRANULARITY.HOURLY ? `/${date.getUTCHours().toString().padStart(2, '0')}` : '';
+
+  return `${BASE_URL}/${domain}/${year}/${month}/${day}${hour}?domainkey=${domainkey}`;
+}
+
+function getUrlChunks(urls, chunkSize) {
+  return Array(Math.ceil(urls.length / chunkSize))
+    .fill()
+    .map((_, index) => urls.slice(index * chunkSize, (index + 1) * chunkSize));
+}
+
+async function fetchBundles(opts = {}) {
+  const {
+    domain,
+    domainkey,
+    interval = 7,
+    granularity = GRANULARITY.DAILY,
+    checkpoints = [],
+  } = opts;
+
+  if (!hasText(domain) || !hasText(domainkey)) {
+    throw new Error('Missing required parameters');
+  }
+
+  const multiplier = granularity.toUpperCase() === GRANULARITY.HOURLY ? ONE_HOUR : ONE_DAY;
+  const range = granularity.toUpperCase() === GRANULARITY.HOURLY
+    ? interval * HOURS_IN_DAY
+    : interval + 1;
+
+  const urls = [];
+  const currentDate = new Date();
+
+  for (let i = 0; i < range; i += 1) {
+    const date = new Date(currentDate.getTime() - i * multiplier);
+    urls.push(constructUrl(domain, date, granularity, domainkey));
+  }
+
+  const chunks = getUrlChunks(urls, CHUNK_SIZE);
+
+  const result = [];
+  for (const chunk of chunks) {
+    const responses = await Promise.all(chunk.map((url) => fetch(url)));
+    const bundles = await Promise.all(responses.map((response) => response.json()));
+    result.push(...bundles.flatMap((b) => b.rumBundles.map(filterBundles(checkpoints))));
+  }
+  return result;
+}
+
+export {
+  fetchBundles,
+};

package/src/functions/404.js

@@ -0,0 +1,46 @@
+/*
+ * Copyright 2024 Adobe. All rights reserved.
+ * This file is licensed to you under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under
+ * the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR REPRESENTATIONS
+ * OF ANY KIND, either express or implied. See the License for the specific language
+ * governing permissions and limitations under the License.
+ */
+
+import { FlatBundle } from '../common/flat-bundle.js';
+
+function collect404s(groupedByUrlAndSource) {
+  const { url, items: itemsByUrl } = groupedByUrlAndSource;
+
+  // find top source which has the most amount of occurrences
+  const { source: topSource } = itemsByUrl.reduce(
+    (max, obj) => (obj.items.length > max.items.length ? obj : max),
+  );
+
+  // calculate the total number of views per 404 event
+  const views = itemsByUrl.flatMap((item) => item.items).reduce((acc, cur) => acc + cur.weight, 0);
+
+  return {
+    url,
+    views,
+    all_sources: itemsByUrl.map((item) => item.source),
+    source_count: itemsByUrl.length,
+    top_source: topSource,
+  };
+}
+
+function handler(bundles) {
+  return FlatBundle.fromArray(bundles)
+    .filter((row) => row.checkpoint === '404')
+    .groupBy('url', 'source')
+    .map(collect404s)
+    .sort((a, b) => b.views - a.views); // sort desc by views
+}
+
+export default {
+  handler,
+  checkpoints: ['404'],
+};

package/src/functions/cwv.js

@@ -0,0 +1,75 @@
+/*
+ * Copyright 2024 Adobe. All rights reserved.
+ * This file is licensed to you under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under
+ * the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR REPRESENTATIONS
+ * OF ANY KIND, either express or implied. See the License for the specific language
+ * governing permissions and limitations under the License.
+ */
+
+import { quantile } from 'd3-array';
+import { pageviewsByUrl } from '../common/aggregateFns.js';
+import { FlatBundle } from '../common/flat-bundle.js';
+
+const CWV_METRICS = ['lcp', 'cls', 'inp', 'ttfb'].map((metric) => `cwv-${metric}`);
+
+function collectCWVs(groupedByUrlIdTime) {
+  const { url, items: itemsByUrl } = groupedByUrlIdTime;
+
+  // first level: grouped by url
+  const CWVs = itemsByUrl.reduce((acc, { items: itemsById }) => {
+    // second level: grouped by id
+    const itemsByTime = itemsById.flatMap((itemById) => itemById.items);
+    // third level: grouped by time
+    const maximums = itemsByTime.reduce((values, item) => {
+      // each session (id-time) can contain multiple measurement for the same metric
+      // we need to find the maximum per metric type
+      // eslint-disable-next-line no-param-reassign
+      values[item.checkpoint] = Math.max(values[item.checkpoint] || 0, item.value);
+      return values;
+    }, {});
+
+    // max values per id for each metric type are collected into an array
+    CWV_METRICS.forEach((metric) => {
+      if (!acc[metric]) acc[metric] = [];
+      if (maximums[metric]) {
+        acc[metric].push(maximums[metric]);
+      }
+    });
+    return acc;
+  }, {});
+
+  return {
+    url,
+    lcp: quantile(CWVs['cwv-lcp'], 0.75) || null,
+    lcpCount: CWVs['cwv-lcp'].length,
+    cls: quantile(CWVs['cwv-cls'], 0.75) || null,
+    clsCount: CWVs['cwv-cls'].length,
+    inp: quantile(CWVs['cwv-inp'], 0.75) || null,
+    inpCount: CWVs['cwv-inp'].length,
+    ttfb: quantile(CWVs['cwv-ttfb'], 0.75) || null,
+    ttfbCount: CWVs['cwv-ttfb'].length,
+  };
+}
+
+function handler(bundles) {
+  const pageviews = pageviewsByUrl(bundles);
+
+  return FlatBundle.fromArray(bundles)
+    .groupBy('url', 'id', 'time')
+    .map(collectCWVs)
+    .filter((row) => row.lcp || row.cls || row.inp || row.ttfb) // filter out pages with no cwv data
+    .map((acc) => {
+      acc.pageviews = pageviews[acc.url];
+      return acc;
+    })
+    .sort((a, b) => b.pageviews - a.pageviews); // sort desc by pageviews
+}
+
+export default {
+  handler,
+  checkpoints: ['cwv-lcp', 'cwv-cls', 'cwv-inp', 'cwv-ttfb'],
+};

package/src/index.d.ts

@@ -1,5 +1,5 @@
 /*
- * Copyright 2023 Adobe. All rights reserved.
+ * Copyright 2024 Adobe. All rights reserved.
  * This file is licensed to you under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License. You may obtain a copy
  * of the License at http://www.apache.org/licenses/LICENSE-2.0
@@ -13,136 +13,34 @@
 import { UniversalContext } from '@adobe/helix-universal';
 
 export interface RUMAPIOptions {
-  interval?: number;
-  startdate?: string,
-  enddate?: string,
-  offset?: number;
-  limit?: number;
-  url?: string;
-}
-
-export interface RUMDashboardOptions {
-  interval?: number;
-  startdate?: string,
-  enddate?: string,
-  offset?: number;
-  limit?: number;
-  domainkey?: string,
-  url?: string;
+    domain: string;
+    domainkey: string;
+    interval?: number;
+    granularity?: 'hourly' | 'daily';
 }
 
 export default class RUMAPIClient {
   /**
-   * Static factory method to create an instance of RUMAPIClient.
-   * @param {UniversalContext} context - An object containing the AWS Lambda context information
-   * @returns An instance of RUMAPIClient.
-   * @remarks This method is designed to create a new instance from an AWS Lambda context.
-   *   The created instance is stored in the Lambda context, and subsequent calls to
-   *   this method will return the singleton instance if previously created.
-   */
+     * Static factory method to create an instance of RUMAPIClient.
+     * @param {UniversalContext} context - An object containing the AWS Lambda context information
+     * @returns An instance of RUMAPIClient.
+     * @remarks This method is designed to create a new instance from an AWS Lambda context.
+     *   The created instance is stored in the Lambda context, and subsequent calls to
+     *   this method will return the singleton instance if previously created.
+     */
   static createFrom(context: UniversalContext): RUMAPIClient;
 
   /**
-   * Constructor for creating an instance of RUMAPIClient.
-   * @param {string} domainkey - A string parameter representing the domain key of the RUM API.
-   *   This key is used to authenticate and interact with the RUM API.
-   * @remarks The domain key is specific to the RUM API.
-   */
-  constructor(domainkey: string);
-
-  /**
-   * Asynchronous method to create a RUM backlink.
-   * @param {string} url - A string representing the URL for the backlink.
-   * @param {number} expiry - An integer representing the expiry value for the backlink.
-   * @param {RUMDashboardOptions} params - An object representing the parameters to be included
-   * @returns A Promise resolving to a string representing url of the created backlink.
-   * @remarks This method creates a backlink to the RUM dashboard, allowing users
-   *   to view their reports and monitor real user activities.
-   */
-  createRUMBacklink(url: string, expiry: number, params?: RUMDashboardOptions): Promise<string>;
-
-  /**
-   * Asynchronous method to create a 404 backlink.
-   * @param {string} url - A string representing the URL for the backlink.
-   * @param {number} expiry - An integer representing the expiry value for the backlink.
-   * @param {RUMDashboardOptions} params - An object representing the parameters to be included
-   * @returns A Promise resolving to a string representing url of the created backlink.
-   * @remarks This method creates a backlink to the 404 report, allowing users
-   *   to view their 404 pages.
-   */
-  create404Backlink(url: string, expiry: number, params?: RUMDashboardOptions): Promise<string>;
-
-  /**
-   * Asynchronous method to return the RUM dashboard API call response data.
-   * @param {RUMAPIOptions} params - An object representing the parameters to be included
-   *  for the RUM Dashboard API call.
-   * @returns A Promise resolving to the RUM dashboard response data.
-   */
-  getRUMDashboard(params?: RUMAPIOptions): Promise<Array<object>>;
-
-  /**
-   * Asynchronous method to return the Experimentation API call response data.
-   * @param {RUMAPIOptions} params - An object representing the parameters to be included
-   *  for the Experimentation data API call.
-   * @returns A Promise resolving to the Experimentation response data.
-   */
-  getExperimentationData(params?: RUMAPIOptions): Promise<Array<object>>;
-
-  /**
-   * Method to return the url composed of params that the Experimentation API is called with.
-   * @param {RUMAPIOptions} params - An object representing the parameters to be included
-   *  for the Experimentation API call.
-   * @returns A string returning the Experimentation url including query parameters.
-   */
-  createExperimentationURL(params?: RUMAPIOptions): string;
-
-  /**
-   * Method to return the url composed of params that the rum-sources API is called with.
-   * @param {RUMAPIOptions} params - An object representing the parameters to be included
-   *  for the rum-sources API call.
-   * @returns A string returning the rum-sources url including query parameters.
-   */
-  createConversionURL(params?: RUMAPIOptions): string;
-
-  /**
-   * Asynchronous method to return the 404 sources API call response data.
-   * @param {RUMAPIOptions} params - An object representing the parameters to be included
-   *  for the 404 sources API call.
-   * @returns A Promise resolving to the 404 sources response data.
-   */
-  get404Sources(params?: RUMAPIOptions): Promise<Array<object>>;
-
-  /**
-   * Method to return the url composed of params that the RUM Dashboard API is called with.
-   * @param {RUMAPIOptions} params - An object representing the parameters to be included
-   *  for the RUM Dashboard API call.
-   * @returns A string returning to the RUM Dashboard url including query parameters.
-   */
-  createRUMURL(params?: RUMAPIOptions): string;
-
-  /**
-   * Method to return the url composed of params that the 404 sources API is called with.
-   * @param {RUMAPIOptions} params - An object representing the parameters to be included
-   *  for the 404 sources API call.
-   * @returns A string returning to the 404 sources url including query parameters.
-   */
-  create404URL(params?: RUMAPIOptions): string;
-
-  /**
-   * Asynchronous method to return an array with the domain for a specific url
-   *  or an array of all domain urls
-   * @param {RUMAPIOptions} params - An object representing the parameters to be included
-   * for the domain list call.
-   * @returns A Promise resolving to an array of the domain for a specific url
-   *  or an array of all domain urls .
-   */
-  getDomainList(params?: RUMAPIOptions): Promise<Array<string>>;
+     * Constructor for creating an instance of RUMAPIClient.
+     */
+  constructor();
 
   /**
-  * Asynchronous method to return the rum-sources API call response data.
-  * @param {RUMAPIOptions} params - An object representing the parameters to be included
-  *  for the rum-sources data API call.
-  * @returns A Promise resolving to the conversion response data.
-  */
-  getConversionData(params?: RUMAPIOptions): Promise<Array<string>>;
+     * Asynchronous method to run queries against RUM Bundler API.
+     * @param {string} query - Name of the query to run.
+     * @param {RUMAPIOptions} opts - A object containing options for query to run.
+     * @returns A Promise resolving to an object with the query results.
+     * @remarks See the README.md for the available queries.
+     */
+  query(query: string, opts?: RUMAPIOptions): Promise<object>;
 }

package/src/index.js

@@ -1,5 +1,5 @@
 /*
- * Copyright 2023 Adobe. All rights reserved.
+ * Copyright 2024 Adobe. All rights reserved.
  * This file is licensed to you under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License. You may obtain a copy
  * of the License at http://www.apache.org/licenses/LICENSE-2.0
@@ -9,192 +9,38 @@
  * OF ANY KIND, either express or implied. See the License for the specific language
  * governing permissions and limitations under the License.
  */
-import { createUrl } from '@adobe/fetch';
-import {
-  hasText, isArray, isInteger, isObject, dateAfterDays,
-} from '@adobe/spacecat-shared-utils';
-import { fetch } from './utils.js';
+import { fetchBundles } from './common/rum-bundler-client.js';
+import notfound from './functions/404.js';
+import cwv from './functions/cwv.js';
 
-const APIS = {
-  ROTATE_DOMAINKEYS: 'https://helix-pages.anywhere.run/helix-services/run-query@v3/rotate-domainkeys',
-  RUM_DASHBOARD_UI: 'https://data.aem.live/rum-dashboard',
-  NOT_FOUND_DASHBOARD_UI: 'https://data.aem.live/404-reports',
-  RUM_DASHBOARD: 'https://helix-pages.anywhere.run/helix-services/run-query@v3/rum-dashboard',
-  DOMAIN_LIST: 'https://helix-pages.anywhere.run/helix-services/run-query@v3/dash/domain-list',
-  RUM_404: 'https://helix-pages.anywhere.run/helix-services/run-query@v3/rum-404',
-  RUM_SOURCES: 'https://helix-pages.anywhere.run/helix-services/run-query@v3/rum-sources',
-  RUM_EXPERIMENTS: 'https://helix-pages.anywhere.run/helix-services/run-query@v3/rum-experiments',
+const HANDLERS = {
+  404: notfound,
+  cwv,
 };
 
-const DOMAIN_LIST_DEFAULT_PARAMS = {
-  interval: 30,
-  offset: 0,
-  limit: 100000,
-};
-
-export const RUM_DEFAULT_PARAMS = {
-  interval: 7,
-  offset: 0,
-  limit: 101,
-};
-
-export const CONVERSION_DEFAULT_PARAMS = {
-  ...RUM_DEFAULT_PARAMS,
-  checkpoint: 'convert',
-  aggregate: false,
-};
-
-export const create404URL = (params = {}) => createUrl(
-  APIS.RUM_404,
-  {
-    ...RUM_DEFAULT_PARAMS, ...params,
-  },
-);
-
-export const createRUMURL = (params = {}) => createUrl(
-  APIS.RUM_DASHBOARD,
-  {
-    ...RUM_DEFAULT_PARAMS, ...params,
-  },
-);
-
-export const createExperimentationURL = (params = {}) => createUrl(
-  APIS.RUM_EXPERIMENTS,
-  {
-    ...RUM_DEFAULT_PARAMS, ...params,
-  },
-);
-
-export const createConversionURL = (params = {}) => createUrl(
-  APIS.RUM_SOURCES,
-  {
-    ...CONVERSION_DEFAULT_PARAMS, ...params,
-  },
-);
-
-export async function sendRequest(url, opts) {
-  let respJson;
-  try {
-    const resp = await (isObject(opts) ? fetch(url, opts) : fetch(url));
-    respJson = await resp.json();
-  } catch (e) {
-    throw new Error(`Error during rum api call: ${e.message}`);
-  }
-
-  const data = respJson?.results?.data;
-  if (!isArray(data)) {
-    throw new Error('Unexpected response from rum api. $.results.data is not array');
-  }
-
-  return data;
-}
-
-async function generateDomainKey(domainkey, url, expiry) {
-  if (!hasText(url) || !isInteger(expiry)) {
-    throw new Error('Invalid input: url and expiry date parameters are required');
-  }
-
-  const params = {
-    domainkey,
-    url,
-    expiry: dateAfterDays(expiry).toISOString(),
-    note: 'generated by spacecat alerting',
-  };
-
-  const data = await sendRequest(createUrl(APIS.ROTATE_DOMAINKEYS, params), { method: 'POST' });
-
-  if (data.length === 0) {
-    throw new Error('Unexpected response: Rum api returned empty result');
-  }
-
-  if (data[0].status !== 'success') {
-    throw new Error('Unexpected response: Response was not successful');
-  }
-
-  if (!hasText(data[0].key)) {
-    throw new Error('Unexpected response: Rum api returned null domain key');
-  }
-
-  return data[0].key;
-}
-
-async function createBacklink(dashboardUrl, domainKey, domainUrl, expiry, params = {}) {
-  const scopedDomainKey = await generateDomainKey(domainKey, domainUrl, expiry);
-  const dataDeskParams = { ...params };
-  if (dataDeskParams.startdate) {
-    delete dataDeskParams.interval;
-  }
-  return createUrl(dashboardUrl, {
-    offset: 0,
-    limit: 100,
-    url: domainUrl,
-    domainkey: scopedDomainKey,
-    ...(dataDeskParams?.startdate ? {} : { interval: expiry }),
-    ...dataDeskParams,
-  });
-}
-
 export default class RUMAPIClient {
   static createFrom(context) {
     if (context.rumApiClient) return context.rumApiClient;
 
-    const { RUM_DOMAIN_KEY: domainkey } = context.env;
-
-    const client = new RUMAPIClient(domainkey);
+    const client = new RUMAPIClient();
     context.rumApiClient = client;
     return client;
   }
 
-  constructor(domainkey) {
-    if (!hasText(domainkey)) {
-      throw Error('RUM API Client needs a domain key to be set');
-    }
-
-    this.domainkey = domainkey;
-  }
-
-  async getRUMDashboard(params = {}) {
-    return sendRequest(createRUMURL({ ...params, domainkey: this.domainkey }));
-  }
-
-  async getExperimentationData(params = {}) {
-    return sendRequest(createExperimentationURL({ ...params, domainkey: this.domainkey }));
-  }
-
-  async getConversionData(params = {}) {
-    return sendRequest(createConversionURL({ ...params, domainkey: this.domainkey }));
-  }
-
-  async get404Sources(params = {}) {
-    return sendRequest(create404URL({ ...params, domainkey: this.domainkey }));
-  }
-
-  async getDomainList(params = {}) {
-    const data = await sendRequest(createUrl(
-      APIS.DOMAIN_LIST,
-      { domainkey: this.domainkey, ...DOMAIN_LIST_DEFAULT_PARAMS, ...params },
-    ));
-
-    return data.map((row) => row.hostname);
-  }
+  // eslint-disable-next-line class-methods-use-this
+  async query(query, opts) {
+    const { handler, checkpoints } = HANDLERS[query] || {};
+    if (!handler) throw new Error(`Unknown query ${query}`);
 
-  async createRUMBacklink(url, expiry, params) {
-    return createBacklink(
-      APIS.RUM_DASHBOARD_UI,
-      this.domainkey,
-      url,
-      expiry,
-      params,
-    );
-  }
+    try {
+      const bundles = await fetchBundles({
+        ...opts,
+        checkpoints,
+      });
 
-  async create404Backlink(url, expiry, params) {
-    return createBacklink(
-      APIS.NOT_FOUND_DASHBOARD_UI,
-      this.domainkey,
-      url,
-      expiry,
-      params,
-    );
+      return handler(bundles);
+    } catch (e) {
+      throw new Error(`Query '${query}' failed. Opts: ${JSON.stringify(opts)}. Reason: ${e.message}`);
+    }
   }
 }