@powerhousedao/academy 0.1.0-dev.0

package/docs/academy/02-AdvancedTutorial/04-WorkWithData/06-Analytics Engine/GraphQL References/QueryingADocumentWithGraphQL.md

@@ -0,0 +1,244 @@
+# Querying a document with GraphQL
+
+## Overview
+
+In this documentation we'll show how to use a **GraphQL** query to query a document model.
+
+We will be looking at 2 different GraphQL queries for retrieving data from a 'Real World Asset Portfolio Report'.
+But the **same queries can be used for any other document model**.
+
+The queries below focus on 2 different ways of receiving the data.
+We will show how you can receive:
+
+> #### 1. The complete state of the document: 
+> Such as the array of accounts, Special Purpose Vehicles (SPVs), fixed income types, fee types, portfolio details, and transactions associated with a particular RWA-report.
+
+> #### 2. Only the latest changes and updates:
+> Or specific operations of a document by registering a listener with a specific filter.
+
+### Adding the specific Document Drive to Connect with an URL
+
+The specific drive where your RWA-report or any other document is being stored can be accessed through Connect. Connect functions as your decentralised document drive to access and edit document models that capture the data present in your organisation.
+
+Get access to an organisations drive instances by adding this drive to your Connect Drive navigation tree view. click the (+) to add a public drive with the help of the correct link. To add a new drive you'll have to know the correct URL for the drive.
+
+## Querying data from Connect with Switchboard
+
+Whenever you want to start a query from a document within connect you can open up switchboard by looking for the switchboard logo in the top right hand corner of the document editor interface, or by clicking a document in the connect drive explorer and opening the document in switchboard. This feature will not be available for your local drives as they are not hosted on switchboard.
+
+![untitled](./rwa-reports/raw-reports1.png)   
+Right click a document and find a direct link to switchboard GraphQL playground*
+
+## Querying data from Connect in Apollo Studio
+
+Aside from switchboard you're able to make use of GraphQL interfaces such as Apollo Studio.
+When opening the document in switchboard the endpoint will be visible at the top of the interface.   Copy this endpoint and use it as your API endpoint in Apollo.
+
+![untitled](./rwa-reports/raw-reports2.png)
+*The endpoint you'll be using for any other GraphQL playgrounds or sandboxes*
+
+### 1. Querying the complete state of a document
+
+This example query is structured to request a document by its unique identifier (id).   
+It extracts common fields such as id, name, documentType, revision, created, and lastModified.
+
+Additionally, it retrieves specific data related to the 'Real World Assets' document model, including accounts, SPVs, fixed income types, fee types, portfolio, and transactions. The RWA section of the query is designed to pull in detailed information about the financial structure and transactions of real-world assets managed within the document model.
+
+#### Common Document Fields
+
+- `name`: The name of the document.
+- `documentType`: The type classification of the document.
+- `revision`: The version number of the document.
+- `created`: The creation timestamp of the document.
+- `lastModified`: The timestamp of when the document was last modified.
+
+<details>
+  <summary>List of document model specific fields.</summary>
+
+#### Real World Assets (RWA) Specific Fields
+
+**State**
+- `accounts`
+  - `id`: Unique identifier for the account.
+  - `reference`: Reference code or number associated with the account.
+  - `label`: A user-friendly name or label for the account.
+- `principalLenderAccountId`: Identifier for the principal lender's account.
+- `spvs`
+  - `id`: Unique identifier for the Special Purpose Vehicle (SPV).
+  - `name`: Name of the SPV.
+- `feeTypes`
+  - `id`: Unique identifier for the fee type.
+  - `name`: Name of the fee type.
+  - `feeType`: The classification or category of the fee.
+  - `accountId`: Identifier for the account associated with the fee.
+- `fixedIncomeTypes`
+  - `id`: Unique identifier for the fixed income type.
+  - `name`: Name of the fixed income type.
+
+  **Portfolio**
+
+  **Fixed Income**
+  - `id`: Unique identifier for the fixed income asset.
+  - `fixedIncomeTypeId`: Identifier for the type of fixed income.
+  - `name`: Name of the fixed income asset.
+  - `spvId`: Identifier for the SPV associated with the fixed income asset.
+  - `maturity`: Maturity date of the fixed income asset.
+  - `purchaseDate`: Purchase date of the fixed income asset.
+  - `notional`: The notional amount of the fixed income asset.
+  - `purchasePrice`: Purchase price of the fixed income asset.
+  - `purchaseProceeds`: Proceeds from the purchase of the fixed income asset.
+  - `totalDiscount`: Total discount received for the fixed income asset.
+  - `annualizedYield`: The annualized yield of the fixed income asset.
+  - `ISIN`: International Securities Identification Number for the fixed income asset.
+  - `CUSIP`: Committee on Uniform Securities Identification Procedures number for the fixed income asset.
+  - `coupon`: Coupon rate of the fixed income asset.
+
+  **Cash**
+
+  - `id`: Unique identifier for the cash holding.
+  - `spvId`: Identifier for the SPV associated with the cash holding.
+  - `currency`: Currency of the cash holding
+
+</details>
+
+```graphql title="An example query for the full state of a document"
+  query {
+  document(id: "") {
+    name
+    documentType
+    revision
+    created
+    lastModified
+    ... on RealWorldAssets {
+      state {
+        accounts {
+          id
+          reference
+          label
+        }
+        principalLenderAccountId
+        spvs {
+          id
+          name
+        }
+        serviceProviderFeeTypes {
+          id
+          name
+          feeType
+          accountId
+        }
+        fixedIncomeTypes {
+          id
+          name
+        }
+        portfolio {
+          ... on FixedIncome {
+            id
+            fixedIncomeTypeId
+            name
+            spvId
+            maturity
+            purchaseDate
+            notional
+            purchasePrice
+            purchaseProceeds
+            totalDiscount
+            annualizedYield
+            ISIN
+            CUSIP
+            coupon
+          }
+          ... on Cash {
+            id
+            spvId
+            currency
+          }
+        }
+      }
+    }
+  }
+}
+
+```
+
+### 2. Querying for the latest updates or specific documents
+
+This query is particularly useful when you only need the latest changes from the document drive. 
+
+### 2.1 Registering a listener
+For this purpose we support adding listeners through a graphQL mutation such as the PullResponderListenerbelow. 
+
+```graphql
+mutation registerListener($filter: InputListenerFilter!) {
+  registerPullResponderListener(filter: $filter) {
+    listenerId
+  }
+}
+```
+
+### 2.2 Defining the filter
+
+Through this listener you can define the filter with query variables.   
+This allows you to filter for specific document ID's or a lists, documentTypes, scopes or branches.   
+Branches allow you to query different versions of a document in case there is a conflict accross different versions of the document or when contributors are maintaining separate versions with the help of branching
+
+    In this case we're filtering by document type makerdao/rwa-portfolio.
+
+```graphql
+{
+  "filter":{
+    "documentId": ["*"],
+    "documentType": ["makerdao/rwa-portfolio"],
+    "scope": "global",
+    "branch": "main"
+  }
+}
+```
+
+This combination of query + query variables will return a listenerID which can be used in the next step to query for specific strands. 
+
+![untitled](./rwa-reports/rwaRegister.png)
+*An example of registering a listener and receiving a listenerId back.*
+
+:::info
+A strand in this scenario can be understood as a list of operations that has been applied to the RWA portfolio or any other document. As a query variable you'll want to add the received listenerId from the previous step together with the pullstrands query below
+:::
+
+```graphql title="Pullstrands query"	
+query pullStrands ($listenerId:ID, $since: Date) {
+  system {
+    sync {
+      strands (listenerId: $listenerId, since: $since) {
+        driveId
+        documentId
+        scope
+        branch
+        operations {
+          index
+          skip
+          type
+          input
+          hash
+          timestamp
+        }
+      }
+    }
+  }
+}
+```
+
+```graphql title="Query variables for pullStrands"
+{
+  "listenerId":"7ef04e24-4466-4642-863d-4a71c29d30c7",
+  "since":"2024-02-21T16:45:30.300Z"
+}
+```
+
+![untitled](./rwa-reports/listener-raw.png)
+*An example of using the ListenerID to pull specific strands (or document operations)* 
+
+In case you'd only like to receive the latest operations of a document the latest timestamp can be used as a filter in the since query variable to only get the most relevant or latest changes. 
+
+:::info
+A "strand" within the context of Powerhouse's Document Synchronization Protocol also refers to a single synchronization channel that connects exactly one unit of synchronization to another, with all four parameters (drive_url, doc_id, scope, branch) set to fixed values. This setup means that synchronization happens at a granular level, focusing specifically on one precise aspect of synchronization between two distinct points of instances of a document or document drive.
+:::

package/docs/academy/02-AdvancedTutorial/04-WorkWithData/06-Analytics Engine/best-practices.md

@@ -0,0 +1,60 @@
+---
+sidebar_position: 1000
+---
+
+# Best Practices
+
+### GraphQL
+
+**Start general, end with detailed queries.** In the case you want to find a specific number for a specific metric and dimension, the rule of thumb is to start general in the filter definitions and slowly start cutting down the paths to a particular team or budget period.
+
+For example: Looking at the dimension section in the filter options: To see available sub paths for the budget, leave the select: "atlas" and lod as 5 . With lod:5 you’ll be able to see the different paths available under your defined filter. Then, later you can apply a more detailed path in the select: "atlas/..." path that will suit your needs.
+
+```graphql
+{
+  "filter": {
+  ...
+    "dimensions": [
+      {
+        "name": "budget",
+        "select": "atlas",
+        "lod": 5
+      }
+    ],
+   ...
+  }
+}
+```
+
+**Follow upper case, lower case rules.** To make sure to get the results you require, you must follow the upper or lower case styling for defining the metrics and granularity. To see the available metrics and dimensions just use the referenced queries. For granularity, check the above granularity section for the different types.
+
+**Fetch per specified path** if performance is an issue.
+
+### Guidelines for Selecting and Combining Dimensions
+
+1. **Understand the Purpose of Analysis**  
+Before selecting dimensions, clarify the objective of your analysis. Are you looking to track expenses for a specific project, analyze budget utilization, or examine transaction patterns? Your objective will guide which dimensions are most relevant.
+
+2. **Choose Relevant Dimensions**  
+Select dimensions that align with your analytical goals. For instance, use the 'Project' dimension for project-based financial tracking or 'Wallet' for blockchain transaction analysis.
+
+3. **Combining Dimensions for Depth**  
+Combine multiple dimensions to gain more nuanced insights. For example, you might combine 'Budget' and 'Category' to understand how different categories of expenses contribute to overall budget utilization within a specific area.
+
+4. **Hierarchy and Path Considerations**  
+Pay attention to the hierarchical structure in dimension paths. For instance, paths like atlas/scopes/SUP/I/PHOENIX/ suggest a structured breakdown that can be crucial for detailed analysis.
+
+5. **Utilize Descriptions for Context**  
+Where available, use the descriptions provided with dimension values to understand the context and relevance of each dimension to your analysis. This is particularly helpful in dimensions with null labels, where the path and description provide critical information.
+
+6. **Avoid Over-Complication**  
+While combining dimensions can provide depth, avoid overly complex combinations that might lead to confusing or inconclusive results. Stick to combinations that directly serve your analysis objectives.
+
+7. **Use Icons for Quick Reference**  
+Where icons are available, they can be used as a quick visual reference to identify different dimensions or categories, particularly in user interfaces where rapid identification is beneficial.
+
+8. **Experiment and Iterate**  
+Don’t hesitate to experiment with different combinations of dimensions to see which provide the most meaningful insights. The flexibility of the dimensions allows for various permutations and combinations to suit diverse analytical needs.
+
+9. **Stay Updated**  
+Keep abreast of any changes or additions to the dimensions within the analytics engine, as this can impact ongoing and future analyses.

package/docs/academy/02-AdvancedTutorial/04-WorkWithData/06-Analytics Engine/graphql/index.md

@@ -0,0 +1,166 @@
+---
+sidebar_position: 3
+---
+
+# GraphQL API
+
+A basic understanding of GraphQL is helpful to understand this document, as we use concepts and vocabulary from the broader ecosystem. For a good primer on the subject, see the [Introduction to GraphQL](https://graphql.org/learn/).
+
+## Basic GraphQL Structure:
+
+- **Query**: At its core, a GraphQL query is about asking or 'querying' for specific fields on objects. The query is structured like a JSON object, but without the values. This structure lets you specify exactly which data you want to fetch from the server.
+
+- **Fields**: In a GraphQL query, you specify a set of fields within braces. Each field corresponds to a piece of data you want to fetch.
+
+- **Nested Fields**: Fields can also be nested, allowing you to fetch related data. For example, if you have a user, you can fetch the user's name and, within the same query, fetch a list of the user's friends and their names.
+
+### Example
+
+Imagine you have a user object and you want to fetch the user's name and email, along with the names of their friends. A basic GraphQL query might look like this:
+
+```graphql
+query {
+  user(id: "1") {
+    name
+    email
+    friends {
+      name
+    }
+  }
+}
+```
+
+In this example, query is the operation type, user is the field on the root type, and name, email, and friends are fields on the user type. The id: `"1"` is a parameter passed to the query, specifying which user to fetch.
+You can find more examples of GraphQL core concept in the section [GraphQL at Powerhouse](/docs/academy/AdvancedTutorial/WorkWithData/GraphQLAtPowerhouse).
+
+## Analytics Query Structure
+
+In the GraphQL query for the Analytics Engine below, each element requests specific pieces of data from the analytics API. Here is a breakdown of the fields and elements with a dropdown with their descriptions below.
+
+```graphql
+query Analytics($filter: AnalyticsFilter) {
+  analytics {
+    series(filter: $filter) {
+      period
+      start
+      end
+      rows {
+        metric
+        unit
+        dimensions {
+          name
+          path
+          label
+          description
+          icon
+        }
+        value
+        sum
+      }
+    }
+  }
+}
+```
+
+<details>
+  <summary>👉 Expand this dropdown to explore the list of fields and retrievable data..</summary>
+
+In the provided GraphQL query, each field and element plays a specific role in determining what data is returned by the analytics engine. Here's a clear and concise description of each:
+
+- `query Analytics($filter: AnalyticsFilter):` This is the declaration of the query named 'Analytics', which accepts a variable `$filter` of type `AnalyticsFilter`. This variable will be used to apply certain filters to the analytics data retrieved.
+
+- `analytics`: This is the main query operation. It calls the analytics field on the GraphQL API.
+
+- `series(filter: $filter)`: This field represents a collection of data points or a data set that matches the criteria specified by the filter. It's an array of results, where each result is a time-bound statistical representation of the filtered data. And passes the `$filter` variable as an argument to determine the scope of the data returned.
+
+- `period`: Within each series, this field denotes the aggregation period for the data (e.g., monthly, quarterly, annually). It's a label describing the time span each series covers.
+start: This is the starting date and time of the data series, indicating when the period begins.
+
+- `end`: This is the ending date and time of the data series, indicating when the period ends.
+
+- `rows`: Represents the individual records or entries in the data series. Each row corresponds to a unique combination of dimensions for the specified period.
+
+- `metric`: Within each row, this field specifies the particular metric being measured (e.g., budget, actuals, forecast).
+
+- `unit`: This indicates the unit of measurement for the metric, such as quantities, currency (e.g., DAI), or percentages.
+
+- `dimensions`: A nested array that provides context for the metric by breaking it down into finer categories or segments, such as 'project' or 'category'. Each dimension can contain:
+    - `name`: The identifier or key for the dimension.
+    - `path`: A structured representation of the dimension's hierarchy or location within a dataset.
+    - `label`: A human-readable label for the dimension, which can be used for display purposes.
+    - `description`: A brief explanation of the dimension to give users an understanding of what it represents.
+    - `icon`: A graphical representation or icon associated with the dimension for easier identification in user interfaces.
+
+- `value`: The actual numerical value of the metric for each row within the specified period.
+- `sum`: A total or aggregated value of the metric over the entire period, providing a summarized figure.
+
+This query structure allows users to extract detailed and summarized analytics data with it's context provided with the help of the the dimensions, tailored to specific time frames and measurements.
+
+</details>
+
+### Filters
+
+In the analytics engine, the filter object in a GraphQL query is crucial for tailoring the data retrieval to specific requirements. For the query to function correctly, a complete and fully structured filter object, containing all necessary parameters, must be used. Here's a guide on how to construct these filter objects and examples of different filter configurations.
+
+### Constructing a Complete Filter Object
+
+A filter object must include all the following parameters:
+
+
+1. Start Date (`start`): The beginning date of the data retrieval period.
+
+2. End Date (`end`): The end date of the data retrieval period.
+
+3. Granularity(`granularity`): Determines the aggregation level of the data (e.g., total, annual, monthly).
+
+4. Metrics (`metrics`): A list of metrics to be analyzed (e.g., Budget, Actuals).
+
+5. Dimensions (`dimensions`): Specifies qualitative data categories for segmentation, each with sub-parameters like `name`, `select`, and level of detail (`lod`).
+
+In the analytics engine, dimensions play a critical role in segmenting and analyzing data. The `select` field within a dimension and its correlation with the level of detail (`lod`)
+parameter are particularly crucial for tailoring your query to retrieve the most relevant and precise data. Here's a detailed explanation of their importance:
+
+The `select` Field in Dimensions
+     - **Function**: The select field specifies the exact segment or category within a dimension that you want to analyze. For example, in a dimension named "project," the select field could specify a particular project like "SES" or "Atlas."
+     - **Path Specification**: The value set in the select field often represents a path in the data hierarchy. This path directs the query to focus on specific segments or nodes within the broader dimension category.
+     - **Precision in Data Retrieval**: By setting the right path in the select field, you ensure that the query fetches data pertinent only to that specific segment, leading to more targeted and relevant insights.
+     - **Example**: If you set the **select** field to "`atlas/scopes/SUP/I/PHOENIX/`" in the "`budget`" dimension, the query will retrieve data related to the budget allocated specifically for the "`PHOENIX`" project under the "`SUP`" scope in the "`Atlas`" system.
+
+The level of detail (`lod`) Parameter
+    - **Granularity** Within Dimensions: While the select field specifies what to select within a dimension, the `lod` parameter determines how detailed or summarized the information should be.
+    - **Hierarchy Levels**: Different levels in `lod` represent different levels of detail in the data hierarchy. A higher `lod` value typically means a more detailed breakdown, while a lower value indicates a more summarized or aggregated view.
+    - **Correlation with `select` Path**: The lod value you choose should correspond appropriately to the path specified in the `select` field. A mismatch might lead to data that is either too granular or too generalized than what is needed.
+    - **Impact on Analysis**: The level of detail can significantly affect the analysis. For instance, a high lod can provide in-depth insights into a specific area, useful for detailed audits or close examination of a particular segment. Conversely, a low lod is better for broader overviews or when comparing larger categories.
+
+Importance of Correct Configuration
+    - **Accuracy of Results**: Setting the correct path in the select field and aligning it with an appropriate lod ensures the accuracy and relevance of the query results. Incorrect or mismatched configurations might lead to misleading insights or data that doesn’t serve the intended analytical purpose.
+    - **Customized Analysis**: Together, the select field and lod allow for a high degree of customization in data queries. Users can tailor their data requests precisely to their specific requirements, whether they need a broad overview or a detailed breakdown.
+    - Follow the right upper or lower case letter style from metrics, granularity and dimensions.
+
+6. **Currency (`currency`)**: The currency format for the financial data (e.g., DAI, MKR).
+
+The filter object can be created by using the UI menu from the graphql apollo studio explorer:
+
+![untitled](../filter.png)
+*Select the filter*
+
+![untitled](../filteroptions.png)
+*Select all filter fields and sub fields*
+
+## Troubleshooting
+
+### Common Issues
+
+**Mistyping the filter options or not using proper upper or lower case when necessary.** Make sure to check this first when running into problems
+
+### Error Messages
+
+Usually, error messages are explicit and easy to understand what is wrong, below you can find some examples.
+
+- `"message": "Cannot read properties of undefined (reading 'filter')"`, → There is something wrong with the filter options, make sure you’ve added all fields in the filter options.
+
+- `"message": "No valid metrics provided, make sure to use metrics from this list: Actuals, Budget, Forecast, FTEs, PaymentsOffChainIncluded, PaymentsOnChain, ProtocolNetOutflow"`, → A wrong format when adding metrics
+
+- `"message": "Variable \"$filter\" got invalid value \"Monthlyu\" at \"filter.granularity\"; Value \"Monthlyu\" does not exist in \"AnalyticsGranularity\" enum. Did you mean the enum value \"monthly\"?"`, → Granularity filter is mistyped.
+
+**If you are receiving error messages with status code of 500, reach out to the dev team.**

package/docs/academy/02-AdvancedTutorial/04-WorkWithData/06-Analytics Engine/graphql/integration.md

@@ -0,0 +1,75 @@
+---
+sidebar_position: 1
+---
+
+# Integration
+
+The `analytics-engine-graphql` module provides types and resolvers needed for a fully-functional GraphQL Server API. This library has no dependencies on any particular server but has been tested using [Apollo Server 3 and 4](https://www.apollographql.com/docs/apollo-server).
+
+## Resolvers and Types
+
+The graphql package contains typical resolvers and typedefs. The resolvers, however, are dependent on another object: `AnalyticsModel`. This object is responsible for connecting GQL types with an `IAnalyticsStore` implementation.
+
+```typescript
+import { AnalyticsModel, AnalyticsResolvers, typedefs } from "@powerhousedao/analytics-engine-graphql";
+import { AnalyticsQueryEngine } from "@powerhousedao/analytics-engine-core";
+import { MemoryAnalyticsStore } from "@powerhousedao/analytics-engine-browser";
+
+const model = new AnalyticsModel(new AnalyticsQueryEngine(new MemoryAnalyticsStore()));
+```
+
+> Note that the analytics resolvers require the passed in context to have an `AnalyticsModel` implementation on it. See below for details.
+
+### Apollo 4
+
+A minimal Apollo 4 example is noted below for completeness, but a full working example can be found in the [MakerDAO-SES Ecosystem API repo](https://github.com/makerdao-ses/ecosystem-api/). Note, in particular, that we pass along a specific context in the Apollo express middleware.
+
+```typescript
+import express from "express";
+import http from "http";
+
+import { makeExecutableSchema } from "@graphql-tools/schema";
+import { ApolloServer } from "@apollo/server";
+import { expressMiddleware } from '@apollo/server/express4';
+import { ApolloServerPluginDrainHttpServer } from '@apollo/server/plugin/drainHttpServer';
+
+import { AnalyticsModel, AnalyticsResolvers, typedefs } from "@powerhousedao/analytics-engine-graphql";
+import { AnalyticsQueryEngine } from "@powerhousedao/analytics-engine-core";
+import { MemoryAnalyticsStore } from "@powerhousedao/analytics-engine-browser";
+
+// create a typical http service
+const app = express();
+const httpServer = http.createServer(app);
+
+// create analytics gql schema from typedefs + resolvers
+const schema = makeExecutableSchema({
+  typeDefs: typedefs,
+  resolvers: AnalyticsResolvers,
+});
+
+// create an apollo instance
+const plugins = [ApolloServerPluginDrainHttpServer({ httpServer }), responseCachePlugin()];
+const server = new ApolloServer({
+  schema,
+  plugins,
+});
+
+await server.start();
+
+// needed by the analytics resolvers
+const model = new AnalyticsModel(new AnalyticsQueryEngine(new MemoryAnalyticsStore()));
+
+app.use("/graphql",
+  express.json(),
+  expressMiddleware(server, {
+    context: ({ req }) => {
+      // the analytics resolvers expect this on the context
+      return {
+        dataSources: { db: { Analytics: model } },
+      };
+    },
+  }),
+);
+
+await new Promise<void>((resolve) => httpServer.listen(options, resolve));
+```