@cccsaurora/howler-ui 2.14.0-dev.263 → 2.14.0-dev.264

package/components/routes/help/markdown/en/client.md

@@ -0,0 +1,213 @@
+# Howler Client Documentation
+
+This documentation will outline how to interact with the howler API using the howler client in both Java and python development environments. We will outline the basic process of creating a new hit in each environment as well as searching howler for hits matching your query.
+
+## Getting started
+
+### Installation
+
+In order to use the howler client, you need to list it as a dependency in your project.
+
+#### **Python**
+
+Simply install through pip:
+
+```bash
+pip install howler-client
+```
+
+You can also add it to your requirements.txt, or whatever dependency management system you use.
+
+### Authentication
+
+As outlined in the [Authentication Documentation](/help/auth), there's a number of ways users can choose to authenticate. In order to interface with the howler client, however, the suggested flow is to use an API key. So before we start, let's generate a key.
+
+1. Open the Howler UI you'd like to interface with.
+2. Log in, then click your profile in the top right.
+3. Under user menu, click Settings.
+4. Under User Security, press the (+) icon on the API Keys row.
+5. Name your key, and give it the requisite permissions.
+6. Press Create, and copy the supplied string somewhere safe. **You will not see this string again.**
+
+This API Key will be supplied to your code later on.
+
+## Python Client
+
+In order to connect with howler using the python client, there is a fairly simple process to follow:
+
+```python
+from howler_client import get_client
+
+USERNAME = 'user' # Obtain this from the user settings page of the Howler UI
+APIKEY = 'apikey_name:apikey_data'
+
+apikey = (USERNAME, APIKEY)
+
+howler = get_client("$CURRENT_URL", apikey=apikey)
+```
+
+```alert
+You can skip generating an API Key and providing it if you're executing this code within HOGWARTS (i.e., on jupyterhub or airflow). OBO will handle authentication for you!
+```
+
+That's it! You can now use the `howler` object to interact with the server. So what does that actually look like?
+
+### Creating hits in Python
+
+For the python client, you can create hits using either the `howler.hit.create` or `howler.hit.create_from_map` functions.
+
+#### `create`
+
+This function takes in a single argument - either a single hit, or a list of them, conforming to the [Howler Schema](/help/hit?tab=schema). Here is a simple example:
+
+```python
+# Some bogus data in the Howler Schema format
+example_hit = {
+  "howler": {
+    "analytic": "example",
+    "score": 10.0
+  },
+  "event": {
+    "reason": "Example hit"
+  }
+}
+
+howler.hit.create(example_hit)
+```
+
+You can also ingest data in a flat format:
+
+```python
+example_hit = {
+  "howler.analytic": "example",
+  "howler.score": 10.0,
+  "event.reason": "Example hit"
+}
+
+howler.hit.create(example_hit)
+```
+
+#### `create_from_map`
+
+This function takes in three arguments:
+
+- `tool name`: The name of the analytic creating the hit
+- `map`: A mapping between the raw data you have and the howler schema
+  - The format is a dictionary where the keys are the flattened path of the raw data, and the values are a list of flattened paths for Howler's fields where the data will be copied into.
+- `documents`: The raw data you want to add to howler
+
+Here is a simple example:
+
+```python
+# The mapping from our data to howler's schema
+hwl_map = {
+  "file.sha256": ["file.hash.sha256", "howler.hash"],
+  "file.name": ["file.name"],
+  "src_ip": ["source.ip", "related.ip"],
+  "dest_ip": ["destination.ip", "related.ip"],
+  "time.created": ["event.start"],
+}
+
+# Some bogus data in a custom format we want to add to howler
+example_hit = {
+  "src_ip": "0.0.0.0",
+  "dest_ip": "8.8.8.8",
+  "file": {
+    "name": "hello.exe",
+    "sha256": sha256(str("hello.exe").encode()).hexdigest()
+  },
+  "time": {
+    "created": datetime.now().isoformat()
+  },
+}
+
+# Note that the third argument is of type list!
+howler.hit.create_from_map("example_ingestor", hwl_map, [example_hit])
+```
+
+### Querying Hits
+
+Querying hits using the howler python client is done using the `howler.search.hit` function. It has a number of required and optional arguments:
+
+- Required:
+  - `query`: lucene query (string)
+- Optional:
+  - `filters`: Additional lucene queries used to filter the data (list of strings)
+  - `fl`: List of fields to return (comma separated string of fields)
+  - `offset`: Offset at which the query items should start (integer)
+  - `rows`: Number of records to return (integer)
+  - `sort`: Field used for sorting with direction (string: ex. 'id desc')
+  - `timeout`: Max amount of milliseconds the query will run (integer)
+  - `use_archive`: Also query the archive
+  - `track_total_hits`: Number of hits to track (default: 10k)
+
+Here are some example queries:
+
+```python
+# Search for all hits created by assemblyline, show the first 50, and return only their ids
+howler.search.hit("howler.analytic:assemblyline", fl="howler.id", rows=50)
+
+# Search for all resolved hits created in the last five days, returning their id and the analytic that created them. Show only ten, offset by 40
+howler.search.hit("howler.status:resolved", filters=['event.created:[now-5d TO now]'] fl="howler.id,howler.analytic", rows=10, offset=40)
+
+# Search for all hits, timeout if the query takes more than 100ms
+howler.search.hit("howler.id:*", track_total_hits=100000000, timeout=100, use_archive=True)
+```
+
+### Updating Hits
+
+In order to update hits, there are a number of supported functions:
+
+- `howler.hit.update(...)`
+- `howler.hit.update_by_query(...)`
+- `howler.hit.overwrite(...)`
+
+#### `update()`
+
+If you want to update a hit in a transactional way, you can use the following code:
+
+```python
+hit_to_update = client.search.hit("howler.id:*", rows=1, sort="event.created desc")["items"][0]
+
+result = client.hit.update(hit_to_update["howler"]["id"], [(UPDATE_SET, "howler.score", hit_to_update["howler"]["score"] + 100)])
+```
+
+The following operations can be run to update a hit.
+
+**List Operations:**
+
+- `UPDATE_APPEND`: Used to append a value to a given list
+- `UPDATE_APPEND_IF_MISSING`: Used to append a value to a given list if the value isn't already in the list
+- `UPDATE_REMOVE`: Will remove a given value from a list
+
+**Numeric Operations:**
+
+- `UPDATE_DEC`: Decrement a numeric value by the specified amount
+- `UPDATE_INC`: Increment a numeric value by the specified amount
+- `UPDATE_MAX`: Will set a numeric value to the maximum of the existing value and the specified value
+- `UPDATE_MIN`: Will set a numeric value to the minimum of the existing value and the specified value
+
+**Multipurpose Operations:**
+
+- `UPDATE_SET`: Set a field's value to the given value
+- `UPDATE_DELETE`: Will delete a given field's value
+
+#### `update_by_query()`
+
+This function allows you to update a large number of hits by a query:
+
+```python
+client.hit.update_by_query(f'howler.analytic:"Example Alert"', [(UPDATE_INC, "howler.score", 100)])
+```
+
+The same operations as in `update()` can be used.
+
+### `overwrite()`
+
+This function allows you to directly overwrite a hit with a partial hit object. This is the most easy to use, but loses some of the validation and additional processing of the update functions.
+
+```python
+hit_to_update = client.search.hit("howler.id:*", rows=1, sort="event.created desc")["items"][0]
+
+result = client.hit.overwrite(hit_to_update["howler"]["id"], {"source.ip": "127.0.0.1", "destination.ip": "8.8.8.8"})
+```

package/components/routes/help/markdown/en/links.md

@@ -0,0 +1,37 @@
+# Hit Links
+
+In order to facilitate the addition of additional tools one can use to triage a hit, Howler allows users to specify a set of links, along with a title and icon to show. This documentation will walk you through how to use these links.
+
+## Specification
+
+In order to add links, you can use the `howler.links` field. This field takes in a list of objects with three keys:
+
+```python
+hit = {
+  "howler.links": [
+    {
+      "title": "Link Title with Internal Image",
+      "href": "https://example.com",
+      # Note that this specifies another application, not an image link
+      "icon": "superset",
+    },
+    {
+      "title": "Link Title with External Image",
+      "href": "https://www.britannica.com/animal/goose-bird",
+      # Note that this specifies an image link. We don't provide hosting, so you'll need to host it somewhere else!
+      "icon": "https://cdn.britannica.com/76/76076-050-39DDCBA1/goose-Canada-North-America.jpg",
+    },
+  ]
+}
+```
+
+Note the icon can either be:
+
+1. A name identifying a linked application (from the app switcher), to use its icon
+2. An external URL
+
+If you'd like to use a linked app, the following values are currently supported:
+
+$APP_LIST
+
+Using any of these values will automatically use the corresponding icon. No need to host your own!

package/components/routes/help/markdown/en/notebook.md

@@ -0,0 +1,157 @@
+# Notebook Integration
+
+Howler provides the option to add notebooks to analytics to aid in triaging hits and alerts. It allows users to quickly spin up a notebook within a jupyter environment with either analytic and/or hit information.
+
+Howler will look for variables to replace within the first code cell of a notebook, giving the flexibility of providing context within the first cells using markdown.
+
+Here an example of how howler will replace the variables within your notebook:
+
+```notebook tab="Template"
+{
+  "cells": [
+    {
+      "cell_type": "code",
+      "id": "fe6f810f-2459-4ad7-92ac-1e925ce892d4",
+      "outputs": [],
+      "source": [
+        "howlerHitId = \"{{hit.howler.id}}\"\n",
+        "howlerAnalyticId = \"{{analytic.analytic_id}}\""
+      ]
+    },
+    {
+      "cell_type": "code",
+      "id": "586470ef-c8e6-45b1-bd17-17ccd083eef1",
+      "outputs": [],
+      "source": [
+        "from howler_client import get_client\n\n",
+        "howler = get_client(\"$CURRENT_URL\")\n",
+        "hit = howler.hit(howlerHitId)"
+      ]
+    }
+  ],
+  "nbformat": 4,
+  "nbformat_minor": 5
+}
+```
+
+```notebook tab="Processed"
+{
+  "cells": [
+    {
+      "cell_type": "code",
+      "id": "fe6f810f-2459-4ad7-92ac-1e925ce892d4",
+      "outputs": [],
+      "source": [
+        "howlerHitId = \"7dxHCat0Y2Sj48qyU7ZkVV\"\n",
+        "howlerAnalyticId = \"2SXKl6Cq4rOxWLps2SFHyB\""
+      ]
+    },
+    {
+      "cell_type": "code",
+      "id": "586470ef-c8e6-45b1-bd17-17ccd083eef1",
+      "outputs": [],
+      "source": [
+        "from howler_client import get_client\n\n",
+        "howler = get_client(\"$CURRENT_URL\")\n",
+        "hit = howler.hit(howlerHitId)"
+      ]
+    }
+  ],
+  "nbformat": 4,
+  "nbformat_minor": 5
+}
+```
+
+or with some markdown in the first cell:
+
+```notebook tab="Template"
+{
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "id": "e17cbaa8-9849-462f-9bd2-bf30943f76b3",
+      "source": [
+        "### Example Notebook"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "id": "fe6f810f-2459-4ad7-92ac-1e925ce892d4",
+      "outputs": [],
+      "source": [
+        "howlerHitId = \"{{hit.howler.id}}\"\n",
+        "howlerAnalyticId = \"{{analytic.analytic_id}}\""
+      ]
+    },
+    {
+      "cell_type": "code",
+      "id": "586470ef-c8e6-45b1-bd17-17ccd083eef1",
+      "outputs": [],
+      "source": [
+        "from howler_client import get_client\n\n",
+        "howler = get_client(\"$CURRENT_URL\")\n",
+        "hit = howler.hit(howlerHitId)"
+      ]
+    }
+  ],
+  "nbformat": 4,
+  "nbformat_minor": 5
+}
+```
+
+```notebook tab="Processed"
+{
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "id": "e17cbaa8-9849-462f-9bd2-bf30943f76b3",
+      "source": [
+        "### Example Notebook"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "id": "fe6f810f-2459-4ad7-92ac-1e925ce892d4",
+      "outputs": [],
+      "source": [
+        "howlerHitId = \"7dxHCat0Y2Sj48qyU7ZkVV\"\n",
+        "howlerAnalyticId = \"2SXKl6Cq4rOxWLps2SFHyB\""
+      ]
+    },
+    {
+      "cell_type": "code",
+      "id": "586470ef-c8e6-45b1-bd17-17ccd083eef1",
+      "outputs": [],
+      "source": [
+        "from howler_client import get_client\n\n",
+        "howler = get_client(\"$CURRENT_URL\")\n",
+        "hit = howler.hit(howlerHitId)"
+      ]
+    }
+  ],
+  "nbformat": 4,
+  "nbformat_minor": 5
+}
+```
+
+Currently, howler will only try to replace hit and analytic objects.
+
+# Requirements for the notebook integration to work
+
+- A working NBGallery setup is required.
+  - If the user can send a notebook from nbgallery to their jupyter environment, it will also work using the open in jupyhub button on Howler.
+- Just like with NBGallery, the user needs to make sure their Jupyter Environment is currently running, otherwise Howler will fail to open the notebook.
+
+Howler will append the id of the Hit/Alert when sending a notebook to Jupyter, making it easy to track for analysis. It is possible to open a notebook from within an analytic page, in this case, no hit id will be appended to the file name of the notebook and Howler won't be able to replace hit informations in the templated notebook since no hit was provided.
+
+# Adding a notebook to an analytic
+
+To add a notebook to an analytic, it's only necessary to provide the NBGallery link of the notebook. The link shoud look like this. The notebook shouldn't be private, otherwise, only the user that uploaded it will be able to use it on Howler.
+
+```
+$NBGALLERY_URL/notebooks/5-example
+```
+
+```alert
+It is advised to clear any outputs from a notebook before adding it on NBGallery to avoid leaking sensitive data.
+```

package/components/routes/help/markdown/en/retention.md

@@ -0,0 +1,15 @@
+# Retention in Howler
+
+In order to comply with organizational policies, Howler is configured to purge stale alerts after a specific amount of time. On this instance, that duration is `duration`.
+
+Howler calculates whether it is time for the removal of an alert by the `event.created` date - once this surpasses the confgured deadline, a nightly automated job will remove the alert.
+
+In order to communicate this to the user, see the example alert below:
+
+`alert`
+
+In the top right, hovering over the timestamp will outline how long users have before the alert is removed. In order to ensure compliance with policy, ensure that `event.created` matches the date the underlying data was collected, allowing howler to ensure data is purged in time.
+
+```alert
+This will soon change - there will be a dedicated field to set that will override this approach.
+```

package/components/routes/help/markdown/en/schema.md

@@ -0,0 +1,20 @@
+# Hit Schema
+
+A howler hit can contain a large number of unique fields, each with a particular definition, in order to make hits across analytics mutually intelligible. Below is a table containing all the given hit fields, as well as their type and a short description of what they are used for. While the vast majority of the fields are based on the Elastic Common Schema (see [this guide on ECS](https://www.elastic.co/guide/en/ecs/8.5/index.html) for documentation), there are also custom fields depending on the plugins enabled in Howler.
+
+## Howler Fields - Best Practices
+
+In order to allow for some consistency between various analytics, there are a number of fields with recommended (but not required) styles. These include:
+
+- `howler.analytic`: Denotes the overarching analytic that generated the hit. For example, if the name of your analytic is Bad Guy Finder, you can set this field to Bad Guy Finder. Examples of use:
+  - Bad Guy Finder (correct)
+  - BadGuyFinder (acceptable, but spaces are preferred)
+  - bad.guy.finder (incorrect, don't use periods)
+  - bad_guy_finder (incorrect, don't use underscores)
+  - in general, you can use [this regex](https://regexr.com/7ikco) to validate your proposed analytic name
+
+- `howler.detection`: Denotes the specific algorithm or portion of the analytic that generated the hit. For example, if your analytic has three ways of detecting hits that should be looked at (Impossible Travel, Incorrect Login Information, XSS Attack Detection), then the manner in which the hit you're creating was detected should be set. Examples of use:
+  - Impossible Travel (correct)
+  - ImpossibleTravel (acceptable, but spaces are preferred)
+  - impossible.travel (incorrect, don't use periods)
+  - impossible_travel (incorrect, don't use underscores)

package/components/routes/help/markdown/en/templates.md

@@ -0,0 +1,23 @@
+# Howler Templates
+
+Howler is, fundamentally, an application that allows analysts to triage hits and alerts. In order to make sure analysts can do this as efficiently as possible, we want to have the ability to present relevant data for a given alert to analysts in an easy, understandable way.
+
+To this end, Howler allows analysts and detection engineers to create **templates**, which allow various analytics and their detections to present fields and data relevant to triaging alerts generated by that analytic/detection. For example, let's consider two different alerts, by two different detections:
+
+```json
+$ALERT_1
+```
+
+```json
+$ALERT_2
+```
+
+Note that while both share some similar fields, they also differ. We want each of these alert cards to present different data - for that, we can use templates. This allows us to show both hits in the same list, but with differing fields displayed:
+
+===SPLIT===
+
+As we can see, by specifying a template for each of the detections, different data will be presented to the analyst. To do so, you can use the template creator [here]($CURRENT_URL/templates/view?type=personal).
+
+```alert
+Note that you must have ingested some hits for the given analytic/detection pair for it to show as an option in the template creation UI!
+```

package/components/routes/help/markdown/en/views.md

@@ -0,0 +1,11 @@
+# Views
+
+Views are a feature in Howler that allows users to create custom, default queries through which they can organize and triage hits. In this document, we will outline how to create and interact with views.
+
+## Using a View
+
+You can use views by navigating to the view manager under [Manage > Views](/views). Clicking on any view will open it in the search page. Here, you can also use the `search` icon to open the view in the search page. You can also edit views that belong to you, and mark them as favourites. This will show them in the `t(route.views.saved)` dropdown in the sidebar. In the top right, you can also choose your "`t(route.views.manager.default)`", that will be selected by default when opening the alerts page.
+
+## Creating Views
+
+In order to create a view, you can use the create view page, located [here](/views/create). This page allows you to modifer your view, specify a `t(hit.search.sort.fields)` and `t(hit.search.span)`, and save the view with a particular name. You can mark a view as global or personal, depending on who you want to be able to see and use the view.

package/components/routes/help/markdown/fr/actionIntroduction.md

@@ -0,0 +1,33 @@
+# Utilisation des actions dans Howler
+
+Les actions sont une fonctionnalité de Howler qui permet aux utilisateurs d'effectuer des tâches particulières sur un grand nombre de hits, en automatisant l'exécution d'une tâche sur chaque hit. Il y a actuellement `action_count` opérations supportées dans Howler :
+
+`action_list`
+
+Toutes ces opérations peuvent être combinées ensemble dans des actions uniques - c'est-à-dire que les opérations sont essentiellement les blocs de construction des actions dans Howler. Chaque opération ne peut apparaître qu'une seule fois par action, et toutes les opérations sont configurées à travers une interface unifiée. Dans ce document, nous allons parcourir les étapes nécessaires à l'exécution et à la sauvegarde d'une action.
+
+## Configuration d'une action
+
+Pour commencer à configurer votre action, décidez s'il s'agit d'une action unique ou d'une action sauvegardée que vous souhaitez exécuter plusieurs fois. Si vous voulez l'exécuter une fois, utilisez l'entrée `t(route.actions.change)` dans la barre latérale, alors qu'une action sauvegardée est mieux configurée sous `t(route.actions.manager)` en appuyant sur "`t(route.actions.create)`".
+
+La première étape de toute action sera de concevoir une requête sur laquelle vous voulez que cette action s'exécute. La boîte de recherche en haut de l'écran accepte n'importe quelle requête lucene - le même format que pour la recherche d'occurrences.
+
+`tui_phrase`
+
+Une fois que vous êtes satisfait des occurrences qui seront incluses dans cette requête, vous pouvez commencer à ajouter des opérations. Vous pouvez le faire en sélectionnant l'opération que vous voulez ajouter dans la liste déroulante:
+
+`operation_select`
+
+Une fois que vous avez sélectionné l'opération que vous souhaitez ajouter, une liste de paramètres à remplir vous est proposée. Voici un exemple d'ajout d'une étiquette.
+
+`operation_configuration`
+
+Une fois que l'opération est validée avec succès, vous pouvez répéter ce processus avec l'opération suivante. Une fois que vous avez ajouté toutes les opérations qui vous intéressent, vous pouvez exécuter ou sauvegarder l'action en utilisant le bouton situé sous la barre de recherche. Vous obtiendrez ainsi un rapport sur les étapes franchies.
+
+`report`
+
+Il peut arriver que des actions génèrent une erreur, soit lors de la validation, soit lors de l'exécution. Dans ce cas, une alerte d'erreur sera affichée, vous aidant à résoudre le problème.
+
+## Automatiser une action
+
+Pour automatiser une action, ouvrez une action sauvegardée. Les options disponibles pour l'automatisation (`automation_options`) apparaîtront sous forme de cases à cocher. En cochant la case, vous vous assurez que l'action s'exécutera ensuite - aucune autre opération n'est nécessaire.