@cccsaurora/howler-ui 2.12.0-dev.38 → 2.12.0-dev.39

package/components/routes/help/markdown/en/actionIntroduction.md

@@ -0,0 +1,33 @@
+# Using Actions in Howler
+
+Actions are a feature in Howler that allow users to perform particular tasks on a large number of hits, through automating the execution of a task on each hit. There are currently `action_count` operations supported in howler:
+
+`action_list`
+
+All of these operations can be combined together into unique actions - that is, operations are essentially the building blocks of actions in Howler. Each operation can only appear once per action, and all operations are configured through one unified UI. In this document, we will walk through the steps necessary to run and save an action.
+
+## Configuring an Action
+
+In order to begin configuring your action, decide if this will be a one-off action or a saved action you want to run several times. If you want to run it once, use the `t(route.actions.change)` entry in the sidebar, while a saved action is best configured under the `t(route.actions.manager)` by pressing "`t(route.actions.create)`".
+
+The first step of any action will be to design a query on which you want this action to run. The search box at the top of the accepts any lucene query - the same format as searching for hits.
+
+`tui_phrase`
+
+Once you are satisfied with the hits that will be included in this query, you can begin adding operations. You can do so by selecting the operation you want to add from the dropdown:
+
+`operation_select`
+
+Once you've selected the operation you want to add, it will prompt you for a list of parameters you will need to fill out. Below is an example for adding a label.
+
+`operation_configuration`
+
+Once the operation validates successfully, you can repeat this process with the next operation. Once you've added all the operations you're interested in, you can execute or save the action using the button below the search bar. This will generate a report of the steps taken.
+
+`report`
+
+Occaisionally, actions will result in an error, either on validation or on execution. In these cases, an error alert will be shown, helping you solve the issue.
+
+## Automating an Action
+
+In order to automate an action, open any saved action. The options available to you for automation (`automation_options`) will show up as checkboxes. Checking the box will ensure this action will run then - no further work required.

package/components/routes/help/markdown/en/authentication.md

@@ -0,0 +1,261 @@
+<!-- docs/ingestion/authentication.md -->
+
+# Howler Authentication
+
+Howler's API supports a number of authentication approaches when accessing the API. This document will outline the
+different approaches, and explain how to use each of them.
+
+## Types of Authentication
+
+There are four methods of authentication one can use to connect to the howler API:
+
+1. Username and Password
+2. Username and API Key
+3. Username and app token (after login)
+4. OAuth Access Token
+
+We will outline the use case for each authentication type next.
+
+### Username/Password Authentication
+
+Username and password authentication is the simplest and least secure method of authentication. Unlikely to be enabled
+in a production environment, it allows users to easily connect to the Howler API and make changes as the given user,
+without having to worry about creating API keys or using an OAuth provider like Keycloak or Azure. Users can connect
+using username and password authentication in one of two ways:
+
+#### Direct Call to the Requisite Endpoint (Password)
+
+This is far and away the simplest method. Simply add a basic Authorization header to the HTTP request you want to
+make, and everything is taken care of:
+
+```bash
+echo -n "user:user" | base64 -w0
+# -> dXNlcjp1c2Vy
+```
+
+```http
+GET $CURRENT_URL/api/v1/user/whoami
+Authorization: Basic dXNlcjp1c2Vy
+```
+
+#### Exchanging for an App Token (Password)
+
+This is a slightly more complex approach, but carries the benefit of not exposing the username and password on every
+request. You can leverage the `v1/auth/login` endpoint to exchange your username and password for an app token. The
+token works similarly to an OAuth access token - you provide it with each subsequent request, and it authenticates you
+until the token expires.
+
+```http
+POST $CURRENT_URL/api/v1/auth/login/
+Content-Type: application/json
+
+{
+  "user": "user",
+  "password": "user"
+}
+```
+
+Will return something like:
+
+```json
+{
+  "api_error_message": "",
+  "api_response": {
+    "app_token": "user:5791a142067745c3af51d6596da7da8f86357a9fa92ad78d1ce118ea7d89d34e",
+    "provider": null,
+    "refresh_token": null
+  },
+  "api_server_version": "0.0.0.dev0",
+  "api_status_code": 200
+}
+```
+
+Using this token in another API call:
+
+```http
+GET $CURRENT_URL/api/v1/user/whoami
+Authorization: Bearer user:5791a142067745c3af51d6596da7da8f86357a9fa92ad78d1ce118ea7d89d34e
+```
+
+### Username/API Key Authentication
+
+The username and API Key authentication works largely the same as username/password from the perspective of the client.
+On the server side, however, API keys have several key advantages. Firstly, they can easily be revoked by the user.
+Secondly, their privileges can be limited, allowing only a subset of permissions.
+
+There are two methods of authentication, mirroring the username and password:
+
+#### Direct Call to the Requisite Endpoint (API Key)
+
+Simply add a basic Authorization header containing the username and API key to the HTTP request you want to make, and
+everything is taken care of:
+
+```bash
+# note the format is <username>:<apikeyname>:<secret>
+echo -n "user:devkey:user" | base64 -w0
+# -> dXNlcjpkZXZrZXk6dXNlcg==
+```
+
+```http
+GET $CURRENT_URL/api/v1/user/whoami
+Authorization: Basic dXNlcjpkZXZrZXk6dXNlcg==
+```
+
+#### Exchanging for an App Token (API Key)
+
+You can also leverage the `v1/auth/login` endpoint to exchange your username and API key for an app token. The
+token works similarly to an OAuth access token - you provide it with each subsequent request, and it authenticates you
+until the token expires.
+
+```http
+POST $CURRENT_URL/api/v1/auth/login/
+Content-Type: application/json
+
+{
+  "user": "user",
+  "apikey": "devkey:user"
+}
+```
+
+Will return something like:
+
+```json
+{
+  "api_error_message": "",
+  "api_response": {
+    "app_token": "user:f220eb76ff8404abfece8c0c2f3368c7d89618c776bedcd3a506843dc4a952e4",
+    "provider": null,
+    "refresh_token": null
+  },
+  "api_server_version": "0.0.0.dev0",
+  "api_status_code": 200
+}
+```
+
+Using this token in another API call:
+
+```http
+GET $CURRENT_URL/api/v1/user/whoami
+Authorization: Bearer user:f220eb76ff8404abfece8c0c2f3368c7d89618c776bedcd3a506843dc4a952e4
+```
+
+Note that when logging in using an api key, the permissions of the api key continue to apply. So if you try and access
+an endpoint that requires the "write" permission with an API key that only has read permission, this will cause a 403
+Forbidden error.
+
+### Access Token Authentication
+
+In addition to the authentication methods provided internally by Howler, you can also authenticate with an external
+OAuth provider like Azure or Keycloak. To do this, you must obtain an access token from one of these providers. In
+order to get an access token, there are generally two flows - On-Behalf-Of, or the user login flow (see [here](https://www.rfc-editor.org/rfc/rfc6749#section-4.1) for a rundown
+of that). In the case of Howler, the `v1/auth/login` endpoint is used for step D & E in the linked user login flow, and
+it returns the following:
+
+```json
+{
+  "api_response": {
+    "app_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ1c2VyIn0",
+    "provider": "keycloak",
+    "refresh_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJyZWZyZXNoIn0"
+  }
+}
+```
+
+In this case, the `app_token` is the JSON Web Token the application uses as an access token. Similarly, the
+`refresh_token` is another JWT that is used to refresh the access token if needed. Finally, the `provider` field
+outlines which provider this access token corresponds to.
+
+Once you have that access token, you can simply pass it along in the Authorization header:
+
+```http
+GET $CURRENT_URL/api/v1/user/whoami
+Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ1c2VyIn0
+```
+
+Howler will automatically detect this is a JWT, and treat it as such. If the token is valid, the user will be
+authenticated.
+
+In order to refresh an expired access token, you can use the following API call:
+
+```http
+POST $CURRENT_URL/api/v1/auth/login/
+Content-Type: application/json
+
+{
+  "oauth_provider": "keycloak",
+  "refresh_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJyZWZyZXNoIn0"
+}
+```
+
+And you'll get something like this back:
+
+```json
+{
+  "api_response": {
+    "app_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ1c2VyMiJ9",
+    "provider": "keycloak",
+    "refresh_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJyZWZyZXNodG9rZW4yIn0"
+  }
+}
+```
+
+## Impersonation
+
+The final important bit of the authentication flow is the impersonation functionality. In order for a service account
+or other user to impersonate you (i.e. to create alerts on your behalf), Howler allows users to provide a second API key
+in order to authenticate as another user.
+
+Before you use another user's API key to authenticate as them, it's important that the API key you're using has been
+marked as valid for impersonation - check with the user that provided it. If it has, you can use it by, formating your
+request as follows:
+
+```bash
+# Your credentials
+echo -n "admin:devkey:admin" | base64 -w0
+# -> YWRtaW46ZGV2a2V5OmFkbWlu
+
+# Their credentials
+echo -n "user:impersonate_admin:user" | base64 -w0
+# -> dXNlcjppbXBlcnNvbmF0ZV9hZG1pbjp1c2Vy
+```
+
+```alert
+Any form of authentication can be used by the impersonator, but validated API keys are the only permitted
+authentication method for the person you're impersonating.
+```
+
+```http
+GET $CURRENT_URL/api/v1/user/whoami
+Authorization: Basic YWRtaW46ZGV2a2V5OmFkbWlu
+X-Impersonating: Basic dXNlcjppbXBlcnNvbmF0ZV9hZG1pbjp1c2Vy
+```
+
+If set up correctly, the result will look like:
+
+```json
+{
+  "api_error_message": "",
+  "api_response": {
+    "avatar": null,
+    "classification": "TLP:W",
+    "email": "user@howler.cyber.gc.ca",
+    "groups": ["USERS"],
+    "is_active": true,
+    "is_admin": false,
+    "name": "User",
+    "roles": ["user"],
+    "username": "user"
+  },
+  "api_server_version": "0.0.0.dev0",
+  "api_status_code": 200
+}
+```
+
+Note that the server returned this as if you were `user`, NOT `admin`. It is clearly marked in the logs, however, that you are making the request:
+
+```log
+22/12/05 14:15:02 INFO howler.api.security | Authenticating user for path /api/v1/user/whoami/
+22/12/05 14:15:03 WARNING howler.api.security | admin is impersonating user
+22/12/05 14:15:03 INFO howler.api.security | Logged in as user from 127.0.0.1
+22/12/05 14:15:03 INFO howler.api | GET /api/v1/user/whoami/ - 200
+```

package/components/routes/help/markdown/en/bundles.md

@@ -0,0 +1,70 @@
+<!-- docs/ingestion/bundles.md -->
+
+# Howler Hit Bundles
+
+Hit bundles can be used to easily package together a large number of similar alerts, allowing analysts to easily triage them as a single incident. For example, consider a single computer that repeatedly makes a network call to `baddomain.ru` - while an alert may be generated for every instance of this computer hitting that domain, it makes sense for analysts to treat all these alerts as a single case.
+
+## Creating bundles through the Howler Client
+
+There are a couple of ways to create a bundle through the howler client:
+
+```python
+from howler_client import get_client
+
+howler = get_client("https://howler.dev.analysis.cyber.gc.ca")
+
+"""Creating a howler bundle and the hits at the same time"""
+howler.bundle.create(
+    # First argument is the bundle hit
+    {
+        "howler.analytic": "example-test",
+        "howler.score": 0
+    },
+    # Second argument is a hit or list of hits to include in the bundle
+    [
+        {
+            "howler.analytic": "example-test",
+            "howler.score": 0
+        },
+        {
+            "howler.analytic": "example-test",
+            "howler.score": 0
+        }
+    ]
+)
+
+"""Creating a howler bundle from existing hits"""
+howler.bundle.create(
+    {
+        "howler.analytic": "example-test",
+        "howler.score": 0,
+        "howler.hits": ["YcUsL8QsjmwwIdstieROk", "6s7MztwuSvz6tM0PgGJhvz"]
+    },
+    # Note: In future releases, you won't need to include this argument
+    []
+)
+
+
+"""Creating from a map"""
+bundle_hit = {
+    "score": 0,
+    "bundle": True
+}
+
+map = {
+    "score": ["howler.score"],
+    "bundle": ["howler.is_bundle"]
+}
+
+howler.bundle.create_from_map("example-test", bundle_hit, map, [{"score": 0}])
+```
+
+## Viewing bundles on the Howler UI
+
+In order to view created bundles on the Howler UI, you can use the query `howler.is_bundle:true`. This will provide a list of created bundles you can look through.
+
+Clicking on a bundle will open up a slightly different search UI to normal. In this case, we automatically filter the search results to include only hits that are included in the bundle. To make this obvious, the header representing the bundle will appear above the search bar.
+
+You can continue to filter through hits using the same queries as usual, and view them as usual. When triaging a bundle, assessing it will apply this assessment to all hits in the bundle, **except those that have already been triaged**. That is, if the bundle is open, all open hits will be assessed when you assess it.
+
+Bundles also have a **Summary** tab not available for regular hits. This summary tab will aid you in aggregating data about all the hits in the bundle. Simply open the tab and click "Create Summary". Note that this may take some time, as a large number of queries are being run to aggregate the data.

package/components/routes/help/markdown/en/client.md

@@ -0,0 +1,213 @@
+# Howler Client Documentation
+
+This documentation will outline how to interact with the howler API using the howler client in both Java and python development environments. We will outline the basic process of creating a new hit in each environment as well as searching howler for hits matching your query.
+
+## Getting started
+
+### Installation
+
+In order to use the howler client, you need to list it as a dependency in your project.
+
+#### **Python**
+
+Simply install through pip:
+
+```bash
+pip install howler-client
+```
+
+You can also add it to your requirements.txt, or whatever dependency management system you use.
+
+### Authentication
+
+As outlined in the [Authentication Documentation](/help/auth), there's a number of ways users can choose to authenticate. In order to interface with the howler client, however, the suggested flow is to use an API key. So before we start, let's generate a key.
+
+1. Open the Howler UI you'd like to interface with.
+2. Log in, then click your profile in the top right.
+3. Under user menu, click Settings.
+4. Under User Security, press the (+) icon on the API Keys row.
+5. Name your key, and give it the requisite permissions.
+6. Press Create, and copy the supplied string somewhere safe. **You will not see this string again.**
+
+This API Key will be supplied to your code later on.
+
+## Python Client
+
+In order to connect with howler using the python client, there is a fairly simple process to follow:
+
+```python
+from howler_client import get_client
+
+USERNAME = 'user' # Obtain this from the user settings page of the Howler UI
+APIKEY = 'apikey_name:apikey_data'
+
+apikey = (USERNAME, APIKEY)
+
+howler = get_client("$CURRENT_URL", apikey=apikey)
+```
+
+```alert
+You can skip generating an API Key and providing it if you're executing this code within HOGWARTS (i.e., on jupyterhub or airflow). OBO will handle authentication for you!
+```
+
+That's it! You can now use the `howler` object to interact with the server. So what does that actually look like?
+
+### Creating hits in Python
+
+For the python client, you can create hits using either the `howler.hit.create` or `howler.hit.create_from_map` functions.
+
+#### `create`
+
+This function takes in a single argument - either a single hit, or a list of them, conforming to the [Howler Schema](/help/hit?tab=schema). Here is a simple example:
+
+```python
+# Some bogus data in the Howler Schema format
+example_hit = {
+  "howler": {
+    "analytic": "example",
+    "score": 10.0
+  },
+  "event": {
+    "reason": "Example hit"
+  }
+}
+
+howler.hit.create(example_hit)
+```
+
+You can also ingest data in a flat format:
+
+```python
+example_hit = {
+  "howler.analytic": "example",
+  "howler.score": 10.0,
+  "event.reason": "Example hit"
+}
+
+howler.hit.create(example_hit)
+```
+
+#### `create_from_map`
+
+This function takes in three arguments:
+
+- `tool name`: The name of the analytic creating the hit
+- `map`: A mapping between the raw data you have and the howler schema
+  - The format is a dictionary where the keys are the flattened path of the raw data, and the values are a list of flattened paths for Howler's fields where the data will be copied into.
+- `documents`: The raw data you want to add to howler
+
+Here is a simple example:
+
+```python
+# The mapping from our data to howler's schema
+hwl_map = {
+  "file.sha256": ["file.hash.sha256", "howler.hash"],
+  "file.name": ["file.name"],
+  "src_ip": ["source.ip", "related.ip"],
+  "dest_ip": ["destination.ip", "related.ip"],
+  "time.created": ["event.start"],
+}
+
+# Some bogus data in a custom format we want to add to howler
+example_hit = {
+  "src_ip": "0.0.0.0",
+  "dest_ip": "8.8.8.8",
+  "file": {
+    "name": "hello.exe",
+    "sha256": sha256(str("hello.exe").encode()).hexdigest()
+  },
+  "time": {
+    "created": datetime.now().isoformat()
+  },
+}
+
+# Note that the third argument is of type list!
+howler.hit.create_from_map("example_ingestor", hwl_map, [example_hit])
+```
+
+### Querying Hits
+
+Querying hits using the howler python client is done using the `howler.search.hit` function. It has a number of required and optional arguments:
+
+- Required:
+  - `query`: lucene query (string)
+- Optional:
+  - `filters`: Additional lucene queries used to filter the data (list of strings)
+  - `fl`: List of fields to return (comma separated string of fields)
+  - `offset`: Offset at which the query items should start (integer)
+  - `rows`: Number of records to return (integer)
+  - `sort`: Field used for sorting with direction (string: ex. 'id desc')
+  - `timeout`: Max amount of milliseconds the query will run (integer)
+  - `use_archive`: Also query the archive
+  - `track_total_hits`: Number of hits to track (default: 10k)
+
+Here are some example queries:
+
+```python
+# Search for all hits created by assemblyline, show the first 50, and return only their ids
+howler.search.hit("howler.analytic:assemblyline", fl="howler.id", rows=50)
+
+# Search for all resolved hits created in the last five days, returning their id and the analytic that created them. Show only ten, offset by 40
+howler.search.hit("howler.status:resolved", filters=['event.created:[now-5d TO now]'] fl="howler.id,howler.analytic", rows=10, offset=40)
+
+# Search for all hits, timeout if the query takes more than 100ms
+howler.search.hit("howler.id:*", track_total_hits=100000000, timeout=100, use_archive=True)
+```
+
+### Updating Hits
+
+In order to update hits, there are a number of supported functions:
+
+- `howler.hit.update(...)`
+- `howler.hit.update_by_query(...)`
+- `howler.hit.overwrite(...)`
+
+#### `update()`
+
+If you want to update a hit in a transactional way, you can use the following code:
+
+```python
+hit_to_update = client.search.hit("howler.id:*", rows=1, sort="event.created desc")["items"][0]
+
+result = client.hit.update(hit_to_update["howler"]["id"], [(UPDATE_SET, "howler.score", hit_to_update["howler"]["score"] + 100)])
+```
+
+The following operations can be run to update a hit.
+
+**List Operations:**
+
+- `UPDATE_APPEND`: Used to append a value to a given list
+- `UPDATE_APPEND_IF_MISSING`: Used to append a value to a given list if the value isn't already in the list
+- `UPDATE_REMOVE`: Will remove a given value from a list
+
+**Numeric Operations:**
+
+- `UPDATE_DEC`: Decrement a numeric value by the specified amount
+- `UPDATE_INC`: Increment a numeric value by the specified amount
+- `UPDATE_MAX`: Will set a numeric value to the maximum of the existing value and the specified value
+- `UPDATE_MIN`: Will set a numeric value to the minimum of the existing value and the specified value
+
+**Multipurpose Operations:**
+
+- `UPDATE_SET`: Set a field's value to the given value
+- `UPDATE_DELETE`: Will delete a given field's value
+
+#### `update_by_query()`
+
+This function allows you to update a large number of hits by a query:
+
+```python
+client.hit.update_by_query(f'howler.analytic:"Example Alert"', [(UPDATE_INC, "howler.score", 100)])
+```
+
+The same operations as in `update()` can be used.
+
+### `overwrite()`
+
+This function allows you to directly overwrite a hit with a partial hit object. This is the most easy to use, but loses some of the validation and additional processing of the update functions.
+
+```python
+hit_to_update = client.search.hit("howler.id:*", rows=1, sort="event.created desc")["items"][0]
+
+result = client.hit.overwrite(hit_to_update["howler"]["id"], {"source.ip": "127.0.0.1", "destination.ip": "8.8.8.8"})
+```

package/components/routes/help/markdown/en/links.md

@@ -0,0 +1,37 @@
+# Hit Links
+
+In order to facilitate the addition of additional tools one can use to triage a hit, Howler allows users to specify a set of links, along with a title and icon to show. This documentation will walk you through how to use these links.
+
+## Specification
+
+In order to add links, you can use the `howler.links` field. This field takes in a list of objects with three keys:
+
+```python
+hit = {
+  "howler.links": [
+    {
+      "title": "Link Title with Internal Image",
+      "href": "https://example.com",
+      # Note that this specifies another application, not an image link
+      "icon": "superset",
+    },
+    {
+      "title": "Link Title with External Image",
+      "href": "https://www.britannica.com/animal/goose-bird",
+      # Note that this specifies an image link. We don't provide hosting, so you'll need to host it somewhere else!
+      "icon": "https://cdn.britannica.com/76/76076-050-39DDCBA1/goose-Canada-North-America.jpg",
+    },
+  ]
+}
+```
+
+Note the icon can either be:
+
+1. A name identifying a linked application (from the app switcher), to use its icon
+2. An external URL
+
+If you'd like to use a linked app, the following values are currently supported:
+
+$APP_LIST
+
+Using any of these values will automatically use the corresponding icon. No need to host your own!