mihari 5.6.2 → 5.7.1

data/docs/emitters/index.md

@@ -1,36 +0,0 @@
-# Emitters
-
-- [Database](database.md)
-- [TheHive](hive.md)
-- [MISP](misp.md)
-- [Slack](slack.md)
-- [Webhook](webhook.md)
-
-## Options
-
-All the emitters can have optional `options`.
-
-```yaml
-emitter: ...
-options:
-  timeout: ...
-  retry_times: ...
-  retry_interval: ...
-  retry_exponential_backoff: ...
-```
-
-### Timeout
-
-`timeout` (`integer`) is an HTTP timeout in seconds. Optional.
-
-### Retry Times
-
-`retry_times` (`integer`) is a number of times of retry when something goes wrong. Optional. Defaults to 3.
-
-### Retry Interval
-
-`retry_interval` (`integer`) is an interval in seconds between retries. Optional. Defaults to 5.
-
-### Retry Exponential Backoff
-
-`retry_exponential_backoff` (`bool`) controls whether to do exponential backoff. Optional. Defaults to `true`.

data/docs/emitters/misp.md

@@ -1,21 +0,0 @@
-# MISP
-
-- [https://www.misp-project.org/](https://www.misp-project.org/)
-
-This emitter creates an event on MISP based on an alert. MISP v2 is supported.
-
-```yaml
-emitter: misp
-url: ...
-api_key: ...
-```
-
-## Components
-
-### URL
-
-`url` (`string`) is a MISP URL. Optional. Defaults to `ENV[MISP_URL]`.
-
-### API Key
-
-`api_key` (`string`) is an API key. Optional. Defaults to `ENV[”MISP_API_KEY”]`.

data/docs/emitters/slack.md

@@ -1,21 +0,0 @@
-# Slack
-
-- [https://slack.com/](https://slack.com/intl/ja-jp/)
-
-This emitter post a message to Slack via incoming webhook.
-
-```yaml
-emitter: slack
-webhook_url: ...
-channel: ...
-```
-
-## Components
-
-### Webhook URL
-
-`url` (`string`) is a Slack's incoming webhook URL. Optional. Defaults to `ENV[SLACK_WEBHOOK_URL]`.
-
-### API Key
-
-`channel` (`string`) is a Slack channel to sent a message. Optional. Defaults to `ENV[SLACK_CHANNEL]` or `#general`.

data/docs/emitters/webhook.md

@@ -1,63 +0,0 @@
-# Webhook
-
-This emitter creates an HTTP request payload based on the specified conditions.
-
-```yaml
-emitter: webhook
-url: ...
-method: ...
-headers: ...
-template: ...
-```
-
-## Components
-
-### URL
-
-`url` (`string`) is a webhook URL.
-
-### Method
-
-`method` (`string`)is an HTTP method. Optional. Defaults to `POST`.
-
-### Headers
-
-`headers` (`hash`) are HTTP headers. Optional.
-
-### Template
-
-`template` (`string`) is an [ERB](https://github.com/ruby/erb) template to customize the payload to sent. A template should generate a valid JSON.
-
-You can use the following parameters inside an ERB template.
-
-- `rule`: a rule
-- `artifacts`: a list of artifacts
-
-## Examples
-
-### ThreatFox
-
-```yaml
-- emitter: webhook
-  url: https://threatfox-api.abuse.ch/api/v1/
-  headers:
-    api-key: YOUR_API_KEY
-  template: threatfox.erb
-```
-
-```ruby
-{
-	"query": "submit_ioc",
-	"threat_type": "payload_delivery",
-	"ioc_type": "ip:port",
-	"malware": "foobar",
-	"confidence_level": 100,
-	"anonymous": 0,
-	"iocs": [
-		<% @artifacts.select { |artifact| artifact.data_type == "ip" }.each_with_index do |artifact, idx| %>
-			"<%= artifact.data %>:80"
-			<%= ',' if idx < (@artifacts.length - 1) %>
-		<% end %>
-	]
-}
-```

data/docs/enrichers/google_public_dns.md

@@ -1,19 +0,0 @@
----
-tags:
-  - Enrichment:DNS_Record
----
-
-# Google Public DNS
-
-- [https://developers.google.com/speed/public-dns](https://developers.google.com/speed/public-dns)
-
-This enricher uses Google Public DNS to enrich an URL and domain artifact.
-
-```yaml
-enricher: google_public_dns
-```
-
-## Supported Artifacts
-
-- URL
-- Domain

data/docs/enrichers/index.md

@@ -1,35 +0,0 @@
-# Enrichers
-
-- [Google Public DNS](google_public_dns.md)
-- [IPInfo](ipinfo.md)
-- [Shodan](shodan.md)
-- [Whois](whois.md)
-
-## Options
-
-All the emitters can have optional `options`.
-
-```yaml
-enricher: ...
-options:
-  timeout: ...
-  retry_times: ...
-  retry_interval: ...
-  retry_exponential_backoff: ...
-```
-
-### Timeout
-
-`timeout` (`integer`) is an HTTP timeout in seconds. Optional.
-
-### Retry Times
-
-`retry_times` (`integer`) is a number of times of retry when something goes wrong. Optional. Defaults to 3.
-
-### Retry Interval
-
-`retry_interval` (`integer`) is an interval in seconds between retries. Optional. Defaults to 5.
-
-### Retry Exponential Backoff
-
-`retry_exponential_backoff` (`bool`) controls whether to do exponential backoff. Optional. Defaults to `true`.

data/docs/enrichers/ipinfo.md

@@ -1,26 +0,0 @@
----
-tags:
-  - Enrichment:Autonomous_System
-  - Enrichment:Geolocation
----
-
-# ipinfo.io
-
-- [https://ipinfo.io/](https://ipinfo.io/)
-
-This enricher uses ipinfo.io API to enrich an IP artifact.
-
-```yaml
-enricher: ipinfo
-api_key: ...
-```
-
-## Components
-
-### API Key
-
-`api_key` (`string`) is an API key. Optional. Defaults to `ENV[”IPINFO_API_KEY”]`.
-
-## Supported Artifacts
-
-- IP address

data/docs/enrichers/shodan.md

@@ -1,22 +0,0 @@
----
-tags:
-  - Enrichment:Port
-  - Enrichment:CPE
-  - Enrichment:DNS_Record
----
-
-# Shodan (The InternetDB API)
-
-- [https://www.shodan.io/](https://www.shodan.io/dashboard)
-
-This enricher uses Shodan InternetDB API to enrich an artifact.
-
-[https://internetdb.shodan.io/](https://internetdb.shodan.io/)
-
-```yaml
-enricher: shodan
-```
-
-## Supported Artifacts
-
-- IP address

data/docs/enrichers/whois.md

@@ -1,17 +0,0 @@
----
-tags:
-  - Enrichment:Whois
----
-
-# Whois
-
-This enricher uses “whois” command to enrich an artifact.
-
-```yaml
-enricher: whois
-```
-
-## Supported Artifacts
-
-- URL
-- Domain

data/docs/github_actions.md

@@ -1,43 +0,0 @@
-# GitHub Actions
-
-GitHub Actions is a good way to run Mihari searches continuously.
-
-The following is an example of a GitHub Actions workflow to run Mihari.
-
-```yaml
-name: Mihari searches
-
-on:
-  workflow_dispatch:
-
-jobs:
-  build:
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v4
-      - name: Install dependencies
-        run: sudo apt-get -yqq install sqlite3 libsqlite3-dev
-      - name: Set up Ruby 3.2
-        uses: ruby/setup-ruby@v1
-        with:
-          ruby-version: "3.2"
-          bundler-cache: true
-      - name: Run Mihari
-        run: |
-          mihari search /path/to/rule.yml
-```
-
-!!! tip
-
-    You need to install `libpq-dev` for PostgreSQL, `libmysqlclient-dev` for MySQL.
-
-This example assumes that you have `Gemfile` in your repository.
-
-```ruby
-source "https://rubygems.org"
-
-gem "pg"     # if you use PostgresSQL
-gem "mysql2" # if you use MySQL
-
-gem "mihari"
-```

data/docs/index.md

@@ -1,11 +0,0 @@
-# Mihari
-
-A query aggregator for OSINT based threat hunting.
-
-Mihari can aggregate multiple searches across multiple services in a single rule & persist findings in a database.
-
-- [Requirements](./requirements.md)
-- [Installation](./installation.md)
-- [How to Write a Rule](./rule.md)
-- [Usage](./usage.md)
-- [Configuration](./configuration.md)

data/docs/installation.md

@@ -1,31 +0,0 @@
-# Installation
-
-## Ruby Gem
-
-Mihari is packaged as a Ruby Gem. Thus you can install it via `gem` command.
-
-```bash
-gem install mihari
-```
-
-Mihari uses SQLite3 as a primary database by default. Thus a gem for SQLite (`sqlite3`) is installed by default.
-
-If you want to use MySQL or PostgreSQL instead of SQLite3, please install a gem for that by yourself.
-
-**MySQL**
-
-```bash
-gem install mysql2
-```
-
-**PostgreSQL**
-
-```bash
-gem install pg
-```
-
-# Docker
-
-You can built the Docker image by yourself.
-
-`Dockerfile` is available at [https://github.com/ninoseki/mihari/tree/master/docker](https://github.com/ninoseki/mihari/tree/master/docker).

data/docs/requirements.md

@@ -1,13 +0,0 @@
-# Requirements
-
-## Runtime
-
-Ruby 2.7+ / 3.0+ (tested with 2.7, 3.0, 3.1 and 3.2)
-
-## Database
-
-- SQLite3
-- PostgreSQL
-- MySQL
-
-You need to have a database to persistent the data. See [Database](./emitters/database.md) for details.

data/docs/rule.md

@@ -1,168 +0,0 @@
-# How to Write a Rule
-
-Mihari has [Sigma](https://github.com/SigmaHQ/sigma) like format to describe a set of search queries to express a rule.
-
-Mihari has three main components to compose a rule.
-
-![](https://imgur.com/BBT99BG.png)
-
-- Analyzers/Queries: a list of queries (analyzers) that builds a list of artifacts
-- Enrichers: a list of enrichers that enriches a list of artifacts
-- Emitters: a list of emitters that emits a list of artifacts as an alert
-
-An artifact has five types:
-
-- IP address (`ip`)
-- Domain (`domain`)
-- URL (`url`)
-- Mail (`mail`)
-- Hash (`hash`)
-
-An alert can have multiple artifacts bundled by a rule.
-
-!!! note
-
-    A rule is assumed to be executed multiple times continuously. An alert generated by a rule will only have new findings at that time.
-
-Let's break down the following example:
-
-```yaml
-id: c7f6968e-dbe1-4612-b0bb-8407a4fe05df
-title: Example
-description: Mihari rule example
-created_on: "2023-01-01"
-updated_on: "2023-01-02"
-author: ninoseki
-references:
-  - https://github.com/ninoseki/mihari
-related:
-  - 6254bb74-5e5d-42ad-bc1e-231da0293b0f
-tags:
-  - foo
-  - bar
-queries:
-  - analyzer: shodan
-    query: ip:1.1.1.1
-  - analyzer: censys
-    query: ip:8.8.8.8
-enrichers:
-  - enricher: whois
-  - enricher: ipinfo
-  - enricher: shodan
-  - enricher: google_public_dns
-emitters:
-  - emitter: database
-  - emitter: misp
-  - emitter: slack
-  - emitter: thehive
-data_types:
-  - hash
-  - ip
-  - domain
-  - url
-  - mail
-falsepositives: []
-```
-
-## Components
-
-### ID
-
-`id` (`string`) is an unique ID of a rule. UUID v4 is recommended.
-
-### Title
-
-`title` (`string`) is a title of a rule.
-
-### Description
-
-`description` (`string`) is a short description of a rule.
-
-### Created/Updated On
-
-`created_on` (`date`) is a date of a rule creation. Optional.
-Also a rule can have `updated_on` that is a date of a rule modification. Optional.
-
-### Tags
-
-`tags` (`array[:string]`) is a list of tags of a rule.
-
-### Author
-
-`author` (`string`) is an author of a rule. Optional.
-
-### References
-
-`references` (`array[:string]`) is a list of a references of a rule. Optional.
-
-### Related
-
-`related` (`array[:string]`) is a list of related rule IDs. Optional.
-
-### Queries
-
-`queries` is a list of queries/analyzers.
-See [Analyzers](./analyzers/index.md) to know details of each analyzer.
-
-### Enrichers
-
-`enrichers` is a list of enrichers.
-See [Enrichers](./enrichers/index.md) to know details of each enricher.
-
-Defaults to:
-
-- `google_public_dns`
-- `ipinfo`
-- `shodan`
-- `whois`
-
-### Emitters
-
-`emitters` is a list of emitters.
-See [Emitters](./emitters/index.md) to know details of each emitter.
-
-Defaults to:
-
-- `database`
-
-### Data Types
-
-`data_types` (`array[:string]`) is a list of data (artifact) types to allow by a rule. Types not defined in here will be automatically rejected.
-
-Defaults to:
-
-- `ip`
-- `domain`
-- `url`
-- `mail`
-- `hash`
-
-### False positives
-
-`falsepositives` (`array[:string]`) is a list of false positive values. A string or regexp can be used in here.
-
-### Artifact TTL
-
-`artifact_ttl` (`integer` / alias: `artifact_lifetime`) is an integer value of artifact TTL (Time-To-Live) in seconds.
-
-Mihari rejects a same artifact in a same rule in general.
-
-But you may want to get a same artifact after a certain period of time. `artifact_ttl` is for that. If a rule finds a same artifact after `artifact_ttl` seconds have been passed, that artifact will be included in an alert.
-
-## How to Run a Rule
-
-Once you finish writing a rule, you can run the rule by `mihari` CLI.
-
-!!! note
-
-    You have to initialize the database by `mihari db migrate` if you haven't already done.
-
-```bash
-mihari search /path/to/rule.yml
-```
-
-The command outputs an alert to the standard output. Also you can confirm it with a built-in web app.
-
-```bash
-mihari web
-```

data/docs/tags.md

@@ -1,3 +0,0 @@
-# Tags
-
-[TAGS]

data/docs/usage.md

@@ -1,103 +0,0 @@
-# Usage
-
-```bash
-$ mihari
-Commands:
-  mihari --version, -v        # Print the version
-  mihari alert                # Sub commands for alert
-  mihari db                   # Sub commands for DB
-  mihari help [COMMAND]       # Describe available commands or one specific command
-  mihari rule                 # Sub commands for rule
-  mihari search [PATH_OR_ID]  # Search by a rule (Outputs null if there is no new finding)
-  mihari web                  # Launch the web app
-
-Options:
-  -d, [--debug], [--no-debug]  # Sets up debug mode
-```
-
-## `mihari db`
-
-This sub command is for initializing/migrating database.
-
-```bash
-mihari db migrate
-```
-
-See [Database](./emitters/database.md) for detailed database configuration.
-
-## `mihari rule`
-
-This sub command is for validating/initializing a rule.
-
-```bash
-mihari rule init /path/to/rule.yml
-mihari rule validate /path/to/rule.yml
-```
-
-## `mihari search`
-
-This is a command for running a rule.
-
-```bash
-mihari search /path/to/rule.yml
-```
-
-Mihari asks whether really you want to update a rule if there is a diff by default.
-
-```bash
-$ mihari search /path/to/rule.yml
-There is a diff in the rule. Are you sure you want to overwrite the rule? (y/n)
-```
-
-It can be suppressed by providing `-f`.
-
-```bash
-mihari search -f /path/to/rule.yml
-```
-
-## `mihari add`
-
-You may want to add an alert manually. You can do that by this command.
-
-```bash
-mihari alert /path/to/alert.yml
-```
-
-## `mihari web`
-
-This command is for launching the built-in web app.
-
-```bash
-mihari web
-```
-
-It stars the app with `localhost:9292`. You can configure it by providing options.
-
-```bash
-$ mihari help web
-Usage:
-  mihari web
-
-Options:
-  [--port=N]                                         # Hostname to listen on
-                                                     # Default: 9292
-  [--host=HOST]                                      # Port to listen on
-                                                     # Default: localhost
-  [--threads=THREADS]                                # min:max threads to use
-                                                     # Default: 0:5
-  [--verbose], [--no-verbose]                        # Report each request
-                                                     # Default: true
-  [--worker-timeout=N]                               # Worker timeout value (in seconds)
-                                                     # Default: 60
-  [--hide-config-values], [--no-hide-config-values]  # Whether to hide config values or not
-                                                     # Default: true
-  [--open], [--no-open]                              # Whether to open the app in browser or not
-                                                     # Default: true
-  [--rack-env=RACK_ENV]                              # Rack environment
-                                                     # Default: production
-```
-
-!!! tip
-
-    The built-in web app offers API to interact with Mihari.
-    The API docs are available on `/redoc-static.html`

data/frontend/.eslintrc.cjs

@@ -1,22 +0,0 @@
-/* eslint-env node */
-require('@rushstack/eslint-patch/modern-module-resolution')
-
-module.exports = {
-  root: true,
-  extends: [
-    'plugin:vue/vue3-essential',
-    'eslint:recommended',
-    '@vue/eslint-config-typescript',
-    '@vue/eslint-config-prettier/skip-formatting'
-  ],
-  plugins: ['simple-import-sort'],
-  rules: {
-    'no-console': process.env.NODE_ENV === 'production' ? 'warn' : 'off',
-    'no-debugger': process.env.NODE_ENV === 'production' ? 'warn' : 'off',
-    'simple-import-sort/imports': 'error',
-    'simple-import-sort/exports': 'error'
-  },
-  parserOptions: {
-    ecmaVersion: 'latest'
-  }
-}

data/frontend/.gitignore

@@ -1,31 +0,0 @@
-# Logs
-logs
-*.log
-npm-debug.log*
-yarn-debug.log*
-yarn-error.log*
-pnpm-debug.log*
-lerna-debug.log*
-
-node_modules
-.DS_Store
-dist
-dist-ssr
-coverage
-*.local
-
-/cypress/videos/
-/cypress/screenshots/
-
-# Editor directories and files
-.vscode/*
-!.vscode/extensions.json
-.idea
-*.suo
-*.ntvs*
-*.njsproj
-*.sln
-*.sw?
-
-# redoc
-public/redoc-static.html