mihari 5.4.9 → 5.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1075febacca36de11c1285b0c08ef54fb41d3c0c7fe3e3df51b2dff28c597fe4
-  data.tar.gz: 56475aef95d5dd5fda32ef7a8633fc2d064219abdb05706d849f15f998c41db2
+  metadata.gz: 9131a7f69be7cde564ec00479ae3fa3723a3e80d28690c3e989119de3feab5f5
+  data.tar.gz: b115531cc635b7767e6bcf75c8ca0376e4ade45772e98aa9fc07df3b2dcc2e96
 SHA512:
-  metadata.gz: e18c315a2389a836aff99fd1da1c50749e83d9272a79c1215d22cd649758c3fd6d74dfabc1445d2c1127b18ea100abebd21326766bc2da79cd5c77a9ba27da3d
-  data.tar.gz: 696b2c8d8e045f647f33a25dceec868676f2e04a8a38cc1e55cfe357356e5a5282f6b42085c2a8c4277d7309e47d8724d86c05d63ec8baf8937bd65dc8cf97cd
+  metadata.gz: 9a893bb138e769bf082bbea057229726f9b2e353fa539c9a1fb64aabcc8a622ed22315a0aa42e5dc873f6a13e3ec145776afe75203db93b7a3d2352d46b026b9
+  data.tar.gz: cd11791f340b58ffc39a03fba8ee2aad1da58fcf47fd8a00281fe8984d0149fa1e448aff4b55860b47e5fb424d994a40ab00f7691b416f315b3fbfdcd5737509

data/docs/analyzers/binaryedge.md

@@ -19,8 +19,8 @@ api_key: ...
 
 ### Query
 
-`query` is a search query.
+`query` (`string`) is a search query.
 
 ### API Key
 
-`api_key` is an API key. Optional. Defaults to `ENV[”BINARYEDGE_API_KEY"]`.
+`api_key` (`string`) is an API key. Optional. Defaults to `ENV[”BINARYEDGE_API_KEY"]`.

data/docs/analyzers/censys.md

@@ -20,12 +20,12 @@ secret: ...
 
 ### Query
 
-`query` is a search query.
+`query` (`string`) is a search query.
 
 ### ID
 
-`id` is a Cencys ID. Optional. Defaults to `ENV[”CENSYS_ID”]`.
+`id` (`string`) is a Cencys ID. Optional. Defaults to `ENV[”CENSYS_ID”]`.
 
 ### Secret
 
-`secret` is a Cencys secret. Optional. Defaults to `ENV[”CENSYS_SECRET”]`.
+`secret` (`string`) is a Cencys secret. Optional. Defaults to `ENV[”CENSYS_SECRET”]`.

data/docs/analyzers/circl.md

@@ -26,12 +26,12 @@ username: ...
 
 ### Query
 
-`query` is a domain or SHA1 certificate fingerprint.
+`query` (`string`) is a domain or SHA1 certificate fingerprint.
 
 ### Username
 
-`username` is a username. Optional. Defaults to `ENV[”CIRCL_PASSIVE_USERNAME”]`.
+`username` (`string`) is a username. Optional. Defaults to `ENV[”CIRCL_PASSIVE_USERNAME”]`.
 
 ### Password
 
-`password` is a password. Optional. Defaults to `ENV[”CIRCL_PASSIVE_PASSWORD”]`.
+`password` (`string`) is a password. Optional. Defaults to `ENV[”CIRCL_PASSIVE_PASSWORD”]`.

data/docs/analyzers/crtsh.md

@@ -19,8 +19,8 @@ exclude_expired: ...
 
 ### Query
 
-`query` is a search query.
+`query` (`string`) is a search query.
 
 ### Exclude Expired
 
-`exclude_expired` (boolean) determines whether to exclude expired domains or not. Optional. Defaults to `true`.
+`exclude_expired` (`boolean`) determines whether to exclude expired domains or not. Optional. Defaults to `true`.

data/docs/analyzers/dnstwister.md

@@ -18,7 +18,7 @@ query: ...
 
 ### Query
 
-`query` is a search query.
+`query` (`string`) is a search query.
 
 !!! tip
 

data/docs/analyzers/feed.md

@@ -19,7 +19,7 @@ json: ...
 
 ### Query
 
-`query` is a URL of a feed.
+`query` (`string`) is a URL of a feed.
 
 !!! note
 
@@ -27,27 +27,27 @@ json: ...
 
 ### Method
 
-`method` is an HTTP method. Defaults to `GET`.
+`method` (`string`) is an HTTP method. Defaults to `GET`.
 
 ### Selector
 
-`selector` is a `jr` selector.
+`selector` (`string`) is a `jr` selector.
 
 ### Headers
 
-`headers` (hash) is an HTTP headers. Optional.
+`headers` (`hash`) is an HTTP headers. Optional.
 
 ### Params
 
-`params` (hash) is an HTTP query params. Optional.
+`params` (`hash`) is an HTTP query params. Optional.
 
 ### Data
 
-`data` (hash) is an HTTP form data. Optional.
+`data` (`hash`) is an HTTP form data. Optional.
 
 ### JSON
 
-`json` (hash) is an JSON body. Optional.
+`json` (`hash`) is an JSON body. Optional.
 
 ## Examples
 

data/docs/analyzers/greynoise.md

@@ -19,8 +19,8 @@ api_key: ...
 
 ### Query
 
-`query` is a GNQL search query.
+`query` (`string`) is a GNQL search query.
 
 ### API Key
 
-`api_key` is an API key. Optional. Defaults to `ENV[”GREYNOISE_API_KEY"]`.
+`api_key` (`string`) is an API key. Optional. Defaults to `ENV[”GREYNOISE_API_KEY"]`.

data/docs/analyzers/hunterhow.md

@@ -21,13 +21,13 @@ end_time: ...
 
 ### Query
 
-`query` is a search query.
+`query` (`string`) is a search query.
 
 ### Start/End Time
 
-- `start_time` (date): Only show results after the given date.
-- `end_time` (date): Only show results after the given date.
+- `start_time` (`date`): Only show results after the given date.
+- `end_time` (`date`): Only show results after the given date.
 
 ### API key
 
-`api_key` is an API key. Optional. Defaults to `ENV[”HUNTERHOW_API_KEY"]`.
+`api_key` (`string`) is an API key. Optional. Defaults to `ENV[”HUNTERHOW_API_KEY"]`.

data/docs/analyzers/index.md

@@ -27,39 +27,44 @@ analyzer: ...
 query: ...
 options:
   timeout: ...
-  interval: ...
+  pagination_interval: ...
   pagination_limit: ...
   retry_times: ...
   retry_interval: ...
+  retry_exponential_backoff: ...
   ignore_error: ...
 ```
 
 ### Timeout
 
-`timeout` is an HTTP timeout in seconds. Optional.
+`timeout` (`integer`) is an HTTP timeout in seconds. Optional.
 
-### Interval
+### Pagination Interval
 
-`interval` is an interval in seconds between pagination. (If an analyzer does pagination). Optional.
+`pagination_interval` (`integer`) is an interval in seconds between pagination. Optional. Defaults to 0.
 
 ### Pagination Limit
 
-`pagination_limit` is an limit for pagination. Defaults to 100.
+`pagination_limit` (`integer`) is an limit for pagination. Optional. Defaults to 100.
 
 In the worst case, if something wrong with Mihari or a service, Mihari can drain API quota by doing pagination forever.
 `pagination_limit` is a safety valve for that. A number of pagination is limited as `pagination_limit` times.
 
 ### Retry Times
 
-`retry_times` is a number of times of retry when something goes wrong. Defaults to 3.
+`retry_times` (`integer`) is a number of times of retry when something goes wrong. Optional. Defaults to 3.
 
 ### Retry Interval
 
-`retry_interval` is an interval in seconds between retries. Defaults to 5.
+`retry_interval` (`integer`) is an interval in seconds between retries. Optional. Defaults to 5.
+
+### Retry Exponential Backoff
+
+`retry_exponential_backoff` (`bool`) controls whether to do exponential backoff. Optional. Defaults to `true`.
 
 ### Ignore Error
 
-`ignore_error` controls whether to ignore an error or not. Defaults to `false`.
+`ignore_error` (`bool`) controls whether to ignore an error or not. Optional. Defaults to `false`.
 
 Mihari uses fail-fast approach. For example, if Shodan returns an error, the Censys query next is not triggered because Mihari raises an error before it.
 

data/docs/analyzers/onyphe.md

@@ -19,8 +19,8 @@ api_key: ...
 
 ### Query
 
-`query` is a search query.
+`query` (`string`) is a search query.
 
 ### API Key
 
-`api_key` is an API key. Optional. Defaults to `ENV[”ONYPHE_API_KEY”"]`.
+`api_key` (`string`) is an API key. Optional. Defaults to `ENV[”ONYPHE_API_KEY”"]`.

data/docs/analyzers/otx.md

@@ -21,8 +21,8 @@ api_key: ...
 
 ### Query
 
-`query` is a passive DNS search query. Domain or IP address.
+`query` (`string`) is a passive DNS search query. Domain or IP address.
 
 ### API Key
 
-`api_key` is an API key. Optional. Defaults to `ENV[”OTX_API_KEY”"]`.
+`api_key` (`string`) is an API key. Optional. Defaults to `ENV[”OTX_API_KEY”"]`.

data/docs/analyzers/passivetotal.md

@@ -31,9 +31,13 @@ api_key: ...
 
 ## Components
 
+### Analyzer
+
+`analyzer` (`string`) should be either of `passivetotal` and `pt`.
+
 ### Query
 
-`query` is a passive DNS/SSL or reverse whois search query. Domain, IP address, mail or SHA1 certificate fingerprint.
+`query` (`string`) is a passive DNS/SSL or reverse whois search query. Domain, IP address, mail or SHA1 certificate fingerprint.
 
 - Passive DNS: Domain, IP Address
 - Passive SSL: SHA1 certificate fingerprint
@@ -41,8 +45,8 @@ api_key: ...
 
 ### Username
 
-`username` is a username. Optional. Defaults to `ENV[”PASSIVETOTAL_USERNAME"]`.
+`username` (`string`) is a username. Optional. Defaults to `ENV[”PASSIVETOTAL_USERNAME"]`.
 
 ### API Key
 
-`api_key` is an API key. Optional. Defaults to `ENV[”PASSIVETOTAL_API_KEY"]`.
+`api_key` (`string`) is an API key. Optional. Defaults to `ENV[”PASSIVETOTAL_API_KEY"]`.

data/docs/analyzers/pulsedive.md

@@ -21,8 +21,8 @@ api_key: ...
 
 ### Query
 
-`query` is a passive DNS search query. Domain or IP address.
+`query` (`string`) is a passive DNS search query. Domain or IP address.
 
 ### API Key
 
-`api_key` is an API key. Optional. Defaults to `ENV[”PULSEDIVE_API_KEY"]`.
+`api_key` (`string`) is an API key. Optional. Defaults to `ENV[”PULSEDIVE_API_KEY"]`.

data/docs/analyzers/securitytrails.md

@@ -28,10 +28,14 @@ api_key: ...
 
 ## Components
 
+### Analyzer
+
+`analyzer` (`string`) should be either of `securitytrails` and `st`.
+
 ### Query
 
-`query` is a passive DNS search/reverse whois query. Domain, IP address or mail.
+`query` (`string`) is a passive DNS search/reverse whois query. Domain, IP address or mail.
 
 ### API Key
 
-`api_key` is an API key. Optional. Defaults to `ENV[”SECURITYTRAILS_API_KEY"]`.
+`api_key` (`string`) is an API key. Optional. Defaults to `ENV[”SECURITYTRAILS_API_KEY"]`.

data/docs/analyzers/shodan.md

@@ -19,8 +19,8 @@ api_key: ...
 
 ### Query
 
-`query` is a search query.
+`query` (`string`) is a search query.
 
 ### API Key
 
-`api_key` is an API key. Optional. Defaults to `ENV[”SHODAN_API_KEY"]`.
+`api_key` (`string`) is an API key. Optional. Defaults to `ENV[”SHODAN_API_KEY"]`.

data/docs/analyzers/urlscan.md

@@ -21,8 +21,8 @@ api_key: ...
 
 ### Query
 
-`query` is a search query.
+`query` (`string`) is a search query.
 
 ### API Key
 
-`api_key` is an API key. Optional. Defaults to `ENV[”URLSCAN_API_KEY"]`.
+`api_key` (`string`) is an API key. Optional. Defaults to `ENV[”URLSCAN_API_KEY"]`.

data/docs/analyzers/virustotal.md

@@ -30,10 +30,14 @@ api_key: ...
 
 ## Components
 
+### Analyzer
+
+`analyzer` (`string`) should be either of `virustoal` and `vt`.
+
 ### Query
 
-`query` is a passive DNS search query. Domain or IP address.
+`query` (`string`) is a passive DNS search query. Domain or IP address.
 
 ### API Key
 
-`api_key` is an API key. Optional. Defaults to `ENV[”VIRUSTOTAL_API_KEY"]`.
+`api_key` (`string`) is an API key. Optional. Defaults to `ENV[”VIRUSTOTAL_API_KEY"]`.

data/docs/analyzers/virustotal_intelligence.md

@@ -20,10 +20,14 @@ api_key: ...
 
 ## Components
 
+### Analyzer
+
+`analyzer` (`string`) should be either of `virustotal_intelligence` and ``.
+
 ### Query
 
-`query` is a search query.
+`query` (`string`) is a search query.
 
 ### API Key
 
-`api_key` is an API key. Optional. Defaults to `ENV[”VIRUSTOTAL_API_KEY"]`.
+`api_key` (`string`) is an API key. Optional. Defaults to `ENV[”VIRUSTOTAL_API_KEY"]`.

data/docs/analyzers/zoomeye.md

@@ -22,12 +22,12 @@ api_key: ...
 
 ### Query
 
-`query` is a search query.
+`query` (`string`) is a search query.
 
 ### Type
 
-`type` determines a search type. `web` or `host`.
+`type` (`string`) determines a search type. `web` or `host`.
 
 ### API Key
 
-`api_key` is an API key. Optional. Defaults to `ENV[”ZOOMEYE_API_KEY"]`.
+`api_key` (`string`) is an API key. Optional. Defaults to `ENV[”ZOOMEYE_API_KEY"]`.

data/docs/emitters/hive.md

@@ -5,7 +5,7 @@
 This emitter creates an alert on TheHive. TheHive v4 & v5 are supported.
 
 ```yaml
-emitter: the_hive
+emitter: thehive
 url: ...
 api_key: ...
 api_version: ...
@@ -15,12 +15,12 @@ api_version: ...
 
 ### URL
 
-`url` is a TheHive URL. Optional. Defaults to `ENV[”THEHIVE_URL”]`.
+`url` (`string`) is a TheHive URL. Optional. Defaults to `ENV[”THEHIVE_URL”]`.
 
 ### API Key
 
-`api_key` is an API key. Optional. Defaults to `ENV[”THEHIVE_API_KEY”]`.
+`api_key` (`string`) is an API key. Optional. Defaults to `ENV[”THEHIVE_API_KEY”]`.
 
 ### API Version
 
-`api_version` is a version of The Hive API. Optional. Defaults to `ENV[”THEHIVE_API_VERSION”]`.
+`api_version` (`string`) is a version of The Hive API. Optional. `v4` or `v5`. Defaults to `ENV[”THEHIVE_API_VERSION”]`.

data/docs/emitters/index.md

@@ -5,3 +5,32 @@
 - [MISP](misp.md)
 - [Slack](slack.md)
 - [Webhook](webhook.md)
+
+## Options
+
+All the emitters can have optional `options`.
+
+```yaml
+emitter: ...
+options:
+  timeout: ...
+  retry_times: ...
+  retry_interval: ...
+  retry_exponential_backoff: ...
+```
+
+### Timeout
+
+`timeout` (`integer`) is an HTTP timeout in seconds. Optional.
+
+### Retry Times
+
+`retry_times` (`integer`) is a number of times of retry when something goes wrong. Optional. Defaults to 3.
+
+### Retry Interval
+
+`retry_interval` (`integer`) is an interval in seconds between retries. Optional. Defaults to 5.
+
+### Retry Exponential Backoff
+
+`retry_exponential_backoff` (`bool`) controls whether to do exponential backoff. Optional. Defaults to `true`.

data/docs/emitters/misp.md

@@ -14,8 +14,8 @@ api_key: ...
 
 ### URL
 
-`url` is a MISP URL. Optional. Defaults to `ENV[MISP_URL]`.
+`url` (`string`) is a MISP URL. Optional. Defaults to `ENV[MISP_URL]`.
 
 ### API Key
 
-`api_key` is an API key. Optional. Defaults to `ENV[”MISP_API_KEY”]`.
+`api_key` (`string`) is an API key. Optional. Defaults to `ENV[”MISP_API_KEY”]`.

data/docs/emitters/slack.md

@@ -10,17 +10,12 @@ webhook_url: ...
 channel: ...
 ```
 
-| Name        | Type   | Required? | Default                         | Desc.             |
-| ----------- | ------ | --------- | ------------------------------- | ----------------- |
-| webhook_url | String | No        | ENV[SLACK_WEBHOOK_URL]          | Slack webhook URL |
-| channel     | String | No        | ENV[SLACK_CHANNEL] / `#general` | Slack channel     |
-
 ## Components
 
 ### Webhook URL
 
-`url` is a Slack's incoming webhook URL. Optional. Defaults to `ENV[SLACK_WEBHOOK_URL]`.
+`url` (`string`) is a Slack's incoming webhook URL. Optional. Defaults to `ENV[SLACK_WEBHOOK_URL]`.
 
 ### API Key
 
-`channel` is a Slack channel to sent a message. Optional. Defaults to `ENV[SLACK_CHANNEL]` or `#general`.
+`channel` (`string`) is a Slack channel to sent a message. Optional. Defaults to `ENV[SLACK_CHANNEL]` or `#general`.

data/docs/emitters/webhook.md

@@ -14,19 +14,19 @@ template: ...
 
 ### URL
 
-`url` is a webhook URL.
+`url` (`string`) is a webhook URL.
 
 ### Method
 
-`method` is an HTTP method. Optional. Defaults to `POST`.
+`method` (`string`)is an HTTP method. Optional. Defaults to `POST`.
 
 ### Headers
 
-`headers` (hash) is HTTP headers. Optional.
+`headers` (`hash`) are HTTP headers. Optional.
 
 ### Template
 
-`template` is an [ERB](https://github.com/ruby/erb) template to customize the payload to sent. A template should generate a valid JSON.
+`template` (`string`) is an [ERB](https://github.com/ruby/erb) template to customize the payload to sent. A template should generate a valid JSON.
 
 You can use the following parameters inside an ERB template.
 

data/docs/enrichers/index.md

@@ -4,3 +4,32 @@
 - [IPInfo](ipinfo.md)
 - [Shodan](shodan.md)
 - [Whois](whois.md)
+
+## Options
+
+All the emitters can have optional `options`.
+
+```yaml
+enricher: ...
+options:
+  timeout: ...
+  retry_times: ...
+  retry_interval: ...
+  retry_exponential_backoff: ...
+```
+
+### Timeout
+
+`timeout` (`integer`) is an HTTP timeout in seconds. Optional.
+
+### Retry Times
+
+`retry_times` (`integer`) is a number of times of retry when something goes wrong. Optional. Defaults to 3.
+
+### Retry Interval
+
+`retry_interval` (`integer`) is an interval in seconds between retries. Optional. Defaults to 5.
+
+### Retry Exponential Backoff
+
+`retry_exponential_backoff` (`bool`) controls whether to do exponential backoff. Optional. Defaults to `true`.

data/docs/enrichers/ipinfo.md

@@ -12,8 +12,15 @@ This enricher uses ipinfo.io API to enrich an IP artifact.
 
 ```yaml
 enricher: ipinfo
+api_key: ...
 ```
 
+## Components
+
+### API Key
+
+`api_key` (`string`) is an API key. Optional. Defaults to `ENV[”IPINFO_API_KEY”]`.
+
 ## Supported Artifacts
 
 - IP address

data/docs/index.md

@@ -9,5 +9,3 @@ Mihari can aggregate multiple searches across multiple services in a single rule
 - [How to Write a Rule](./rule.md)
 - [Usage](./usage.md)
 - [Configuration](./configuration.md)
-- [GitHub Actions](./github_actions.md)
-- [Alternatives](./alternatives.md)

data/docs/installation.md

@@ -2,7 +2,7 @@
 
 ## Ruby Gem
 
-Mihari is packaged as a Ruby Gem.
+Mihari is packaged as a Ruby Gem. Thus you can install it via `gem` command.
 
 ```bash
 gem install mihari

data/docs/rule.md

@@ -54,7 +54,7 @@ emitters:
   - emitter: database
   - emitter: misp
   - emitter: slack
-  - emitter: the_hive
+  - emitter: thehive
 data_types:
   - hash
   - ip
@@ -68,36 +68,36 @@ falsepositives: []
 
 ### ID
 
-`id` is an unique ID of a rule. UUID v4 is recommended.
+`id` (`string`) is an unique ID of a rule. UUID v4 is recommended.
 
 ### Title
 
-`title` is a title of a rule.
+`title` (`string`) is a title of a rule.
 
 ### Description
 
-`description` is a short description of a rule.
+`description` (`string`) is a short description of a rule.
 
 ### Created/Updated On
 
-`created_on` is a date of a rule creation. Optional.
+`created_on` (`date`) is a date of a rule creation. Optional.
 Also a rule can have `updated_on` that is a date of a rule modification. Optional.
 
 ### Tags
 
-`tags` is a list of tags of a rule.
+`tags` (`array[:string]`) is a list of tags of a rule.
 
 ### Author
 
-`author` is an author of a rule. Optional.
+`author` (`string`) is an author of a rule. Optional.
 
 ### References
 
-`references` is a list of a references of a rule. Optional.
+`references` (`array[:string]`) is a list of a references of a rule. Optional.
 
 ### Related
 
-`related` is a list of related rule IDs. Optional.
+`related` (`array[:string]`) is a list of related rule IDs. Optional.
 
 ### Queries
 
@@ -124,13 +124,10 @@ See [Emitters](./emitters/index.md) to know details of each emitter.
 Defaults to:
 
 - `database`
-- `misp`
-- `slack`
-- `the_hive`
 
 ### Data Types
 
-`data_types` is a list of data (artifact) types to allow by a rule. Types not defined in here will be automatically rejected.
+`data_types` (`array[:string]`) is a list of data (artifact) types to allow by a rule. Types not defined in here will be automatically rejected.
 
 Defaults to:
 
@@ -142,11 +139,11 @@ Defaults to:
 
 ### False positives
 
-`falsepositives` is a list of false positive values. A string or regexp can be used in here.
+`falsepositives` (`array[:string]`) is a list of false positive values. A string or regexp can be used in here.
 
 ### Artifact TTL
 
-`artifact_ttl` (alias: `artifact_lifetime`) is an integer value of artifact TTL (Time-To-Live) in seconds.
+`artifact_ttl` (`integer` / alias: `artifact_lifetime`) is an integer value of artifact TTL (Time-To-Live) in seconds.
 
 Mihari rejects a same artifact in a same rule in general.
 

data/docs/usage.md

@@ -8,8 +8,11 @@ Commands:
   mihari db                   # Sub commands for DB
   mihari help [COMMAND]       # Describe available commands or one specific command
   mihari rule                 # Sub commands for rule
-  mihari search [PATH_OR_ID]  # Search by a rule
+  mihari search [PATH_OR_ID]  # Search by a rule (Outputs null if there is no new finding)
   mihari web                  # Launch the web app
+
+Options:
+  -d, [--debug], [--no-debug]  # Sets up debug mode
 ```
 
 ## `mihari db`
@@ -43,7 +46,7 @@ Mihari asks whether really you want to update a rule if there is a diff by defau
 
 ```bash
 $ mihari search /path/to/rule.yml
-There is a diff in the rule (6254bb74-5e5d-42ad-bc1e-231da0293b0f). Are you sure you want to overwrite the rule? (y/n)
+There is a diff in the rule. Are you sure you want to overwrite the rule? (y/n)
 ```
 
 It can be suppressed by providing `-f`.