txt2stix 1.0.1.post3__tar.gz → 1.0.3__tar.gz

{txt2stix-1.0.1.post3 → txt2stix-1.0.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: txt2stix
-Version: 1.0.1.post3
+Version: 1.0.3
 Summary: txt2stix is a Python script that is designed to identify and extract IoCs and TTPs from text files, identify the relationships between them, convert them to STIX 2.1 objects, and output as a STIX 2.1 bundle.
 Project-URL: Homepage, https://github.com/muchdogesec/txt2stix
 Project-URL: Issues, https://github.com/muchdogesec/txt2stix/issues
@@ -171,8 +171,9 @@ If any AI extractions, or AI relationship mode is set, you must set the followin
 #### Other AI related settings
 
 * `--ai_content_check_provider` (`model:provider`, required if passed): Passing this flag will get the AI to try and classify the text in the input to 1) determine if it is talking about threat intelligence, and 2) what type of threat intelligence it is talking about. For context, we use this to filter out non-threat intel posts in Obstracts and Stixify. You pass `provider:model` with this flag to determine the AI model you wish to use to perform the check. It will also create a summary of the content passed (and store this into a STIX Note).
-* `--ai_extract_if_no_incidence` (boolean, default `true`) if content check decides the report is not related to cyber security intelligence (e.g. vendor marketing), then you can use this setting to decide wether or not script should proceed. Setting to `false` will stop processing. It is designed to save AI tokens processing unknown content at scale in an automated way.
-* `--ai_create_attack_flow` (boolean): passing this flag will also prompt the AI model (the same entered for `--ai_settings_relationships`) to generate an [Attack Flow](https://center-for-threat-informed-defense.github.io/attack-flow/) for the MITRE ATT&CK extractions to define the logical order in which they are being described. You must pass `--ai_settings_relationships` for this to work.
+* `--ai_extract_if_no_incidence` (boolean, default `true`, will only work if `ai_content_check_provider` set) if content check decides the report is not related to cyber security intelligence (e.g. vendor marketing), then you can use this setting to decide wether or not script should proceed. Setting to `false` will stop processing. It is designed to save AI tokens processing unknown content at scale in an automated way.
+* `--ai_create_attack_flow` (boolean): passing this flag will also prompt the AI model (the same entered for `--ai_settings_relationships`, default `false`) to generate an [Attack Flow](https://center-for-threat-informed-defense.github.io/attack-flow/) for the MITRE ATT&CK extractions to define the logical order in which they are being described. You must pass `--ai_settings_relationships` for this to work.
+* `--ai_create_attack_navigator_layer` (boolean, default `false`): passing this flag will generate [MITRE ATT&CK Navigator layers](https://mitre-attack.github.io/attack-navigator/) for MITRE ATT&CK extractions. For each ATT&CK domain (Enterprise, ICS, Mobile) txt2stix will generate a layer. You must pass `--ai_settings_relationships` for this to work because the AI is tasked with linking extracted Techniques to the correct Tactic. Known issues with `openai:gpt-3.5` (avoid using this model if possible when using ATT&CK Navigator).
 
 ## Adding new extractions
 

{txt2stix-1.0.1.post3 → txt2stix-1.0.3}/README.md

@@ -127,8 +127,9 @@ If any AI extractions, or AI relationship mode is set, you must set the followin
 #### Other AI related settings
 
 * `--ai_content_check_provider` (`model:provider`, required if passed): Passing this flag will get the AI to try and classify the text in the input to 1) determine if it is talking about threat intelligence, and 2) what type of threat intelligence it is talking about. For context, we use this to filter out non-threat intel posts in Obstracts and Stixify. You pass `provider:model` with this flag to determine the AI model you wish to use to perform the check. It will also create a summary of the content passed (and store this into a STIX Note).
-* `--ai_extract_if_no_incidence` (boolean, default `true`) if content check decides the report is not related to cyber security intelligence (e.g. vendor marketing), then you can use this setting to decide wether or not script should proceed. Setting to `false` will stop processing. It is designed to save AI tokens processing unknown content at scale in an automated way.
-* `--ai_create_attack_flow` (boolean): passing this flag will also prompt the AI model (the same entered for `--ai_settings_relationships`) to generate an [Attack Flow](https://center-for-threat-informed-defense.github.io/attack-flow/) for the MITRE ATT&CK extractions to define the logical order in which they are being described. You must pass `--ai_settings_relationships` for this to work.
+* `--ai_extract_if_no_incidence` (boolean, default `true`, will only work if `ai_content_check_provider` set) if content check decides the report is not related to cyber security intelligence (e.g. vendor marketing), then you can use this setting to decide wether or not script should proceed. Setting to `false` will stop processing. It is designed to save AI tokens processing unknown content at scale in an automated way.
+* `--ai_create_attack_flow` (boolean): passing this flag will also prompt the AI model (the same entered for `--ai_settings_relationships`, default `false`) to generate an [Attack Flow](https://center-for-threat-informed-defense.github.io/attack-flow/) for the MITRE ATT&CK extractions to define the logical order in which they are being described. You must pass `--ai_settings_relationships` for this to work.
+* `--ai_create_attack_navigator_layer` (boolean, default `false`): passing this flag will generate [MITRE ATT&CK Navigator layers](https://mitre-attack.github.io/attack-navigator/) for MITRE ATT&CK extractions. For each ATT&CK domain (Enterprise, ICS, Mobile) txt2stix will generate a layer. You must pass `--ai_settings_relationships` for this to work because the AI is tasked with linking extracted Techniques to the correct Tactic. Known issues with `openai:gpt-3.5` (avoid using this model if possible when using ATT&CK Navigator).
 
 ## Adding new extractions
 

{txt2stix-1.0.1.post3 → txt2stix-1.0.3}/includes/lookups/_generate_lookups.py

@@ -1,3 +1,5 @@
+## IMPORTANT: if using CTI Butler database locally in arangodb (i.e is not app.ctibutler.com in .env) you need to follow these steps to import the data needed to populate these lookups: https://github.com/muchdogesec/stix2arango/blob/main/utilities/arango_cti_processor/README.md (use `--database ctibutler_database` in the s2a script or change it in this script)
+
 import os
 from arango import ArangoClient
 

{txt2stix-1.0.1.post3 → txt2stix-1.0.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "txt2stix"
-version = "1.0.1-3"
+version = "1.0.3"
 authors = [{ name = "dogesec" }]
 maintainers = [{ name = "dogesec" }]
 description = "txt2stix is a Python script that is designed to identify and extract IoCs and TTPs from text files, identify the relationships between them, convert them to STIX 2.1 objects, and output as a STIX 2.1 bundle."

{txt2stix-1.0.1.post3 → txt2stix-1.0.3}/requirements.txt

@@ -6,7 +6,7 @@
 #
 aiohappyeyeballs==2.6.1
     # via aiohttp
-aiohttp==3.12.13
+aiohttp==3.12.14
     # via llama-index-core
 aiosignal==1.4.0
     # via aiohttp

txt2stix-1.0.3/tests/data/manually_generated_reports/attack_navigator_demo.txt

@@ -0,0 +1,9 @@
+Enterprise
+
+T1595 is used during TA0043
+
+T1587.001 is then used to T1587.
+
+Mobile
+
+T1451 is used for TA0027 to achieve T1662

{txt2stix-1.0.1.post3 → txt2stix-1.0.3}/tests/manual-tests/cases-standard-tests.md

@@ -433,6 +433,25 @@ python3 txt2stix.py \
     --report_id ed6039d6-699c-44f0-9bf0-957d4d0ff99f
 ```
 
+ Will pass but still process, as `ai_content_check_provider` is omitted
+
+```shell
+python3 txt2stix.py \
+    --relationship_mode standard \
+    --input_file tests/data/extraction_types/all_cases.txt \
+    --name 'Test AI Content check failure' \
+    --tlp_level clear \
+    --confidence 100 \
+	--use_extractions ai_ipv4_address_only \
+	--ai_settings_extractions openai:gpt-4o \
+    --tlp_level clear \
+    --confidence 100 \
+	--use_extractions ai_ipv4_address_only \
+	--ai_settings_extractions openai:gpt-4o \
+	--ai_extract_if_no_incidence false \
+    --report_id 2880d1c1-0211-45b6-8565-befe596ff81f
+```
+
 ### attack flow demo
 
 no indicators
@@ -465,4 +484,42 @@ python3 txt2stix.py \
     --ai_settings_extractions openai:gpt-4o \
     --ai_create_attack_flow \
     --report_id 3b160a8d-12dd-4e7c-aee8-5af6e371b425
+```
+
+### attack navigator demo
+
+```shell
+python3 txt2stix.py \
+    --relationship_mode ai \
+    --ai_settings_relationships openai:gpt-4o \
+    --input_file tests/data/manually_generated_reports/attack_navigator_demo.txt \
+    --name 'Test MITRE ATT&CK Navigator' \
+    --tlp_level clear \
+    --confidence 100 \
+    --use_extractions 'ai_mitre_attack_*' \
+    --ai_settings_extractions openai:gpt-4o \
+    --ai_create_attack_navigator_layer \
+    --ai_content_check_provider openai:gpt-4o \
+    --report_id b599f044-f22c-4e38-a2ed-3ef43442ccd2
+```
+
+`ai_content_check_provider` checked to ensure summary is used as description
+
+### attack navigator and attack flow
+
+used to check prompts only sent once
+
+```shell
+python3 txt2stix.py \
+    --relationship_mode ai \
+    --ai_settings_relationships openai:gpt-4o \
+    --input_file tests/data/manually_generated_reports/attack_navigator_demo.txt \
+    --name 'Test MITRE ATT&CK Flow and Navigator' \
+    --tlp_level clear \
+    --confidence 100 \
+    --use_extractions 'ai_mitre_attack_enterprise' \
+    --ai_settings_extractions openai:gpt-4o \
+    --ai_create_attack_flow \
+    --ai_create_attack_navigator_layer \
+    --report_id c0d48262-1d9f-42d2-aa29-f0cba1bfa2e0
 ```