reait 0.0.19__tar.gz → 1.0.0__tar.gz

{reait-0.0.19 → reait-1.0.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: reait
-Version: 0.0.19
+Version: 1.0.0
 Home-page: https://github.com/RevEng-AI/reait
 Author: James Patrick-Evans
 Author-email: James Patrick-Evans <james@reveng.ai>
@@ -685,13 +685,13 @@ Project-URL: Bug Tracker, https://github.com/RevEng-AI/reait/issues
 Project-URL: Organisation Homepage, https://reveng.ai
 Project-URL: Documentation, https://docs.reveng.ai
 Keywords: reverse,engineering,reveng.ai,reveng,machine,learning,binary,analysis,ml,ai,vector,embedding
+Platform: Cross Platform
 Classifier: Programming Language :: Python :: 3
 Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
 Classifier: Operating System :: OS Independent
 Requires-Python: >=3.7
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: tqdm
 Requires-Dist: requests
 Requires-Dist: rich
 Requires-Dist: argparse
@@ -713,20 +713,22 @@ Analyse compiled executable binaries using the RevEng.AI API. This tool allows y
 NB: We are in Alpha. We support GNU/Linux ELF and Windows PE executables for x86_64, and focus our support for x86_64 Linux ELF executables. 
 
 ## Installation
+Install the latest stable version using `pip3`.
 
-Install the latest stable version using pip.
-
-`pip install reait`
+```shell
+pip3 install reait
+```
 
 ### Latest development version
-
-`pip install -e .`
+```shell
+pip3 install -e .
+```
 
 or 
 
-```
+```shell
 python3 -m build .
-pip install -U dist/reait-*.whl
+pip3 install -U dist/reait-*.whl
 ```
 
 ## Using reait
@@ -734,7 +736,9 @@ pip install -U dist/reait-*.whl
 ### Analysing binaries
 To submit a binary for analysis, run `reait` with the `-a` flag:
 
-`reait -b /usr/bin/true -a`
+```shell
+reait -b /usr/bin/true -a
+```
 
 This uploads the binary specified by `-b` to RevEng.AI servers for analysis. Depending on the size of the binary, it may take several hours. You may check an analysis jobs progress with the `-l` flag e.g. `reait -b /usr/bin/true -l`.
 
@@ -742,30 +746,36 @@ This uploads the binary specified by `-b` to RevEng.AI servers for analysis. Dep
 Symbol embeddings are numerical vector representations of each component that capture their semantic understanding. Similar functions should be similar to each other in our embedded vector space. They can be thought of as *advanced* AI-based IDA FLIRT signatures or Radare2 Zignatures.
 Once an analysis is complete, you may access RevEng.AI's BinNet embeddings for all symbols extracted with the `-x` flag. 
 
-`reait -b /usr/bin/true -x > embeddings.json`
-
-#### Extract embedding for symbol at vaddr 0x19f0
-`reait -b /usr/bin/true -x | jq ".[] | select(.vaddr==$((0x19f0))).embedding" > embedding.json`
-
+```shell
+reait -b /usr/bin/true -x > embeddings.json
+```
 
 ### Search for similar symbols using an embedding
 To query our database of similar symbols based on an embedding, use `-n` to search using Approximate Nearest Neighbours. The `--nns` allows you to specify the number of results returned. A list of symbols with their names, distance (similarity), RevEng.AI collection set, source code filename, source code line number, and file creation timestamp is returned. 
 
-`reait -e embedding.json -n`
+```shell
+reait --embedding embedding.json -n
+```
 
-The following command searches for the top 10 most similar symbols found in md5sum.gcc.og.dynamic to the symbol starting at 0x4037e0 in md5sum.clang.og.dynamic. You may need to pass `--image-base` to ensure virtual addresses are mapped correctly.
+The following command searches for the top 10 most similar symbols found in md5sum.gcc.og.dynamic to the symbol starting at _0x33E6_ in md5sum.clang.og.dynamic. You may need to pass `--image-base` to ensure virtual addresses are mapped correctly.
 
-`reait -b md5sum.gcc.og.dynamic -n --start-vaddr 0x33e6 --found-in md5sum.gcc.o2.dynamic --nns 10 --base-address 0x100000`
+```shell
+reait -b md5sum.gcc.og.dynamic -n --start-vaddr 0x33E6 --found-in md5sum.gcc.o2.dynamic --nns 10 --base-address 0x100000
+```
 
 Search NN by symbol name.
-`reait -b md5sum.gcc.og.dynamic -n --symbol md5_buffer --found-in md5sum.gcc.o2.dynamic --nns 5`
+```shell
+reait -b md5sum.gcc.og.dynamic -n --symbol md5_buffer --found-in md5sum.gcc.o2.dynamic --nns 5
+```
 
 NB: A smaller distance indicates a higher degree of similarity.
 
 #### Specific Search
 To search for the most similar symbols found in a specific binary, use the `--found-in` option with a path to the executable to search from.
 
-`reait -n --embedding /tmp/sha256_init.json --found-in ~/malware.exe --nns 5` 
+```shell
+reait -n --embedding /tmp/sha256_init.json --found-in ~/malware.exe --nns 5
+``` 
 
 This downloads embeddings from `malware.exe` and computes the cosine similarity between all symbols and `sha256_init.json`. The returned results lists the most similar symbol locations by cosine similarity score (1.0 most similar, -1.0 dissimilar).
 
@@ -775,7 +785,9 @@ The `--from-file` option may also be used to limit the search to a custom file c
 #### Limited Search
 To search for most similar symbols from a set of RevEng.AI collections, use the `--collections` options with a RegEx to match collection names. For example:
 
-`reait -n --embedding my_func.json --collections "(libc.*|lib.*crypt.*)"`
+```shell
+reait -n --embedding my_func.json --collections "(libc.*|lib.*crypt.*)"
+```
 
 RevEng.AI collections are sets of pre-analysed executable objects. To create custom collection sets e.g., malware collections, please create a RevEng.AI account.
 
@@ -786,47 +798,25 @@ Find common components between binaries, RevEng.AI collections, or global search
 
 Example usage: 
 
-```
+```shell
 reait -M -b 05ff897f430fec0ac17f14c89181c76961993506e5875f2987e9ead13bec58c2.exe --from-file 755a4b2ec15da6bb01248b2dfbad206c340ba937eae9c35f04f6cedfe5e99d63.embeddings.json --confidence high
 ```
 
 ### RevEng.AI embedding models
 To use specific RevEng.AI AI models, or for training custom models, use `-m` to specify the model. The default option is to use the latest development model. Available models are `binnet-0.1` and `dexter`.
 
-`reait -b /usr/bin/true -m dexter -a`
+```shell
+reait -b /usr/bin/true -m dexter -a
+```
 
 ### Software Composition Analysis
 To identify known open source software components embedded inside a binary, use the `-C` flag.
 
-#### Stripped Binary CVE Checker
-To check for known vulnerabilities found with embedded software components, use `-c` or `--cves`.
-
-
-### REAI Signatures
-To generate an AI functional description of an entire binary file, use the `-s` flag. This will return the REAI signature of the file.
-
-REAI signatures can be used to compute the binary similarity between entire executables with the `-S` flag. For example:
-
-```
-reait -b d24ccf73aabca4192d33a07b4a238c8d40ac97a550c2e65b8074f03455a981ca.exe -S -t 00062cb01088cea245cd5f3eb03f65a0e6b11a8126ce00034d87935a451cf99c.exe,438d64bb831555caadaa92a32c9d62e255001bc8d524721c885f37d750ec3476.exe,755a4b2ec15da6bb01248b2dfbad206c340ba937eae9c35f04f6cedfe5e99d63.exe,05ff897f430fec0ac17f14c89181c76961993506e5875f2987e9ead13bec58c2.exe
-Computing Binary Similarity... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:01
-                      Binary Similarity to RedlineInfoStealer/d24ccf73aabca4192d33a07b4a238c8d40ac97a550c2e65b8074f03455a981ca.exe                      
-┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
-┃                                                               Binary ┃ SHA3-256                                                         ┃ Similarity ┃
-┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
-│ 00062cb01088cea245cd5f3eb03f65a0e6b11a8126ce00034d87935a451cf99c.exe │ 00062cb01088cea245cd5f3eb03f65a0e6b11a8126ce00034d87935a451cf99c │ 0.99907    │
-│ 438d64bb831555caadaa92a32c9d62e255001bc8d524721c885f37d750ec3476.exe │ 438d64bb831555caadaa92a32c9d62e255001bc8d524721c885f37d750ec3476 │ 1.00000    │
-│ 755a4b2ec15da6bb01248b2dfbad206c340ba937eae9c35f04f6cedfe5e99d63.exe │ 755a4b2ec15da6bb01248b2dfbad206c340ba937eae9c35f04f6cedfe5e99d63 │ 0.80522    │
-│ 05ff897f430fec0ac17f14c89181c76961993506e5875f2987e9ead13bec58c2.exe │ 05ff897f430fec0ac17f14c89181c76961993506e5875f2987e9ead13bec58c2 │ 0.94701    │
-└──────────────────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────┴────────────┘
-```
-
 
 ### Binary ANN Search
-
 To perform binary ANN search, pass in `-n` and `-s` flag at the same time. For example:
 
-```
+```shell
 reait -b /usr/bin/true -s -n
 Found /usr/bin/true:elf-x86_64
 [
@@ -855,13 +845,12 @@ Found /usr/bin/true:elf-x86_64
 
 
 ## Configuration
-
 `reait` reads the config file stored at `~/.reait.toml`. An example config file looks like:
 
-```
+```shell
 apikey = "l1br3"
 host = "https://api.reveng.ai"
-model = "binnet-0.1"
+model = "binnet-0.3-x86"
 ```
 
 ## Contact

{reait-0.0.19 → reait-1.0.0}/README.md

@@ -9,20 +9,22 @@ Analyse compiled executable binaries using the RevEng.AI API. This tool allows y
 NB: We are in Alpha. We support GNU/Linux ELF and Windows PE executables for x86_64, and focus our support for x86_64 Linux ELF executables. 
 
 ## Installation
+Install the latest stable version using `pip3`.
 
-Install the latest stable version using pip.
-
-`pip install reait`
+```shell
+pip3 install reait
+```
 
 ### Latest development version
-
-`pip install -e .`
+```shell
+pip3 install -e .
+```
 
 or 
 
-```
+```shell
 python3 -m build .
-pip install -U dist/reait-*.whl
+pip3 install -U dist/reait-*.whl
 ```
 
 ## Using reait
@@ -30,7 +32,9 @@ pip install -U dist/reait-*.whl
 ### Analysing binaries
 To submit a binary for analysis, run `reait` with the `-a` flag:
 
-`reait -b /usr/bin/true -a`
+```shell
+reait -b /usr/bin/true -a
+```
 
 This uploads the binary specified by `-b` to RevEng.AI servers for analysis. Depending on the size of the binary, it may take several hours. You may check an analysis jobs progress with the `-l` flag e.g. `reait -b /usr/bin/true -l`.
 
@@ -38,30 +42,36 @@ This uploads the binary specified by `-b` to RevEng.AI servers for analysis. Dep
 Symbol embeddings are numerical vector representations of each component that capture their semantic understanding. Similar functions should be similar to each other in our embedded vector space. They can be thought of as *advanced* AI-based IDA FLIRT signatures or Radare2 Zignatures.
 Once an analysis is complete, you may access RevEng.AI's BinNet embeddings for all symbols extracted with the `-x` flag. 
 
-`reait -b /usr/bin/true -x > embeddings.json`
-
-#### Extract embedding for symbol at vaddr 0x19f0
-`reait -b /usr/bin/true -x | jq ".[] | select(.vaddr==$((0x19f0))).embedding" > embedding.json`
-
+```shell
+reait -b /usr/bin/true -x > embeddings.json
+```
 
 ### Search for similar symbols using an embedding
 To query our database of similar symbols based on an embedding, use `-n` to search using Approximate Nearest Neighbours. The `--nns` allows you to specify the number of results returned. A list of symbols with their names, distance (similarity), RevEng.AI collection set, source code filename, source code line number, and file creation timestamp is returned. 
 
-`reait -e embedding.json -n`
+```shell
+reait --embedding embedding.json -n
+```
 
-The following command searches for the top 10 most similar symbols found in md5sum.gcc.og.dynamic to the symbol starting at 0x4037e0 in md5sum.clang.og.dynamic. You may need to pass `--image-base` to ensure virtual addresses are mapped correctly.
+The following command searches for the top 10 most similar symbols found in md5sum.gcc.og.dynamic to the symbol starting at _0x33E6_ in md5sum.clang.og.dynamic. You may need to pass `--image-base` to ensure virtual addresses are mapped correctly.
 
-`reait -b md5sum.gcc.og.dynamic -n --start-vaddr 0x33e6 --found-in md5sum.gcc.o2.dynamic --nns 10 --base-address 0x100000`
+```shell
+reait -b md5sum.gcc.og.dynamic -n --start-vaddr 0x33E6 --found-in md5sum.gcc.o2.dynamic --nns 10 --base-address 0x100000
+```
 
 Search NN by symbol name.
-`reait -b md5sum.gcc.og.dynamic -n --symbol md5_buffer --found-in md5sum.gcc.o2.dynamic --nns 5`
+```shell
+reait -b md5sum.gcc.og.dynamic -n --symbol md5_buffer --found-in md5sum.gcc.o2.dynamic --nns 5
+```
 
 NB: A smaller distance indicates a higher degree of similarity.
 
 #### Specific Search
 To search for the most similar symbols found in a specific binary, use the `--found-in` option with a path to the executable to search from.
 
-`reait -n --embedding /tmp/sha256_init.json --found-in ~/malware.exe --nns 5` 
+```shell
+reait -n --embedding /tmp/sha256_init.json --found-in ~/malware.exe --nns 5
+``` 
 
 This downloads embeddings from `malware.exe` and computes the cosine similarity between all symbols and `sha256_init.json`. The returned results lists the most similar symbol locations by cosine similarity score (1.0 most similar, -1.0 dissimilar).
 
@@ -71,7 +81,9 @@ The `--from-file` option may also be used to limit the search to a custom file c
 #### Limited Search
 To search for most similar symbols from a set of RevEng.AI collections, use the `--collections` options with a RegEx to match collection names. For example:
 
-`reait -n --embedding my_func.json --collections "(libc.*|lib.*crypt.*)"`
+```shell
+reait -n --embedding my_func.json --collections "(libc.*|lib.*crypt.*)"
+```
 
 RevEng.AI collections are sets of pre-analysed executable objects. To create custom collection sets e.g., malware collections, please create a RevEng.AI account.
 
@@ -82,47 +94,25 @@ Find common components between binaries, RevEng.AI collections, or global search
 
 Example usage: 
 
-```
+```shell
 reait -M -b 05ff897f430fec0ac17f14c89181c76961993506e5875f2987e9ead13bec58c2.exe --from-file 755a4b2ec15da6bb01248b2dfbad206c340ba937eae9c35f04f6cedfe5e99d63.embeddings.json --confidence high
 ```
 
 ### RevEng.AI embedding models
 To use specific RevEng.AI AI models, or for training custom models, use `-m` to specify the model. The default option is to use the latest development model. Available models are `binnet-0.1` and `dexter`.
 
-`reait -b /usr/bin/true -m dexter -a`
+```shell
+reait -b /usr/bin/true -m dexter -a
+```
 
 ### Software Composition Analysis
 To identify known open source software components embedded inside a binary, use the `-C` flag.
 
-#### Stripped Binary CVE Checker
-To check for known vulnerabilities found with embedded software components, use `-c` or `--cves`.
-
-
-### REAI Signatures
-To generate an AI functional description of an entire binary file, use the `-s` flag. This will return the REAI signature of the file.
-
-REAI signatures can be used to compute the binary similarity between entire executables with the `-S` flag. For example:
-
-```
-reait -b d24ccf73aabca4192d33a07b4a238c8d40ac97a550c2e65b8074f03455a981ca.exe -S -t 00062cb01088cea245cd5f3eb03f65a0e6b11a8126ce00034d87935a451cf99c.exe,438d64bb831555caadaa92a32c9d62e255001bc8d524721c885f37d750ec3476.exe,755a4b2ec15da6bb01248b2dfbad206c340ba937eae9c35f04f6cedfe5e99d63.exe,05ff897f430fec0ac17f14c89181c76961993506e5875f2987e9ead13bec58c2.exe
-Computing Binary Similarity... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:01
-                      Binary Similarity to RedlineInfoStealer/d24ccf73aabca4192d33a07b4a238c8d40ac97a550c2e65b8074f03455a981ca.exe                      
-┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
-┃                                                               Binary ┃ SHA3-256                                                         ┃ Similarity ┃
-┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
-│ 00062cb01088cea245cd5f3eb03f65a0e6b11a8126ce00034d87935a451cf99c.exe │ 00062cb01088cea245cd5f3eb03f65a0e6b11a8126ce00034d87935a451cf99c │ 0.99907    │
-│ 438d64bb831555caadaa92a32c9d62e255001bc8d524721c885f37d750ec3476.exe │ 438d64bb831555caadaa92a32c9d62e255001bc8d524721c885f37d750ec3476 │ 1.00000    │
-│ 755a4b2ec15da6bb01248b2dfbad206c340ba937eae9c35f04f6cedfe5e99d63.exe │ 755a4b2ec15da6bb01248b2dfbad206c340ba937eae9c35f04f6cedfe5e99d63 │ 0.80522    │
-│ 05ff897f430fec0ac17f14c89181c76961993506e5875f2987e9ead13bec58c2.exe │ 05ff897f430fec0ac17f14c89181c76961993506e5875f2987e9ead13bec58c2 │ 0.94701    │
-└──────────────────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────┴────────────┘
-```
-
 
 ### Binary ANN Search
-
 To perform binary ANN search, pass in `-n` and `-s` flag at the same time. For example:
 
-```
+```shell
 reait -b /usr/bin/true -s -n
 Found /usr/bin/true:elf-x86_64
 [
@@ -151,13 +141,12 @@ Found /usr/bin/true:elf-x86_64
 
 
 ## Configuration
-
 `reait` reads the config file stored at `~/.reait.toml`. An example config file looks like:
 
-```
+```shell
 apikey = "l1br3"
 host = "https://api.reveng.ai"
-model = "binnet-0.1"
+model = "binnet-0.3-x86"
 ```
 
 ## Contact

{reait-0.0.19 → reait-1.0.0}/pyproject.toml

@@ -1,10 +1,10 @@
 [build-system]
-requires = ["setuptools >= 40.9.0", "tqdm", "argparse", "requests", "rich", "tomli", "pandas", "numpy", "scipy", "lief", "scikit-learn"]
+requires = ["setuptools >= 40.9.0", "argparse", "requests", "rich", "tomli", "pandas", "numpy", "scipy", "lief", "scikit-learn"]
 build-backend = "setuptools.build_meta"
 
 [project]
 name = "reait"
-version = "0.0.19"
+version = "1.0.0"
 readme = "README.md"
 classifiers=[
   "Programming Language :: Python :: 3",
@@ -12,7 +12,6 @@ classifiers=[
   "Operating System :: OS Independent"
 ]
 dependencies = [
-    "tqdm",
     "requests",
     "rich",
     "argparse",
@@ -39,7 +38,5 @@ maintainers = [
 "Organisation Homepage" = "https://reveng.ai"
 "Documentation" = "https://docs.reveng.ai"
 
-
-
 [project.scripts]
 reait = "reait.main:main"

reait-1.0.0/setup.py

@@ -0,0 +1,31 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+from setuptools import setup, find_packages
+
+
+with open("requirements.txt") as fd:
+    required = fd.read().splitlines()
+
+with open("README.md", encoding="utf-8") as fd:
+    long_description = fd.read()
+
+
+setup(
+    name="reait",
+    long_description=long_description,
+    long_description_content_type="text/markdown",
+    url="https://github.com/RevEng-AI/reait",
+    author="James Patrick-Evans",
+    author_email="james@reveng.ai",
+    platforms="Cross Platform",
+    packages=find_packages(where="src", exclude=["tests",]),
+    package_dir={
+        "": "src",
+    },
+    classifiers=[
+        "Operating System :: OS Independent",
+        "Programming Language :: Python :: 3",
+        "License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
+    ],
+    install_requires=required,
+)

{reait-0.0.19 → reait-1.0.0}/src/reait/__init__.py

@@ -1,2 +1,4 @@
+# -*- coding: utf-8 -*-
 from reait import api
+
 api.parse_config()