firegex 2.5.3__tar.gz → 3.0.0__tar.gz

firegex-3.0.0/PKG-INFO

@@ -0,0 +1,170 @@
+Metadata-Version: 2.2
+Name: firegex
+Version: 3.0.0
+Summary: Firegex client
+Home-page: https://github.com/pwnzer0tt1/firegex
+Author: Pwnzer0tt1
+Author-email: pwnzer0tt1@poliba.it
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: typer==0.15.2
+Requires-Dist: pydantic>=2
+Requires-Dist: typing-extensions>=4.7.1
+Requires-Dist: watchfiles
+Requires-Dist: fgex
+Requires-Dist: pyllhttp
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+# Firegex Python Library and CLI
+
+This is the Python library for Firegex. It is used to get additional features of Firegex and use the feature of the command `fgex`.
+
+## Installation
+
+```bash
+pip install -U firegex
+```
+
+fgex is an alias of firegex. You can use `fgex` instead of `firegex`.
+
+## Command line usage:
+
+```text
+➤ fgex nfproxy -h
+
+ Usage: fgex nfproxy [OPTIONS] FILTER_FILE ADDRESS PORT
+
+ Run an nfproxy simulation
+
+╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
+│ *    filter_file      TEXT     The path to the filter file [default: None] [required]                                                               │
+│ *    address          TEXT     The address of the target to proxy [default: None] [required]                                                        │
+│ *    port             INTEGER  The port of the target to proxy [default: None] [required]                                                           │
+╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
+│ --proto                 [tcp|http]  The protocol to proxy [default: tcp]                                                                            │
+│ --from-address          TEXT        The address of the local server [default: None]                                                                 │
+│ --from-port             INTEGER     The port of the local server [default: 7474]                                                                    │
+│                 -6                  Use IPv6 for the connection                                                                                     │
+│ --help          -h                  Show this message and exit.                                                                                     │
+╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+```
+
+## Library usage:
+
+## NfProxy decorator
+
+```python
+from firegex.nfproxy import pyfilter
+```
+This decorator is used to create a filter for the nfproxy.
+Example:
+```python
+@pyfilter
+def my_filter(raw_packet: RawPacket): #Logging filter
+    print(raw_packet.data)
+```
+
+## Data handlers
+
+### RawPacket 
+```python
+from firegex.nfproxy import RawPacket
+```
+This handler will be called every time arrives a packet from the network. It will receive a RawPacket object with the following properties:
+- is_input: bool - It's true if the packet is an input packet, false if it's an output packet
+- is_ipv6: bool - It's true if the packet is an ipv6 packet, false if it's an ipv4 packet
+- is_tcp: bool - It's true if the packet is a tcp packet, false if it's an udp packet
+- data: bytes - The data of the packet assembled and sorted from TCP
+- l4_size: int - The size of the layer 4 data
+- raw_packet_header_len: int - The size of the original packet header
+- l4_data: bytes - The layer 4 payload of the packet
+- raw_packet: bytes - The raw packet with IP and TCP headers
+
+### TCPInputStream
+Alias: TCPClientStream
+```python
+from firegex.nfproxy import TCPInputStream
+```
+This handler will be called every time a TCP stream is assembled in input. It will receive a TCPInputStream object with the following properties:
+- data: bytes - The data of the packets assembled and sorted from TCP
+- is_ipv6: bool - It's true if the packet is an ipv6 packet, false if it's an ipv4 packet
+- total_stream_size: int - The size of the stream
+
+### TCPOutputStream
+Alias: TCPServerStream
+```python
+from firegex.nfproxy import TCPOutputStream
+```
+This handler will be called every time a TCP stream is assembled in output. It will receive a TCPOutputStream object with the following properties:
+- data: bytes - The data of the packets assembled and sorted from TCP
+- is_ipv6: bool - It's true if the packet is an ipv6 packet, false if it's an ipv4 packet
+- total_stream_size: int - The size of the stream
+
+### HttpRequest
+```python
+from firegex.nfproxy import HttpRequest
+```
+This handler will be called twice: one for the request headers and one for the request body. It will receive a HttpRequest object with the following properties:
+- method: bytes - The method of the request
+- url: str - The url of the request
+- headers: dict - The headers of the request
+- user_agent: str - The user agent of the request
+- content_encoding: str - The content encoding of the request
+- has_begun: bool - It's true if the request has begun
+- body: bytes - The body of the request
+- headers_complete: bool - It's true if the headers are complete
+- message_complete: bool - It's true if the message is complete
+- http_version: str - The http version of the request
+- keep_alive: bool - It's true if the request should keep alive
+- should_upgrade: bool - It's true if the request should upgrade
+- content_length: int - The content length of the request
+- get_header(header: str, default=None): str - Get a header from the request without caring about the case
+- total_size: int - The total size of the stream
+- stream: bytes - The stream of the request
+
+### HttpRequestHeader
+```python
+from firegex.nfproxy import HttpRequestHeader
+```
+This handler will be called only when the request headers are complete. It will receive a HttpRequestHeader object with the same properties as HttpRequest.
+
+### HttpResponse
+```python
+from firegex.nfproxy import HttpResponse
+```
+This handler will be called twice: one for the response headers and one for the response body. It will receive a HttpResponse object with the following properties:
+- status_code: int - The status code of the response
+- url: str - The url of the response
+- headers: dict - The headers of the response
+- user_agent: str - The user agent of the response
+- content_encoding: str - The content encoding of the response
+- has_begun: bool - It's true if the response has begun
+- body: bytes - The body of the response
+- headers_complete: bool - It's true if the headers are complete
+- message_complete: bool - It's true if the message is complete
+- http_version: str - The http version of the response
+- keep_alive: bool - It's true if the response should keep alive
+- should_upgrade: bool - It's true if the response should upgrade
+- content_length: int - The content length of the response
+- get_header(header: str, default=None): str - Get a header from the response without caring about the case
+- total_size: int - The total size of the stream
+- stream: bytes - The stream of the response
+
+### HttpResponseHeader
+```python
+from firegex.nfproxy import HttpResponseHeader
+```
+This handler will be called only when the response headers are complete. It will receive a HttpResponseHeader object with the same properties as HttpResponse.
+

firegex-3.0.0/README.md

@@ -0,0 +1,142 @@
+# Firegex Python Library and CLI
+
+This is the Python library for Firegex. It is used to get additional features of Firegex and use the feature of the command `fgex`.
+
+## Installation
+
+```bash
+pip install -U firegex
+```
+
+fgex is an alias of firegex. You can use `fgex` instead of `firegex`.
+
+## Command line usage:
+
+```text
+➤ fgex nfproxy -h
+
+ Usage: fgex nfproxy [OPTIONS] FILTER_FILE ADDRESS PORT
+
+ Run an nfproxy simulation
+
+╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
+│ *    filter_file      TEXT     The path to the filter file [default: None] [required]                                                               │
+│ *    address          TEXT     The address of the target to proxy [default: None] [required]                                                        │
+│ *    port             INTEGER  The port of the target to proxy [default: None] [required]                                                           │
+╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
+│ --proto                 [tcp|http]  The protocol to proxy [default: tcp]                                                                            │
+│ --from-address          TEXT        The address of the local server [default: None]                                                                 │
+│ --from-port             INTEGER     The port of the local server [default: 7474]                                                                    │
+│                 -6                  Use IPv6 for the connection                                                                                     │
+│ --help          -h                  Show this message and exit.                                                                                     │
+╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+```
+
+## Library usage:
+
+## NfProxy decorator
+
+```python
+from firegex.nfproxy import pyfilter
+```
+This decorator is used to create a filter for the nfproxy.
+Example:
+```python
+@pyfilter
+def my_filter(raw_packet: RawPacket): #Logging filter
+    print(raw_packet.data)
+```
+
+## Data handlers
+
+### RawPacket 
+```python
+from firegex.nfproxy import RawPacket
+```
+This handler will be called every time arrives a packet from the network. It will receive a RawPacket object with the following properties:
+- is_input: bool - It's true if the packet is an input packet, false if it's an output packet
+- is_ipv6: bool - It's true if the packet is an ipv6 packet, false if it's an ipv4 packet
+- is_tcp: bool - It's true if the packet is a tcp packet, false if it's an udp packet
+- data: bytes - The data of the packet assembled and sorted from TCP
+- l4_size: int - The size of the layer 4 data
+- raw_packet_header_len: int - The size of the original packet header
+- l4_data: bytes - The layer 4 payload of the packet
+- raw_packet: bytes - The raw packet with IP and TCP headers
+
+### TCPInputStream
+Alias: TCPClientStream
+```python
+from firegex.nfproxy import TCPInputStream
+```
+This handler will be called every time a TCP stream is assembled in input. It will receive a TCPInputStream object with the following properties:
+- data: bytes - The data of the packets assembled and sorted from TCP
+- is_ipv6: bool - It's true if the packet is an ipv6 packet, false if it's an ipv4 packet
+- total_stream_size: int - The size of the stream
+
+### TCPOutputStream
+Alias: TCPServerStream
+```python
+from firegex.nfproxy import TCPOutputStream
+```
+This handler will be called every time a TCP stream is assembled in output. It will receive a TCPOutputStream object with the following properties:
+- data: bytes - The data of the packets assembled and sorted from TCP
+- is_ipv6: bool - It's true if the packet is an ipv6 packet, false if it's an ipv4 packet
+- total_stream_size: int - The size of the stream
+
+### HttpRequest
+```python
+from firegex.nfproxy import HttpRequest
+```
+This handler will be called twice: one for the request headers and one for the request body. It will receive a HttpRequest object with the following properties:
+- method: bytes - The method of the request
+- url: str - The url of the request
+- headers: dict - The headers of the request
+- user_agent: str - The user agent of the request
+- content_encoding: str - The content encoding of the request
+- has_begun: bool - It's true if the request has begun
+- body: bytes - The body of the request
+- headers_complete: bool - It's true if the headers are complete
+- message_complete: bool - It's true if the message is complete
+- http_version: str - The http version of the request
+- keep_alive: bool - It's true if the request should keep alive
+- should_upgrade: bool - It's true if the request should upgrade
+- content_length: int - The content length of the request
+- get_header(header: str, default=None): str - Get a header from the request without caring about the case
+- total_size: int - The total size of the stream
+- stream: bytes - The stream of the request
+
+### HttpRequestHeader
+```python
+from firegex.nfproxy import HttpRequestHeader
+```
+This handler will be called only when the request headers are complete. It will receive a HttpRequestHeader object with the same properties as HttpRequest.
+
+### HttpResponse
+```python
+from firegex.nfproxy import HttpResponse
+```
+This handler will be called twice: one for the response headers and one for the response body. It will receive a HttpResponse object with the following properties:
+- status_code: int - The status code of the response
+- url: str - The url of the response
+- headers: dict - The headers of the response
+- user_agent: str - The user agent of the response
+- content_encoding: str - The content encoding of the response
+- has_begun: bool - It's true if the response has begun
+- body: bytes - The body of the response
+- headers_complete: bool - It's true if the headers are complete
+- message_complete: bool - It's true if the message is complete
+- http_version: str - The http version of the response
+- keep_alive: bool - It's true if the response should keep alive
+- should_upgrade: bool - It's true if the response should upgrade
+- content_length: int - The content length of the response
+- get_header(header: str, default=None): str - Get a header from the response without caring about the case
+- total_size: int - The total size of the stream
+- stream: bytes - The stream of the response
+
+### HttpResponseHeader
+```python
+from firegex.nfproxy import HttpResponseHeader
+```
+This handler will be called only when the response headers are complete. It will receive a HttpResponseHeader object with the same properties as HttpResponse.
+

{firegex-2.5.3 → firegex-3.0.0}/fgex

@@ -1,6 +1,5 @@
 #!/usr/bin/env python3
 
-# TODO implement cli start function
 from firegex.cli import run
 
 if __name__ == "__main__":

firegex-3.0.0/firegex/__init__.py

@@ -0,0 +1,5 @@
+
+__version__ = "3.0.0" if "{" not in "3.0.0" else "0.0.0"
+
+#Exported functions
+__all__ = []

{firegex-2.5.3 → firegex-3.0.0}/firegex/__main__.py

@@ -1,6 +1,5 @@
 #!/usr/bin/env python3
 
-# TODO implement cli start function
 from firegex.cli import run
 
 if __name__ == "__main__":

firegex-3.0.0/firegex/cli.py

@@ -0,0 +1,76 @@
+
+#!/usr/bin/env python3
+
+import typer
+from rich import print
+from rich.markup import escape
+from typer import Exit
+from firegex import __version__
+from firegex.nfproxy.proxysim import run_proxy_simulation
+from firegex.nfproxy.models import Protocols
+import os
+import socket
+
+app = typer.Typer(
+    no_args_is_help=True,
+    context_settings={"help_option_names": ["-h", "--help"]}
+)
+
+def close_cli(code:int=1):
+    raise Exit(code)
+
+DEV_MODE = __version__ == "0.0.0"
+
+def test_connection(host, port, use_ipv6=False):
+    family = socket.AF_INET6 if use_ipv6 else socket.AF_INET
+    sock = socket.socket(family, socket.SOCK_STREAM)
+    
+    try:
+        sock.settimeout(3)
+        sock.connect((host, port))
+        return True
+    except Exception:
+        return False
+    finally:
+        sock.close()
+
+@app.command(help="Run an nfproxy simulation")
+def nfproxy(
+    filter_file: str = typer.Argument(..., help="The path to the filter file"),
+    address: str = typer.Argument(..., help="The address of the target to proxy"),
+    port: int = typer.Argument(..., help="The port of the target to proxy"),
+    
+    proto: Protocols = typer.Option(Protocols.TCP.value, help="The protocol to proxy"),
+    from_address: str = typer.Option(None, help="The address of the local server"),
+    from_port: int = typer.Option(7474, help="The port of the local server"),    
+    ipv6: bool = typer.Option(False, "-6", help="Use IPv6 for the connection"),
+):
+    if from_address is None:
+        from_address = "::1" if ipv6 else "127.0.0.1"
+    if not os.path.isfile(filter_file):
+        print(f"[bold red]'{escape(os.path.abspath(filter_file))}' not found[/]")
+        close_cli()
+    if not test_connection(address, port, ipv6):
+        print(f"[bold red]Can't connect to {escape(address)}:{port}[/]")
+        close_cli()
+    run_proxy_simulation(filter_file, proto.value, address, port, from_address, from_port, ipv6)
+
+def version_callback(verison: bool):
+    if verison:
+        print(__version__, "Development Mode" if DEV_MODE else "Release")
+        raise typer.Exit()
+
+@app.callback()
+def main(
+    verison: bool = typer.Option(False, "--version", "-v", help="Show the version of the client", callback=version_callback),
+):
+    pass
+
+def run():
+    try:
+        app()
+    except KeyboardInterrupt:
+        print("[bold yellow]Operation cancelled[/]") 
+ 
+if __name__ == "__main__":
+    run()

firegex-3.0.0/firegex/nfproxy/__init__.py

@@ -0,0 +1,39 @@
+import functools
+from firegex.nfproxy.models import RawPacket, TCPInputStream, TCPOutputStream, TCPClientStream, TCPServerStream
+from firegex.nfproxy.internals.models import Action, FullStreamAction
+
+ACCEPT = Action.ACCEPT
+DROP = Action.DROP
+REJECT = Action.REJECT
+UNSTABLE_MANGLE = Action.MANGLE
+
+def pyfilter(func):
+    """
+    Decorator to mark functions that will be used in the proxy.
+    Stores the function reference in a global registry.
+    """
+    if not hasattr(pyfilter, "registry"):
+        pyfilter.registry = set()
+    
+    pyfilter.registry.add(func.__name__)
+    
+    @functools.wraps(func)
+    def wrapper(*args, **kwargs):
+        return func(*args, **kwargs)
+    
+    return wrapper
+
+def get_pyfilters():
+    """Returns the list of functions marked with @pyfilter."""
+    return list(pyfilter.registry)
+
+def clear_pyfilter_registry():
+    """Clears the pyfilter registry."""
+    if hasattr(pyfilter, "registry"):
+        pyfilter.registry.clear()
+
+__all__ = [
+    "ACCEPT", "DROP", "REJECT", "UNSTABLE_MANGLE"
+    "Action", "FullStreamAction", "pyfilter",
+    "RawPacket", "TCPInputStream", "TCPOutputStream", "TCPClientStream", "TCPServerStream"
+]

firegex-3.0.0/firegex/nfproxy/internals/__init__.py

@@ -0,0 +1,160 @@
+from inspect import signature
+from firegex.nfproxy.internals.models import Action, FullStreamAction
+from firegex.nfproxy.internals.models import FilterHandler, PacketHandlerResult
+import functools
+from firegex.nfproxy.internals.data import DataStreamCtx
+from firegex.nfproxy.internals.exceptions import NotReadyToRun, StreamFullReject, DropPacket, RejectConnection, StreamFullDrop
+from firegex.nfproxy.internals.data import RawPacket
+
+def context_call(glob, func, *args, **kargs):
+    glob["__firegex_tmp_args"] = args
+    glob["__firegex_tmp_kargs"] = kargs
+    glob["__firege_tmp_call"] = func
+    res = eval("__firege_tmp_call(*__firegex_tmp_args, **__firegex_tmp_kargs)", glob, glob)
+    if "__firegex_tmp_args" in glob.keys():
+        del glob["__firegex_tmp_args"]
+    if "__firegex_tmp_kargs" in glob.keys():
+        del glob["__firegex_tmp_kargs"]
+    if "__firege_tmp_call" in glob.keys():
+        del glob["__firege_tmp_call"]
+    return res
+
+def generate_filter_structure(filters: list[str], proto:str, glob:dict) -> list[FilterHandler]:
+    from firegex.nfproxy.models import type_annotations_associations
+    if proto not in type_annotations_associations.keys():
+        raise Exception("Invalid protocol")
+    res = []
+    valid_annotation_type = type_annotations_associations[proto]
+    def add_func_to_list(func):
+        if not callable(func):
+            raise Exception(f"{func} is not a function")
+        sig = signature(func)
+        params_function = {}
+        
+        for k, v in sig.parameters.items():
+            if v.annotation in valid_annotation_type.keys():
+                params_function[v.annotation] = valid_annotation_type[v.annotation]
+            else:
+                raise Exception(f"Invalid type annotation {v.annotation} for function {func.__name__}")
+        
+        res.append(
+            FilterHandler(
+                func=func,
+                name=func.__name__,
+                params=params_function,
+                proto=proto
+            )
+        )
+    
+    for filter in filters:
+        if not isinstance(filter, str):
+            raise Exception("Invalid filter list: must be a list of strings")
+        if filter in glob.keys():
+            add_func_to_list(glob[filter])
+        else:
+            raise Exception(f"Filter {filter} not found")
+    return res
+
+def get_filters_info(code:str, proto:str) -> list[FilterHandler]:
+    glob = {}
+    exec("import firegex.nfproxy", glob, glob)
+    exec("firegex.nfproxy.clear_pyfilter_registry()", glob, glob)
+    exec(code, glob, glob)
+    filters = eval("firegex.nfproxy.get_pyfilters()", glob, glob)
+    try:
+        return generate_filter_structure(filters, proto, glob)
+    finally:
+        exec("firegex.nfproxy.clear_pyfilter_registry()", glob, glob)
+        
+
+def get_filter_names(code:str, proto:str) -> list[str]:
+    return [ele.name for ele in get_filters_info(code, proto)]    
+
+def handle_packet(glob: dict) -> None:
+    internal_data = DataStreamCtx(glob)
+    
+    cache_call = {} # Cache of the data handler calls
+    cache_call[RawPacket] = internal_data.current_pkt
+    
+    final_result = Action.ACCEPT
+    result = PacketHandlerResult(glob)
+    
+    func_name = None
+    mangled_packet = None
+    for filter in internal_data.filter_call_info:
+        final_params = []
+        skip_call = False
+        for data_type, data_func in filter.params.items():
+            if data_type not in cache_call.keys():
+                try:
+                    cache_call[data_type] = data_func(internal_data)
+                except NotReadyToRun:
+                    cache_call[data_type] = None
+                    skip_call = True
+                    break
+                except StreamFullDrop:
+                    result.action = Action.DROP
+                    result.matched_by = "@MAX_STREAM_SIZE_REACHED"
+                    return result.set_result()
+                except StreamFullReject:
+                    result.action = Action.REJECT
+                    result.matched_by = "@MAX_STREAM_SIZE_REACHED"
+                    return result.set_result()
+                except DropPacket:
+                    result.action = Action.DROP
+                    result.matched_by = filter.name
+                    return result.set_result()
+                except RejectConnection:
+                    result.action = Action.REJECT
+                    result.matched_by = filter.name
+                    return result.set_result()
+            if cache_call[data_type] is None:
+                skip_call = True
+                break
+            final_params.append(cache_call[data_type])
+            
+        if skip_call:
+            continue
+        
+        res = context_call(glob, filter.func, *final_params)
+        
+        if res is None:
+            continue #ACCEPTED
+        if not isinstance(res, Action):
+            raise Exception(f"Invalid return type {type(res)} for function {filter.name}")
+        if res == Action.MANGLE:
+            mangled_packet = internal_data.current_pkt.raw_packet
+        if res != Action.ACCEPT:
+            func_name = filter.name
+            final_result = res
+            break
+    
+    result.action = final_result
+    result.matched_by = func_name
+    result.mangled_packet = mangled_packet
+    
+    return result.set_result()
+
+
+def compile(glob:dict) -> None:
+    internal_data = DataStreamCtx(glob, init_pkt=False)
+
+    glob["print"] = functools.partial(print, flush = True)
+    
+    filters = glob["__firegex_pyfilter_enabled"]
+    proto = glob["__firegex_proto"]
+    
+    internal_data.filter_call_info = generate_filter_structure(filters, proto, glob)
+
+    if "FGEX_STREAM_MAX_SIZE" in glob and int(glob["FGEX_STREAM_MAX_SIZE"]) > 0:
+        internal_data.stream_max_size = int(glob["FGEX_STREAM_MAX_SIZE"])
+    else:
+        internal_data.stream_max_size = 1*8e20 # 1MB default value
+    
+    if "FGEX_FULL_STREAM_ACTION" in glob and isinstance(glob["FGEX_FULL_STREAM_ACTION"], FullStreamAction):
+        internal_data.full_stream_action = glob["FGEX_FULL_STREAM_ACTION"]
+    else:
+        internal_data.full_stream_action = FullStreamAction.FLUSH
+    
+    PacketHandlerResult(glob).reset_result()
+