graphsense-python 2.13.5__tar.gz → 2.15.1__tar.gz

{graphsense_python-2.13.5 → graphsense_python-2.15.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: graphsense-python
-Version: 2.13.5
+Version: 2.15.1
 Summary: GraphSense API
 Author-email: Iknaio Cryptoasset Analytics GmbH <contact@iknaio.com>
 License-Expression: MIT
@@ -58,11 +58,13 @@ your behalf.
 To use it, add that `/mcp` URL to an MCP-capable assistant as a new
 connector or tool source; the assistant then handles the rest.
 
+**Authentication:** the MCP endpoint is protected by OAuth. When adding the connector in your assistant, use the client ID `iknaio-mcp`.
+
 
 This Python package is automatically generated by the [OpenAPI Generator](https://openapi-generator.tech) project:
 
-- API version: 2.13.5
-- Package version: 2.13.5
+- API version: 2.15.1
+- Package version: 2.15.1
 - Generator version: 7.19.0
 - Build package: org.openapitools.codegen.languages.PythonClientCodegen
 For more information, please visit [https://www.iknaio.com/](https://www.iknaio.com/)
@@ -188,6 +190,8 @@ Class | Method | HTTP request | Description
 *EntitiesApi* | [**search_entity_neighbors**](docs/EntitiesApi.md#search_entity_neighbors) | **GET** /{currency}/entities/{entity}/search | Search entity neighborhood
 *GeneralApi* | [**get_statistics**](docs/GeneralApi.md#get_statistics) | **GET** /stats | Get platform statistics for supported currencies
 *GeneralApi* | [**search**](docs/GeneralApi.md#search) | **GET** /search | Search addresses, transactions, actors, and labels
+*GraphApi* | [**graph_compare**](docs/GraphApi.md#graph_compare) | **POST** /graph/compare | Compare multiple transactions (beta)
+*GraphApi* | [**graph_summary**](docs/GraphApi.md#graph_summary) | **POST** /graph/summary | Summarize a set of transactions and/or addresses (beta)
 *RatesApi* | [**get_exchange_rates**](docs/RatesApi.md#get_exchange_rates) | **GET** /{currency}/rates/{height} | Get fiat exchange rates at a block height
 *TagsApi* | [**get_actor**](docs/TagsApi.md#get_actor) | **GET** /tags/actors/{actor} | Get actor details by ID
 *TagsApi* | [**get_actor_tags**](docs/TagsApi.md#get_actor_tags) | **GET** /tags/actors/{actor}/tags | List tags associated with an actor
@@ -230,6 +234,25 @@ Class | Method | HTTP request | Description
  - [Entity](docs/Entity.md)
  - [EntityAddresses](docs/EntityAddresses.md)
  - [ExternalConversion](docs/ExternalConversion.md)
+ - [GraphAddressNetworkSummary](docs/GraphAddressNetworkSummary.md)
+ - [GraphAddressOverall](docs/GraphAddressOverall.md)
+ - [GraphAddressRef](docs/GraphAddressRef.md)
+ - [GraphAddressSummary](docs/GraphAddressSummary.md)
+ - [GraphCompareNote](docs/GraphCompareNote.md)
+ - [GraphCompareRequest](docs/GraphCompareRequest.md)
+ - [GraphCompareSignal](docs/GraphCompareSignal.md)
+ - [GraphCompareVerdict](docs/GraphCompareVerdict.md)
+ - [GraphComparedTx](docs/GraphComparedTx.md)
+ - [GraphComparison](docs/GraphComparison.md)
+ - [GraphLineageEdge](docs/GraphLineageEdge.md)
+ - [GraphNote](docs/GraphNote.md)
+ - [GraphSummary](docs/GraphSummary.md)
+ - [GraphSummaryRequest](docs/GraphSummaryRequest.md)
+ - [GraphTxCharacteristics](docs/GraphTxCharacteristics.md)
+ - [GraphTxNetworkSummary](docs/GraphTxNetworkSummary.md)
+ - [GraphTxOverall](docs/GraphTxOverall.md)
+ - [GraphTxRef](docs/GraphTxRef.md)
+ - [GraphTxSummary](docs/GraphTxSummary.md)
  - [HTTPValidationError](docs/HTTPValidationError.md)
  - [JoinMarketHeuristic](docs/JoinMarketHeuristic.md)
  - [LabelSummary](docs/LabelSummary.md)
@@ -247,6 +270,7 @@ Class | Method | HTTP request | Description
  - [NeighborEntities](docs/NeighborEntities.md)
  - [NeighborEntity](docs/NeighborEntity.md)
  - [OneTimeChangeHeuristic](docs/OneTimeChangeHeuristic.md)
+ - [PerTxInner](docs/PerTxInner.md)
  - [Rate](docs/Rate.md)
  - [Rates](docs/Rates.md)
  - [RelatedAddress](docs/RelatedAddress.md)

{graphsense_python-2.13.5 → graphsense_python-2.15.1}/README.md

@@ -36,11 +36,13 @@ your behalf.
 To use it, add that `/mcp` URL to an MCP-capable assistant as a new
 connector or tool source; the assistant then handles the rest.
 
+**Authentication:** the MCP endpoint is protected by OAuth. When adding the connector in your assistant, use the client ID `iknaio-mcp`.
+
 
 This Python package is automatically generated by the [OpenAPI Generator](https://openapi-generator.tech) project:
 
-- API version: 2.13.5
-- Package version: 2.13.5
+- API version: 2.15.1
+- Package version: 2.15.1
 - Generator version: 7.19.0
 - Build package: org.openapitools.codegen.languages.PythonClientCodegen
 For more information, please visit [https://www.iknaio.com/](https://www.iknaio.com/)
@@ -166,6 +168,8 @@ Class | Method | HTTP request | Description
 *EntitiesApi* | [**search_entity_neighbors**](docs/EntitiesApi.md#search_entity_neighbors) | **GET** /{currency}/entities/{entity}/search | Search entity neighborhood
 *GeneralApi* | [**get_statistics**](docs/GeneralApi.md#get_statistics) | **GET** /stats | Get platform statistics for supported currencies
 *GeneralApi* | [**search**](docs/GeneralApi.md#search) | **GET** /search | Search addresses, transactions, actors, and labels
+*GraphApi* | [**graph_compare**](docs/GraphApi.md#graph_compare) | **POST** /graph/compare | Compare multiple transactions (beta)
+*GraphApi* | [**graph_summary**](docs/GraphApi.md#graph_summary) | **POST** /graph/summary | Summarize a set of transactions and/or addresses (beta)
 *RatesApi* | [**get_exchange_rates**](docs/RatesApi.md#get_exchange_rates) | **GET** /{currency}/rates/{height} | Get fiat exchange rates at a block height
 *TagsApi* | [**get_actor**](docs/TagsApi.md#get_actor) | **GET** /tags/actors/{actor} | Get actor details by ID
 *TagsApi* | [**get_actor_tags**](docs/TagsApi.md#get_actor_tags) | **GET** /tags/actors/{actor}/tags | List tags associated with an actor
@@ -208,6 +212,25 @@ Class | Method | HTTP request | Description
  - [Entity](docs/Entity.md)
  - [EntityAddresses](docs/EntityAddresses.md)
  - [ExternalConversion](docs/ExternalConversion.md)
+ - [GraphAddressNetworkSummary](docs/GraphAddressNetworkSummary.md)
+ - [GraphAddressOverall](docs/GraphAddressOverall.md)
+ - [GraphAddressRef](docs/GraphAddressRef.md)
+ - [GraphAddressSummary](docs/GraphAddressSummary.md)
+ - [GraphCompareNote](docs/GraphCompareNote.md)
+ - [GraphCompareRequest](docs/GraphCompareRequest.md)
+ - [GraphCompareSignal](docs/GraphCompareSignal.md)
+ - [GraphCompareVerdict](docs/GraphCompareVerdict.md)
+ - [GraphComparedTx](docs/GraphComparedTx.md)
+ - [GraphComparison](docs/GraphComparison.md)
+ - [GraphLineageEdge](docs/GraphLineageEdge.md)
+ - [GraphNote](docs/GraphNote.md)
+ - [GraphSummary](docs/GraphSummary.md)
+ - [GraphSummaryRequest](docs/GraphSummaryRequest.md)
+ - [GraphTxCharacteristics](docs/GraphTxCharacteristics.md)
+ - [GraphTxNetworkSummary](docs/GraphTxNetworkSummary.md)
+ - [GraphTxOverall](docs/GraphTxOverall.md)
+ - [GraphTxRef](docs/GraphTxRef.md)
+ - [GraphTxSummary](docs/GraphTxSummary.md)
  - [HTTPValidationError](docs/HTTPValidationError.md)
  - [JoinMarketHeuristic](docs/JoinMarketHeuristic.md)
  - [LabelSummary](docs/LabelSummary.md)
@@ -225,6 +248,7 @@ Class | Method | HTTP request | Description
  - [NeighborEntities](docs/NeighborEntities.md)
  - [NeighborEntity](docs/NeighborEntity.md)
  - [OneTimeChangeHeuristic](docs/OneTimeChangeHeuristic.md)
+ - [PerTxInner](docs/PerTxInner.md)
  - [Rate](docs/Rate.md)
  - [Rates](docs/Rates.md)
  - [RelatedAddress](docs/RelatedAddress.md)

{graphsense_python-2.13.5 → graphsense_python-2.15.1}/graphsense/__init__.py

@@ -5,14 +5,14 @@
 """
     GraphSense API
 
-    GraphSense API provides programmatic access to blockchain analytics data across multiple ledgers. Use it to explore addresses, clusters, blocks, transactions, tags, token activity, and exchange-rate context, and to integrate investigation workflows into your own applications and automation.  ## Versioning & deprecation  The API follows semantic versioning. Minor releases are additive and backwards-compatible; breaking changes only happen in major releases, which are rare and announced in advance.  Deprecated endpoints and fields remain fully functional for at least six months after they are marked deprecated. During that window they are highlighted with a strikethrough in the docs and in generated clients, and responses from deprecated endpoints carry a `Deprecation` HTTP header that client tooling can detect. Replacement endpoints and fields are always introduced before the deprecated surface is removed.  See the [full versioning and deprecation policy](https://github.com/graphsense/graphsense-lib/blob/master/README.md#rest-api-evolution-and-deprecation-policy) for details.  ## AI assistant access (MCP)  In addition to this REST API, this deployment exposes a **Model Context Protocol (MCP)** interface that lets AI assistants query GraphSense directly in natural language, without writing code against the endpoints below.  The MCP interface is available at the `/mcp` path of this server, relative to the address this documentation is served from. It offers a curated selection of the most useful operations, so an assistant can answer questions about addresses, transactions, clusters, and tags on your behalf.  To use it, add that `/mcp` URL to an MCP-capable assistant as a new connector or tool source; the assistant then handles the rest.   # noqa: E501
+    GraphSense API provides programmatic access to blockchain analytics data across multiple ledgers. Use it to explore addresses, clusters, blocks, transactions, tags, token activity, and exchange-rate context, and to integrate investigation workflows into your own applications and automation.  ## Versioning & deprecation  The API follows semantic versioning. Minor releases are additive and backwards-compatible; breaking changes only happen in major releases, which are rare and announced in advance.  Deprecated endpoints and fields remain fully functional for at least six months after they are marked deprecated. During that window they are highlighted with a strikethrough in the docs and in generated clients, and responses from deprecated endpoints carry a `Deprecation` HTTP header that client tooling can detect. Replacement endpoints and fields are always introduced before the deprecated surface is removed.  See the [full versioning and deprecation policy](https://github.com/graphsense/graphsense-lib/blob/master/README.md#rest-api-evolution-and-deprecation-policy) for details.  ## AI assistant access (MCP)  In addition to this REST API, this deployment exposes a **Model Context Protocol (MCP)** interface that lets AI assistants query GraphSense directly in natural language, without writing code against the endpoints below.  The MCP interface is available at the `/mcp` path of this server, relative to the address this documentation is served from. It offers a curated selection of the most useful operations, so an assistant can answer questions about addresses, transactions, clusters, and tags on your behalf.  To use it, add that `/mcp` URL to an MCP-capable assistant as a new connector or tool source; the assistant then handles the rest.  **Authentication:** the MCP endpoint is protected by OAuth. When adding the connector in your assistant, use the client ID `iknaio-mcp`.   # noqa: E501
 
     Contact: contact@iknaio.com
     Generated by: https://openapi-generator.tech
 """
 
 
-__version__ = "2.13.5"
+__version__ = "2.15.1"
 
 # Define package exports
 __all__ = [
@@ -22,6 +22,7 @@ __all__ = [
     "ClustersApi",
     "EntitiesApi",
     "GeneralApi",
+    "GraphApi",
     "RatesApi",
     "TagsApi",
     "TokensApi",
@@ -58,6 +59,25 @@ __all__ = [
     "Entity",
     "EntityAddresses",
     "ExternalConversion",
+    "GraphAddressNetworkSummary",
+    "GraphAddressOverall",
+    "GraphAddressRef",
+    "GraphAddressSummary",
+    "GraphCompareNote",
+    "GraphCompareRequest",
+    "GraphCompareSignal",
+    "GraphCompareVerdict",
+    "GraphComparedTx",
+    "GraphComparison",
+    "GraphLineageEdge",
+    "GraphNote",
+    "GraphSummary",
+    "GraphSummaryRequest",
+    "GraphTxCharacteristics",
+    "GraphTxNetworkSummary",
+    "GraphTxOverall",
+    "GraphTxRef",
+    "GraphTxSummary",
     "HTTPValidationError",
     "JoinMarketHeuristic",
     "LabelSummary",
@@ -75,6 +95,7 @@ __all__ = [
     "NeighborEntities",
     "NeighborEntity",
     "OneTimeChangeHeuristic",
+    "PerTxInner",
     "Rate",
     "Rates",
     "RelatedAddress",
@@ -120,6 +141,7 @@ from graphsense.api.bulk_api import BulkApi as BulkApi
 from graphsense.api.clusters_api import ClustersApi as ClustersApi
 from graphsense.api.entities_api import EntitiesApi as EntitiesApi
 from graphsense.api.general_api import GeneralApi as GeneralApi
+from graphsense.api.graph_api import GraphApi as GraphApi
 from graphsense.api.rates_api import RatesApi as RatesApi
 from graphsense.api.tags_api import TagsApi as TagsApi
 from graphsense.api.tokens_api import TokensApi as TokensApi
@@ -160,6 +182,25 @@ from graphsense.models.direct_change_heuristic import DirectChangeHeuristic as D
 from graphsense.models.entity import Entity as Entity
 from graphsense.models.entity_addresses import EntityAddresses as EntityAddresses
 from graphsense.models.external_conversion import ExternalConversion as ExternalConversion
+from graphsense.models.graph_address_network_summary import GraphAddressNetworkSummary as GraphAddressNetworkSummary
+from graphsense.models.graph_address_overall import GraphAddressOverall as GraphAddressOverall
+from graphsense.models.graph_address_ref import GraphAddressRef as GraphAddressRef
+from graphsense.models.graph_address_summary import GraphAddressSummary as GraphAddressSummary
+from graphsense.models.graph_compare_note import GraphCompareNote as GraphCompareNote
+from graphsense.models.graph_compare_request import GraphCompareRequest as GraphCompareRequest
+from graphsense.models.graph_compare_signal import GraphCompareSignal as GraphCompareSignal
+from graphsense.models.graph_compare_verdict import GraphCompareVerdict as GraphCompareVerdict
+from graphsense.models.graph_compared_tx import GraphComparedTx as GraphComparedTx
+from graphsense.models.graph_comparison import GraphComparison as GraphComparison
+from graphsense.models.graph_lineage_edge import GraphLineageEdge as GraphLineageEdge
+from graphsense.models.graph_note import GraphNote as GraphNote
+from graphsense.models.graph_summary import GraphSummary as GraphSummary
+from graphsense.models.graph_summary_request import GraphSummaryRequest as GraphSummaryRequest
+from graphsense.models.graph_tx_characteristics import GraphTxCharacteristics as GraphTxCharacteristics
+from graphsense.models.graph_tx_network_summary import GraphTxNetworkSummary as GraphTxNetworkSummary
+from graphsense.models.graph_tx_overall import GraphTxOverall as GraphTxOverall
+from graphsense.models.graph_tx_ref import GraphTxRef as GraphTxRef
+from graphsense.models.graph_tx_summary import GraphTxSummary as GraphTxSummary
 from graphsense.models.http_validation_error import HTTPValidationError as HTTPValidationError
 from graphsense.models.join_market_heuristic import JoinMarketHeuristic as JoinMarketHeuristic
 from graphsense.models.label_summary import LabelSummary as LabelSummary
@@ -177,6 +218,7 @@ from graphsense.models.neighbor_clusters import NeighborClusters as NeighborClus
 from graphsense.models.neighbor_entities import NeighborEntities as NeighborEntities
 from graphsense.models.neighbor_entity import NeighborEntity as NeighborEntity
 from graphsense.models.one_time_change_heuristic import OneTimeChangeHeuristic as OneTimeChangeHeuristic
+from graphsense.models.per_tx_inner import PerTxInner as PerTxInner
 from graphsense.models.rate import Rate as Rate
 from graphsense.models.rates import Rates as Rates
 from graphsense.models.related_address import RelatedAddress as RelatedAddress

graphsense_python-2.15.1/graphsense/address_scan/__init__.py

@@ -0,0 +1,11 @@
+# AUTO-GENERATED — DO NOT EDIT.
+# Synced from src/graphsenselib/convert/address_scan/__init__.py via
+# clients/python/scripts/sync_address_scan.py. Edit the source and re-run
+# `make -C clients/python sync-address-scan`.
+"""Scan text/SQL files (and compressed containers) for cryptocurrency addresses.
+
+Extraction is deliberately permissive; the real filter is checksum validation
+(reusing :mod:`graphsenselib.utils.address`), which removes the many
+address-shaped false positives found in database dumps (hashes, session ids,
+base64 blobs, filenames).
+"""

graphsense_python-2.15.1/graphsense/address_scan/decompress.py

@@ -0,0 +1,175 @@
+# AUTO-GENERATED — DO NOT EDIT.
+# Synced from src/graphsenselib/convert/address_scan/decompress.py via
+# clients/python/scripts/sync_address_scan.py. Edit the source and re-run
+# `make -C clients/python sync-address-scan`.
+"""Transparently unwrap compressed containers before scanning.
+
+Addresses often hide inside compressed data (.gz dumps, zip exports, or raw
+zlib/gzip blobs embedded in a binary). We unwrap known containers, recurse
+(so .tar.gz works), and optionally carve embedded streams. A byte budget guards
+against decompression bombs.
+
+Supported: gzip, zlib, bz2, xz, zip, tar (recursively), GraphSense ``.gs`` save
+files (LZW + base64 + JSON, decoded via
+:func:`graphsenselib.convert.gs_files.parser.lzw_unpack`), and -- with
+``carve=True`` -- zlib/gzip streams embedded anywhere in a binary.
+"""
+
+from __future__ import annotations
+
+import base64
+import bz2
+import io
+import lzma
+import re
+import struct
+import tarfile
+import zipfile
+import zlib
+from typing import Iterator
+
+from graphsense.gs_files.parser import lzw_unpack
+
+MAX_DEPTH = 6
+DEFAULT_BUDGET = 1024 * 1024 * 1024  # 1 GiB of decompressed output per file
+GS_AUTO_MAX = 64 * 1024 * 1024  # only auto-probe .gs on smallish files
+
+
+def _decode_gs(data: bytes, name: str) -> "bytes | None":
+    """Return the inner JSON bytes of a GraphSense ``.gs`` file, or None.
+
+    Headerless format, so we validate structurally: size is a non-zero multiple
+    of 4, the first LZW code is a base-dictionary char (<256), and the payload
+    base64-decodes to something that starts like JSON. That combination makes
+    false positives on arbitrary binaries essentially nil.
+    """
+    if not data or len(data) % 4 != 0:
+        return None
+    forced = name.lower().endswith(".gs")
+    if not forced and len(data) > GS_AUTO_MAX:
+        return None  # avoid an expensive LZW pass on large non-.gs blobs
+    codes = struct.unpack(f"<{len(data) // 4}I", data)
+    if codes[0] >= 256:
+        return None
+    try:
+        payload = base64.b64decode(lzw_unpack(list(codes)), validate=True)
+    except Exception:
+        return None
+    return payload if payload.lstrip()[:1] in (b"[", b"{") else None
+
+
+def _bounded(decompressor: object, data: bytes, limit: int) -> bytes:
+    """Streaming-decompress ``data``, stopping once ``limit`` bytes are produced."""
+    out = bytearray()
+    step = 1 << 20
+    try:
+        for pos in range(0, len(data), step):
+            out += decompressor.decompress(data[pos : pos + step])  # type: ignore[attr-defined]
+            if len(out) >= limit:
+                return bytes(out[:limit])
+        flush = getattr(decompressor, "flush", None)
+        if callable(flush):
+            out += flush()
+    except Exception:
+        pass  # truncated/garbage tail -> keep whatever decoded so far
+    return bytes(out[:limit])
+
+
+def _is_zlib(data: bytes) -> bool:
+    return (
+        len(data) > 2 and (data[0] & 0x0F) == 8 and ((data[0] << 8) | data[1]) % 31 == 0
+    )
+
+
+def _decompress_layer(
+    data: bytes, name: str, budget: list[int]
+) -> Iterator[tuple[str, bytes]]:
+    """Yield (label, bytes) for each container format ``data`` unwraps into."""
+    if budget[0] <= 0:
+        return
+    limit = budget[0]
+
+    if data[:2] == b"\x1f\x8b":  # gzip
+        out = _bounded(zlib.decompressobj(zlib.MAX_WBITS | 16), data, limit)
+        if out:
+            budget[0] -= len(out)
+            yield f"{name} :: gzip", out
+    elif _is_zlib(data):  # raw zlib
+        out = _bounded(zlib.decompressobj(), data, limit)
+        if out:
+            budget[0] -= len(out)
+            yield f"{name} :: zlib", out
+    elif data[:3] == b"BZh":  # bzip2
+        out = _bounded(bz2.BZ2Decompressor(), data, limit)
+        if out:
+            budget[0] -= len(out)
+            yield f"{name} :: bz2", out
+    elif data[:6] == b"\xfd7zXZ\x00":  # xz
+        out = _bounded(lzma.LZMADecompressor(), data, limit)
+        if out:
+            budget[0] -= len(out)
+            yield f"{name} :: xz", out
+
+    if data[:4] in (b"PK\x03\x04", b"PK\x05\x06"):  # zip
+        try:
+            with zipfile.ZipFile(io.BytesIO(data)) as zf:
+                for info in zf.infolist():
+                    if info.is_dir() or budget[0] <= 0:
+                        continue
+                    with zf.open(info) as fh:
+                        out = fh.read(budget[0] + 1)[: budget[0]]
+                    budget[0] -= len(out)
+                    yield f"{name} :: zip[{info.filename}]", out
+        except Exception:
+            pass
+    elif tarfile.is_tarfile(io.BytesIO(data)):  # tar (incl. plain, not gz here)
+        try:
+            with tarfile.open(fileobj=io.BytesIO(data)) as tf:
+                for member in tf:
+                    if not member.isfile() or budget[0] <= 0:
+                        continue
+                    fh = tf.extractfile(member)
+                    if fh is None:
+                        continue
+                    out = fh.read(budget[0] + 1)[: budget[0]]
+                    budget[0] -= len(out)
+                    yield f"{name} :: tar[{member.name}]", out
+        except Exception:
+            pass
+
+    gs = _decode_gs(data, name)  # GraphSense .gs (LZW + base64 + JSON)
+    if gs is not None and budget[0] > 0:
+        gs = gs[: budget[0]]
+        budget[0] -= len(gs)
+        yield f"{name} :: gs/lzw", gs
+
+
+def _carve_zlib(
+    data: bytes, name: str, budget: list[int]
+) -> Iterator[tuple[str, bytes]]:
+    """Best-effort: find gzip/zlib streams anywhere in ``data`` and inflate them."""
+    seen = 0
+    for m in re.finditer(rb"\x1f\x8b\x08|\x78[\x01\x9c\xda]", data):
+        if budget[0] <= 0 or seen >= 200:
+            break
+        off = m.start()
+        wbits = zlib.MAX_WBITS | 16 if data[off] == 0x1F else zlib.MAX_WBITS
+        out = _bounded(zlib.decompressobj(wbits), data[off:], budget[0])
+        if len(out) >= 16:  # ignore trivially short / bogus inflations
+            budget[0] -= len(out)
+            seen += 1
+            yield f"{name} :: carved@{off}", out
+
+
+def iter_streams(
+    data: bytes, name: str, carve: bool, budget: list[int], depth: int = 0
+) -> Iterator[tuple[str, bytes]]:
+    """Yield the raw file plus every decompressed stream, recursively."""
+    yield name, data
+    if depth >= MAX_DEPTH:
+        return
+    for label, inner in _decompress_layer(data, name, budget):
+        yield from iter_streams(inner, label, carve, budget, depth + 1)
+    if carve:
+        for label, inner in _carve_zlib(data, name, budget):
+            yield from iter_streams(inner, label, carve, budget, depth + 1)

graphsense_python-2.15.1/graphsense/address_scan/detectors.py

@@ -0,0 +1,124 @@
+# AUTO-GENERATED — DO NOT EDIT.
+# Synced from src/graphsenselib/convert/address_scan/detectors.py via
+# clients/python/scripts/sync_address_scan.py. Edit the source and re-run
+# `make -C clients/python sync-address-scan`.
+"""Regex detectors and validators for the address scanner.
+
+Each detector pairs a permissive regex (to *extract* address-shaped runs) with
+a validator (to *confirm* them). The validators reuse the checksum logic in
+:mod:`graphsenselib.utils.address` wherever possible; only XMR (no cheap
+checksum) and the opt-in tx-hash candidates are format-level heuristics.
+
+Supported with full checksum validation:
+  - Bitcoin  : legacy (1.../3...), bech32/bech32m (bc1...)
+  - Litecoin : legacy (L/M...), bech32 (ltc1...)
+  - Ethereum : 0x + 40 hex (EIP-55 mixed-case checksum verified when present)
+  - TRON     : T... base58check
+  - Zcash    : transparent t1/t3 base58check
+  - XRP      : classic r-addresses (base58check, ripple alphabet)
+Reported as candidates only (no cheap checksum):
+  - Monero   : 4.../8... 95-char addresses
+"""
+
+from __future__ import annotations
+
+from typing import Callable, NamedTuple, Optional
+
+from graphsense.address_scan.validators import validate_address, validate_xrp_address
+
+
+def decodes_to_text(hex_body: str, threshold: float = 0.7) -> bool:
+    """True if ``hex_body`` decodes mostly to printable ASCII.
+
+    SQL dumps store blobs as ``0x...``/hex literals; a run that decodes to
+    readable text is encoded content (e.g. stored HTML/JS), not an address.
+    Real ETH/XMR addresses are effectively random bytes and fail this test.
+    """
+    try:
+        raw = bytes.fromhex(hex_body)
+    except ValueError:
+        return False
+    if not raw:
+        return False
+    printable = sum(1 for b in raw if 0x20 <= b <= 0x7E)
+    return printable / len(raw) >= threshold
+
+
+def _eth_valid(addr: str) -> bool:
+    # Drop hex-encoded text blobs before the (stricter) EIP-55 check.
+    if decodes_to_text(addr[2:]):
+        return False
+    return validate_address("eth", addr)
+
+
+def _xmr_candidate_valid(s: str) -> bool:
+    """Monero has no cheap checksum, so reject the common false positives:
+    pure-hex strings (SQL blob literals) and hex that decodes to text."""
+    if all(c in "0123456789abcdefABCDEF" for c in s):
+        return False  # a real base58 Monero address is ~never all-hex
+    if decodes_to_text(s):
+        return False
+    return True
+
+
+def txhash_valid(s: str) -> bool:
+    """A tx hash is 32 random bytes with NO checksum -- unverifiable by form.
+    We can only drop the obvious non-hashes: hex that decodes to ASCII text."""
+    return not decodes_to_text(s[2:] if s[:2].lower() == "0x" else s)
+
+
+class Detector(NamedTuple):
+    label: str
+    pattern: str
+    validator: Optional[Callable[[str], bool]]
+
+
+# Lookarounds keep matches from being slices of a longer hex/base58 run.
+_H = r"[a-fA-F0-9]"
+
+DETECTORS: list[Detector] = [
+    Detector(
+        "BTC legacy/P2SH",
+        r"[13][1-9A-HJ-NP-Za-km-z]{25,34}",
+        lambda m: validate_address("btc", m),
+    ),
+    Detector(
+        "BTC bech32", r"bc1[0-9ac-hj-np-z]{11,87}", lambda m: validate_address("btc", m)
+    ),
+    Detector(
+        "LTC legacy",
+        r"[LM][1-9A-HJ-NP-Za-km-z]{25,34}",
+        lambda m: validate_address("ltc", m),
+    ),
+    Detector(
+        "LTC bech32",
+        r"ltc1[0-9ac-hj-np-z]{11,87}",
+        lambda m: validate_address("ltc", m),
+    ),
+    Detector("ETH", rf"(?<!{_H})0x{_H}{{40}}(?!{_H})", _eth_valid),
+    Detector("TRX", r"T[1-9A-HJ-NP-Za-km-z]{33}", lambda m: validate_address("trx", m)),
+    Detector(
+        "ZEC transparent",
+        r"t[13][1-9A-HJ-NP-Za-km-z]{33}",
+        lambda m: validate_address("zec", m),
+    ),
+    Detector("XRP", r"r[1-9A-HJ-NP-Za-km-z]{24,34}", validate_xrp_address),
+    Detector(
+        "XMR (candidate)", r"[48][0-9AB][1-9A-HJ-NP-Za-km-z]{93}", _xmr_candidate_valid
+    ),
+]
+
+# Opt-in only (--tx-hashes): format-level, NOT checksum-verifiable. Every 64-hex
+# digest (SHA-256 file hashes, tokens, etc.) matches, so results are candidates.
+# The 0x form is matched first; the bare form excludes an 'x'-prefixed run so a
+# single 0x<64hex> is not double-counted.
+TX_DETECTORS: list[Detector] = [
+    Detector(
+        "TX-hash candidate (0x+64hex)", rf"(?<!{_H})0x{_H}{{64}}(?!{_H})", txhash_valid
+    ),
+    Detector(
+        "TX-hash candidate (64hex)",
+        rf"(?<![a-fA-F0-9xX]){_H}{{64}}(?!{_H})",
+        txhash_valid,
+    ),
+]