eth-portfolio 1.1.0__tar.gz

eth_portfolio-1.1.0/PKG-INFO

@@ -0,0 +1,174 @@
+Metadata-Version: 2.2
+Name: eth-portfolio
+Version: 1.1.0
+Summary: eth-portfolio makes it easy to analyze your portfolio.
+Home-page: https://github.com/BobTheBuidler/eth-portfolio
+Author: BobTheBuidler
+Author-email: bobthebuidlerdefi@gmail.com
+Project-URL: Homepage, https://github.com/BobTheBuidler/eth-portfolio
+Project-URL: Documentation, https://bobthebuidler.github.io/eth-portfolio
+Project-URL: Source Code, https://github.com/BobTheBuidler/eth-portfolio
+Description-Content-Type: text/markdown
+Requires-Dist: checksum_dict>=1.1.2
+Requires-Dist: dank_mids>=4.20.109
+Requires-Dist: eth-brownie<1.21,>=1.19.3
+Requires-Dist: eth_retry<1,>=0.1.15
+Requires-Dist: evmspec>=0.2.0
+Requires-Dist: ez-a-sync>=0.24.43
+Requires-Dist: numpy<2
+Requires-Dist: pandas<1.6,>=1.4.3
+Requires-Dist: ypricemagic<5,>=4.0.57
+Dynamic: author
+Dynamic: author-email
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: project-url
+Dynamic: requires-dist
+Dynamic: summary
+
+# eth-portfolio
+Use `eth-portfolio` to output information about your portfolio in a streamlined, speed-optimized way.
+
+### NOTES:
+- This lib is still a WIP and the provided API is subject to change without notice.
+
+### INSTALLATION:
+- You should start with a fresh virtual environment, and then just...
+```
+pip install git+https://github.com/BobTheBuidler/eth-portfolio
+```
+
+**Known Issues**  
+Make sure you are using Python >= 3.8 and < 3.13
+If you have a PyYaml Issue with 3.4.1 not installing due to an issue with cython, try the following:  
+```
+pip install wheel
+pip install --no-build-isolation "Cython<3" "pyyaml==5.4.1"
+```
+then try again
+`
+pip install git+https://github.com/BobTheBuidler/eth-portfolio
+`
+
+
+### USE:
+For basic use, input each of your addresses as environment variables using the following pattern:
+```
+PORTFOLIO_ADDRESS_0=0x123...
+PORTFOLIO_ADDRESS_1=0x234...
+PORTFOLIO_ADDRESS_2=0x345...
+```
+
+Then do...
+```
+from eth_portfolio import portfolio
+portfolio.eth_balance(block)
+
+>>> {
+        0xaddress0: _BalanceItem(balance=1234, usd_value=5678)
+        0xaddress1: _BalanceItem(balance=1234, usd_value=5678)
+        0xaddress2: _BalanceItem(balance=1234, usd_value=5678)
+    }
+```
+
+Getting token transfers:
+```
+from eth_portfolio import portfolio
+token_transfers = portfolio.token_transfers.get(start_block, end_block)
+token_transfers.df()
+
+>>> {
+        0xaddress0: AddressTokenTransfersLedger(...)  # Each of these contains the token transfers for the specified address
+        0xaddress1: AddressTokenTransfersLedger(...)
+        0xaddress2: AddressTokenTransfersLedger(...)
+    }
+```
+
+Getting transactions as a DataFrame:
+```
+from eth_portfolio import portfolio
+txs = portfolio.transactions.get(start_block, end_block)
+txs.df()
+
+>>> [I am a pretend DataFrame]
+```
+
+Getting assets:
+```
+from eth_portfolio import portfolio
+portfolio.describe(start_block, end_block)
+
+>>>  {
+        'assets': {
+            'wallet0_address': {
+                'token0': {
+                    'amount': 123,
+                    'value usd: 456,
+                },
+                'token1': {
+                    'amount': 123,
+                    'value usd: 456,
+                },
+            },
+            'wallet0_address': {
+                'token0': {
+                    'amount': 123,
+                    'value usd: 456,
+                },
+                'token1': {
+                    'amount': 123,
+                    'value usd: 456,
+                },
+            },
+        },
+        'debt': {
+            'wallet0_address': {
+                'token0': {
+                    'amount': 123,
+                    'value usd: 456,
+                },
+                'token1': {
+                    'amount': 123,
+                    'value usd: 456,
+                },
+            },
+            'wallet1_address': {
+                'token0': {
+                    'amount': 123,
+                    'value usd: 456,
+                },
+                'token1': {
+                    'amount': 123,
+                    'value usd: 456,
+                },
+            },
+        },
+    }
+```
+
+Getting net worth:
+```
+from eth_portfolio import portfolio
+desc = portfolio.describe(block)
+assets = desc['assets']     # OR you can do `assets = portfolio.assets(block)`
+debt = desc['debt']         # OR you can do `debt = portfolio.debt(block)`
+assets = sum(assets.values())
+debt = sum(debt.values())
+net = assets - debt
+net.sum_usd()
+
+>>> Decimal("123456.78900")
+```
+
+### ADVANCED:
+You also have more granular control available using the Portfolio object:
+```
+from eth_portfolio import Portfolio
+port = Portfolio([0xaddress0, 0xaddress1, 0xaddress2])
+port.describe(chain.height)
+
+# Or for async code
+async_port = Portfolio([0xaddress0, 0xaddress1, 0xaddress2], asynchronous=True)
+await port.describe(chain.height)
+```

eth_portfolio-1.1.0/README.md

@@ -0,0 +1,145 @@
+# eth-portfolio
+Use `eth-portfolio` to output information about your portfolio in a streamlined, speed-optimized way.
+
+### NOTES:
+- This lib is still a WIP and the provided API is subject to change without notice.
+
+### INSTALLATION:
+- You should start with a fresh virtual environment, and then just...
+```
+pip install git+https://github.com/BobTheBuidler/eth-portfolio
+```
+
+**Known Issues**  
+Make sure you are using Python >= 3.8 and < 3.13
+If you have a PyYaml Issue with 3.4.1 not installing due to an issue with cython, try the following:  
+```
+pip install wheel
+pip install --no-build-isolation "Cython<3" "pyyaml==5.4.1"
+```
+then try again
+`
+pip install git+https://github.com/BobTheBuidler/eth-portfolio
+`
+
+
+### USE:
+For basic use, input each of your addresses as environment variables using the following pattern:
+```
+PORTFOLIO_ADDRESS_0=0x123...
+PORTFOLIO_ADDRESS_1=0x234...
+PORTFOLIO_ADDRESS_2=0x345...
+```
+
+Then do...
+```
+from eth_portfolio import portfolio
+portfolio.eth_balance(block)
+
+>>> {
+        0xaddress0: _BalanceItem(balance=1234, usd_value=5678)
+        0xaddress1: _BalanceItem(balance=1234, usd_value=5678)
+        0xaddress2: _BalanceItem(balance=1234, usd_value=5678)
+    }
+```
+
+Getting token transfers:
+```
+from eth_portfolio import portfolio
+token_transfers = portfolio.token_transfers.get(start_block, end_block)
+token_transfers.df()
+
+>>> {
+        0xaddress0: AddressTokenTransfersLedger(...)  # Each of these contains the token transfers for the specified address
+        0xaddress1: AddressTokenTransfersLedger(...)
+        0xaddress2: AddressTokenTransfersLedger(...)
+    }
+```
+
+Getting transactions as a DataFrame:
+```
+from eth_portfolio import portfolio
+txs = portfolio.transactions.get(start_block, end_block)
+txs.df()
+
+>>> [I am a pretend DataFrame]
+```
+
+Getting assets:
+```
+from eth_portfolio import portfolio
+portfolio.describe(start_block, end_block)
+
+>>>  {
+        'assets': {
+            'wallet0_address': {
+                'token0': {
+                    'amount': 123,
+                    'value usd: 456,
+                },
+                'token1': {
+                    'amount': 123,
+                    'value usd: 456,
+                },
+            },
+            'wallet0_address': {
+                'token0': {
+                    'amount': 123,
+                    'value usd: 456,
+                },
+                'token1': {
+                    'amount': 123,
+                    'value usd: 456,
+                },
+            },
+        },
+        'debt': {
+            'wallet0_address': {
+                'token0': {
+                    'amount': 123,
+                    'value usd: 456,
+                },
+                'token1': {
+                    'amount': 123,
+                    'value usd: 456,
+                },
+            },
+            'wallet1_address': {
+                'token0': {
+                    'amount': 123,
+                    'value usd: 456,
+                },
+                'token1': {
+                    'amount': 123,
+                    'value usd: 456,
+                },
+            },
+        },
+    }
+```
+
+Getting net worth:
+```
+from eth_portfolio import portfolio
+desc = portfolio.describe(block)
+assets = desc['assets']     # OR you can do `assets = portfolio.assets(block)`
+debt = desc['debt']         # OR you can do `debt = portfolio.debt(block)`
+assets = sum(assets.values())
+debt = sum(debt.values())
+net = assets - debt
+net.sum_usd()
+
+>>> Decimal("123456.78900")
+```
+
+### ADVANCED:
+You also have more granular control available using the Portfolio object:
+```
+from eth_portfolio import Portfolio
+port = Portfolio([0xaddress0, 0xaddress1, 0xaddress2])
+port.describe(chain.height)
+
+# Or for async code
+async_port = Portfolio([0xaddress0, 0xaddress1, 0xaddress2], asynchronous=True)
+await port.describe(chain.height)
+```

eth_portfolio-1.1.0/eth_portfolio/__init__.py

@@ -0,0 +1,16 @@
+import a_sync._smart
+
+a_sync._smart.set_smart_task_factory()
+
+
+from eth_portfolio.portfolio import Portfolio, portfolio
+
+# make sure we init the extended db before we init ypm somewhere
+from eth_portfolio._db import utils
+from eth_portfolio._shitcoins import SHITCOINS
+
+__all__ = [
+    "Portfolio",
+    "portfolio",
+    "SHITCOINS",
+]

eth_portfolio-1.1.0/eth_portfolio/_argspec.py

@@ -0,0 +1,42 @@
+from inspect import getfullargspec
+from typing import Any, Callable, List, Tuple, Type
+
+# WIP:
+
+
+def get_args_type(sample: Callable) -> Tuple[Type, ...]:
+    argspec = getfullargspec(sample)
+    args = {
+        arg_name: argspec.annotations[arg_name] if arg_name in argspec.annotations else Any
+        for arg_name in argspec.args
+        if arg_name != "self"
+    }
+    return tuple(*args.values())
+
+
+def get_kwargs_type(sample: Callable) -> Tuple[Type, ...]:
+    argspec = getfullargspec(sample)
+    num_kwargs = len(argspec.args) - len(argspec.defaults or [])
+    kwarg_names = argspec.args[num_kwargs:]
+    kwargs = {
+        kwarg_name: argspec.annotations[kwarg_name] if kwarg_name in argspec.annotations else Any
+        for kwarg_name in kwarg_names
+    }
+    _kwarg_types: List[Type[object]] = list(*kwargs.values())
+    if num_kwargs == 1:
+        return Tuple[_kwarg_types[0]]  # type: ignore [valid-type,return-value]
+    elif num_kwargs == 2:
+        return Tuple[_kwarg_types[0], _kwarg_types[1]]  # type: ignore [misc,return-value]
+    elif num_kwargs == 3:
+        return Tuple[_kwarg_types[0], _kwarg_types[1], _kwarg_types[2]]  # type: ignore [misc,return-value]
+    else:
+        return Any  # type: ignore [misc,return-value]
+
+
+def get_return_type(sample: Callable) -> Type:
+    argspec = getfullargspec(sample)
+    return argspec.annotations["return"] if "return" in argspec.annotations else Any
+
+
+def get_types(sample: Callable) -> Tuple[Type, Type, Type]:
+    return get_args_type(sample), get_kwargs_type(sample), get_return_type(sample)  # type: ignore [return-value]

eth_portfolio-1.1.0/eth_portfolio/_cache.py

@@ -0,0 +1,116 @@
+import functools
+import inspect
+from asyncio import PriorityQueue, Task, current_task, get_event_loop
+from concurrent.futures import Executor
+from hashlib import md5
+from logging import getLogger
+from os import makedirs
+from os.path import exists, join
+from pickle import dumps, load, loads
+from random import random
+from typing import Any, Callable, List, NoReturn
+
+from a_sync import PruningThreadPoolExecutor
+from a_sync._typing import P, T
+from a_sync.asyncio import create_task
+from a_sync.primitives.queue import log_broken
+from aiofiles import open as _aio_open
+from brownie import chain
+
+BASE_PATH = f"./cache/{chain.id}/"
+_THREAD_NAME_PREFIX = "eth-portfolio-cache-decorator"
+_EXISTS_EXECUTOR = PruningThreadPoolExecutor(8, f"{_THREAD_NAME_PREFIX}-exists")
+
+
+def cache_to_disk(fn: Callable[P, T]) -> Callable[P, T]:
+    # sourcery skip: use-contextlib-suppress
+    name = fn.__name__
+    cache_path_for_fn = f"{BASE_PATH}{fn.__module__.replace('.', '/')}/{name}"
+    logger = getLogger(f"eth_portfolio.cache_to_disk.{name}")
+
+    def get_cache_file_path(args, kwargs):
+        # Create a unique filename based on the function arguments
+        key = md5(dumps((args, sorted(kwargs.items())))).hexdigest()
+        return join(cache_path_for_fn, f"{key}.json")
+
+    write_executor = PruningThreadPoolExecutor(8, f"{_THREAD_NAME_PREFIX}-{fn.__qualname__}-write")
+
+    makedirs(cache_path_for_fn, exist_ok=True)
+
+    if inspect.iscoroutinefunction(fn):
+        read_executor = PruningThreadPoolExecutor(
+            8, f"{_THREAD_NAME_PREFIX}-{fn.__qualname__}-read"
+        )
+
+        queue = PriorityQueue()
+
+        async def cache_deco_worker_coro(func) -> NoReturn:
+            try:
+                while True:
+                    _, fut, cache_path, args, kwargs = await queue.get()
+                    try:
+                        async with _aio_open(cache_path, "rb", executor=read_executor) as f:
+                            fut.set_result(loads(await f.read()))
+                    except Exception as e:
+                        fut.set_exception(e)
+            except Exception as e:
+                logger.error("%s for %s is broken!!!", current_task(), func)
+                logger.exception(e)
+                raise
+
+        workers: List[Task[NoReturn]] = []
+
+        @functools.wraps(fn)
+        async def disk_cache_wrap(*args: P.args, **kwargs: P.kwargs) -> T:
+            cache_path = get_cache_file_path(args, kwargs)
+            if await _EXISTS_EXECUTOR.run(exists, cache_path):
+                fut = get_event_loop().create_future()
+                # we intentionally mix up the order to break up heavy load block ranges
+                queue.put_nowait((random(), fut, cache_path, args, kwargs))
+                if not workers:
+                    workers.extend(create_task(cache_deco_worker_coro(fn)) for _ in range(100))
+                try:
+                    return await fut
+                except EOFError:
+                    pass
+
+            async_result: T = await fn(*args, **kwargs)
+            try:
+                await __cache_write(cache_path, async_result, write_executor)
+            except OSError as e:
+                # I was having some weird issues in docker that I don't want to debug,
+                # so I'm going to assume you have another means to let you know you're
+                # out of disk space and will pass right on through here so my script
+                # can continue
+                if e.strerror != "No space left on device":
+                    raise
+            return async_result
+
+    else:
+
+        @functools.wraps(fn)
+        def disk_cache_wrap(*args: P.args, **kwargs: P.kwargs) -> T:
+            cache_path = get_cache_file_path(args, kwargs)
+            try:
+                with open(cache_path, "rb") as f:
+                    return load(f)
+            except (FileNotFoundError, EOFError):
+                pass
+
+            sync_result: T = fn(*args, **kwargs)  # type: ignore [assignment, return-value]
+            try:
+                create_task(
+                    coro=__cache_write(cache_path, sync_result, write_executor),
+                    skip_gc_until_done=True,
+                )
+            except RuntimeError:
+                pass
+            return sync_result
+
+    return disk_cache_wrap
+
+
+async def __cache_write(cache_path: str, result: Any, executor: Executor) -> None:
+    result = dumps(result)
+    async with _aio_open(cache_path, "wb", executor=executor) as f:
+        await f.write(result)

eth_portfolio-1.1.0/eth_portfolio/_config.py

@@ -0,0 +1,3 @@
+import os
+
+REORG_BUFFER = int(os.environ.get("REORG_BUFFER", 30))

eth_portfolio-1.1.0/eth_portfolio/_db/decorators.py

@@ -0,0 +1,147 @@
+from asyncio import iscoroutinefunction
+from asyncio import sleep as aio_sleep
+from functools import wraps
+from logging import DEBUG, getLogger
+from random import random
+from time import sleep as time_sleep
+from typing import Callable, TypeVar
+
+from a_sync._typing import AnyFn
+from pony.orm import OperationalError, TransactionError
+from typing_extensions import ParamSpec
+
+
+P = ParamSpec("P")
+T = TypeVar("T")
+
+
+logger = getLogger(__name__)
+__logger_is_enabled_for = logger.isEnabledFor
+__logger_warning = logger.warning
+__logger_log = logger._log
+
+
+def break_locks(fn: AnyFn[P, T]) -> AnyFn[P, T]:
+    """
+    Decorator to handle database lock errors by retrying the function.
+
+    This decorator is designed to wrap functions that interact with a database
+    and may encounter `OperationalError` due to database locks. It will retry
+    the function indefinitely if a "database is locked" error occurs. After
+    5 attempts, a warning is logged, but the function will continue to retry
+    until it succeeds or a non-lock-related error occurs.
+
+    Args:
+        fn: The function to be wrapped, which may be a coroutine or a regular function.
+
+    Examples:
+        Basic usage with a regular function:
+
+        >>> @break_locks
+        ... def my_function():
+        ...     # Function logic that may encounter a database lock
+        ...     pass
+
+        Basic usage with an asynchronous function:
+
+        >>> @break_locks
+        ... async def my_async_function():
+        ...     # Async function logic that may encounter a database lock
+        ...     pass
+
+    See Also:
+        - :func:`pony.orm.db_session`: For managing database sessions.
+    """
+    if iscoroutinefunction(fn):
+
+        @wraps(fn)
+        async def break_locks_wrap(*args: P.args, **kwargs: P.kwargs) -> T:
+            debug_logs_enabled = None
+            tries = 0
+            while True:
+                try:
+                    return await fn(*args, **kwargs)
+                except OperationalError as e:
+                    if str(e) != "database is locked":
+                        raise e
+
+                    if debug_logs_enabled is None:
+                        debug_logs_enabled = __logger_is_enabled_for(DEBUG)
+
+                    if debug_logs_enabled is True:
+                        __logger_log(DEBUG, "%s.%s %s", (fn.__module__, fn.__name__, e))
+
+                    await aio_sleep(tries * random())
+                    tries += 1
+                    if tries > 5:
+                        __logger_warning("%s caught in err loop with %s", fn, e)
+
+    else:
+
+        @wraps(fn)
+        def break_locks_wrap(*args: P.args, **kwargs: P.kwargs) -> T:
+            debug_logs_enabled = None
+            tries = 0
+            while True:
+                try:
+                    return fn(*args, **kwargs)  # type: ignore [return-value]
+                except OperationalError as e:
+                    if str(e) != "database is locked":
+                        raise e
+
+                    if debug_logs_enabled is None:
+                        debug_logs_enabled = __logger_is_enabled_for(DEBUG)
+
+                    if debug_logs_enabled is True:
+                        __logger_log(DEBUG, "%s.%s %s", (fn.__module__, fn.__name__, e))
+
+                    time_sleep(tries * random())
+                    tries += 1
+                    if tries > 5:
+                        __logger_warning("%s caught in err loop with %s", fn, e)
+
+    return break_locks_wrap
+
+
+def requery_objs_on_diff_tx_err(fn: Callable[P, T]) -> Callable[P, T]:
+    """
+    Decorator to handle transaction errors by retrying the function.
+
+    This decorator is designed to wrap functions that may encounter
+    `TransactionError` due to mixing objects from different transactions.
+    It will retry the function until it succeeds or a non-transaction-related
+    error occurs.
+
+    Args:
+        fn: The function to be wrapped, which must not be a coroutine.
+
+    Raises:
+        TypeError: If the function is a coroutine.
+
+    Examples:
+        Basic usage with a function that may encounter transaction errors:
+
+        >>> @requery_objs_on_diff_tx_err
+        ... def my_function():
+        ...     # Function logic that may encounter a transaction error
+        ...     pass
+
+    See Also:
+        - :func:`pony.orm.db_session`: For managing database sessions.
+    """
+    if iscoroutinefunction(fn):
+        raise TypeError(f"{fn} must not be async")
+
+    @wraps(fn)
+    def requery_wrap(*args: P.args, **kwargs: P.kwargs) -> T:
+        while True:
+            try:
+                return fn(*args, **kwargs)
+            except TransactionError as e:
+                if str(e) != "An attempt to mix objects belonging to different transactions":
+                    raise e
+                # The error occurs if you committed new objs to the db and started a new transaction while still inside of a `db_session`,
+                # and then tried to use the newly committed objects in the next transaction. Now that the objects are in the db this will
+                # not reoccur. The next iteration will be successful.
+
+    return requery_wrap