cryptofeed 2.4.0__tar.gz → 2.5.0__tar.gz

{cryptofeed-2.4.0 → cryptofeed-2.5.0}/AUTHORS.md

@@ -14,3 +14,8 @@ Cryptofeed was originally created by Bryant Moscon, but many others have contrib
 * [O. Janche](https://github.com/toyan) - <toyan@yandex.ru>
 * [Bastien Enjalbert](https://github.com/bastienjalbert) - <bastienjalbert@gmail.com>
 * [Jonggyun Kim](https://github.com/gyunt) - <truth0233@gmail.com>
+* [QuasarDB](https://quasar.ai/)
+* [Thomas Bouamoud](https://github.com/thomasbs17) - <thomasbs17@yahoo.fr>
+* [Carlo Eugster](https://github.com/carloe) - <carlo@relaun.ch>
+* [Marten Schlüter](https://github.com/maschlr)
+* [Handel Scholze](https://github.com/HandelSM) - <handel.scholze@gmail.com>

{cryptofeed-2.4.0 → cryptofeed-2.5.0}/CHANGES.md

@@ -1,5 +1,28 @@
 ## Changelog
 
+### 2.5.0 (2026-08-08)
+ * Update: Python 3.12+ required, completed move to uv, `setup.py` removed
+ * Update: CI/CD overhaul
+ * Bugfix: various bugs in FeedHandler that prevented it from running on 3.14+
+ * Bugfix: various backend issues that prevented them from working with later versions of Python
+ * Bugfix: Gemini authenticated issue with newer version of websocket library
+ * Bugfix: Binance listenKey refresh now uses aiohttp
+ * Update: Binance per connection stream limit (issue #1087)
+ * Bugfix: Bybit L2_BOOK callback timestamp normalization (issue #1083)
+
+### 2.4.1 (2025-02-08)
+ * Update: Added `is_data_json` to `write()` in `HTTPSync` from `connection.py` to support JSON payloads (#1071)
+ * Bugfix: Handle empty nextFundingRate in OKX
+ * Bugfix: Handle null next_funding_time and estimated_rate in HuobiSwap funding
+ * Update: transitioned from Coinbase Pro (retired) to Coinbase Advanced Trade
+ * Feature: Bybit spot support
+ * Update: Bybit migrate to API V5 for public streams
+ * Bugfix: Handle None ids for Kraken trades in QuestDB
+ * Bugfix: Handle OrderChanged event in IndependentReserve
+ * Bugfix: Remove deprecated `USD` currency from bit.com
+ * Bugfix: Make `entry` key optional when retrieving symbols for BitMex
+ * Update: Changes to work with latest version of websockets
+
 ### 2.4.0 (2024-01-07)
  * Update: Fix tests
  * Update: Okcoin moved to v5 API used by OKX

{cryptofeed-2.4.0 → cryptofeed-2.5.0}/INSTALL.md

@@ -47,6 +47,11 @@ for the exhaustive list of these *extra* dependencies.
 
          pip install --user --upgrade cryptofeed[postgres]
 
+* QuasarDB backend  
+  To install Cryptofeed along with [QuasarDB](https://quasar.ai/) in one bundle:
+
+         pip install --user --upgrade cryptofeed[quasardb]
+
 * RabbitMQ backend
 
          pip install --user --upgrade cryptofeed[rabbit]

{cryptofeed-2.4.0 → cryptofeed-2.5.0}/LICENSE

@@ -1,4 +1,4 @@
-Copyright (C) 2017-2024 Bryant Moscon - bmoscon@gmail.com
+Copyright (C) 2017-2025 Bryant Moscon - bmoscon@gmail.com
 
  Permission is hereby granted, free of charge, to any person obtaining a copy
  of this software and associated documentation files (the "Software"), to

cryptofeed-2.5.0/PKG-INFO

@@ -0,0 +1,257 @@
+Metadata-Version: 2.4
+Name: cryptofeed
+Version: 2.5.0
+Summary: Cryptocurrency Exchange Websocket Data Feed Handler
+Author-email: Bryant Moscon <bmoscon@gmail.com>
+License-Expression: XFree86-1.1
+Project-URL: Homepage, https://github.com/bmoscon/cryptofeed
+Keywords: cryptocurrency,bitcoin,btc,feed handler,market feed,market data,crypto assets,Trades,Tickers,BBO,Funding,Open Interest,Liquidation,Order book,Bid,Ask,fmfw.io,Bitfinex,bitFlyer,AscendEX,Bitstamp,Blockchain.com,Bybit,Binance,Binance Delivery,Binance Futures,Binance US,BitMEX,Coinbase,Deribit,EXX,Gate.io,Gemini,HitBTC,Huobi,Huobi DM,Huobi Swap,Kraken,Kraken Futures,OKCoin,OKX,Poloniex,ProBit,Upbit
+Classifier: Intended Audience :: Developers
+Classifier: Development Status :: 4 - Beta
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Framework :: AsyncIO
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+License-File: LICENSE
+License-File: AUTHORS.md
+Requires-Dist: requests>=2.18.4
+Requires-Dist: websockets>=14.1
+Requires-Dist: pyyaml
+Requires-Dist: aiohttp>=3.11.6
+Requires-Dist: aiofile>=2.0.0
+Requires-Dist: yapic.json>=1.6.3
+Requires-Dist: uvloop; platform_system != "Windows"
+Requires-Dist: order_book>=0.6.0
+Requires-Dist: aiodns>=1.1
+Provides-Extra: arctic
+Requires-Dist: arctic; extra == "arctic"
+Requires-Dist: pandas; extra == "arctic"
+Provides-Extra: gcp-pubsub
+Requires-Dist: google_cloud_pubsub>=2.4.1; extra == "gcp-pubsub"
+Requires-Dist: gcloud_aio_pubsub; extra == "gcp-pubsub"
+Provides-Extra: kafka
+Requires-Dist: aiokafka>=0.7.0; extra == "kafka"
+Provides-Extra: mongo
+Requires-Dist: motor; extra == "mongo"
+Provides-Extra: postgres
+Requires-Dist: asyncpg; extra == "postgres"
+Provides-Extra: quasardb
+Requires-Dist: quasardb; extra == "quasardb"
+Requires-Dist: numpy; extra == "quasardb"
+Provides-Extra: rabbit
+Requires-Dist: aio_pika; extra == "rabbit"
+Requires-Dist: pika; extra == "rabbit"
+Provides-Extra: redis
+Requires-Dist: hiredis; extra == "redis"
+Requires-Dist: redis>=4.5.1; extra == "redis"
+Provides-Extra: zmq
+Requires-Dist: pyzmq; extra == "zmq"
+Provides-Extra: all
+Requires-Dist: arctic; extra == "all"
+Requires-Dist: google_cloud_pubsub>=2.4.1; extra == "all"
+Requires-Dist: gcloud_aio_pubsub; extra == "all"
+Requires-Dist: aiokafka>=0.7.0; extra == "all"
+Requires-Dist: motor; extra == "all"
+Requires-Dist: asyncpg; extra == "all"
+Requires-Dist: aio_pika; extra == "all"
+Requires-Dist: pika; extra == "all"
+Requires-Dist: hiredis; extra == "all"
+Requires-Dist: redis>=4.5.1; extra == "all"
+Requires-Dist: pyzmq; extra == "all"
+Dynamic: license-file
+
+# Cryptocurrency Exchange Feed Handler
+[![License](https://img.shields.io/badge/license-XFree86-blue.svg)](LICENSE)
+![Python](https://img.shields.io/badge/Python-3.12+-green.svg)
+[![PyPi](https://img.shields.io/badge/PyPi-cryptofeed-brightgreen.svg)](https://pypi.python.org/pypi/cryptofeed)
+[![Codacy Badge](https://api.codacy.com/project/badge/Grade/efa4e0d6e10b41d0b51454d08f7b33b1)](https://www.codacy.com/app/bmoscon/cryptofeed?utm_source=github.com&amp;utm_medium=referral&amp;utm_content=bmoscon/cryptofeed&amp;utm_campaign=Badge_Grade)
+
+Handles multiple cryptocurrency exchange data feeds and returns normalized and standardized results to client registered callbacks for events like trades, book updates, ticker updates, etc. Utilizes websockets when possible, but can also poll data via REST endpoints if a websocket is not provided.
+
+## Supported exchanges
+
+* [AscendEX](https://ascendex.com/)
+* [Bequant](https://bequant.io/)
+* [Bitfinex](https://bitfinex.com)
+* [bitFlyer](https://bitflyer.com/)
+* [Bithumb](https://en.bithumb.com/)
+* [Bitstamp](https://www.bitstamp.net/)
+* [Blockchain.com](https://www.blockchain.com/)
+* [Bybit](https://www.bybit.com/)
+* [Binance](https://www.binance.com/en)
+* [Binance Delivery](https://binance-docs.github.io/apidocs/delivery/en/)
+* [Binance Futures](https://www.binance.com/en/futures)
+* [Binance US](https://www.binance.us/en)
+* [Bit.com](https://www.bit.com)
+* [Bitget](https://www.bitget.com/)
+* [BitMEX](https://www.bitmex.com/)
+* [Coinbase](https://www.coinbase.com/)
+* [Crypto.com](https://www.crypto.com)
+* [Delta](https://www.delta.exchange/)
+* [Deribit](https://www.deribit.com/)
+* [dYdX](https://dydx.exchange/)
+* [FMFW.io](https://www.fmfw.io/)
+* [EXX](https://www.exx.com/)
+* [Gate.io](https://www.gate.io/)
+* [Gate.io Futures](https://www.gate.io/futures_center)
+* [Gemini](https://gemini.com/)
+* [HitBTC](https://hitbtc.com/)
+* [Huobi](https://www.hbg.com/)
+* [Huobi DM](https://www.huobi.com/en-us/markets/hb_dm/)
+* Huobi Swap (Coin-M and USDT-M)
+* [Independent Reserve](https://www.independentreserve.com/) 
+* [Kraken](https://www.kraken.com/)
+* [Kraken Futures](https://futures.kraken.com/)
+* [KuCoin](https://www.kucoin.com/)
+* [OKCoin](http://okcoin.com/)
+* [OKX](https://www.okx.com/)
+* [Phemex](https://phemex.com/)
+* [Poloniex](https://www.poloniex.com/)
+* [ProBit](https://www.probit.com/)
+* [Upbit](https://sg.upbit.com/home)
+
+
+## Basic Usage
+
+Create a FeedHandler object and add subscriptions. For the various data channels that an exchange supports, you can supply callbacks for data events, or use provided backends (described below) to handle the data for you. Start the feed handler and you're done!
+
+```python
+from cryptofeed import FeedHandler
+# not all imports shown for clarity
+
+fh = FeedHandler()
+
+# ticker, trade, and book are user defined functions that
+# will be called when ticker, trade and book updates are received
+ticker_cb = {TICKER: ticker}
+trade_cb = {TRADES: trade}
+gemini_cb = {TRADES: trade, L2_BOOK: book}
+
+
+fh.add_feed(Coinbase(symbols=['BTC-USD'], channels=[TICKER], callbacks=ticker_cb))
+fh.add_feed(Bitfinex(symbols=['BTC-USD'], channels=[TICKER], callbacks=ticker_cb))
+fh.add_feed(Poloniex(symbols=['BTC-USDT'], channels=[TRADES], callbacks=trade_cb))
+fh.add_feed(Gemini(symbols=['BTC-USD', 'ETH-USD'], channels=[TRADES, L2_BOOK], callbacks=gemini_cb))
+
+fh.run()
+```
+
+Please see the [examples](https://github.com/bmoscon/cryptofeed/tree/master/examples) for more code samples and the [documentation](https://github.com/bmoscon/cryptofeed/blob/master/docs/README.md) for more information about the library usage.
+
+
+For an example of a containerized application using cryptofeed to store data to a backend, please see [Cryptostore](https://github.com/bmoscon/cryptostore).
+
+
+## National Best Bid/Offer (NBBO)
+
+Cryptofeed also provides a synthetic [NBBO](examples/demo_nbbo.py) (National Best Bid/Offer) feed that aggregates the best bids and asks from the user specified feeds.
+
+```python
+from cryptofeed import FeedHandler
+from cryptofeed.exchanges import Coinbase, Gemini, Kraken
+
+
+def nbbo_update(symbol, bid, bid_size, ask, ask_size, bid_feed, ask_feed):
+    print(f'Pair: {symbol} Bid Price: {bid:.2f} Bid Size: {bid_size:.6f} Bid Feed: {bid_feed} Ask Price: {ask:.2f} Ask Size: {ask_size:.6f} Ask Feed: {ask_feed}')
+
+
+def main():
+    f = FeedHandler()
+    f.add_nbbo([Coinbase, Kraken, Gemini], ['BTC-USD'], nbbo_update)
+    f.run()
+```
+
+## Supported Channels
+
+Cryptofeed supports the following channels from exchanges:
+
+### Market Data Channels (Public)
+
+* L1_BOOK - Top of book
+* L2_BOOK - Price aggregated sizes. Some exchanges provide the entire depth, some provide a subset.
+* L3_BOOK - Price aggregated orders. Like the L2 book, some exchanges may only provide partial depth.
+* TRADES - Note this reports the taker's side, even for exchanges that report the maker side.
+* TICKER
+* FUNDING
+* OPEN_INTEREST - Open interest data.
+* LIQUIDATIONS
+* INDEX
+* CANDLES - Candlestick / K-Line data.
+
+### Authenticated Data Channels
+
+* ORDER_INFO - Order status updates
+* TRANSACTIONS - Real-time updates on account deposits and withdrawals
+* BALANCES - Updates on wallet funds
+* FILLS - User's executed trades
+
+
+## Backends
+
+Cryptofeed supports `backend` callbacks that will write directly to storage or other interfaces.
+
+Supported Backends:
+* Redis (Streams and Sorted Sets)
+* [Arctic](https://github.com/manahl/arctic)
+* ZeroMQ
+* UDP Sockets
+* TCP Sockets
+* Unix Domain Sockets
+* [InfluxDB v2](https://github.com/influxdata/influxdb)
+* MongoDB
+* Kafka
+* RabbitMQ
+* PostgreSQL
+* [QuasarDB](https://quasar.ai/)
+* GCP Pub/Sub
+* [QuestDB](https://questdb.io/)
+
+
+## Installation
+
+**Note:** cryptofeed requires Python 3.12+
+
+Cryptofeed can be installed from PyPi. (It's recommended that you install in a virtual environment of your choosing).
+
+    pip install cryptofeed
+
+Cryptofeed has optional dependencies, depending on the backends used. You can install them individually, or all at once. To install Cryptofeed along with all its optional dependencies in one bundle:
+
+    pip install cryptofeed[all]
+
+If you wish to clone the repository and work on it (development uses [uv](https://docs.astral.sh/uv/)), run this from the root of the cloned repository:
+
+    uv sync
+
+Alternatively, you can install from source in editable mode with pip:
+
+    pip install -e .
+
+See more discussion of package installation in [INSTALL.md](https://github.com/bmoscon/cryptofeed/blob/master/INSTALL.md).
+
+
+
+## Rest API
+
+Cryptofeed supports some REST interfaces for retrieving real-time and historical data, as well as order placement and account management. These are integrated into the exchange classes directly. You can view the supported methods by calling the `info()` method on any exchange. The methods for interacting with the exchange RET endpoints exist in two flavors, the synchronous methods (suffixed with `_sync`) as well as the asynchronous which can be utilized with asyncio. For more information see the [documentation](docs/rest.md).
+
+
+## Future Work
+
+There are a lot of planned features, new exchanges, etc planned! If you'd like to discuss ongoing development, please join the [discord](https://discord.gg/zaBYaGAYfR) or open a thread in the [discussions](https://github.com/bmoscon/cryptofeed/discussions) in GitHub.
+
+## Contributing
+
+Issues and PRs are welcomed!
+
+Cryptofeed wouldn't be possible without the help of many [contributors](AUTHORS.md)! I owe them and all other contributors my thanks!
+
+## Donations / Support
+
+Support and donations are appreciated but not required. You can donate via [GitHub Sponsors](https://github.com/sponsors/bmoscon), or via the addresses below:
+
+* Bitcoin: bc1qm0kxz8hqacaglku5fjhfe9a5hjnuyfwk02lsyr
+* Ethereum: 0x690709FEe13eEce9E7852089BB2D53Ae5D073154

{cryptofeed-2.4.0 → cryptofeed-2.5.0}/README.md

@@ -1,6 +1,6 @@
 # Cryptocurrency Exchange Feed Handler
 [![License](https://img.shields.io/badge/license-XFree86-blue.svg)](LICENSE)
-![Python](https://img.shields.io/badge/Python-3.8+-green.svg)
+![Python](https://img.shields.io/badge/Python-3.12+-green.svg)
 [![PyPi](https://img.shields.io/badge/PyPi-cryptofeed-brightgreen.svg)](https://pypi.python.org/pypi/cryptofeed)
 [![Codacy Badge](https://api.codacy.com/project/badge/Grade/efa4e0d6e10b41d0b51454d08f7b33b1)](https://www.codacy.com/app/bmoscon/cryptofeed?utm_source=github.com&utm_medium=referral&utm_content=bmoscon/cryptofeed&utm_campaign=Badge_Grade)
 
@@ -140,13 +140,14 @@ Supported Backends:
 * Kafka
 * RabbitMQ
 * PostgreSQL
+* [QuasarDB](https://quasar.ai/)
 * GCP Pub/Sub
 * [QuestDB](https://questdb.io/)
 
 
 ## Installation
 
-**Note:** cryptofeed requires Python 3.8+
+**Note:** cryptofeed requires Python 3.12+
 
 Cryptofeed can be installed from PyPi. (It's recommended that you install in a virtual environment of your choosing).
 
@@ -156,13 +157,13 @@ Cryptofeed has optional dependencies, depending on the backends used. You can in
 
     pip install cryptofeed[all]
 
-If you wish to clone the repository and install from source, run this command from the root of the cloned repository.
+If you wish to clone the repository and work on it (development uses [uv](https://docs.astral.sh/uv/)), run this from the root of the cloned repository:
 
-    python setup.py install
+    uv sync
 
-Alternatively, you can install in 'edit' mode (also called development mode):
+Alternatively, you can install from source in editable mode with pip:
 
-    python setup.py develop
+    pip install -e .
 
 See more discussion of package installation in [INSTALL.md](https://github.com/bmoscon/cryptofeed/blob/master/INSTALL.md).
 
@@ -175,7 +176,7 @@ Cryptofeed supports some REST interfaces for retrieving real-time and historical
 
 ## Future Work
 
-There are a lot of planned features, new exchanges, etc planned! If you'd like to discuss ongoing development, please join the [slack](https://join.slack.com/t/cryptofeed-dev/shared_invite/enQtNjY4ODIwODA1MzQ3LTIzMzY3Y2YxMGVhNmQ4YzFhYTc3ODU1MjQ5MDdmY2QyZjdhMGU5ZDFhZDlmMmYzOTUzOTdkYTZiOGUwNGIzYTk) or open a thread in the [discussions](https://github.com/bmoscon/cryptofeed/discussions) in GitHub.
+There are a lot of planned features, new exchanges, etc planned! If you'd like to discuss ongoing development, please join the [discord](https://discord.gg/zaBYaGAYfR) or open a thread in the [discussions](https://github.com/bmoscon/cryptofeed/discussions) in GitHub.
 
 ## Contributing
 

{cryptofeed-2.4.0 → cryptofeed-2.5.0}/cryptofeed/__init__.py

@@ -1,5 +1,5 @@
 '''
-Copyright (C) 2017-2024 Bryant Moscon - bmoscon@gmail.com
+Copyright (C) 2017-2025 Bryant Moscon - bmoscon@gmail.com
 
 Please see the LICENSE file for the terms and conditions
 associated with this software.

{cryptofeed-2.4.0 → cryptofeed-2.5.0}/cryptofeed/backends/_util.py

@@ -1,5 +1,5 @@
 '''
-Copyright (C) 2017-2024 Bryant Moscon - bmoscon@gmail.com
+Copyright (C) 2017-2025 Bryant Moscon - bmoscon@gmail.com
 
 Please see the LICENSE file for the terms and conditions
 associated with this software.

{cryptofeed-2.4.0 → cryptofeed-2.5.0}/cryptofeed/backends/aggregate.py

@@ -1,5 +1,5 @@
 '''
-Copyright (C) 2017-2024 Bryant Moscon - bmoscon@gmail.com
+Copyright (C) 2017-2025 Bryant Moscon - bmoscon@gmail.com
 
 Please see the LICENSE file for the terms and conditions
 associated with this software.
@@ -13,9 +13,9 @@ import numpy as np
 class AggregateCallback:
     def __init__(self, handler):
         self.handler = handler
-        if not callable(self.handler):
-            setattr(self, 'start', self.handler.start)
-            setattr(self, 'stop', self.handler.stop)
+        if hasattr(self.handler, "__class__") and hasattr(self.handler, "start") and hasattr(self.handler, "stop"):
+            setattr(self, "start", self.handler.start)
+            setattr(self, "stop", self.handler.stop)
             self.__name__ = self.handler.__class__
 
 

{cryptofeed-2.4.0 → cryptofeed-2.5.0}/cryptofeed/backends/arctic.py

@@ -1,5 +1,5 @@
 '''
-Copyright (C) 2017-2024 Bryant Moscon - bmoscon@gmail.com
+Copyright (C) 2017-2025 Bryant Moscon - bmoscon@gmail.com
 
 Please see the LICENSE file for the terms and conditions
 associated with this software.

{cryptofeed-2.4.0 → cryptofeed-2.5.0}/cryptofeed/backends/backend.py

@@ -1,5 +1,5 @@
 '''
-Copyright (C) 2017-2024 Bryant Moscon - bmoscon@gmail.com
+Copyright (C) 2017-2025 Bryant Moscon - bmoscon@gmail.com
 
 Please see the LICENSE file for the terms and conditions
 associated with this software.

{cryptofeed-2.4.0 → cryptofeed-2.5.0}/cryptofeed/backends/gcppubsub.py

@@ -1,5 +1,5 @@
 '''
-Copyright (C) 2017-2024 Bryant Moscon - bmoscon@gmail.com
+Copyright (C) 2017-2025 Bryant Moscon - bmoscon@gmail.com
 
 Please see the LICENSE file for the terms and conditions
 associated with this software.

{cryptofeed-2.4.0 → cryptofeed-2.5.0}/cryptofeed/backends/http.py

@@ -1,5 +1,5 @@
 '''
-Copyright (C) 2017-2024 Bryant Moscon - bmoscon@gmail.com
+Copyright (C) 2017-2025 Bryant Moscon - bmoscon@gmail.com
 
 Please see the LICENSE file for the terms and conditions
 associated with this software.

{cryptofeed-2.4.0 → cryptofeed-2.5.0}/cryptofeed/backends/influxdb.py

@@ -1,5 +1,5 @@
 '''
-Copyright (C) 2017-2024 Bryant Moscon - bmoscon@gmail.com
+Copyright (C) 2017-2025 Bryant Moscon - bmoscon@gmail.com
 
 Please see the LICENSE file for the terms and conditions
 associated with this software.

{cryptofeed-2.4.0 → cryptofeed-2.5.0}/cryptofeed/backends/kafka.py

@@ -1,5 +1,5 @@
 '''
-Copyright (C) 2017-2024 Bryant Moscon - bmoscon@gmail.com
+Copyright (C) 2017-2025 Bryant Moscon - bmoscon@gmail.com
 
 Please see the LICENSE file for the terms and conditions
 associated with this software.
@@ -56,11 +56,10 @@ class KafkaCallback(BackendQueue):
 
     async def _connect(self):
         if not self.producer:
-            loop = asyncio.get_event_loop()
             try:
                 config_keys = ', '.join([k for k in self.producer_config.keys()])
                 LOG.info(f'{self.__class__.__name__}: Configuring AIOKafka with the following parameters: {config_keys}')
-                self.producer = AIOKafkaProducer(**self.producer_config, loop=loop)
+                self.producer = AIOKafkaProducer(**self.producer_config)
             # Quit if invalid config option passed to AIOKafka
             except (TypeError, ValueError) as e:
                 LOG.error(f'{self.__class__.__name__}: Invalid AIOKafka configuration: {e.args}{chr(10)}See https://aiokafka.readthedocs.io/en/stable/api.html#aiokafka.AIOKafkaProducer for list of configuration options')

{cryptofeed-2.4.0 → cryptofeed-2.5.0}/cryptofeed/backends/mongo.py

@@ -1,5 +1,5 @@
 '''
-Copyright (C) 2017-2024 Bryant Moscon - bmoscon@gmail.com
+Copyright (C) 2017-2025 Bryant Moscon - bmoscon@gmail.com
 
 Please see the LICENSE file for the terms and conditions
 associated with this software.

{cryptofeed-2.4.0 → cryptofeed-2.5.0}/cryptofeed/backends/postgres.py

@@ -1,5 +1,5 @@
 '''
-Copyright (C) 2017-2024 Bryant Moscon - bmoscon@gmail.com
+Copyright (C) 2017-2025 Bryant Moscon - bmoscon@gmail.com
 
 Please see the LICENSE file for the terms and conditions
 associated with this software.

cryptofeed-2.5.0/cryptofeed/backends/quasardb.py

@@ -0,0 +1,175 @@
+from datetime import datetime, timedelta
+import quasardb.pool as pool
+import quasardb.numpy as qdbnp
+import numpy as np
+from cryptofeed.backends.backend import BackendCallback
+
+
+class QuasarCallback(BackendCallback):
+    def __init__(self, uri="qdb://127.0.0.1:2836", username: str = "", private_key: str = "", public_key: str = "", none_to=None, shard_size: timedelta = timedelta(minutes=15)):
+        self.numeric_type = float
+        self.table = ""
+        self.running = True
+        self.none_to = none_to
+        self.shard_size = self._get_str_timedelta(shard_size)
+
+        pool.initialize(uri=uri, user_name=username, user_private_key=private_key, cluster_public_key=public_key)
+
+    def _get_str_timedelta(self, delta: timedelta):
+        # calculate the number of hours, minutes, and remaining seconds from timedelta, return it in correct format for query
+        hours, remainder = divmod(delta.total_seconds(), 3600)
+        minutes, seconds = divmod(remainder, 60)
+        return f"{int(hours)}hour {int(minutes)}min {int(seconds)}s"
+
+    def format(self, data: dict):
+        data['timestamp'] = np.datetime64(datetime.utcfromtimestamp(data['timestamp']), 'ns')
+        data['receipt_timestamp'] = np.datetime64(datetime.utcfromtimestamp(data['receipt_timestamp']), 'ns')
+        data['timestamp'], data['receipt_timestamp'] = data['receipt_timestamp'], data['timestamp']
+        index = data['timestamp']
+        data.pop('timestamp')
+        return index, data
+
+    def _set_table_name(self, data: dict):
+        # setting table name
+        # {channel}/{exchange}/{symbol_1-symbol_2}
+        # eg. ticker/coinbase/btc-usd
+        self.table = f"{self.table_prefix.lower()}/{data['exchange'].lower()}/{data['symbol'].lower()}"
+
+    def _create_table(self, conn):
+        if not conn.table(self.table).exists():
+            conn.query(self.query)
+
+    def _insert_format(self, date: np.datetime64, data: dict):
+        # converts values to np.array
+        for key, value in data.items():
+            data[key] = np.array([value])
+        return np.array([date]), data
+
+    async def write(self, data: dict):
+        self._set_table_name(data)
+        self._create_query()
+        index, data = self.format(data)
+        idx, np_array = self._insert_format(index, data)
+        # write to table, if table doesnt exist it will be created with specified shard_size value
+        with pool.instance().connect() as conn:
+            self._create_table(conn)
+            qdbnp.write_arrays(np_array, conn, conn.table(self.table), index=idx, fast=True, _async=True)
+
+
+class TickerQuasar(QuasarCallback):
+    table_prefix = "ticker"
+
+    def _create_query(self):
+        self.query = f'CREATE TABLE "{self.table}" (exchange SYMBOL(exchange), symbol SYMBOL(symbol), bid DOUBLE, ask DOUBLE, receipt_timestamp TIMESTAMP) SHARD_SIZE = {self.shard_size}'
+
+
+class TradeQuasar(QuasarCallback):
+    table_prefix = "trades"
+
+    def _create_query(self):
+        self.query = f'CREATE TABLE "{self.table}" (exchange SYMBOL(exchange), symbol SYMBOL(symbol), side SYMBOL(side), amount DOUBLE, price DOUBLE, id STRING, type SYMBOL(type), receipt_timestamp TIMESTAMP) SHARD_SIZE = {self.shard_size}'
+
+
+class CandlesQuasar(QuasarCallback):
+    table_prefix = "candles"
+
+    def format(self, data: dict):
+        index, data = super().format(data)
+        data['start'] = datetime.utcfromtimestamp(data['start'])
+        data['stop'] = datetime.utcfromtimestamp(data['stop'])
+        data['closed'] = int(data['closed'])
+        return index, data
+
+    def _create_query(self):
+        self.query = f'CREATE TABLE "{self.table}" (exchange SYMBOL(exchange), symbol SYMBOL(symbol), start TIMESTAMP, stop TIMESTAMP, interval STRING, trades STRING, open DOUBLE, close DOUBLE, high DOUBLE, low DOUBLE, volume DOUBLE, closed INT64, receipt_timestamp TIMESTAMP) SHARD_SIZE = {self.shard_size}'
+
+
+class FundingQuasar(QuasarCallback):
+    table_prefix = "funding"
+
+    def format(self, data: dict):
+        index, data = super().format(data)
+        data['next_funding_time'] = datetime.utcfromtimestamp(data['next_funding_time'])
+        return index, data
+
+    def _create_query(self):
+        self.query = f'CREATE TABLE "{self.table}" (exchange SYMBOL(exchange), symbol SYMBOL(symbol), mark_price DOUBLE, rate DOUBLE, next_funding_time TIMESTAMP, predicted_rate DOUBLE, receipt_timestamp TIMESTAMP) SHARD_SIZE = {self.shard_size}'
+
+
+class BookQuasar(QuasarCallback):
+    table_prefix = "book"
+
+    def format(self, data: dict):
+        index, data = super().format(data)
+        # store only best bid and best ask
+        if not data['book']:
+            best_bid = max(data["delta"]["bid"], key=lambda x: x[0])
+            best_ask = min(data["delta"]["ask"], key=lambda x: x[0])
+
+            data['best_bid_price'] = best_bid[0]
+            data['best_bid_amount'] = best_bid[1]
+            data['best_ask_price'] = best_ask[0]
+            data['best_ask_amount'] = best_ask[1]
+            data.pop('delta')
+        else:
+            best_bid = max(data["book"]["bid"].keys())
+            best_ask = min(data["book"]["ask"].keys())
+
+            data['best_bid_price'] = best_bid
+            data['best_bid_amount'] = data["book"]["bid"][best_bid]
+            data['best_ask_price'] = best_ask
+            data['best_ask_amount'] = data["book"]["ask"][best_ask]
+            data.pop('book')
+        return index, data
+
+    def _create_query(self):
+        self.query = f'CREATE TABLE "{self.table}" (exchange SYMBOL(exchange), symbol SYMBOL(symbol), best_bid_price DOUBLE, best_bid_amount DOUBLE, best_ask_price DOUBLE, best_ask_amount DOUBLE, receipt_timestamp TIMESTAMP) SHARD_SIZE = {self.shard_size}'
+
+
+class LiquidationsQuasar(QuasarCallback):
+    table_prefix = "liquidations"
+
+    def _create_query(self):
+        self.query = f'CREATE TABLE "{self.table}" (exchange SYMBOL(exchange), symbol SYMBOL(symbol), side SYMBOL(side), quantity DOUBLE, price DOUBLE, id STRING, status SYMBOL(type), receipt_timestamp TIMESTAMP) SHARD_SIZE = {self.shard_size}'
+
+
+class OpenInterestQuasar(QuasarCallback):
+    table_prefix = "open_interest"
+
+    def _create_query(self):
+        self.query = f'CREATE TABLE "{self.table}" (exchange SYMBOL(exchange), symbol SYMBOL(symbol), open_interest FLOAT, receipt_timestamp TIMESTAMP) SHARD_SIZE = {self.shard_size}'
+
+
+class OrderInfoQuasar(QuasarCallback):
+    table_prefix = "order_info"
+
+    def _create_query(self):
+        self.query = f'CREATE TABLE "{self.table}" (exchange SYMBOL(exchange), symbol SYMBOL(symbol), id STRING, client_order_id STRING, side SYMBOL(side), status SYMBOL(type), type SYMBOL(type), price DOUBLE, amount DOUBLE, remaining DOUBLE, account STRING, receipt_timestamp TIMESTAMP) SHARD_SIZE = {self.shard_size}'
+
+
+class TransactionsQuasar(QuasarCallback):
+    table_prefix = "transactions"
+
+    def _create_query(self):
+        self.query = f'CREATE TABLE "{self.table}" (exchange SYMBOL(exchange), currency SYMBOL(currency), type SYMBOL(type), status SYMBOL(type), amount DOUBLE, receipt_timestamp TIMESTAMP) SHARD_SIZE = {self.shard_size}'
+
+
+class BalancesQuasar(QuasarCallback):
+    table_prefix = "balances"
+
+    def _create_query(self):
+        self.query = f'CREATE TABLE "{self.table}" (exchange SYMBOL(exchange), currency SYMBOL(currency), balance DOUBLE, reserved DOUBLE, receipt_timestamp TIMESTAMP) SHARD_SIZE = {self.shard_size}'
+
+
+class FillsQuasar(QuasarCallback):
+    table_prefix = "fills"
+
+    def _create_query(self):
+        self.query = f'CREATE TABLE "{self.table}" (exchange SYMBOL(exchange), symbol SYMBOL(symbol), price DOUBLE, amount DOUBLE, side SYMBOL(side), fee DOUBLE, id STRING, order_id STRING, liquidity DOUBLE, type SYMBOL(type), account STRING, receipt_timestamp TIMESTAMP) SHARD_SIZE = {self.shard_size}'
+
+
+class IndexQuasar(QuasarCallback):
+    table_prefix = "index"
+
+    def _create_query(self):
+        self.query = f'CREATE TABLE "{self.table}" (exchange SYMBOL(exchange), symbol SYMBOL(symbol), price DOUBLE, receipt_timestamp TIMESTAMP) SHARD_SIZE = {self.shard_size}'