vix_utils_tddschn 0.2.0__tar.gz

vix_utils_tddschn-0.2.0/PKG-INFO

@@ -0,0 +1,110 @@
+Metadata-Version: 2.3
+Name: vix_utils_tddschn
+Version: 0.2.0
+Summary: Provide VIX Cash and Futures Term Structure as Pandas dataframes
+Keywords: vix,volatility,pandas,vix term structure,cboe
+Author: Doug Ransom, T
+Author-email: Doug Ransom <doug@ransom.vip>, T <45612704+tddschn@users.noreply.github.com>
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Financial and Insurance Industry
+Classifier: Topic :: Office/Business :: Financial :: Investment
+Requires-Dist: pandas
+Requires-Dist: pandas-market-calendars>=4.4.0
+Requires-Dist: aiofiles
+Requires-Dist: aiohttp[speedups]
+Requires-Dist: openpyxl
+Requires-Dist: appdirs
+Requires-Dist: more-itertools
+Requires-Dist: icecream
+Requires-Dist: matplotlib ; extra == 'examples'
+Requires-Dist: scipy ; extra == 'examples'
+Requires-Dist: pytest ; extra == 'test'
+Requires-Python: >=3.11
+Project-URL: home-page, https://github.com/tddschn/vix_utils
+Provides-Extra: examples
+Provides-Extra: test
+Description-Content-Type: text/markdown
+
+# VIX Utils
+## Overview
+
+*vix_utils* provides some tools for preparing data for analysing  the VIX Futures and Cash Term structures.
+
+The futures can also contain a 30 day continuous maturity weighting of front two months of vix futures.
+
+VIX Futures Data downloaded from [CBOE Futures Historical Data](https://www.cboe.com/us/futures/market_statistics/historical_data/).
+
+Vix Cash Data are downloaded from [CBOE Historical Volatility Indexes](https://www.cboe.com/tradable_products/vix/vix_historical_data/).
+
+
+There is an API for Python to load the data into Pandas DataFrames.  If you do your analysis in Python, use the API.
+
+Since there is no documentation yet, look at the examples in the src/vix_utils/examples folder.
+There is a Jupyter Notebook vix_utils.ipynb in that folder.
+
+*Important note for Juypter notebooks.*  
+You must use  async_get_vix_index_histories and async_load_vix_term_structure 
+rather than get_vix_index_histories and load_vix_term_structure.  There is an example Jupyter notebook "vix_utils use in Jupyter.ipynb" in the src/vix_utils/examples folder. 
+ 
+If you do your analysis in other tools such as R or excel, you can use the command line tool vixutil.
+
+`vixutil -h` will give the help.  The data are availble in record and wide formats.  Just run it and look at the excel or csv output to see what they look like.
+
+ 
+
+
+## Installation
+
+You will need a Python 3.11 or later instalation.
+
+### Install from the Python Packaging Index
+ 
+Install using pip from [The Python Package Index ](https://www.pypi.org):
+
+`pip install vix_utils`
+
+If you want to run the samples, install like this:
+`pip install vix_utils[examples]`
+
+The sample to load all the various data frames can be run as:
+'vix_sample_load_data'
+
+The sample to plot the history of futures and cash term structures:
+`vix_sample_plots`
+
+To load the sample Jupyter notebook, run vix_sample_load_data to figure out where the examples folder is. Browse there with Jupyter and open a notebook.   
+
+
+### Development 
+
+Clone from  [github repository](https://github.com/dougransom/vix_utils).
+
+ 
+`pip install -e .[test,examples]` will:
+- install vix_utils into your python environment, including any command line scripts. 
+- install the necessary prequisites for running any 
+tests in the `test` folder, and for running the programs in the `src/vixutils/examples` folder.
+
+#### Testing
+
+The tests directory contains a few tests.  This project wasn't developed with 
+[Test Driven Development](https://www.agilealliance.org/glossary/tdd/), unit tests have been added to isolate
+and fix defects. 
+
+However, new features and bug fixes should be developed with [Test Driven Development](https://www.agilealliance.org/glossary/tdd/) practices when practical.
+
+
+## Examples
+Source is in `src/vix_utils/examples`
+ 
+~~~
+## Data Notes
+These dates appear to be missing from the CBOE Data.
+At some point they need to be patched in if they exist.
+```
+[Timestamp('2006-11-10 00:00:00'), Timestamp('2007-01-03 00:00:00'), Timestamp('2021-04-02 00:00:00'), Timestamp('2021-12-24 00:00:00')]
+```
+There seem to be  a few dates where spot indexes are missing, you will have to workaround by using fill feature of Pandas datafame, or skip those days, in any analysis.
+~~~
+

vix_utils_tddschn-0.2.0/README.md

@@ -0,0 +1,82 @@
+# VIX Utils
+## Overview
+
+*vix_utils* provides some tools for preparing data for analysing  the VIX Futures and Cash Term structures.
+
+The futures can also contain a 30 day continuous maturity weighting of front two months of vix futures.
+
+VIX Futures Data downloaded from [CBOE Futures Historical Data](https://www.cboe.com/us/futures/market_statistics/historical_data/).
+
+Vix Cash Data are downloaded from [CBOE Historical Volatility Indexes](https://www.cboe.com/tradable_products/vix/vix_historical_data/).
+
+
+There is an API for Python to load the data into Pandas DataFrames.  If you do your analysis in Python, use the API.
+
+Since there is no documentation yet, look at the examples in the src/vix_utils/examples folder.
+There is a Jupyter Notebook vix_utils.ipynb in that folder.
+
+*Important note for Juypter notebooks.*  
+You must use  async_get_vix_index_histories and async_load_vix_term_structure 
+rather than get_vix_index_histories and load_vix_term_structure.  There is an example Jupyter notebook "vix_utils use in Jupyter.ipynb" in the src/vix_utils/examples folder. 
+ 
+If you do your analysis in other tools such as R or excel, you can use the command line tool vixutil.
+
+`vixutil -h` will give the help.  The data are availble in record and wide formats.  Just run it and look at the excel or csv output to see what they look like.
+
+ 
+
+
+## Installation
+
+You will need a Python 3.11 or later instalation.
+
+### Install from the Python Packaging Index
+ 
+Install using pip from [The Python Package Index ](https://www.pypi.org):
+
+`pip install vix_utils`
+
+If you want to run the samples, install like this:
+`pip install vix_utils[examples]`
+
+The sample to load all the various data frames can be run as:
+'vix_sample_load_data'
+
+The sample to plot the history of futures and cash term structures:
+`vix_sample_plots`
+
+To load the sample Jupyter notebook, run vix_sample_load_data to figure out where the examples folder is. Browse there with Jupyter and open a notebook.   
+
+
+### Development 
+
+Clone from  [github repository](https://github.com/dougransom/vix_utils).
+
+ 
+`pip install -e .[test,examples]` will:
+- install vix_utils into your python environment, including any command line scripts. 
+- install the necessary prequisites for running any 
+tests in the `test` folder, and for running the programs in the `src/vixutils/examples` folder.
+
+#### Testing
+
+The tests directory contains a few tests.  This project wasn't developed with 
+[Test Driven Development](https://www.agilealliance.org/glossary/tdd/), unit tests have been added to isolate
+and fix defects. 
+
+However, new features and bug fixes should be developed with [Test Driven Development](https://www.agilealliance.org/glossary/tdd/) practices when practical.
+
+
+## Examples
+Source is in `src/vix_utils/examples`
+ 
+~~~
+## Data Notes
+These dates appear to be missing from the CBOE Data.
+At some point they need to be patched in if they exist.
+```
+[Timestamp('2006-11-10 00:00:00'), Timestamp('2007-01-03 00:00:00'), Timestamp('2021-04-02 00:00:00'), Timestamp('2021-12-24 00:00:00')]
+```
+There seem to be  a few dates where spot indexes are missing, you will have to workaround by using fill feature of Pandas datafame, or skip those days, in any analysis.
+~~~
+

vix_utils_tddschn-0.2.0/pyproject.toml

@@ -0,0 +1,66 @@
+[build-system]
+requires = ["uv_build"]
+build-backend = "uv_build"
+
+[project]
+keywords=["vix","volatility","pandas","vix term structure","cboe"]
+name = "vix_utils_tddschn"
+authors = [
+    {name="Doug Ransom", email="doug@ransom.vip"},
+    {name="T", email="45612704+tddschn@users.noreply.github.com"}
+]
+description="Provide VIX Cash and Futures Term Structure as Pandas dataframes"
+classifiers = [
+	"License :: OSI Approved :: MIT License",
+	"Development Status :: 3 - Alpha",
+	"Intended Audience :: Financial and Insurance Industry",
+	"Topic :: Office/Business :: Financial :: Investment"
+]
+
+readme="README.md"
+requires-python=">=3.11"
+version = "0.2.0"
+
+dependencies= [
+    "pandas",
+    "pandas-market-calendars >= 4.4.0",
+    "aiofiles",
+    "aiohttp[speedups]",
+    "openpyxl",
+    "appdirs",
+    "more_itertools",
+    "icecream"
+]
+
+[project.optional-dependencies]
+test=["pytest"]
+examples=["matplotlib","scipy"]
+
+[tool.pytest.ini_options]
+minversion = "7.1.2"
+addopts = "--capture=tee-sys "
+# very important
+#the pythonpath lets pytest load code in your source area
+#in addition to that in site-packages etc.
+#you may want to run your tests without install natlinkcore with flit or pip
+pythonpath = [
+]
+testpaths= [
+		"test",
+]
+python_files = [
+			"test_*.py",
+]
+
+[project.scripts]
+vixutil = "vix_utils.vixutil:main"
+
+#samples
+vix_sample_plots="vix_utils.examples.sample_plots:main"
+vix_sample_load_data="vix_utils.examples.sample_load_data:main"
+
+[project.urls]
+home-page = "https://github.com/tddschn/vix_utils"
+
+[tool.uv.build-backend]
+module-name = "vix_utils"

vix_utils_tddschn-0.2.0/src/vix_utils/__init__.py

@@ -0,0 +1,42 @@
+"""
+A library for preparing   VIX Futures and Cash Term Structures for analysis,
+including a  continuous maturity VIX Futures term structure.
+"""
+
+from importlib.metadata import version, PackageNotFoundError
+
+try:
+    __version__ = version("vix_utils_tddschn")
+except PackageNotFoundError:
+    __version__ = "0.1.7"
+
+from .download_vix_futures import (
+    pivot_futures_on_monthly_tenor,
+    select_monthly_futures,
+    async_load_vix_term_structure,
+    load_vix_term_structure,
+)
+
+from .vix_cash_term_structure import (
+    async_get_vix_index_histories,
+    get_vix_index_histories,
+    pivot_spot_term_structure_on_symbol,
+)
+
+from .vix_futures_dates import (
+    vix_futures_expiry_date_monthly,
+    vix_futures_expiry_date_from_trade_date,
+    vix_futures_trade_dates_and_expiry_dates,
+    vix_constant_maturity_weights,
+)
+
+from .continuous_maturity import (
+    continuous_maturity_one_month,
+    append_continuous_maturity_one_month,
+)
+
+from .cboe_equity_history import (
+    async_get_cboe_equity_histories,
+    get_cboe_equity_histories,
+    pivot_equity_histories_on_symbol,
+)

vix_utils_tddschn-0.2.0/src/vix_utils/cboe_equity_history.py

@@ -0,0 +1,187 @@
+import json
+import logging
+from pathlib import Path
+
+import aiofiles
+import aiohttp
+import asyncio
+import pandas as pd
+
+from .location import data_dir, make_dir
+
+# Browse equities at https://www.cboe.com/us/equities/
+_cboe_equity_history_url = (
+    "https://cdn.cboe.com/api/global/delayed_quotes/charts/historical/{symbol}.json"
+)
+
+
+def _normalize_equity_symbols(symbols):
+    if isinstance(symbols, str):
+        symbols = [symbols]
+    out = []
+    for symbol in symbols:
+        symbol_str = str(symbol).upper().strip()
+        if not symbol_str:
+            continue
+        if symbol_str in out:
+            continue
+        out.append(symbol_str)
+    return out
+
+
+def _read_equity_json(fname, symbol):
+    try:
+        with open(fname) as f:
+            data = json.load(f)
+    except Exception as e:
+        logging.warning(f"Skipping symbol {symbol}: failed to read {fname}: {e}")
+        return pd.DataFrame(
+            columns=["Trade Date", "Open", "High", "Low", "Close", "Volume", "Symbol"]
+        )
+
+    records = data.get("data", [])
+    if not records:
+        return pd.DataFrame(
+            columns=["Trade Date", "Open", "High", "Low", "Close", "Volume", "Symbol"]
+        )
+
+    df = pd.DataFrame(records)
+
+    col_map = {
+        "date": "Trade Date",
+        "open": "Open",
+        "high": "High",
+        "low": "Low",
+        "close": "Close",
+        "volume": "Volume",
+    }
+    df = df.rename(columns={k: v for k, v in col_map.items() if k in df.columns})
+    df["Symbol"] = symbol
+
+    for col in ["Open", "High", "Low", "Close"]:
+        if col in df.columns:
+            df[col] = pd.to_numeric(df[col], errors="coerce")
+    if "Volume" in df.columns:
+        df["Volume"] = pd.to_numeric(df["Volume"], errors="coerce")
+
+    df["Trade Date"] = pd.to_datetime(df["Trade Date"], errors="coerce")
+    df = df.dropna(subset=["Trade Date", "Close"])
+
+    ordered_cols = ["Trade Date"]
+    for col in ["Open", "High", "Low", "Close", "Volume", "Symbol"]:
+        if col in df.columns:
+            ordered_cols.append(col)
+    df = df[ordered_cols]
+    return df
+
+
+async def async_get_cboe_equity_histories(symbols):
+    """
+    Download historical daily OHLCV data for equities/ETFs from CBOE.
+
+    Uses the endpoint:
+        https://cdn.cboe.com/api/global/delayed_quotes/charts/historical/{symbol}.json
+
+    Parameters
+    ----------
+    symbols : str or list of str
+        Ticker symbol(s) to download (e.g. ``"SPY"`` or ``["SPY", "NVDA"]``).
+
+    Returns
+    -------
+    pd.DataFrame
+        Records-format DataFrame with columns:
+        Trade Date, Open, High, Low, Close, Volume, Symbol.
+    """
+    symbols = _normalize_equity_symbols(symbols)
+
+    equity_download_dir = Path(data_dir()) / "equity" / "download"
+    make_dir(equity_download_dir)
+
+    async with aiohttp.ClientSession() as session:
+
+        async def download_one(symbol):
+            url = _cboe_equity_history_url.format(symbol=symbol)
+            logging.debug(f"Reading URL {url}")
+            cache_path = equity_download_dir / f"{symbol}_History.json"
+
+            async with session.get(url) as resp:
+                if resp.status != 200:
+                    logging.warning(
+                        f"Skipping symbol {symbol}: {url} returned HTTP {resp.status}"
+                    )
+                    return None
+                text = await resp.text()
+
+            async with aiofiles.open(cache_path, mode="w") as f:
+                await f.write(text)
+            logging.debug(f"Wrote {cache_path}")
+
+            return symbol, cache_path
+
+        downloaded = await asyncio.gather(*(download_one(s) for s in symbols))
+
+    downloaded = [entry for entry in downloaded if entry is not None]
+    if not downloaded:
+        return pd.DataFrame(
+            columns=["Trade Date", "Open", "High", "Low", "Close", "Volume", "Symbol"]
+        )
+
+    frames = [
+        frame
+        for frame in (_read_equity_json(path, symbol) for symbol, path in downloaded)
+        if not frame.empty
+    ]
+    if not frames:
+        return pd.DataFrame(
+            columns=["Trade Date", "Open", "High", "Low", "Close", "Volume", "Symbol"]
+        )
+
+    result = pd.concat(frames)
+    result["Trade Date"] = pd.to_datetime(result["Trade Date"])
+    result = result.sort_values(["Trade Date", "Symbol"]).reset_index(drop=True)
+    logging.debug(f"Equity histories shape: {result.shape}")
+    return result
+
+
+def get_cboe_equity_histories(symbols):
+    """
+    Download historical daily OHLCV data for equities/ETFs from CBOE.
+
+    Synchronous wrapper around :func:`async_get_cboe_equity_histories`.
+
+    Parameters
+    ----------
+    symbols : str or list of str
+        Ticker symbol(s) to download (e.g. ``"SPY"`` or ``["SPY", "NVDA"]``).
+
+    Returns
+    -------
+    pd.DataFrame
+        Records-format DataFrame with columns:
+        Trade Date, Open, High, Low, Close, Volume, Symbol.
+    """
+    return asyncio.run(async_get_cboe_equity_histories(symbols=symbols))
+
+
+def pivot_equity_histories_on_symbol(equity_histories):
+    """
+    Pivot equity histories from record format to wide format.
+
+    Parameters
+    ----------
+    equity_histories : pd.DataFrame
+        Records-format DataFrame as returned by :func:`get_cboe_equity_histories`.
+
+    Returns
+    -------
+    pd.DataFrame
+        Wide-format DataFrame with Trade Date as index and a MultiIndex of
+        (price column, Symbol) as columns.
+    """
+    try:
+        wide = equity_histories.set_index(["Trade Date", "Symbol"]).unstack()
+    except Exception as e:
+        logging.error(f"{e} in pivot_equity_histories_on_symbol")
+        raise
+    return wide

vix_utils_tddschn-0.2.0/src/vix_utils/continuous_maturity.py

@@ -0,0 +1,146 @@
+from operator import mul
+from functools import partial
+import numpy as np
+from .download_vix_futures import  \
+pivot_futures_on_monthly_tenor
+import pandas as pd
+import logging
+
+from .vix_futures_dates import vix_futures_expiry_date_monthly, \
+    vix_futures_expiry_date_from_trade_date, \
+    vix_futures_trade_dates_and_expiry_dates, \
+    vix_constant_maturity_weights   
+
+
+
+_weighted_column_names=['Open','High','Low','Close','Settle','Change']
+
+
+def do_weight(trades_df:pd.DataFrame,weight_df:pd.DataFrame,weight_column_name:str,tenor:int):
+    """ 
+    Produces a data frame containing a scaled set of select columns.
+    parameters:
+    ----------
+    trades_df:     a DataFrame indexed by trade date on axis 0, and by an integer (which represents a tenor which could be monthly or weekly) 
+    as the first level index on axis 1.
+    weight_df:  a DataFrame with columns that represent the desired weights.
+    weight_column_name: the name of the column with the desired weights.
+    tenor:  the column index of trades_df for the select columns to be scaled.
+
+    """    
+    w=weight_df[weight_column_name]
+    weight_fn=partial(mul,w)            #a function to multiply by w
+    tenor_df=trades_df[tenor]
+    v=tenor_df.apply(weight_fn)         #apply to the relevant columns.
+    return v
+
+def do_weighting_months(trades_df : pd.DataFrame,weight_df:pd.DataFrame,weights_and_tenors : list[(str,int)]) ->pd.DataFrame:
+    """ Produces a data frame containing the weighted mean of select columns.
+    parameters:
+    ----------
+    trades_df:     a DataFrame indexed by trade date on axis 0, and by an integer (which represents a tenor which could be monthly or weekly) 
+    as the first level index on axis 1.
+    weight_df:  a dataframe with columns that represent the desired weights.
+    weights_and_tenors:  a list of tuples of the name of the column weighting and the tenor to be used to find the columns in trades_df.
+
+    """
+    return sum(do_weight(trades_df,weight_df,n,t) for n,t in weights_and_tenors )
+
+_weights_and_tenors_vix_front_months=[(f"T{i}W",i) for i in range(1,4)]
+
+def do_weighting_front_three_months(trades_df : pd.DataFrame,weight_df : pd.DataFrame) -> pd.DataFrame:
+    """
+    produces a weighted mean of the nearest monthly futures resulting in an average of maturity one month.
+    parameters:
+    ----------
+    trades_df:  a DataFrame indexed by trade date on axis 0, and by Tenor_Monthly (as the first level index) on axis 1.
+
+    """
+    return do_weighting_months(trades_df,weight_df,_weights_and_tenors_vix_front_months)
+
+def append_continuous_maturity_one_month(monthly_wide_records : pd.DataFrame)->pd.DataFrame:
+    """
+    produces a weighted mean of the two nearest monthly futures (using continous_maturity_30day)
+    appends it to the monthly_wide_records, with Monthly_Tenor of 1.5 (for ease of sorting)
+    There will be fewer columns as the result of continous_maturity_30day has fewer columns for a tenor than 
+    monthly_wide_records.
+    parameters:
+    -----------
+    monthly_records:
+        A DataFrame in a wide format of Monthly records, as returned by  pivot_futures_on_monthly_tenor     
+   
+    """
+
+    cm=continuous_maturity_one_month(monthly_wide_records)
+    wide_columns=monthly_wide_records[1].columns
+    #intersection of columns in the two data frames
+     
+    cols=[col for col in cm.columns if col in wide_columns ]
+    new_df1=monthly_wide_records.swaplevel(axis=1)[cols]
+    new_cm=cm[cols]
+    new_df2=new_df1.swaplevel(axis=1)
+    #add a level to new_cm
+    d={}
+    d[1.5]=new_cm
+    e=pd.concat(d,axis=1)
+
+    #concatenate new_cm (after adding the level of idexing)
+    new_df3=pd.concat([new_df2,e],axis=1).sort_index(axis=1)
+    return new_df3
+
+
+
+
+
+
+
+
+
+def continuous_maturity_one_month(monthly_wide_records : pd.DataFrame)->pd.DataFrame:   
+    """
+    produces a weighted mean of the two nearest monthly futures resulting in an average of maturity one month.
+    parameters:
+    -----------
+    monthly_records:
+        A DataFrame in a wide format of Monthly records, as returned by  pivot_futures_on_monthly_tenor     
+    """
+    
+    df=vix_futures_trade_dates_and_expiry_dates()
+
+  
+
+    futures_history=pd.DataFrame(monthly_wide_records)
+
+ 
+       
+
+ 
+    weights_all=vix_constant_maturity_weights(df)
+
+    #we only need the weigths  for dates we have trades
+    weights=weights_all[weights_all.index.isin(futures_history.index)]
+   
+    #select the front three months and the columns that have trade values
+    #we need three because at times teh front month is ignored and the back two contracts are used.
+
+    with pd.option_context('display.max_columns',None): 
+        logging.debug(f"\n{'*'*50}\nColumns to weight:\n{futures_history}")
+    front_three_months=futures_history[[1,2,3]]
+    futures_history_trade_value_columns=front_three_months.swaplevel(axis=1)[_weighted_column_names].swaplevel(axis=1)
+ 
+    weighted_values=do_weighting_front_three_months(futures_history_trade_value_columns,weights)
+
+    #should have the same number of rows
+    assert weighted_values.shape[0]==futures_history_trade_value_columns.shape[0]
+
+    
+    df_file=front_three_months.swaplevel(axis=1)["File"]
+    weighted_values["File"]=df_file[1]+"+"+df_file[2] 
+    
+    weighted_values['Expiry']=weights['Expiry']
+
+
+    weighted_values["Tenor_Days"]=(weighted_values['Expiry']-weighted_values.index).dt.days
+
+
+    return weighted_values