bulum 0.0.0__tar.gz → 0.2.9__tar.gz

bulum-0.2.9/PKG-INFO

@@ -0,0 +1,100 @@
+Metadata-Version: 2.1
+Name: bulum
+Version: 0.2.9
+Summary: Open source python library for assessing hydrologic model results in Queensland
+Home-page: https://bitbucket.org/odhydrology/bulum.git
+Author: Chas Egan
+Author-email: chas@odhydrology.com
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: altair[all]>=5.5.0
+Requires-Dist: folium>=0.14
+Requires-Dist: matplotlib>=3.8.3
+Requires-Dist: numpy>=1.26.4
+Requires-Dist: pandas>=2.2.0
+Requires-Dist: plotly>=5.18.0
+
+# bulum
+
+## Installation
+
+This package may be installed using pip from Bitbucket (requires authentication), or directly from PyPi (public), or from a .tar.gz. Examples are shown below.
+
+```bash
+pip install git+https://bitbucket.org/odhydrology/bulum.git
+```
+
+```bash
+pip install bulum
+```
+
+```bash
+pip install .\dist\bulum-0.0.32.tar.gz
+```
+
+## Usage
+
+```python
+import bulum
+
+# returns the package version
+bulum.__version__
+
+# prints 'Hello world!' to the console
+bulum.hello_world()
+```
+
+## Build and Upload to PyPi
+
+First build a distribution from an anaconda prompt in the root of your project, and then upload the dist to PyPi using Twine.
+
+```bash
+python setup.py sdist
+```
+
+```bash
+twine upload dist\bulum-0.0.32.tar.gz
+```
+
+As of Nov 2023, PyPi uses an API token instead of a conventional password. You can still use Twine, but the username is "__token__", and password is the API token which is very long string starting with "pypi-". 
+
+``` bash
+username = __token__
+password = pypi-#####################################################################################
+```
+
+Where can I find the API token password? Chas has it in his emails. It is also here on the network at *.\ODH working files\Professional development, reading, etc\Software\ODHSoftware\bulum\PyPi_password_and_instructions.txt*.
+
+How do I make a new API token? Go to your PyPi account settings, and click on "API tokens". Then click on "Add API token", and give it a name. The token will be displayed on the next screen.
+
+## Unit Tests
+
+WARNING: Run unit tests from an anaconda environment with compatable dependencies!
+
+Install the nose2 test-runner framework. 
+
+```bash
+pip install nose2
+```
+
+Then from the root project folder run the nose2 module. You can do this as a python modules, or just direcly from the anaconda prompt (both examples given below). This will automatically find and run tests in any modules named "test_*".
+
+```bash
+python -m nose2
+```
+
+```bash
+nose2
+```
+
+You can run specific tests by specifying the module name. Example below.
+
+```bash
+nose2 src.bulum.stats.tests
+```
+
+## License
+
+None.

bulum-0.2.9/README.md

@@ -0,0 +1,82 @@
+# bulum
+
+## Installation
+
+This package may be installed using pip from Bitbucket (requires authentication), or directly from PyPi (public), or from a .tar.gz. Examples are shown below.
+
+```bash
+pip install git+https://bitbucket.org/odhydrology/bulum.git
+```
+
+```bash
+pip install bulum
+```
+
+```bash
+pip install .\dist\bulum-0.0.32.tar.gz
+```
+
+## Usage
+
+```python
+import bulum
+
+# returns the package version
+bulum.__version__
+
+# prints 'Hello world!' to the console
+bulum.hello_world()
+```
+
+## Build and Upload to PyPi
+
+First build a distribution from an anaconda prompt in the root of your project, and then upload the dist to PyPi using Twine.
+
+```bash
+python setup.py sdist
+```
+
+```bash
+twine upload dist\bulum-0.0.32.tar.gz
+```
+
+As of Nov 2023, PyPi uses an API token instead of a conventional password. You can still use Twine, but the username is "__token__", and password is the API token which is very long string starting with "pypi-". 
+
+``` bash
+username = __token__
+password = pypi-#####################################################################################
+```
+
+Where can I find the API token password? Chas has it in his emails. It is also here on the network at *.\ODH working files\Professional development, reading, etc\Software\ODHSoftware\bulum\PyPi_password_and_instructions.txt*.
+
+How do I make a new API token? Go to your PyPi account settings, and click on "API tokens". Then click on "Add API token", and give it a name. The token will be displayed on the next screen.
+
+## Unit Tests
+
+WARNING: Run unit tests from an anaconda environment with compatable dependencies!
+
+Install the nose2 test-runner framework. 
+
+```bash
+pip install nose2
+```
+
+Then from the root project folder run the nose2 module. You can do this as a python modules, or just direcly from the anaconda prompt (both examples given below). This will automatically find and run tests in any modules named "test_*".
+
+```bash
+python -m nose2
+```
+
+```bash
+nose2
+```
+
+You can run specific tests by specifying the module name. Example below.
+
+```bash
+nose2 src.bulum.stats.tests
+```
+
+## License
+
+None.

bulum-0.2.9/setup.py

@@ -0,0 +1,32 @@
+import setuptools
+
+with open("README.md", "r") as fh:
+    long_description = fh.read()
+
+exec(open('src/bulum/version.py').read())
+
+setuptools.setup(
+    name="bulum",
+    version=__version__,
+    python_requires=">=3.9",
+    author="Chas Egan",
+    author_email="chas@odhydrology.com",
+    description="Open source python library for assessing hydrologic model results in Queensland",
+    long_description=long_description,
+    long_description_content_type="text/markdown",
+    url="https://bitbucket.org/odhydrology/bulum.git",
+    package_dir={'': 'src'},
+    packages=setuptools.find_packages('src'),
+    classifiers=[
+        "Programming Language :: Python :: 3",
+        "Operating System :: OS Independent",
+    ],
+    install_requires=[
+        'altair[all]>=5.5.0',
+        'folium>=0.14',
+        'matplotlib>=3.8.3',
+        'numpy>=1.26.4',
+        'pandas>=2.2.0',
+        'plotly>=5.18.0',
+    ],    
+)

bulum-0.2.9/src/bulum/__init__.py

@@ -0,0 +1,3 @@
+from .version import __version__
+from .demo import *
+#from .stoch import *

bulum-0.2.9/src/bulum/clim/__init__.py

@@ -0,0 +1 @@
+from .clim import *

bulum-0.2.9/src/bulum/clim/clim.py

@@ -0,0 +1,150 @@
+import numpy as np
+import pandas as pd
+
+
+def derive_transformation_curves(original_ts: pd.Series, augmented_ts: pd.Series, season_start_months=[1,2,3,4,5,6,7,8,9,10,11,12], epsilon=1e-3) -> dict:
+    """Returns a dictionary of exceedence-based transformation curves - one for each season 
+    with the season's start month as the key. These are tables that map from exceedance 
+    (cunnane plotting position as a fraction) to a scaling factor. These are intended to 
+    be used to
+    effectively summarise climate-change adjustments, and allow them to be transported from
+    one timeseries to another.
+
+    Args:
+        original_ts (pd.Series): _description_
+        augmented_ts (pd.Series): _description_
+        season_start_months (list, optional): _description_. Defaults to [1,2,3,4,5,6,7,8,9,10,11,12].
+
+    Returns:
+        dict: _description_
+    """
+    df = pd.DataFrame()
+    df["x"] = original_ts
+    df["y"] = augmented_ts
+    df = df.dropna() # Force common period
+    answer = {}
+    for i in range(len(season_start_months)):
+        # Get a list of the months in this season
+        start_month = season_start_months[i]
+        season_len = (season_start_months + [m + 12 for m in season_start_months])[i + 1] - start_month
+        months_in_this_season = [1,2,3,4,5,6,7,8,9,10,11,12,1,2,3,4,5,6,7,8,9,10,11,12][start_month - 1: start_month - 1 + season_len]
+        # Find the data for this season
+        df_m = df[[int(d[5:7]) in months_in_this_season for d in df.index]] #d[5:7] is the month part of the date string
+        x = np.sort(df_m.x.values)
+        y = np.sort(df_m.y.values)
+        # The transformation factor is y/x except when the original value x is zero (<epsilon) in which case we default to 1.0
+        f = np.where(x < epsilon, 1.0, y / x)
+        n = len(x)
+        ii = [i + 1 for i in range(n)] #index starting at 1
+        p = [(i - 0.4)/(n + 0.2) for i in ii]
+        answer[start_month] = [p,f]
+    return answer     
+
+    
+def apply_transformation_curves(tranformation_curves: dict, series: pd.Series) -> pd.Series:
+    """Applies seasonal transformation curves to an input series.
+    Refer to the function 'derive_transformation_curves(...)'.
+
+    Args:
+        tranformation_curves (dict): _description_
+        series (pd.Series): _description_
+
+    Returns:
+        pd.Series: _description_
+    """
+    dates = series.index
+    answer = series.copy()    
+    # Apply each transformation curves to the whole series. Splice the appropriate 
+    # parts (seasons) into the 'answer' series as we go. 
+    season_start_months = sorted(tranformation_curves.keys())
+    for i in range(len(season_start_months)):
+        # Identify the transform curve for this season
+        start_month = season_start_months[i]
+        t = tranformation_curves[start_month]
+        xp = t[0] 
+        fp = t[1]
+        # Get a list of the months in this season
+        season_len = (season_start_months + [m + 12 for m in season_start_months])[i + 1] - start_month
+        months_in_this_season = [1,2,3,4,5,6,7,8,9,10,11,12,1,2,3,4,5,6,7,8,9,10,11,12][start_month - 1: start_month - 1 + season_len]
+        # Find the data for this season
+        m = len(series)
+        season_dates = pd.Series([d for d in dates if int(d[5:7]) in months_in_this_season]) #d[5:7] is the month part of the date string
+        values = answer[season_dates]
+        # And get their ranks and plotting positions
+        rank_starting_at_one = values.rank(ascending=True) # This function is nice because equal values are assigned the same (averaged) rank.
+        n = len(values)
+        p = [(r - 0.4)/(n + 0.2) for r in rank_starting_at_one] # plotting position
+        f = np.interp(p, xp, fp) # interpolated scaling factors
+        # Calcualte new values and update the answer
+        new_values = pd.Series([values.iloc[i] * f[i] for i in range(n)], index=season_dates)
+        answer.update(new_values)
+    # Return a pd.Series so user can easily join it back into a dataframe
+    return pd.Series(answer, index=dates, name=series.name)     
+    
+    
+def derive_transformation_factors(original_ts: pd.Series, augmented_ts: pd.Series, season_start_months=[1,2,3,4,5,6,7,8,9,10,11,12], epsilon=1e-3) -> dict:
+    """Returns a dictionary of transformation factors - one for each season 
+    with the season's start month as the key. These scaling factors are intended to 
+    be used to effectively summarise climate-change adjustments, and allow them to be 
+    transported from one timeseries to another.
+
+    Args:
+        original_ts (pd.Series): _description_
+        augmented_ts (pd.Series): _description_
+        season_start_months (list, optional): _description_. Defaults to [1,2,3,4,5,6,7,8,9,10,11,12].
+        epsilon: Threshold below which values are treated as zero, and the associated factor defaults to 1.
+
+    Returns:
+        dict: _description_
+    """
+    # Create a map of month -> season_start_month (for all months)
+    month_to_season_map = {}
+    key = max(season_start_months)
+    for m in [1,2,3,4,5,6,7,8,9,10,11,12]:
+        if m in season_start_months:
+            key = m
+        month_to_season_map[m] = key
+    # Put the data in a dataframe, and groupby season start month
+    df = pd.DataFrame()
+    df["x"] = original_ts
+    df["y"] = augmented_ts
+    df = df.dropna() # Force common period
+    df['m'] = df.index.month
+    df['s'] = df['m'].map(month_to_season_map)
+    df2 = df.groupby('s').agg('sum')
+    df2['f'] = np.where(df2.x < epsilon, 1.0, df2.y / df2.x)
+    return df2['f'].to_dict()
+
+    
+def apply_transformation_factors(tranformation_factors: dict, series: pd.Series) -> pd.Series:
+    """Applies seasonal transformation factors to an input series.
+    Refer to the function 'derive_transformation_factors(...)'.
+
+    Args:
+        tranformation_curves (dict): _description_
+        series (pd.Series): _description_
+
+    Returns:
+        pd.Series: _description_
+    """
+    # Create a map of month -> factor (containing all months)
+    season_start_months = sorted(tranformation_factors.keys())
+    month_to_factor_map = {}
+    key = max(season_start_months)
+    for m in [1,2,3,4,5,6,7,8,9,10,11,12]:
+        if m in season_start_months:
+            key = m
+        month_to_factor_map[m] = tranformation_factors[key]
+    # Apply transformation factors to the whole series. Splice the appropriate 
+    df = pd.DataFrame()
+    df['x'] = series
+    df['m'] = df.index.month
+    df['f'] = df['m'].map(month_to_factor_map)
+    df['y'] = df['x'] * df['f']
+    answer = df['y']
+    answer.name = series.name
+    return answer
+
+    
+    
+    

bulum-0.2.9/src/bulum/demo.py

@@ -0,0 +1,6 @@
+def hello_world():
+    print("Hello world!")
+    
+if __name__ == "__main__":
+    print(type(hello_world))
+    hello_world()

bulum-0.2.9/src/bulum/io/__init__.py

@@ -0,0 +1,7 @@
+from .csv_io import *
+from .res_csv_io import *
+from .idx_io import *
+from .idx_io_native import *
+from .iqqm_out_reader import *
+from .lqn_io import *
+from .general_io import *

bulum-0.2.9/src/bulum/io/csv_io.py

@@ -0,0 +1,59 @@
+import numpy as np
+import pandas as pd
+from bulum import utils
+na_values = ['', ' ', 'null', 'NULL', 'NAN', 'NaN', 'nan', 'NA', 'na', 'N/A' 'n/a', '#N/A', '#NA', '-NaN', '-nan']
+
+
+def read_ts_csv(filename, date_format=None, df=None, colprefix=None, allow_nonnumeric=False, assert_date=True, **kwargs) -> utils.TimeseriesDataframe:
+    """Reads a daily timeseries csv into a DataFrame, and sets the index to string dates in the "%Y-%m-%d" format. 
+    The method assumes the first column are dates.
+
+    Args:
+        filename (_type_): _description_
+        date_format (str, optional): defaults to "%d/%m/%Y" as per Fors. Other common formats include "%Y-%m-%d", "%Y/%m/%d".
+        df (pd.DataFrame, optional): If provided, the reader will append columns to this dataframe. Defaults to None.
+        colprefix (str, optional): If provided, the reader will append this prefix to the start of each column name. Defaults to None.
+        allow_nonnumeric (bool, optional): If false, the method will assert that all columns are numerical. Defaults to False.
+        assert_date (bool, optional): If true, the method will assert that date index meets "%Y-%m-%d" format. Defaults to True.         
+
+    Returns:
+        pd.DataFrame: Dataframe containing the data from the csv file.
+    """
+    new_df = pd.read_csv(filename, na_values=na_values, **kwargs)
+    # Date index
+    new_df.set_index(new_df.columns[0], inplace=True)
+    if assert_date:
+        new_df.index = utils.standardize_datestring_format(new_df.index)
+    new_df.index.name = "Date"
+    # df = df.replace(r'^\s*$', np.nan, regex=True)
+    # Check values
+    if not allow_nonnumeric:
+        for col in new_df.columns:
+            if not np.issubdtype(new_df[col].dtype, np.number):
+                raise Exception(f"ERROR: Column '{col}' is not numeric!")
+    # Rename columns if required
+    if colprefix is not None:
+        for c in new_df.columns:
+            new_df.rename(columns = {c:f"{colprefix}{c}"}, inplace = True)
+    # Join to existing dataframe if required
+    if df is None:
+        df = new_df
+    else:
+        if len(df) > 0:
+            # Check that the dates overlap
+            newdf_ends_before_df_starts = new_df.index[0] < df.index[-1]
+            df_ends_before_newdf_starts = df.index[-1] < new_df.index[0]
+            if newdf_ends_before_df_starts or df_ends_before_newdf_starts:
+                raise Exception("ERROR: The dates in the new dataframe do not overlap with the existing dataframe!")
+        df = df.join(new_df, how="outer")
+    return utils.TimeseriesDataframe.from_dataframe(df)
+
+
+def write_ts_csv(df: pd.DataFrame, filename: str):
+    """_summary_
+
+    Args:
+        df (pd.DataFrame): _description_
+        filename (str): _description_
+    """
+    df.to_csv(filename)

bulum-0.2.9/src/bulum/io/general_io.py

@@ -0,0 +1,24 @@
+import pandas as pd
+import bulum.io as oio
+from bulum import utils
+import re
+
+
+def read(filename: str, **kwargs) -> utils.TimeseriesDataframe:
+    filename_lower = filename.lower()
+    df = None
+    if filename_lower.endswith(".res.csv"):
+        df = oio.read_res_csv(filename, **kwargs)
+        if df is None:
+            raise ValueError("Res csv could not be read.")
+    elif filename_lower.endswith(".csv"):
+        df = oio.read_ts_csv(filename, **kwargs)
+    elif filename_lower.endswith(".idx"):
+        df = oio.read_idx(filename, **kwargs)
+    elif re.search(".[0-9]{2}d$", filename_lower):
+        df = oio.read_iqqm_lqn_output(filename, **kwargs)
+    else:
+        raise ValueError(f"Unknown file extension: {filename}")
+    assert isinstance(df, utils.TimeseriesDataframe), \
+        "Output of `read` is not a TimeseriesDataframe."
+    return df

bulum-0.2.9/src/bulum/io/idx_io.py

@@ -0,0 +1,64 @@
+import os
+import pandas as pd
+import uuid
+import shutil
+import subprocess
+from bulum import utils
+from .csv_io import *
+
+
+
+def write_idx(df, filename, cleanup_tempfile=True):
+    """_summary_
+
+    Args:
+        df (_type_): _description_
+        filename (_type_): _description_
+    """
+    if shutil.which('csvidx') is None:
+        raise Exception("This method relies on the external program 'csvidx.exe'. Please ensure it is in your path.")
+    temp_filename = f"{uuid.uuid4().hex}.tempfile.csv"
+    write_area_ts_csv(df, temp_filename)
+    command = f"csvidx {temp_filename} {filename}"
+    process = subprocess.Popen(command)
+    process.wait()
+    if cleanup_tempfile:
+        os.remove(temp_filename)
+
+
+
+def write_area_ts_csv(df, filename, units = "(mm.d^-1)"):
+    """_summary_
+
+    Args:
+        df (_type_): _description_
+        filename (_type_): _description_
+        units (str, optional): _description_. Defaults to "(mm.d^-1)".
+
+    Raises:
+        Exception: If shortenned field names are going to clash in output file.
+    """
+    # ensures dataframe adheres to standards
+    utils.assert_df_format_standards(df)
+    # convert field names to 12 chars and check for collisions
+    fields = {}
+    for c in df.columns:
+        c12 = f"{c[:12]:<12}"
+        if c12 in fields.keys():
+            raise Exception(f"Field names clash when shortenned to 12 chars: {c} and {fields[c12]}")
+        fields[c12] = c
+    # create the header text
+    header = f"{units}"
+    for k in fields.keys():
+        header += f',"{k}"'
+    header += os.linesep
+    header += "Catchment area (km^2)"
+    for k in fields.keys():
+        header += f", 1.00000000"
+    header += os.linesep
+    # open a file and write the header and the csv body
+    with open(filename, "w+", newline='', encoding='utf-8') as file:        
+        file.write(header)
+        df.to_csv(file, header=False, na_rep=' NaN')
+        
+        

bulum-0.2.9/src/bulum/io/idx_io_native.py

@@ -0,0 +1,136 @@
+import os
+import pandas as pd
+import numpy as np
+from bulum import utils
+
+
+def _detect_header_bytes(b_data: np.ndarray) -> bool:
+    """
+    Helper function for read_idx. Detects whether the OUT file was written with
+    a version of IQQM with an old compiler with metadata/junk data as a header.
+    Fails if (not necessarily only if) the run was undertaken with only one
+    source of data, i.e. the .idx file has only one entry.
+
+    Args:
+        b_data (np.ndarray): 2d array of binary data filled with float32 data. 
+    """
+    b_data_slice: tuple[np.float32] = b_data[0]
+    first_non_zero = b_data_slice[0] != 0.0
+    rest_zeroes = not np.any(list(b_data_slice)[1:])
+    return first_non_zero and rest_zeroes
+
+
+def read_idx(filename, skip_header_bytes=None) -> utils.TimeseriesDataframe:
+    """_summary_
+
+    Args:
+        filename (_type_): Name of the IDX file.
+        skip_header_bytes (bool | None): Whether to skip header bytes in the IDX
+          file (related to the compiler used for IQQM). If set to None, attempt
+          to detect the presence of header bytes automatically.
+
+    Returns:
+        utils.TimeseriesDataframe: _description_
+    """
+    if not os.path.exists(filename):
+        raise FileNotFoundError(f"File does not exist: {filename}")
+    # Read ".idx" file
+    with open(filename, 'r') as f:
+        # Skip line
+        stmp = f.readline()
+        # Start date, end date, date interval
+        stmp = f.readline().split()
+        date_start = utils.standardize_datestring_format([stmp[0]])[0]
+        date_end = utils.standardize_datestring_format([stmp[1]])[0]        
+        date_flag = int(stmp[2])
+        snames = []
+        for n, line in enumerate(f):
+            sfile = line[0:13].strip()
+            sdesc = line[13:54].strip()
+            sname = f"{n + 1}>{sfile}>{sdesc}"
+            snames.append(sname)
+    # Read ".out" file
+    out_filename = filename.lower().replace('.idx', '.out')
+    if not os.path.exists(out_filename):
+        raise FileNotFoundError(f"File does not exist: {out_filename}")
+    # 4-byte reals
+    b_types = [(s, 'f4') for s in snames]
+    # Read all data in, drop header bytes (first row) if necessary
+    b_data = np.fromfile(out_filename, dtype=np.dtype(b_types))
+    # Detection of header bytes
+    if skip_header_bytes is None:
+        skip_header_bytes = _detect_header_bytes(b_data)
+    if skip_header_bytes:
+        b_data = b_data[1:]  # skip header bytes
+    # Read data
+    if date_flag == 0:
+        daily_date_values = utils.datetime_functions.get_dates(
+            date_start, end_date=date_end, include_end_date=True)         
+        df = pd.DataFrame.from_records(b_data, index=daily_date_values)
+        df.columns = snames
+        df.index.name = "Date"
+        # Check data types. If not 'float64' or 'int64', convert to 'float64'
+        x = df.select_dtypes(exclude=['int64','float64']).columns
+        if x.__len__()>0:
+            df=df.astype({i: 'float64' for i in x})
+    elif date_flag == 1:
+        raise NotImplementedError("Monthly data not yet supported")
+    elif date_flag == 3:
+        raise NotImplementedError("Annual data not yet supported")
+    else:
+        raise ValueError(f"Unsupported date interval: {date_flag}")
+    utils.assert_df_format_standards(df)
+    return utils.TimeseriesDataframe.from_dataframe(df)
+
+
+def write_idx_native(df: pd.DataFrame, filepath, type="None", units="None") -> None:
+    """Writer for .IDX and corresponding .OUT binary files written in native Python.
+    Currently only supports daily data (date flag 0), as with the reader read_idx(...). 
+
+    Assumes that data are homogeneous in units and type e.g. Precipitation & mm resp., or Flow & ML/d.
+
+    Args:
+        df (pd.Dataframe): DataFrame as per the output of read_idx(...).
+        filepath (str)   : Path to the IDX file to be written to including .IDX extension.
+        units (str, optional)      : Units for data in df. 
+        type (str, optional)       : Data specifier for data in df, e.g. Gauged Flow, Precipitation, etc.
+    """
+    date_flag = 0
+    # TODO: When generalising to other frequencies, we may be able to simply read the data type off the time delta in df.index values
+    # As is, I've essentially copied what was done in the reader to flag that this should be implemented at the "same time".
+    # Verify valid date_flag
+    match date_flag:
+        case 0:
+            pass  # valid
+        case 1:
+            raise NotImplementedError("Monthly data not yet supported")
+        case 3:
+            raise NotImplementedError("Annual data not yet supported")
+        case _:
+            raise ValueError(f"Unsupported date interval: {date_flag}")
+
+    utils.assert_df_format_standards(df)
+    first_date = df.index[0]
+    last_date = df.index[-1]
+    col_names = df.columns
+
+    # write index
+    with open(filepath, 'w') as f:
+        # TODO: check whether this "skipped" line has important info
+        # For now I've just copied the data from ./tests/BUR_FLWX.IDX as it's likely just metadata.
+        f.write('6.36.1 06/11/2006 10:48:30.64\n')
+        f.write(f"{first_date} {last_date} {date_flag}\n")
+        # data
+        # inline fn to ensure padded string is exactly l characters long
+        def ljust_or_truncate(s, l): return s.ljust(l)[0:l]
+        for idx, col_name in enumerate(col_names):
+            source_entry = ljust_or_truncate(f"df_col{idx+1}", 12)
+            name_entry = ljust_or_truncate(f"{col_name}", 40)
+            type_entry = ljust_or_truncate(f"{type}", 15)
+            units_entry = ljust_or_truncate(f"{units}", 15)
+            f.write(f"{source_entry} {name_entry}" +
+                    f" {type_entry} {units_entry}\n")
+    # write binary
+    out_filepath = filepath.lower().replace('.idx', '.out')
+    df.to_numpy().tofile(out_filepath)
+    return