neurowarp 0.0.1__tar.gz

neurowarp-0.0.1/.gitignore

@@ -0,0 +1,9 @@
+.DS_Store
+.gitattributes
+.gitignore
+*__pycache__
+*README~.md
+*dist
+*instructions_for_myself.docx
+~$structions_for_myself.docx
+nans_data.mat

neurowarp-0.0.1/PKG-INFO

@@ -0,0 +1,107 @@
+Metadata-Version: 2.3
+Name: neurowarp
+Version: 0.0.1
+Summary: Toolbox for Dynamic Time Warping based latency differences and temporal correlations between two time series for neuroscience
+Project-URL: Homepage, https://github.com/mahan-hosseini/NeuroWarp/
+Project-URL: Issues, https://github.com/mahan-hosseini/NeuroWarp/Issues
+Author-email: Mahan Hosseini <m.hosseini@fz-juelich.de>
+Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.10
+Requires-Dist: matplotlib>=3.7
+Requires-Dist: numpy>=1.24
+Requires-Dist: scipy>=1.11
+Requires-Dist: tslearn>=0.6.3
+Description-Content-Type: text/markdown
+
+# Python NeuroWarp
+
+Python NeuroWarp is provided as a package that can be installed via PyPi. It consists of two functions that can be called after importing neurowarp: timeseries_correlation and latency_difference -  these are the general-purpose scripts enable the DTW analyses presented in *Transient Attention Gates Access Consciousness: Coupling N2pc and P3 Latencies using Dynamic Time Warping*.
+
+## Installation
+
+We recommend that you create a new virtual environment for our module via:
+1. Open a terminal / cmd.exe window, navigate (via `cd`) to the directory you want to create the environment and enter:
+`python -m venv neurowarp_env`
+2. Activate the neurowarp_env via:
+
+	**Windows:** `path\to\neurowarp_env\Scripts\activate`
+
+	**MacOS:** `source neurowarp_env/bin/activate`
+
+3. Install neurowarp via pip (enter while neurowarp_env is active):
+`pip install neurowarp`
+4. The functions can be used as explained below after importing the module
+
+For more detail on virtual environments & pip [click here](https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/)
+
+
+## DTW Temporal Correlation - neurowarp.timeseries_correlation()
+Perform a **DTW based bootstrap analysis** to assess the **temporal correlation between two time series** (Figure 5 of our paper).
+
+### Important Notes
+- Time series must be **2D matrices**
+	- I.e., data points (e.g. time) x subjects (i.e., replications)
+- We provide the ERPs of correct and intrusion trials for users to explore this function
+
+### Running timeseries_correlation using our ERPs
+*Enter the following into Python and make sure to enter your actual paths!*
+1. `import neurowarp`	
+2. `from scipy.io import loadmat`
+3. `data = loadmat("your/path/to/example_series_N2pcP3s")`
+4. `series_1 = data["P3_Correct"]`
+5. `series_2 = data["N2pc_Correct"]`
+6. `name_1 = "P3"`
+7. `name_2 = "N2pc"`
+8. `savepath = "where/to/store/results/to"`
+9. `num_boots = 10000`
+	- The number of bootstrap samples that you want to implement
+10. `outlier = 0`
+	- Exclude outliers if their DTW area is +-5 standard deviations from the mean
+11. `try_to_fix_ylims = 1`
+	- Attempt to standardise y-limits of marginals
+12. `neurowarp.timeseries_correlation(series_1, series_2, name_1, name_2, savepath, num_boots, outlier, try_to_fix_ylims)`
+
+*Note that the figure will look slightly different to that of our paper due to different x/y limits. See the replicate_figures folder if you want to replicate our figure as it was printed.*
+
+## DTW Latency Difference - neurowarp.latency_difference()
+Assess the **latency difference** between **two conditions** (i.e., within-subjects effect) or between **two groups** (i.e., across-subjects effect) of any signal of interest (in milliseconds).
+
+*Figures 3 & 4 of our paper show a two conditions analysis*
+
+### Important Notes
+- Reference and query time series must be **2D matrices**
+	- I.e., data points (e.g., time) x subjects (i.e., replications)
+	- Time series have to be of **equal sizes**
+- **analysis_design** determines whether you want to assess a within- or between-subjects latency effect (can only take “within” or “between” as input)
+- We provide the ERPs of correct and intrusion trials for users to explore this function
+
+### Running timeseries_correlation using our ERPs
+*Enter the following into Python and make sure to enter your actual paths!*
+1. `import neurowarp`	
+2. `from scipy.io import loadmat`
+3. `data = loadmat("your/path/to/example_series_N2pcP3s")`
+4. `analysis_design = "within"`
+5. `query = data["N2pc_Intrusion"]`
+6. `reference = data["N2pc_Correct"]`
+7. `name_query = "Intrusion"`
+8. `name_reference = "Correct"`
+9. `units = "\u03BCV"`
+	- The units that your signals are measured in (in our case micro volts)
+10. `sampling_rate = 500`
+	- The number of data points per second in Hertz
+11. `savepath = "where/to/store/results/to"`
+12. `permutations = 10000`
+	- The number of permutations you would like to implement in statistical testing (we recommend >=10000)
+13. `neurowarp.latency_difference(analysis_design, query, reference, name_query, name_reference,units, sampling_rate, savepath, permutations)`
+
+## Dependencies
+*Python NeuroWarp requires the following toolboxes which are automatically installed via `pip install neurowarp`*
+- Numpy
+- Matplotlib
+- Tslearn
+- Scipy
+
+## Tests
+Python NeuroWarp was tested with Python 3.10.9.

neurowarp-0.0.1/README.md

@@ -0,0 +1,90 @@
+# Python NeuroWarp
+
+Python NeuroWarp is provided as a package that can be installed via PyPi. It consists of two functions that can be called after importing neurowarp: timeseries_correlation and latency_difference -  these are the general-purpose scripts enable the DTW analyses presented in *Transient Attention Gates Access Consciousness: Coupling N2pc and P3 Latencies using Dynamic Time Warping*.
+
+## Installation
+
+We recommend that you create a new virtual environment for our module via:
+1. Open a terminal / cmd.exe window, navigate (via `cd`) to the directory you want to create the environment and enter:
+`python -m venv neurowarp_env`
+2. Activate the neurowarp_env via:
+
+	**Windows:** `path\to\neurowarp_env\Scripts\activate`
+
+	**MacOS:** `source neurowarp_env/bin/activate`
+
+3. Install neurowarp via pip (enter while neurowarp_env is active):
+`pip install neurowarp`
+4. The functions can be used as explained below after importing the module
+
+For more detail on virtual environments & pip [click here](https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/)
+
+
+## DTW Temporal Correlation - neurowarp.timeseries_correlation()
+Perform a **DTW based bootstrap analysis** to assess the **temporal correlation between two time series** (Figure 5 of our paper).
+
+### Important Notes
+- Time series must be **2D matrices**
+	- I.e., data points (e.g. time) x subjects (i.e., replications)
+- We provide the ERPs of correct and intrusion trials for users to explore this function
+
+### Running timeseries_correlation using our ERPs
+*Enter the following into Python and make sure to enter your actual paths!*
+1. `import neurowarp`	
+2. `from scipy.io import loadmat`
+3. `data = loadmat("your/path/to/example_series_N2pcP3s")`
+4. `series_1 = data["P3_Correct"]`
+5. `series_2 = data["N2pc_Correct"]`
+6. `name_1 = "P3"`
+7. `name_2 = "N2pc"`
+8. `savepath = "where/to/store/results/to"`
+9. `num_boots = 10000`
+	- The number of bootstrap samples that you want to implement
+10. `outlier = 0`
+	- Exclude outliers if their DTW area is +-5 standard deviations from the mean
+11. `try_to_fix_ylims = 1`
+	- Attempt to standardise y-limits of marginals
+12. `neurowarp.timeseries_correlation(series_1, series_2, name_1, name_2, savepath, num_boots, outlier, try_to_fix_ylims)`
+
+*Note that the figure will look slightly different to that of our paper due to different x/y limits. See the replicate_figures folder if you want to replicate our figure as it was printed.*
+
+## DTW Latency Difference - neurowarp.latency_difference()
+Assess the **latency difference** between **two conditions** (i.e., within-subjects effect) or between **two groups** (i.e., across-subjects effect) of any signal of interest (in milliseconds).
+
+*Figures 3 & 4 of our paper show a two conditions analysis*
+
+### Important Notes
+- Reference and query time series must be **2D matrices**
+	- I.e., data points (e.g., time) x subjects (i.e., replications)
+	- Time series have to be of **equal sizes**
+- **analysis_design** determines whether you want to assess a within- or between-subjects latency effect (can only take “within” or “between” as input)
+- We provide the ERPs of correct and intrusion trials for users to explore this function
+
+### Running timeseries_correlation using our ERPs
+*Enter the following into Python and make sure to enter your actual paths!*
+1. `import neurowarp`	
+2. `from scipy.io import loadmat`
+3. `data = loadmat("your/path/to/example_series_N2pcP3s")`
+4. `analysis_design = "within"`
+5. `query = data["N2pc_Intrusion"]`
+6. `reference = data["N2pc_Correct"]`
+7. `name_query = "Intrusion"`
+8. `name_reference = "Correct"`
+9. `units = "\u03BCV"`
+	- The units that your signals are measured in (in our case micro volts)
+10. `sampling_rate = 500`
+	- The number of data points per second in Hertz
+11. `savepath = "where/to/store/results/to"`
+12. `permutations = 10000`
+	- The number of permutations you would like to implement in statistical testing (we recommend >=10000)
+13. `neurowarp.latency_difference(analysis_design, query, reference, name_query, name_reference,units, sampling_rate, savepath, permutations)`
+
+## Dependencies
+*Python NeuroWarp requires the following toolboxes which are automatically installed via `pip install neurowarp`*
+- Numpy
+- Matplotlib
+- Tslearn
+- Scipy
+
+## Tests
+Python NeuroWarp was tested with Python 3.10.9.

neurowarp-0.0.1/pyproject.toml

@@ -0,0 +1,28 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "neurowarp"
+version = "0.0.1"
+authors = [
+  { name="Mahan Hosseini", email="m.hosseini@fz-juelich.de" },
+]
+description = "Toolbox for Dynamic Time Warping based latency differences and temporal correlations between two time series for neuroscience"
+dependencies = [
+    "numpy>=1.24",
+    "matplotlib>=3.7",
+    "scipy>=1.11",
+    "tslearn>=0.6.3",
+]
+readme = "README.md"
+requires-python = ">=3.10"
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
+    "Operating System :: OS Independent",
+]
+
+[project.urls]
+Homepage = "https://github.com/mahan-hosseini/NeuroWarp/"
+Issues = "https://github.com/mahan-hosseini/NeuroWarp/Issues"

neurowarp-0.0.1/src/neurowarp/__init__.py

@@ -0,0 +1,2 @@
+from .latency_difference import latency_difference
+from .timeseries_correlation import timeseries_correlation

neurowarp-0.0.1/src/neurowarp/latency_difference.py

@@ -0,0 +1,481 @@
+# %% ************  LATENCY DIFFERENCES WITH DYNAMIC TIME WARPING (IN MS)  **************
+
+# %% Imports
+import pdb
+import os
+import pickle
+import numpy as np
+import matplotlib.pyplot as plt
+from tslearn.metrics import dtw_path
+import scipy.stats as stats
+
+# %% Global constants
+TITLE_FONTSIZE = 20
+SUPLABEL_FONTSIZE = 15
+
+
+# %% Main function
+def latency_difference(
+    analysis_design,
+    query,
+    reference,
+    name_query,
+    name_reference,
+    units,
+    sampling_rate,
+    savepath,
+    permutations,
+):
+    """
+    Conceptual Approach
+    -------------------
+    To assess a latency difference between 2 time series:
+      1) Average across subjects & compute DTW between the averages
+      2) The median of the distribution of the warping path's x/y distances
+          is used to compute the latency difference in milliseconds
+      3) The area between the warping path and the main diagonal is used
+          in a permutation procedure for assigning a p value to the latency
+          difference
+
+    Note
+    ----
+    1) This function allows assessment of latency differences in within-
+        or between-subjects designs. Set the "analysis_design" input parameter
+        accordingly. This affects permutation procedure, adopting the
+        permutation equivalents of paired- or independent-sample t-tests
+        respectively.
+    2) This whole function is in line with the proposition of
+        Zoumpoulaki et al. (2015)
+        Latency as a region contrast: Measuring ERP latency differences with
+        Dynamic Time Warping
+    3) Your full time series are analysed.
+        If you would like to assess only a specific interval of your time
+        series, use indexing before calling this function.
+
+    Input Parameters
+    ----------------
+    1) analysis_design
+        => Character array or string informing about your analysis design
+        => Has to be either "within" or "between", based on whether you have
+          a within- or between-subjects design
+    2) query
+        => Query time series (matrix w. dimensions: datapoints x subjects)
+    3) reference
+        => Reference time series (matrix w. dimensions: datapoints x subjects)
+    4) name_query
+        => Name of the query time series
+    5) name_reference
+        => Name of the reference time series
+    6) units
+        => In what units was your data measured?
+          - For microvolts enter '\muV'
+    7) sampling_rate
+        => What was the sampling rate of your signal? In Hertz
+        => Used to compute the latency difference in milliseconds
+    8) savepath
+        => Path with files
+    9) permutations
+        => Number of permutations to adopt in statistical testing
+          - We recommend at least 10000
+    """
+
+    # %% Preparation
+
+    analysis_design = str(analysis_design)
+    units = str(units)
+    name_query = str(name_query)
+    name_reference = str(name_reference)
+    savepath = str(savepath)
+    plotpath = os.path.join(savepath, "Plots/")
+    varpath = os.path.join(savepath, "Variables/")
+    if not os.path.exists(plotpath):
+        os.makedirs(plotpath)
+    if not os.path.exists(varpath):
+        os.makedirs(varpath)
+
+    # Sanity check on analysis_design value
+    if analysis_design not in ["between", "within"]:
+        print('analysis_design must be either "between" or "within"! Fix & re-run')
+        return
+
+    # Sanity check if series are identical for within and have same length
+    try:
+        query = np.array(query)
+        reference = np.array(reference)
+    except:
+        raise ValueError(
+            "Query & Ref input vars must be convertible to np arrays - fix & rerun!"
+        )
+    # => can have different number of subjects if between
+    if analysis_design == "within":
+        if query.shape != reference.shape:
+            print(
+                "Note - query & reference don't have identical shapes - fix & re-run!"
+            )
+            return
+    else:  # has to be 'between' now (
+        if query.shape[0] != reference.shape[0]:
+            print(
+                "Note - query & reference do not have same datapoint-length - fix & "
+                + "re-run!"
+            )
+            return
+
+    # %% MAIN DTW COMPUTATION
+    # We need indices of x & y coordinates of warping path!
+    #   ORIGINAL FORM: WP, score = dtw(x,y) (we don't need score)
+    #   ==> QUERY IS ON X-AXIS OF WP, SO FIRST INPUT
+    #   ==> REFERENCE IS ON Y-AXIS OF WP, SO SECOND INPUT!
+    #   ==> WP is a list of lists with [ix, iy] coordinates of WP
+    #   ==> WP of this will be below main diagonal if QUERY is
+    #       LATER than REFERENCE
+    #   ==> I.e. AREA will be positive in this case because area = areaDIAG -
+    #       areaWP (and hence areaDIAG will be larger than areaWP if WP is below
+    #       DIAG [and QUERY is indeed LATER than REFERENCE!]
+
+    # average across subjects
+    plot_avg_query = query.mean(axis=1)
+    plot_avg_reference = reference.mean(axis=1)
+
+    # DTW has to be done on standardised averages
+    z_avg_query = stats.zscore(plot_avg_query)
+    z_avg_reference = stats.zscore(plot_avg_reference)
+    WP, score = dtw_path(z_avg_query, z_avg_reference)
+
+    # extract indices of WP's x & y coordinates
+    i_query_x = []
+    i_reference_y = []
+    for WP_idxs in WP:
+        i_query_x.append(WP_idxs[0])
+        i_reference_y.append(WP_idxs[1])
+    warped_query = z_avg_query[i_query_x]  # apply indices to get aligned time series
+    warped_reference = z_avg_reference[i_reference_y]
+
+    # plot standardised & aligned time series
+    plot_standardised_and_aligned_series(
+        z_avg_query, z_avg_reference, warped_query, warped_reference, units, plotpath
+    )
+
+    # Latency in MS is distribution of point-wise distances:
+    # (distance = ix - iy) / sampling rate (Hz) / 1000
+    distance = np.array(i_query_x) - np.array(i_reference_y)
+    latency = np.median(distance)
+    lat_in_ms = latency / (sampling_rate / 1000)
+
+    print("\n *** Latency Difference: %i ms! *** \n" % (lat_in_ms))
+
+    # Plot WP compared to main diagonal
+    plotWP(
+        i_query_x,
+        i_reference_y,
+        plot_avg_query,
+        plot_avg_reference,
+        name_query,
+        name_reference,
+        units,
+        lat_in_ms,
+        plotpath,
+    )
+
+    # %% PERMUTATION TEST ON AREA MEASURE
+    # ==> have this for median(distance), i.e., lat_in_ms_measure, too
+
+    # compute true observed area
+    xmax = i_query_x[-1] + 1  # because this is used with range
+    DIAG = np.arange(xmax)
+    areaDIAG = np.trapz(DIAG)
+    # IMPORTANT NOTE ABOUT NP.TRAPZ - TAKES IN (Y) & (Y,X) WHEREAS MATLAB'S TRAPZ
+    # TAKES (Y) & (X,Y) !!!!!
+    areaWP = np.trapz(i_reference_y, i_query_x)
+    area = (areaDIAG - areaWP) / areaDIAG
+
+    # prepare vars for permuting
+    original_query = query.copy()  # copy so it's not just a reference
+    original_reference = reference.copy()
+    perm_x = [[] for _ in range(permutations)]
+    perm_y = [[] for _ in range(permutations)]
+    perm_area = [[] for _ in range(permutations)]
+    datapoints = np.shape(query)[0]
+    subjects = np.shape(query)[1]
+
+    # permutation loop
+    for perms in range(permutations):
+        # for a within-subject contrast, do the coin flipping for permuting class labels
+        if analysis_design == "within":
+            perm_query = np.zeros((datapoints, subjects))
+            perm_reference = np.zeros((datapoints, subjects))
+            for s in range(subjects):
+                coin = np.random.randint(1, 3, subjects)  # 3 bc. indexing exclusive
+                if coin[s] == 1:
+                    perm_query[:, s] = original_query[:, s]
+                    perm_reference[:, s] = original_reference[:, s]
+                elif coin[s] == 2:
+                    perm_query[:, s] = original_reference[:, s]
+                    perm_reference[:, s] = original_query[:, s]
+        # for a between-subject contrast, randomly assign subjects to groups, preserving
+        # the original group sizes of query & reference
+        else:
+            subjects_query = np.shape(query)[1]
+            subjects_reference = np.shape(reference)[1]
+            subjects_total = subjects_query + subjects_reference
+            this_perm = np.random.permutation(subjects_total)
+            data_total = np.concatenate((query, reference), axis=1)
+            perm_query = data_total[:, this_perm[:subjects_query]]
+            perm_reference = data_total[:, this_perm[subjects_query:]]
+
+        # get averages across subjects
+        perm_query_avg = perm_query.mean(axis=1)
+        perm_reference_avg = perm_reference.mean(axis=1)
+
+        # DTW has to be done on standardised averages
+        perm_query_z_avg = stats.zscore(perm_query_avg)
+        perm_reference_z_avg = stats.zscore(perm_reference_avg)
+        perm_WP, perm_score = dtw_path(perm_query_z_avg, perm_reference_z_avg)
+
+        # extract indices of permuted WP's x & y coordinates - store in lists
+        perm_i_query_x = []
+        perm_i_reference_y = []
+        for WP_idxs in perm_WP:
+            perm_i_query_x.append(WP_idxs[0])
+            perm_i_reference_y.append(WP_idxs[1])
+        perm_x[perms] = perm_i_query_x
+        perm_y[perms] = perm_i_reference_y
+
+        # compute permuted area - store results in list
+        perm_xmax = perm_i_query_x[-1] + 1  # because this is used with range
+        perm_DIAG = np.arange(perm_xmax)
+        perm_areaDIAG = np.trapz(perm_DIAG)
+        # IMPORTANT NOTE ABOUT NP.TRAPZ - TAKES IN (Y) & (Y,X) WHEREAS MATLAB'S TRAPZ
+        # TAKES (Y) & (X,Y) !!!!!
+        perm_areaWP = np.trapz(perm_i_reference_y, perm_i_query_x)
+        perm_area[perms] = (perm_areaDIAG - perm_areaWP) / perm_areaDIAG
+
+    # permutation loop end
+    permdist_areas = np.sort(perm_area)  # sort nonabsolutes for idxing thresholds
+    sort_abs_permdist_areas = np.sort(abs(permdist_areas))
+    thresh_abs = round(len(permdist_areas) * 0.95)
+    # find 5% sig. thresh using sorted absolutes
+    thresh_abs_val = sort_abs_permdist_areas[thresh_abs]
+    thresh1 = -thresh_abs_val
+    thresh2 = thresh_abs_val
+
+    # p value (two tailed test)
+    p = sum(abs(permdist_areas) >= abs(area)) / permutations
+    if p < 0.0001:
+        plot_p = "p < 0.0001"
+    else:
+        plot_p = "p = " + str(round(p, 3))
+    print("\n *** %s *** \n" % (plot_p))
+
+    # Plot figure of observed area-size & its permutation distribution
+    f, ax = plt.subplots()
+    plt.tight_layout()
+    ax.hist(permdist_areas, bins=30, facecolor="#c7fdb5", edgecolor="#e4cbff")
+    ax.axvline(thresh1, linewidth=1.5, linestyle="-", color="#ff796c")
+    ax.axvline(thresh2, linewidth=1.5, linestyle="-", color="#ff796c")
+    ax.axvline(area, linewidth=1.5, linestyle="-", color="#8ab8fe")
+    yticks = ax.get_yticks()
+    yticklabels_num = yticks / permutations
+    yticklabels_str = [str(tick) for tick in yticklabels_num]
+    ax.set_yticks(yticks, labels=yticklabels_str)
+    ax.set_ylabel("Proportion", fontsize=12)
+    ax.set_xlabel("DTW - Area", fontsize=12)
+    ax.set_title(
+        "Latency diff.: %i ms - %s" % (lat_in_ms, plot_p),
+        fontweight="bold",
+        fontsize=13,
+    )
+    f.savefig(
+        os.path.join(plotpath, "Permutation Distribution.png"),
+        bbox_inches="tight",
+        dpi=300,
+    )
+    f.savefig(
+        os.path.join(plotpath, "Permutation Distribution.svg"),
+        bbox_inches="tight",
+        dpi=300,
+    )
+    plt.show(block=False)
+    plt.pause(0.001)
+
+    # Plot figure of observed area-size & its permutation distribution
+    f, ax = plt.subplots()
+    plt.tight_layout()
+    ax.hist(abs(permdist_areas), bins=30, facecolor="#c7fdb5", edgecolor="#e4cbff")
+    ax.axvline(thresh_abs_val, linewidth=1.5, linestyle="-", color="#ff796c")
+    ax.axvline(abs(area), linewidth=1.5, linestyle="-", color="#8ab8fe")
+    yticks = ax.get_yticks()
+    yticklabels_num = yticks / permutations
+    yticklabels_str = [str(tick) for tick in yticklabels_num]
+    ax.set_yticks(yticks, labels=yticklabels_str)
+    ax.set_ylabel("Proportion", fontsize=12)
+    ax.set_xlabel("DTW - Area (absolute)", fontsize=12)
+    ax.set_title(
+        "Absolute Areas - Latency diff.: %i ms - %s" % (lat_in_ms, plot_p),
+        fontweight="bold",
+        fontsize=13,
+    )
+    f.savefig(
+        os.path.join(plotpath, "Permutation Distribution (absolutes).png"),
+        bbox_inches="tight",
+        dpi=300,
+    )
+    f.savefig(
+        os.path.join(plotpath, "Permutation Distribution (absolutes).svg"),
+        bbox_inches="tight",
+        dpi=300,
+    )
+    plt.show(block=False)
+    plt.pause(0.001)
+
+    # store variables to a pickle file
+    vars_to_save = {
+        "area": area,
+        "p": p,
+        "thresh1": thresh1,
+        "thresh2": thresh2,
+        "perm_x": perm_x,
+        "perm_y": perm_y,
+        "permdist_areas": permdist_areas,
+    }
+    picklepath = os.path.join(
+        varpath, ("DTW Permutation Test - %s subjects.pkl" % analysis_design)
+    )
+    with open(picklepath, "wb") as file:
+        pickle.dump(vars_to_save, file)
+    print(
+        "\n\n****************************"
+        + "\nDTW Latency Analysis is done"
+        + "\n****************************\n\nResults were saved to:\n"
+        + savepath
+    )
+
+
+# %% Local functions - plotting
+def plot_standardised_and_aligned_series(
+    z_avg_query, z_avg_reference, warped_query, warped_reference, units, plotpath
+):
+    """Plots a figure of original/standardised and aligned time series"""
+    f, ax = plt.subplots(4, 1)
+    plt.tight_layout()
+    plt.subplots_adjust(left=0.1, bottom=0.1, top=0.9)
+    ax[0].plot(z_avg_query, color="k")  # plot the data
+    ax[1].plot(z_avg_reference, color="r")
+    ax[2].plot(warped_query, color="k")
+    ax[3].plot(warped_reference, color="r")
+    # ylims of standardised time series
+    stand_min = min([z_avg_query.min(), z_avg_reference.min(), -1])
+    stand_max = max([z_avg_query.max(), z_avg_reference.max(), 1])
+    ax[0].set_ylim([stand_min, stand_max])
+    ax[1].set_ylim([stand_min, stand_max])
+    # ylims of warped time series
+    warped_min = min([warped_query.min(), warped_reference.min(), -1])
+    warped_max = max([warped_query.max(), warped_reference.max(), -1])
+    ax[2].set_ylim([warped_min, warped_max])
+    ax[3].set_ylim([warped_min, warped_max])
+    # labels, titles & save
+    ax[-1].set_xlabel("Datapoints", fontsize=SUPLABEL_FONTSIZE)
+    f.suptitle("Alignment", y=0.99, fontsize=TITLE_FONTSIZE)
+    f.supylabel(units + " (z-scored)", x=0.01, fontsize=SUPLABEL_FONTSIZE)
+    f.savefig(os.path.join(plotpath, "Alignment.png"), bbox_inches="tight", dpi=300)
+    f.savefig(os.path.join(plotpath, "Alignment.svg"), bbox_inches="tight", dpi=300)
+    plt.show(block=False)  # otherwise stopped further code if python via terminal
+    plt.pause(0.001)
+
+
+def plotWP(
+    i_query_x,
+    i_reference_y,
+    plot_avg_query,
+    plot_avg_reference,
+    name_query,
+    name_reference,
+    units,
+    lat_in_ms,
+    plotpath,
+):
+    """Plot the warping path, the main diagonal and both original time series"""
+    f, ax_global = plt.subplots()
+    ax_global.set_xticks([])
+    ax_global.set_yticks([])
+    gs = f.add_gridspec(4, 4)  # prepare the grid
+    plt.tight_layout()
+    plt.subplots_adjust(left=0.1, bottom=0.1, top=0.9, hspace=0.5)
+
+    # *** grid space 1: WP & main diagonal ***
+    # main diagonal
+    ax = f.add_subplot(gs[0:3, 1:4])  # set wanted gridcells as ax
+    ax.set_xlim(right=i_query_x[-1])
+    ax.set_ylim(top=i_reference_y[-1])
+    xmax = i_query_x[-1] + 1  # bc. indexing starts at 0 & we use this var only w. range
+    ax.plot(
+        np.arange(xmax), np.arange(xmax), color="#ff796c", linewidth=1.5
+    )  # c==salmon
+    # warping path
+    ax.plot(i_query_x, i_reference_y, color="#8ab8fe", linewidth=1.5)  # c==carolinablue
+    # fill area between WP & main diagonal
+    ax.fill_between(
+        i_query_x, i_query_x, i_reference_y, color="#c7fdb5", alpha=0.5, edgecolor="k"
+    )
+    # remove ticklabels
+    ax.set_xticklabels([])
+    ax.set_yticklabels([])
+
+    # *** grid space 2: query on x axis ***
+    ax_2 = f.add_subplot(gs[-1, 1:])
+    ax_2.plot(plot_avg_query, color="r")
+
+    # *** grid space 3: reference on y axis ***
+    ax_3 = f.add_subplot(gs[0:-1, 0])
+    # IMPORTANT NOTE - plt does not have camroll as matlab does. Thus we:
+    # 1. plot reference's y values (e.g. in mV) on "x axis"
+    # 2. plot len of idxs (xmax) as "y values"
+    # 3. we now HAVE TO invert the xaxis to get the same effect as with camroll
+    ax_3.plot(plot_avg_reference, np.arange(xmax), color="k")
+    ax_3.invert_xaxis()
+
+    # axis stuff
+    ylimmax = max([plot_avg_query.max(), plot_avg_reference.max()])
+    ylimmax = ylimmax + (ylimmax * 0.1)
+    ax_2.set_ylim(top=ylimmax)
+    ax_3.set_ylim(top=ylimmax)
+    xticks = ax.get_xticks()  # set xaxis of ax_2 and yaxis of ax_3 according to WP axis
+    xlims = ax.get_xlim()
+    ax_2.set_xticks(xticks)
+    ax_2.set_xlim(xlims)
+    ax_3.set_yticks(xticks)
+    ax_3.set_ylim(xlims)
+    # query & reference text & labels
+    ax_2.text(
+        0.77,
+        1.1,
+        name_query,
+        transform=ax_2.transAxes,
+        fontsize=12,
+        fontweight="bold",
+        fontstyle="italic",
+    )
+    ax_3.text(
+        0.2,
+        1.03,
+        name_reference,
+        transform=ax_3.transAxes,
+        fontsize=12,
+        fontweight="bold",
+        fontstyle="italic",
+    )
+    ax_2.set_ylabel(units, labelpad=1)
+    ax_3.set_xlabel(units, labelpad=1)
+    for spine in ax_global.spines.values():  # despine outmost axis
+        spine.set_visible(False)
+
+    # suplabels & save the figure
+    ax.set_title(
+        "Latency difference: %i ms" % lat_in_ms, fontweight="bold", fontsize=13
+    )
+    ax_2.set_xlabel("Datapoints", fontsize=11)
+    ax_3.set_ylabel("Datapoints", fontsize=11)
+    f.savefig(os.path.join(plotpath, "WarpPath.png"), bbox_inches="tight", dpi=300)
+    f.savefig(os.path.join(plotpath, "WarpPath.svg"), bbox_inches="tight", dpi=300)
+    plt.show(block=False)
+    plt.pause(0.001)

neurowarp-0.0.1/src/neurowarp/timeseries_correlation.py

@@ -0,0 +1,424 @@
+# %% *************  CORRELATE TWO TIME SERIES WITH DYNAMIC TIME WARPING  ***************
+
+# %% Imports
+import pdb
+import os
+import pickle
+import numpy as np
+import matplotlib.pyplot as plt
+from tslearn.metrics import dtw_path
+import scipy.stats as stats
+
+# %% Global constants
+TITLE_FONTSIZE = 20
+SUPLABEL_FONTSIZE = 15
+INFO_TXT_FILENAME = "CorrelateAnalysis_Info.txt"
+
+
+# %% Main function
+def timeseries_correlation(
+    series_1, series_2, name_1, name_2, savepath, num_boots, outlier, try_to_fix_ylims
+):
+    """
+    Conceptual Approach
+    -------------------
+    To assess latency correlations between 2 time series:
+      1) Get Single Subject (or group) time series
+      2) Get N bootstrap time series (w. same bootstrapping
+          subject-distributions for the two time series)
+      3) DTW of 2) with 1) using AREA measure!
+      4) Gives you two numbers for each bootstrap sample -> DTW result of
+          time series 1 & 2 latency (lag)
+      5) Correlate these with Pearson and Spearman (latter preferred for
+          presentations)
+
+    Input Parameters
+    ----------------
+    1) series_1, series_2
+       => Timeseries as two matrices of shape:
+          dataindices (e.g., timepoints) x subjects (or repetitions)
+    2) name_1, name_2
+       => Names of your timeseries
+    3) savepath
+       => Where to save output to
+    4) num_boots
+       => Number of bootstraps (we recommend at least 10000)
+    5) outlier
+       => 1: to exclude +-5SD outliers from your scatterplot
+       => 0: don't exclude outliers
+    6) try_to_fix_ylims
+       => 1: try to fix ylimits of marginals and include lines for easier
+             comparison
+             - based on num_boots
+             - if you want to modify this yourself (e.g., hardcode lines &
+               maxima, change the if statement blocks for the two marginals)
+       => 0: let matlab handle ylimits of marginals (likely won't allow
+             comparisons
+
+    Note
+    ----
+    1) Number of subjects has to be equal in both time series!
+    2) Your full time series are analysed.
+       If you would like to assess only a specific interval of your time
+       series, use indexing before calling this function.
+    3) We fix marginals' x-axis ranges to +-8 SD. This should be okay for
+       most data-sets since we zscore DTW area distributions. However, if you
+       do not want to standardise like this (which will make your marginals
+       harder to compare) or you want to implement different values, change
+       the lines that use xlimits & ylimits (see line 180)
+    """
+
+    # %% Preparation
+
+    # first check if user tried to compare groups with unequal sample sizes
+    if np.shape(series_1)[1] != np.shape(series_2)[1]:
+        print("Your time series have a different number of subjects - cancelling!")
+        return
+
+    # prepare some vars
+    name_1 = str(name_1)
+    name_2 = str(name_2)
+    plotpath = os.path.join(savepath, "Plots/")
+    varpath = os.path.join(savepath, "Variables/")
+    if not os.path.exists(plotpath):
+        os.makedirs(plotpath)
+    if not os.path.exists(varpath):
+        os.makedirs(varpath)
+    DTW_marginals = {"name_1": [], "name_2": []}
+    DTW_stats = dict()
+
+    # %% Run the analysis
+    series_1_dtw_area, series_2_dtw_area = run_bootstrap_dtw(
+        series_1, series_2, num_boots
+    )
+
+    # %% Outlier Rejection
+    if outlier:
+
+        # exclude outliers if +-5 SD (change in std_thresh line if wanted)
+        series_1_std = np.std(series_1_dtw_area)
+        series_2_std = np.std(series_2_dtw_area)
+        std_thresh = 5  # Threshold for outlier rejection
+
+        # *****************************************************************
+        #                           IMPORTANT
+        # This might seem like a strange way of removing outliers
+        # ==> However it ensures that we check all 4 directions of the
+        #   scatterplot for outliers, save indices as we do so, then just
+        #   take those unique indices (removing duplicates) and (IMPORTANTLY)
+        #   IN THE END removing these indices FROM BOTH DTW distributions
+        # ==> THIS IS IMPORTANT BECAUSE DTW DISTRIBUTIONS HAVE TO STAY
+        #   LINKED DUE TO THE BOOTSTRAP
+        # *****************************************************************
+
+        outlier_idx = []
+        # series 1 - positive outliers
+        series_1_pos_outlier_idx = np.where(
+            series_1_dtw_area
+            > (np.mean(series_1_dtw_area) + (std_thresh * series_1_std))
+        )[0]
+        if len(series_1_pos_outlier_idx) > 0:
+            outlier_idx.extend(series_1_pos_outlier_idx)
+        # series 1 - negative outliers
+        series_1_neg_outlier_idx = np.where(
+            series_1_dtw_area
+            < (np.mean(series_1_dtw_area) - (std_thresh * series_1_std))
+        )[0]
+        if len(series_1_neg_outlier_idx) > 0:
+            outlier_idx.extend(series_1_neg_outlier_idx)
+        # series 2 - positive outliers
+        series_2_pos_outlier_idx = np.where(
+            series_2_dtw_area
+            > (np.mean(series_2_dtw_area) + (std_thresh * series_2_std))
+        )[0]
+        if len(series_2_pos_outlier_idx) > 0:
+            outlier_idx.extend(series_2_pos_outlier_idx)
+        # series 2 - negative outliers
+        series_2_neg_outlier_idx = np.where(
+            series_2_dtw_area
+            < (np.mean(series_2_dtw_area) - (std_thresh * series_2_std))
+        )[0]
+        if len(series_2_neg_outlier_idx) > 0:
+            outlier_idx.extend(series_1_neg_outlier_idx)
+
+        # remove duplicates
+        outlier_idx = np.unique(outlier_idx)
+
+        # remove outliers from BOTH series!
+        if len(outlier_idx) > 0:
+            series_1_dtw_area = np.delete(series_1_dtw_area, outlier_idx)
+            series_2_dtw_area = np.delete(series_2_dtw_area, outlier_idx)
+
+    # %% Save var, skew & kurtosis values of DTW Area Distributions
+    DTW_stats["series_1_var"] = np.var(series_1_dtw_area)
+    DTW_stats["series_1_skew"] = stats.skew(series_1_dtw_area)
+    DTW_stats["series_1_kurt"] = stats.kurtosis(series_1_dtw_area)
+    DTW_stats["series_2_var"] = np.var(series_2_dtw_area)
+    DTW_stats["series_2_skew"] = stats.skew(series_2_dtw_area)
+    DTW_stats["series_2_kurt"] = stats.kurtosis(series_2_dtw_area)
+    if outlier:
+        picklepath = os.path.join(
+            varpath, ("%iSD OutlierRejected Marginal Stats.pkl" % (std_thresh))
+        )
+        with open(picklepath, "wb") as file:
+            pickle.dump(DTW_stats, file)
+    else:
+        picklepath = os.path.join(varpath, ("Marginal Stats.pkl"))
+        with open(picklepath, "wb") as file:
+            pickle.dump(DTW_stats, file)
+
+    # %% Standardise DTW Area Distributions & correlate
+    # So line of linear fit in scatter & marginal distributions reflect correlation
+    # values better (since correlations have internal standardisation)
+
+    # zscore
+    z_x = stats.zscore(series_1_dtw_area)
+    z_y = stats.zscore(series_2_dtw_area)
+
+    # pearson corr
+    pears_res = stats.pearsonr(z_x, z_y)
+    pears_r = pears_res[0]
+    pears_p = pears_res[1]
+
+    # spearman corr
+    spear_res = stats.spearmanr(z_x, z_y)
+    spear_r = spear_res[0]
+    spear_p = spear_res[1]
+
+    # prepare strings
+    str_df = str(num_boots - 2)  # degrees of freedom
+    pears_str_p = ""
+    spear_str_p = ""
+    if pears_p < 0.0001:
+        pears_str_p = "p < .0001"
+    else:
+        pears_str_p = "p = " + str(pears_p.round(3))
+    if spear_p < 0.0001:
+        spear_str_p = "p < .0001"
+    else:
+        spear_str_p = "p = " + str(spear_p.round(3))
+
+    # %% Plot one large figure
+    f, ax_global = plt.subplots()
+    ax_global.set_xticks([])
+    ax_global.set_yticks([])
+    for spine in ax_global.spines.values():  # despine outmost axis
+        spine.set_visible(False)
+    gs = f.add_gridspec(4, 4, wspace=0, hspace=0)  # prepare the grid
+    scatter_color = "#c79fef"  # lavender
+    line_color = "#8ab8fe"  # carolina blue
+    xlimits = [-8, 8]
+    xticks = np.squeeze([-8, -4, 0, 4, 8])
+    ylimits = [-8, 8]
+    yticks = xticks.copy()
+    if try_to_fix_ylims:
+        ylimmax = num_boots / 8
+        ref_val_1 = round(ylimmax * 0.25)
+        ref_val_2 = round(ylimmax * 0.5)
+        ref_val_3 = round(ylimmax * 0.75)
+
+    # %% first, scatterplot
+    ax = f.add_subplot(gs[0:3, 1:])  # set wanted gridcells as ax
+    ax.scatter(
+        z_x, z_y, c="white", edgecolor=scatter_color, s=7, linewidth=0.5, alpha=0.6
+    )
+    ax.set_ylim(ylimits)
+    ax.set_xlim(xlimits)
+    ax.set_xticks(xticks)
+    ax.set_yticks(yticks)
+    ax.set_xticklabels([])
+    ax.set_yticklabels([])
+    series_1_xlimits = ax.get_xlim()  # for plotting marginals later
+    series_1_xticks = ax.get_xticks()
+    series_2_xlimits = ax.get_ylim()
+    series_2_xticks = ax.get_yticks()
+    for spine in ax.spines.values():
+        spine.set_visible(False)
+    # ax.set_title("DTW area-diff correlations of %s & %s.\nPearson: r(%s) = %s: %s & Spearman: r(%s) = %s: %s" % (name_1, name_2, str_df, pears_str_r, pears_str_p, str_df, spear_str_r, spear_str_p))
+    ax.set_title(
+        "DTW area-diff correlations of "
+        + name_1
+        + " & "
+        + name_2
+        + "\nPearson: r("
+        + str(str_df)
+        + ") = "
+        + str(pears_r.round(2))
+        + ": "
+        + pears_str_p
+        + "\nSpearman: r("
+        + str(str_df)
+        + ") = "
+        + str(spear_r.round(2))
+        + ": "
+        + spear_str_p
+    )
+    # have to create the line of least squares manually
+    coefficients = np.polyfit(z_x, z_y, 1)  # 1 == linear (fit coefficients to orig z_x)
+    slope, intercept = coefficients
+    # create y_fit using new z_x that spans over the full xlimits (so line isnt cut-off)
+    z_x_new = np.arange(xlimits[0], xlimits[1] + 1, 1)
+    y_fit = intercept + (slope * z_x_new)
+    ax.plot(z_x_new, y_fit, linewidth=1.5, color=line_color)
+
+    # %% second, series_2 marginal distribution
+    ax_series_2 = f.add_subplot(gs[:-1, 0])
+    ax_series_2.hist(
+        z_y,
+        100,
+        orientation="horizontal",
+        facecolor=scatter_color,
+        edgecolor=line_color,
+        linewidth=0.15,
+    )
+    ax_series_2.set_ylim(series_2_xlimits)  # set ylim because horizontal hist!
+    ax_series_2.set_yticks(series_2_xticks)
+    ax_series_2.invert_xaxis()
+    ax_series_2.set_ylabel(name_2 + " Marginal Distribution")
+    for side in ax_series_2.spines.keys():  # side is left/right/top/bottom
+        ax_series_2.spines[side].set_linewidth(0.2)
+        if side == "top":
+            ax_series_2.spines[side].set_visible(False)
+    if try_to_fix_ylims:
+        ax_series_2.set_xlim(left=ylimmax)
+        ref_val_strings = []
+        for val in [ref_val_1, ref_val_2, ref_val_3]:
+            ax_series_2.axvline(val, color="#029386", linewidth=0.2, linestyle="--")
+            ref_val_strings.append(str(val))
+        ax_series_2.set_xticks(
+            [ref_val_1, ref_val_2, ref_val_3], labels=ref_val_strings
+        )
+
+    # %% third, series_1 marginal distribution
+    ax_series_1 = f.add_subplot(gs[-1, 1:])
+    ax_series_1.hist(
+        z_x, 100, facecolor=scatter_color, edgecolor=line_color, linewidth=0.15
+    )
+    ax_series_1.set_xlim(series_1_xlimits)
+    ax_series_1.set_xticks(series_1_xticks)
+    ax_series_1.set_xlabel(name_1 + " Marginal Distribution")
+    for side in ax_series_1.spines.keys():  # side is left/right/top/bottom
+        ax_series_1.spines[side].set_linewidth(0.2)
+        if side == "left":
+            ax_series_1.spines[side].set_visible(False)
+    if try_to_fix_ylims:
+        ax_series_1.set_ylim(top=ylimmax)
+        ref_val_strings = []
+        for val in [ref_val_1, ref_val_2, ref_val_3]:
+            ax_series_1.axhline(val, color="#029386", linewidth=0.2, linestyle="--")
+            ref_val_strings.append(str(val))
+        ax_series_1.set_yticks(
+            [ref_val_1, ref_val_2, ref_val_3], labels=ref_val_strings
+        )
+    ax_series_1.yaxis.set_ticks_position("right")
+
+    # %% save figures & marginal distributions
+    # figures
+    if outlier:
+        fig_filename = (
+            "DTW Latency Correlation - "
+            + name_1
+            + " & "
+            + name_2
+            + " - "
+            + str(std_thresh)
+            + "std outlier removed"
+        )
+        marginals_filename = (
+            "Marginal Distributions - " + str(std_thresh) + "std outlier removed"
+        )
+    else:
+        fig_filename = "DTW Latency Correlation - " + name_1 + " & " + name_2
+        marginals_filename = "Marginal Distributions"
+    f.savefig(
+        os.path.join(plotpath, fig_filename + ".png"),
+        bbox_inches="tight",
+        dpi=300,
+    )
+    f.savefig(
+        os.path.join(plotpath, fig_filename + ".svg"),
+        bbox_inches="tight",
+        dpi=300,
+    )
+    # marginal distributions
+    series_2_marginals = series_2_dtw_area
+    series_1_marginals = series_1_dtw_area
+    z_series_2_marginals = z_y
+    z_series_1_marginals = z_x
+    vars_to_save = {
+        "series_1_marginals": series_1_marginals,
+        "series_2_marginals": series_2_marginals,
+        "z_series_1_marginals": z_series_1_marginals,
+        "z_series_2_marginals": z_series_2_marginals,
+    }
+
+    picklepath = os.path.join(varpath, (marginals_filename + ".pkl"))
+    with open(picklepath, "wb") as file:
+        pickle.dump(vars_to_save, file)
+    plt.show(block=False)  # otherwise stopped further code if python via terminal
+    plt.pause(0.001)
+
+
+# %% Local functions
+def run_bootstrap_dtw(series_1, series_2, num_boots):
+    """Run the main bootstrap DTW analysis"""
+    # compute standardised (subjects / reps) averages of both time series
+    avg_series_1 = series_1.mean(axis=1)
+    avg_series_2 = series_2.mean(axis=1)
+    z_avg_series_1 = stats.zscore(avg_series_1)
+    z_avg_series_2 = stats.zscore(avg_series_2)
+
+    # bootstrap manually using randint & run dtw between bootstrap & observed z_avg
+    num_subs = np.shape(series_1)[1]
+    series_1_dtw_area = np.zeros(num_boots)
+    series_2_dtw_area = np.zeros(num_boots)
+
+    # bootstrap loop
+    for i in range(num_boots):
+        print("\nboots num #%s" % (i))
+
+        # get num_subj bootstrap samples (using randint == we bootstrap with duplicates)
+        this_trap = np.random.randint(0, num_subs, num_subs)
+        boot_series_1 = series_1[:, this_trap]
+        boot_series_2 = series_2[:, this_trap]
+
+        # average & standardise trapped series
+        avg_boot_series_1 = boot_series_1.mean(axis=1)
+        avg_boot_series_2 = boot_series_2.mean(axis=1)
+        z_avg_boot_series_1 = stats.zscore(avg_boot_series_1)
+        z_avg_boot_series_2 = stats.zscore(avg_boot_series_2)
+
+        # ***** dtw between z-scored averages of series & bootstrapped series *****
+
+        # series_1
+        WP_1, score_1 = dtw_path(z_avg_boot_series_1, z_avg_series_1)
+        ix_1 = []
+        iy_1 = []
+        for WP_idxs in WP_1:
+            ix_1.append(WP_idxs[0])
+            iy_1.append(WP_idxs[1])
+        xmax_1 = ix_1[-1] + 1  # because this is used with range
+        DIAG_1 = np.arange(xmax_1)
+        areaDIAG_1 = np.trapz(DIAG_1)
+        # IMPORTANT NOTE ABOUT NP.TRAPZ - TAKES IN (Y) & (Y,X) WHEREAS MATLAB'S TRAPZ
+        # TAKES (Y) & (X,Y) !!!!!
+        areaWP_1 = np.trapz(iy_1, ix_1)
+        series_1_dtw_area[i] = (areaDIAG_1 - areaWP_1) / areaDIAG_1
+
+        # series_2
+        WP_2, score_2 = dtw_path(z_avg_boot_series_2, z_avg_series_2)
+        ix_2 = []
+        iy_2 = []
+        for WP_idxs in WP_2:
+            ix_2.append(WP_idxs[0])
+            iy_2.append(WP_idxs[1])
+        xmax_2 = ix_2[-1] + 1  # because this is used with range
+        DIAG_2 = np.arange(xmax_2)
+        areaDIAG_2 = np.trapz(DIAG_2)
+        # IMPORTANT NOTE ABOUT NP.TRAPZ - TAKES IN (Y) & (Y,X) WHEREAS MATLAB'S TRAPZ
+        # TAKES (Y) & (X,Y) !!!!!
+        areaWP_2 = np.trapz(iy_2, ix_2)
+        series_2_dtw_area[i] = (areaDIAG_2 - areaWP_2) / areaDIAG_2
+
+    # bootstrap loop end
+    return series_1_dtw_area, series_2_dtw_area