diff-diff 3.5.0__tar.gz → 3.5.1__tar.gz

{diff_diff-3.5.0 → diff_diff-3.5.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: diff-diff
-Version: 3.5.0
+Version: 3.5.1
 Classifier: Development Status :: 5 - Production/Stable
 Classifier: Intended Audience :: Science/Research
 Classifier: Operating System :: OS Independent
@@ -161,7 +161,7 @@ Full guide: `diff_diff.get_llm_guide("practitioner")`.
 - [TwoStageDiD](https://diff-diff.readthedocs.io/en/stable/api/two_stage.html) - Gardner (2022) two-stage estimator with GMM sandwich variance
 - [SpilloverDiD](https://diff-diff.readthedocs.io/en/stable/api/spillover.html) - Butts (2021) ring-indicator spillover-aware DiD identifying direct effect on treated + per-ring spillover on near-control units; handles non-staggered and staggered timing; supports survey-design variance under `survey_design=` for HC1 / CR1 (Wave E.1 Binder TSL) and Conley (Wave E.2 panel-aware stratified-Conley sandwich on per-period PSU totals; extended in Wave E.2 follow-up to `conley_lag_cutoff > 0` via panel-block composition with within-PSU serial Bartlett HAC — `lag>0` requires an effective PSU via explicit `survey_design.psu` or injected `cluster=<col>`); `SurveyDesign.subpopulation()` preserves full-design `n_psu` / `df_survey` via zero-padded scores (Wave E.3, R `svyrecvar(subset())` form)
 - [SyntheticDiD](https://diff-diff.readthedocs.io/en/stable/api/estimators.html) - Synthetic DiD combining standard DiD and synthetic control for few treated units
-- [SyntheticControl](https://diff-diff.readthedocs.io/en/stable/api/synthetic_control.html) - Abadie, Diamond & Hainmueller (2010) classic synthetic control for a single treated unit (donor-weight counterfactual, nested/custom V; in-space placebo permutation inference via `in_space_placebo()`)
+- [SyntheticControl](https://diff-diff.readthedocs.io/en/stable/api/synthetic_control.html) - Abadie, Diamond & Hainmueller (2010) classic synthetic control for a single treated unit (donor-weight counterfactual, nested/cv/inverse-variance/custom V; in-space placebo permutation inference via `in_space_placebo()`, plus ADH-2015 `leave_one_out()` + `in_time_placebo()` robustness)
 - [TripleDifference](https://diff-diff.readthedocs.io/en/stable/api/triple_diff.html) - triple difference (DDD) estimator for designs requiring two criteria for treatment eligibility
 - [ContinuousDiD](https://diff-diff.readthedocs.io/en/stable/api/continuous_did.html) - Callaway, Goodman-Bacon & Sant'Anna (2024) continuous treatment DiD with dose-response curves
 - [HeterogeneousAdoptionDiD](https://diff-diff.readthedocs.io/en/stable/api/had.html) - de Chaisemartin, Ciccia, D'Haultfœuille & Knau (2026) for designs where **no unit remains untreated**; local-linear estimator at the dose support boundary returning Weighted Average Slope (WAS) on Design 1' (`d̲ = 0` / QUG) or `WAS_{d̲}` on Design 1 (`d̲ > 0`, continuous-near-d̲ or mass-point), with a multi-period event-study extension (last-treatment cohort, pointwise CIs). **Panel-only** in this release - repeated cross-sections rejected by the validator. Alias `HAD`.

{diff_diff-3.5.0 → diff_diff-3.5.1}/README.md

@@ -108,7 +108,7 @@ Full guide: `diff_diff.get_llm_guide("practitioner")`.
 - [TwoStageDiD](https://diff-diff.readthedocs.io/en/stable/api/two_stage.html) - Gardner (2022) two-stage estimator with GMM sandwich variance
 - [SpilloverDiD](https://diff-diff.readthedocs.io/en/stable/api/spillover.html) - Butts (2021) ring-indicator spillover-aware DiD identifying direct effect on treated + per-ring spillover on near-control units; handles non-staggered and staggered timing; supports survey-design variance under `survey_design=` for HC1 / CR1 (Wave E.1 Binder TSL) and Conley (Wave E.2 panel-aware stratified-Conley sandwich on per-period PSU totals; extended in Wave E.2 follow-up to `conley_lag_cutoff > 0` via panel-block composition with within-PSU serial Bartlett HAC — `lag>0` requires an effective PSU via explicit `survey_design.psu` or injected `cluster=<col>`); `SurveyDesign.subpopulation()` preserves full-design `n_psu` / `df_survey` via zero-padded scores (Wave E.3, R `svyrecvar(subset())` form)
 - [SyntheticDiD](https://diff-diff.readthedocs.io/en/stable/api/estimators.html) - Synthetic DiD combining standard DiD and synthetic control for few treated units
-- [SyntheticControl](https://diff-diff.readthedocs.io/en/stable/api/synthetic_control.html) - Abadie, Diamond & Hainmueller (2010) classic synthetic control for a single treated unit (donor-weight counterfactual, nested/custom V; in-space placebo permutation inference via `in_space_placebo()`)
+- [SyntheticControl](https://diff-diff.readthedocs.io/en/stable/api/synthetic_control.html) - Abadie, Diamond & Hainmueller (2010) classic synthetic control for a single treated unit (donor-weight counterfactual, nested/cv/inverse-variance/custom V; in-space placebo permutation inference via `in_space_placebo()`, plus ADH-2015 `leave_one_out()` + `in_time_placebo()` robustness)
 - [TripleDifference](https://diff-diff.readthedocs.io/en/stable/api/triple_diff.html) - triple difference (DDD) estimator for designs requiring two criteria for treatment eligibility
 - [ContinuousDiD](https://diff-diff.readthedocs.io/en/stable/api/continuous_did.html) - Callaway, Goodman-Bacon & Sant'Anna (2024) continuous treatment DiD with dose-response curves
 - [HeterogeneousAdoptionDiD](https://diff-diff.readthedocs.io/en/stable/api/had.html) - de Chaisemartin, Ciccia, D'Haultfœuille & Knau (2026) for designs where **no unit remains untreated**; local-linear estimator at the dose support boundary returning Weighted Average Slope (WAS) on Design 1' (`d̲ = 0` / QUG) or `WAS_{d̲}` on Design 1 (`d̲ > 0`, continuous-near-d̲ or mass-point), with a multi-period event-study extension (last-treatment cohort, pointwise CIs). **Panel-only** in this release - repeated cross-sections rejected by the validator. Alias `HAD`.

{diff_diff-3.5.0 → diff_diff-3.5.1}/diff_diff/__init__.py

@@ -298,7 +298,7 @@ ETWFE = WooldridgeDiD
 DCDH = ChaisemartinDHaultfoeuille
 HAD = HeterogeneousAdoptionDiD
 
-__version__ = "3.5.0"
+__version__ = "3.5.1"
 __all__ = [
     # Estimators
     "DifferenceInDifferences",

{diff_diff-3.5.0 → diff_diff-3.5.1}/diff_diff/business_report.py

@@ -1019,6 +1019,9 @@ def _lift_robustness(dr: Optional[Dict[str, Any]]) -> Dict[str, Any]:
         native_block["pre_rmspe"] = native.get("pre_rmspe")
         native_block["weight_concentration"] = native.get("weight_concentration")
         native_block["in_space_placebo"] = native.get("in_space_placebo")
+        # ADH-2015 robustness diagnostics (opt-in; "not_run" stub until run).
+        native_block["leave_one_out"] = native.get("leave_one_out")
+        native_block["in_time_placebo"] = native.get("in_time_placebo")
     return {
         "bacon": {
             "status": bacon.get("status"),

{diff_diff-3.5.0 → diff_diff-3.5.1}/diff_diff/diagnostic_report.py

@@ -2152,8 +2152,10 @@ class DiagnosticReport:
         selected ``lambda_*``).
 
         SyntheticControl: pre-treatment fit (``pre_rmspe``), donor-weight
-        concentration, and — when already computed — the in-space placebo
-        permutation p-value (``in_space_placebo``).
+        concentration, and — each surfaced only when already computed — the
+        in-space placebo permutation p-value (``in_space_placebo``), the ADH-2015
+        leave-one-out donor robustness (``leave_one_out``), and the in-time
+        backdating placebo (``in_time_placebo``).
         """
         r = self._results
         name = type(r).__name__
@@ -2251,8 +2253,11 @@ class DiagnosticReport:
         DR never triggers it implicitly, because it refits one synthetic control
         per donor (potentially many nested V searches) and the placebo layer is
         opt-in by design. (This differs from SDiD's cheaper in-time-placebo sweep,
-        which ``_sdid_native`` runs inline.) Only the in-space placebo is exposed;
-        in-time placebo and leave-one-out are ADH 2015 (not implemented).
+        which ``_sdid_native`` runs inline.) The ADH-2015 §4 diagnostics —
+        leave-one-out donor robustness (``leave_one_out()``) and the in-time
+        (backdating) placebo (``in_time_placebo()``) — are surfaced the same way:
+        opt-in, reported only once the user has run them (each refits the synthetic
+        control one or more times), else a ``status="not_run"`` stub.
         """
         out: Dict[str, Any] = {"status": "ran", "estimator": "SyntheticControl"}
         out["pre_rmspe"] = _to_python_float(getattr(r, "pre_rmspe", None))
@@ -2327,6 +2332,132 @@ class DiagnosticReport:
                     "per donor)."
                 ),
             }
+
+        # Leave-one-out donor robustness (ADH 2015 §4): opt-in, surfaced once run.
+        if getattr(r, "_loo_df", None) is not None:
+            loo_status = getattr(r, "_loo_status", None)
+            if loo_status == "ran":
+                att_range = getattr(r, "_loo_att_range", None)
+                out["leave_one_out"] = {
+                    "status": "ran",
+                    # Headline single-donor-dependence metric: the largest baseline-
+                    # relative swing (max |delta_att|). Preferred over att_range, which
+                    # can look narrow even when every drop shifts the ATT far from the
+                    # full-fit baseline in the same direction.
+                    "max_abs_delta_att": _to_python_float(
+                        getattr(r, "_loo_max_abs_delta_att", None)
+                    ),
+                    "att_range": (
+                        [_to_python_float(att_range[0]), _to_python_float(att_range[1])]
+                        if att_range is not None
+                        else None
+                    ),
+                    "n_failed": _to_python_scalar(getattr(r, "_loo_n_failed", None)),
+                }
+            else:
+                _loo_reasons = {
+                    "treated_fit_nonconverged": (
+                        "leave_one_out() was run but the treated unit's own SCM fit "
+                        "did not converge at fit time, so the baseline ATT is not a "
+                        "valid reference for the leave-one-out deltas."
+                    ),
+                    "too_few_donors": (
+                        "leave_one_out() was run but fewer than 2 donors are available "
+                        "(dropping one must leave a non-empty pool)."
+                    ),
+                    "all_refits_failed": (
+                        "leave_one_out() was run but every donor-drop refit failed to "
+                        "converge, so no valid leave-one-out estimate was produced "
+                        "(see the status='failed' rows); raise n_starts or loosen the "
+                        "optimizer tolerances."
+                    ),
+                }
+                out["leave_one_out"] = {
+                    "status": "infeasible",
+                    # Machine-readable code so consumers can distinguish a numerical
+                    # convergence failure ("all_refits_failed") from structural
+                    # infeasibility ("too_few_donors") without parsing `reason`.
+                    "reason_code": loo_status,
+                    "reason": _loo_reasons.get(
+                        loo_status, "leave_one_out() produced no valid refits."
+                    ),
+                }
+        else:
+            out["leave_one_out"] = {
+                "status": "not_run",
+                "reason": (
+                    "Call results.leave_one_out() to run leave-one-out donor "
+                    "robustness (opt-in; refits once per reportably-weighted donor)."
+                ),
+            }
+
+        # In-time (backdating) placebo (ADH 2015 §4): opt-in, surfaced once run.
+        if getattr(r, "_in_time_df", None) is not None:
+            in_time_status = getattr(r, "_in_time_status", None)
+            if in_time_status == "ran":
+                itp = r._in_time_df
+                ran = itp[itp["status"] == "ran"] if "status" in itp else itp
+                max_abs_att = float(ran["placebo_att"].abs().max()) if len(ran) else None
+                out["in_time_placebo"] = {
+                    "status": "ran",
+                    # Full coverage breakdown so a partially-usable sweep is not
+                    # overstated: n_dates is the requested grid; n_ran are the usable
+                    # placebos; n_failed / n_infeasible are the dropped remainder.
+                    "n_dates": _to_python_scalar(int(len(itp))),
+                    "n_ran": _to_python_scalar(int(len(ran))),
+                    "n_failed": _to_python_scalar(getattr(r, "_in_time_n_failed", None)),
+                    "n_infeasible": _to_python_scalar(getattr(r, "_in_time_n_infeasible", None)),
+                    "max_abs_placebo_att": _to_python_float(max_abs_att),
+                }
+            else:
+                _it_reasons = {
+                    "treated_fit_nonconverged": (
+                        "in_time_placebo() was run but the treated unit's own SCM fit "
+                        "did not converge at fit time."
+                    ),
+                    "too_few_pre_periods": (
+                        "in_time_placebo() was run but there are too few pre-treatment "
+                        "periods for any feasible placebo date (need >=3)."
+                    ),
+                    "all_dates_infeasible": (
+                        "in_time_placebo() was run but every placebo date was "
+                        "infeasible (no pre-fake period, all predictors dropped, or "
+                        "the supplied custom_v had zero mass on the surviving "
+                        "predictors after truncation)."
+                    ),
+                    "all_dates_failed": (
+                        "in_time_placebo() was run but every placebo refit failed to "
+                        "converge (none was dimensionally infeasible); raise n_starts "
+                        "or loosen the optimizer tolerances."
+                    ),
+                    "all_dates_unusable": (
+                        "in_time_placebo() was run but no placebo date produced a usable "
+                        "result: some refits failed to converge AND some dates were "
+                        "dimensionally infeasible (see n_failed / n_infeasible)."
+                    ),
+                }
+                out["in_time_placebo"] = {
+                    "status": "infeasible",
+                    # Machine-readable code distinguishing a numerical convergence
+                    # failure ("all_dates_failed") from structural infeasibility
+                    # ("all_dates_infeasible" / "too_few_pre_periods") or a mix
+                    # ("all_dates_unusable"), without parsing `reason`. The n_failed /
+                    # n_infeasible counts give the exact breakdown.
+                    "reason_code": in_time_status,
+                    "n_failed": _to_python_scalar(getattr(r, "_in_time_n_failed", None)),
+                    "n_infeasible": _to_python_scalar(getattr(r, "_in_time_n_infeasible", None)),
+                    "reason": _it_reasons.get(
+                        in_time_status, "in_time_placebo() produced no valid refits."
+                    ),
+                }
+        else:
+            out["in_time_placebo"] = {
+                "status": "not_run",
+                "reason": (
+                    "Call results.in_time_placebo() to run the in-time (backdating) "
+                    "placebo (opt-in; refits per backdated date)."
+                ),
+            }
         return out
 
     # -- Heterogeneity helpers --------------------------------------------

{diff_diff-3.5.0 → diff_diff-3.5.1}/diff_diff/efficient_did.py

@@ -4,8 +4,8 @@ Efficient Difference-in-Differences estimator.
 Implements the ATT estimator from Chen, Sant'Anna & Xie (2025).
 Without covariates, achieves the semiparametric efficiency bound via
 closed-form within-group covariances.  With covariates, uses a doubly
-robust path with OLS outcome regression, sieve propensity ratios, and
-kernel-smoothed conditional Omega*(X) (see class docstring for caveats).
+robust path with sieve outcome regressions, sieve propensity ratios, and
+kernel-smoothed conditional Omega*(X) (see class docstring for details).
 
 Under PT-All the model is overidentified and EDiD exploits this for
 tighter inference; under PT-Post it reduces to the standard
@@ -139,6 +139,69 @@ def _compute_se_from_eif(
     return float(np.sqrt(np.mean(eif**2) / n_units))
 
 
+def _hausman_quadratic_form(
+    delta: np.ndarray,
+    cov_post: np.ndarray,
+    cov_all: np.ndarray,
+) -> Tuple[float, int, float, int, bool]:
+    """Hausman statistic from the event-study delta and the two ES covariances.
+
+    Implements the Theorem A.1 test statistic of Chen, Sant'Anna & Xie (2025,
+    arXiv:2506.17729v1).  The variance-difference matrix is
+
+        V = aCov(ES_post) - aCov(ES_all) = cov_post - cov_all
+
+    (restricted minus efficient, PSD under H0 because the efficient estimator has
+    the smaller variance), and the statistic is ``H = delta' V^+ delta`` with
+    ``delta = ES_post - ES_all``.  ``V`` is inverted by Moore-Penrose pseudoinverse
+    and the number of strictly positive eigenvalues is used as the chi-square
+    degrees of freedom -- a finite-sample safeguard for a non-PSD ``V`` that equals
+    ``|E|`` (the number of post-treatment horizons) when ``V`` is well-conditioned.
+
+    Parameters
+    ----------
+    delta : ndarray, shape (|E|,)
+        Event-study difference ``ES_post - ES_all`` (restricted minus efficient).
+    cov_post, cov_all : ndarray, shape (|E|, |E|)
+        Estimator-scale covariances of the restricted (PT-Post) and efficient
+        (PT-All) event-study vectors.
+
+    Returns
+    -------
+    H : float
+        The Hausman statistic (``max(delta' V^+ delta, 0)``); NaN if ``V`` is
+        non-finite or has no positive eigenvalues.
+    effective_rank : int
+        Number of positive eigenvalues of ``V`` (the chi-square degrees of freedom).
+    p_value : float
+        Upper-tail ``chi2(effective_rank)`` p-value; NaN when ``H`` is NaN.
+    n_negative : int
+        Number of substantially negative eigenvalues of ``V`` (efficiency-reversal
+        diagnostic).
+    finite_ok : bool
+        False when ``V`` contains non-finite entries.
+    """
+    from scipy.stats import chi2
+
+    V = cov_post - cov_all
+    if not np.all(np.isfinite(V)):
+        return np.nan, 0, np.nan, 0, False
+
+    eigvals = np.linalg.eigvalsh(V)
+    max_eigval = float(np.max(np.abs(eigvals))) if len(eigvals) > 0 else 0.0
+    tol = max(1e-10 * max_eigval, 1e-15)
+
+    n_negative = int(np.sum(eigvals < -tol))
+    effective_rank = int(np.sum(eigvals > tol))
+    if effective_rank == 0:
+        return np.nan, 0, np.nan, n_negative, True
+
+    V_pinv = np.linalg.pinv(V, rcond=tol / max_eigval if max_eigval > 0 else 1e-10)
+    H = max(float(delta @ V_pinv @ delta), 0.0)
+    p_value = float(chi2.sf(H, df=effective_rank))
+    return H, effective_rank, p_value, n_negative, True
+
+
 class EfficientDiD(EfficientDiDBootstrapMixin):
     """Efficient DiD estimator (Chen, Sant'Anna & Xie 2025).
 
@@ -147,13 +210,15 @@ class EfficientDiD(EfficientDiDBootstrapMixin):
     means and covariances.
 
     With covariates, uses a doubly robust path: sieve-based propensity
-    score ratios (Eq 4.1-4.2), OLS outcome regression, sieve-estimated
-    inverse propensities (algorithm step 4), and kernel-smoothed
-    conditional Omega*(X) with per-unit efficient weights (Eq 3.12).
-    The DR property ensures consistency if either the OLS outcome model
-    or the sieve propensity ratio is correctly specified.  The OLS
-    working model for outcome regressions does not generically guarantee
-    the semiparametric efficiency bound (see REGISTRY.md).
+    score ratios (Eq 4.1-4.2), sieve outcome regressions (polynomial
+    basis, AIC/BIC order selection), sieve-estimated inverse propensities
+    (algorithm step 4), and kernel-smoothed conditional Omega*(X) with
+    per-unit efficient weights (Eq 3.12).  The DR property ensures
+    consistency if either the outcome regression or the sieve propensity
+    ratio is correctly specified; because all nuisances are sieves /
+    kernel smoothers (the paper's flexible-nuisance specification), the
+    covariate path attains the semiparametric efficiency bound under the
+    paper's regularity conditions (see REGISTRY.md).
 
     Parameters
     ----------
@@ -199,10 +264,21 @@ class EfficientDiD(EfficientDiDBootstrapMixin):
         ``control_group="last_cohort"``, also trims the pseudo-control
         period set at ``t >= last_g - anticipation`` (see REGISTRY.md).
     sieve_k_max : int or None
-        Maximum polynomial degree for sieve ratio estimation. None = auto
-        (``min(floor(n_gp^{1/5}), 5)``). Only used with covariates.
+        Maximum polynomial degree for the covariate-path sieves — the
+        propensity-ratio, inverse-propensity, AND outcome-regression fits all
+        use it. None = auto (``floor(n_pos^{1/5})`` over each group's
+        positive-weight support ``n_pos`` — the raw group size when unweighted —
+        a growing sieve with no fixed ceiling, bounded by ``n_basis < n_pos``;
+        zero-weight survey rows do not affect order selection). Only
+        used with covariates. ``sieve_k_max=1`` forces every covariate-path
+        sieve (outcome regression and both propensity sieves) to degree 1: it
+        recovers the pre-sieve linear-OLS *outcome regression* but also
+        degree-1-constrains the propensity sieves, so it does not reproduce the
+        exact pre-sieve estimator.
     sieve_criterion : str, default ``"bic"``
-        Information criterion for sieve degree selection: ``"aic"`` or ``"bic"``.
+        Information criterion (``"aic"`` or ``"bic"``) for the order selection
+        of all covariate-path sieves (propensity ratio, inverse propensity, and
+        outcome regression).
     ratio_clip : float, default 20.0
         Clip sieve propensity ratios to ``[1/ratio_clip, ratio_clip]``.
     kernel_bandwidth : float or None
@@ -855,6 +931,8 @@ class EfficientDiD(EfficientDiDBootstrapMixin):
                                 never_treated_mask,
                                 t_col_val,
                                 tpre_col_val,
+                                k_max=self.sieve_k_max,
+                                criterion=self.sieve_criterion,
                                 unit_weights=unit_level_weights,
                             )
                         # m_{g', tpre, 1}(X)
@@ -869,6 +947,8 @@ class EfficientDiD(EfficientDiDBootstrapMixin):
                                 gp_mask_for_reg,
                                 tpre_col_val,
                                 effective_p1_col,
+                                k_max=self.sieve_k_max,
+                                criterion=self.sieve_criterion,
                                 unit_weights=unit_level_weights,
                             )
                         # r_{g, inf}(X) and r_{g, g'}(X) via sieve (Eq 4.1-4.2)
@@ -1672,8 +1752,6 @@ class EfficientDiD(EfficientDiDBootstrapMixin):
         -------
         HausmanPretestResult
         """
-        from scipy.stats import chi2
-
         # Fit under both assumptions (analytical SEs only, no bootstrap)
         common_kwargs = dict(
             cluster=cluster,
@@ -1836,22 +1914,16 @@ class EfficientDiD(EfficientDiDBootstrapMixin):
                 cov_all = (eif_all_mat.T @ eif_all_mat) / (n_units**2)
                 cov_post = (eif_post_mat.T @ eif_post_mat) / (n_units**2)
 
-        V = cov_post - cov_all
-
-        if not np.all(np.isfinite(V)):
+        H, effective_rank, p_value, n_negative, finite_ok = _hausman_quadratic_form(
+            delta, cov_post, cov_all
+        )
+        if not finite_ok:
             warnings.warn(
                 "Hausman covariance matrix contains non-finite values. " "The test is unreliable.",
                 UserWarning,
                 stacklevel=2,
             )
             return _nan_result()
-
-        # Eigendecompose V — check for non-PSD
-        eigvals = np.linalg.eigvalsh(V)
-        max_eigval = np.max(np.abs(eigvals)) if len(eigvals) > 0 else 0.0
-        tol = max(1e-10 * max_eigval, 1e-15)
-
-        n_negative = int(np.sum(eigvals < -tol))
         if n_negative > 0:
             warnings.warn(
                 f"Hausman variance-difference matrix V has {n_negative} "
@@ -1860,16 +1932,8 @@ class EfficientDiD(EfficientDiDBootstrapMixin):
                 UserWarning,
                 stacklevel=2,
             )
-
-        effective_rank = int(np.sum(eigvals > tol))
         if effective_rank == 0:
             return _nan_result()
-
-        V_pinv = np.linalg.pinv(V, rcond=tol / max_eigval if max_eigval > 0 else 1e-10)
-        H = float(delta @ V_pinv @ delta)
-        H = max(H, 0.0)
-
-        p_value = float(chi2.sf(H, df=effective_rank))
         reject = p_value < alpha
 
         es_details = pd.DataFrame(