diff-diff 0.5.0__tar.gz → 1.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {diff_diff-0.5.0 → diff_diff-1.0.0}/PKG-INFO +244 -8
- {diff_diff-0.5.0 → diff_diff-1.0.0}/README.md +241 -5
- {diff_diff-0.5.0 → diff_diff-1.0.0}/diff_diff/__init__.py +87 -30
- diff_diff-1.0.0/diff_diff/bacon.py +1027 -0
- {diff_diff-0.5.0 → diff_diff-1.0.0}/diff_diff/diagnostics.py +3 -0
- {diff_diff-0.5.0 → diff_diff-1.0.0}/diff_diff/estimators.py +144 -5
- diff_diff-1.0.0/diff_diff/honest_did.py +1491 -0
- diff_diff-1.0.0/diff_diff/power.py +1350 -0
- {diff_diff-0.5.0 → diff_diff-1.0.0}/diff_diff/prep.py +1 -1
- diff_diff-1.0.0/diff_diff/staggered.py +1822 -0
- {diff_diff-0.5.0 → diff_diff-1.0.0}/diff_diff/utils.py +21 -21
- diff_diff-1.0.0/diff_diff/visualization.py +1388 -0
- {diff_diff-0.5.0 → diff_diff-1.0.0}/diff_diff.egg-info/PKG-INFO +244 -8
- {diff_diff-0.5.0 → diff_diff-1.0.0}/diff_diff.egg-info/SOURCES.txt +7 -0
- {diff_diff-0.5.0 → diff_diff-1.0.0}/pyproject.toml +12 -4
- diff_diff-1.0.0/tests/test_bacon.py +679 -0
- {diff_diff-0.5.0 → diff_diff-1.0.0}/tests/test_diagnostics.py +5 -8
- diff_diff-1.0.0/tests/test_honest_did.py +699 -0
- diff_diff-1.0.0/tests/test_power.py +691 -0
- diff_diff-1.0.0/tests/test_staggered.py +1121 -0
- diff_diff-1.0.0/tests/test_utils.py +1270 -0
- {diff_diff-0.5.0 → diff_diff-1.0.0}/tests/test_wild_bootstrap.py +3 -4
- diff_diff-0.5.0/diff_diff/staggered.py +0 -982
- diff_diff-0.5.0/diff_diff/visualization.py +0 -471
- diff_diff-0.5.0/tests/test_staggered.py +0 -390
- {diff_diff-0.5.0 → diff_diff-1.0.0}/diff_diff/results.py +0 -0
- {diff_diff-0.5.0 → diff_diff-1.0.0}/diff_diff.egg-info/dependency_links.txt +0 -0
- {diff_diff-0.5.0 → diff_diff-1.0.0}/diff_diff.egg-info/requires.txt +0 -0
- {diff_diff-0.5.0 → diff_diff-1.0.0}/diff_diff.egg-info/top_level.txt +0 -0
- {diff_diff-0.5.0 → diff_diff-1.0.0}/setup.cfg +0 -0
- {diff_diff-0.5.0 → diff_diff-1.0.0}/tests/test_estimators.py +1 -1
- {diff_diff-0.5.0 → diff_diff-1.0.0}/tests/test_prep.py +7 -7
- {diff_diff-0.5.0 → diff_diff-1.0.0}/tests/test_visualization.py +1 -1
|
@@ -1,15 +1,15 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: diff-diff
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 1.0.0
|
|
4
4
|
Summary: A library for Difference-in-Differences causal inference analysis
|
|
5
5
|
Author: diff-diff contributors
|
|
6
6
|
License-Expression: MIT
|
|
7
7
|
Project-URL: Homepage, https://github.com/igerber/diff-diff
|
|
8
|
-
Project-URL: Documentation, https://
|
|
8
|
+
Project-URL: Documentation, https://diff-diff.readthedocs.io
|
|
9
9
|
Project-URL: Repository, https://github.com/igerber/diff-diff
|
|
10
10
|
Project-URL: Issues, https://github.com/igerber/diff-diff/issues
|
|
11
11
|
Keywords: causal-inference,difference-in-differences,econometrics,statistics,treatment-effects
|
|
12
|
-
Classifier: Development Status ::
|
|
12
|
+
Classifier: Development Status :: 5 - Production/Stable
|
|
13
13
|
Classifier: Intended Audience :: Science/Research
|
|
14
14
|
Classifier: Operating System :: OS Independent
|
|
15
15
|
Classifier: Programming Language :: Python :: 3
|
|
@@ -109,9 +109,25 @@ Signif. codes: '***' 0.001, '**' 0.01, '*' 0.05, '.' 0.1
|
|
|
109
109
|
- **Synthetic DiD**: Combined DiD with synthetic control for improved robustness
|
|
110
110
|
- **Event study plots**: Publication-ready visualization of treatment effects
|
|
111
111
|
- **Parallel trends testing**: Multiple methods including equivalence tests
|
|
112
|
+
- **Goodman-Bacon decomposition**: Diagnose TWFE bias by decomposing into 2x2 comparisons
|
|
112
113
|
- **Placebo tests**: Comprehensive diagnostics including fake timing, fake group, permutation, and leave-one-out tests
|
|
114
|
+
- **Honest DiD sensitivity analysis**: Rambachan-Roth (2023) bounds and breakdown analysis for parallel trends violations
|
|
115
|
+
- **Power analysis**: MDE, sample size, and power calculations for study design; simulation-based power for any estimator
|
|
113
116
|
- **Data prep utilities**: Helper functions for common data preparation tasks
|
|
114
117
|
|
|
118
|
+
## Tutorials
|
|
119
|
+
|
|
120
|
+
We provide Jupyter notebook tutorials in `docs/tutorials/`:
|
|
121
|
+
|
|
122
|
+
| Notebook | Description |
|
|
123
|
+
|----------|-------------|
|
|
124
|
+
| `01_basic_did.ipynb` | Basic 2x2 DiD, formula interface, covariates, fixed effects, cluster-robust SE, wild bootstrap |
|
|
125
|
+
| `02_staggered_did.ipynb` | Staggered adoption with Callaway-Sant'Anna, group-time effects, aggregation methods, Bacon decomposition |
|
|
126
|
+
| `03_synthetic_did.ipynb` | Synthetic DiD, unit/time weights, inference methods, regularization |
|
|
127
|
+
| `04_parallel_trends.ipynb` | Testing parallel trends, equivalence tests, placebo tests, diagnostics |
|
|
128
|
+
| `05_honest_did.ipynb` | Honest DiD sensitivity analysis, bounds, breakdown values, visualization |
|
|
129
|
+
| `06_power_analysis.ipynb` | Power analysis, MDE, sample size calculations, simulation-based power |
|
|
130
|
+
|
|
115
131
|
## Data Preparation
|
|
116
132
|
|
|
117
133
|
diff-diff provides utility functions to help prepare your data for DiD analysis. These functions handle common data transformation tasks like creating treatment indicators, reshaping panel data, and validating data formats.
|
|
@@ -720,14 +736,66 @@ CallawaySantAnna(
|
|
|
720
736
|
estimation_method='dr', # 'dr', 'ipw', or 'reg'
|
|
721
737
|
alpha=0.05, # Significance level
|
|
722
738
|
cluster=None, # Column for cluster SEs
|
|
723
|
-
n_bootstrap=0, #
|
|
739
|
+
n_bootstrap=0, # Bootstrap iterations (0 = analytical SEs)
|
|
740
|
+
bootstrap_weight_type='rademacher', # 'rademacher', 'mammen', or 'webb'
|
|
724
741
|
seed=None # Random seed
|
|
725
742
|
)
|
|
726
743
|
```
|
|
727
744
|
|
|
728
|
-
**
|
|
729
|
-
|
|
730
|
-
|
|
745
|
+
**Multiplier bootstrap for inference:**
|
|
746
|
+
|
|
747
|
+
With few clusters or when analytical standard errors may be unreliable, use the multiplier bootstrap for valid inference. This implements the approach from Callaway & Sant'Anna (2021).
|
|
748
|
+
|
|
749
|
+
```python
|
|
750
|
+
# Bootstrap inference with 999 iterations
|
|
751
|
+
cs = CallawaySantAnna(
|
|
752
|
+
n_bootstrap=999,
|
|
753
|
+
bootstrap_weight_type='rademacher', # or 'mammen', 'webb'
|
|
754
|
+
seed=42
|
|
755
|
+
)
|
|
756
|
+
results = cs.fit(
|
|
757
|
+
data,
|
|
758
|
+
outcome='sales',
|
|
759
|
+
unit='firm_id',
|
|
760
|
+
time='year',
|
|
761
|
+
first_treat='first_treat',
|
|
762
|
+
aggregate='event_study'
|
|
763
|
+
)
|
|
764
|
+
|
|
765
|
+
# Access bootstrap results
|
|
766
|
+
print(f"Overall ATT: {results.overall_att:.3f}")
|
|
767
|
+
print(f"Bootstrap SE: {results.bootstrap_results.overall_att_se:.3f}")
|
|
768
|
+
print(f"Bootstrap 95% CI: {results.bootstrap_results.overall_att_ci}")
|
|
769
|
+
print(f"Bootstrap p-value: {results.bootstrap_results.overall_att_p_value:.4f}")
|
|
770
|
+
|
|
771
|
+
# Event study bootstrap inference
|
|
772
|
+
for rel_time, se in results.bootstrap_results.event_study_ses.items():
|
|
773
|
+
ci = results.bootstrap_results.event_study_cis[rel_time]
|
|
774
|
+
print(f"e={rel_time}: SE={se:.3f}, 95% CI=[{ci[0]:.3f}, {ci[1]:.3f}]")
|
|
775
|
+
```
|
|
776
|
+
|
|
777
|
+
**Bootstrap weight types:**
|
|
778
|
+
- `'rademacher'` - Default, ±1 with p=0.5, good for most cases
|
|
779
|
+
- `'mammen'` - Two-point distribution matching first 3 moments
|
|
780
|
+
- `'webb'` - Six-point distribution, recommended for very few clusters (<10)
|
|
781
|
+
|
|
782
|
+
**Covariate adjustment for conditional parallel trends:**
|
|
783
|
+
|
|
784
|
+
When parallel trends only holds conditional on covariates, use the `covariates` parameter:
|
|
785
|
+
|
|
786
|
+
```python
|
|
787
|
+
# Doubly robust estimation with covariates
|
|
788
|
+
cs = CallawaySantAnna(estimation_method='dr') # 'dr', 'ipw', or 'reg'
|
|
789
|
+
results = cs.fit(
|
|
790
|
+
data,
|
|
791
|
+
outcome='sales',
|
|
792
|
+
unit='firm_id',
|
|
793
|
+
time='year',
|
|
794
|
+
first_treat='first_treat',
|
|
795
|
+
covariates=['size', 'age', 'industry'], # Covariates for conditional PT
|
|
796
|
+
aggregate='event_study'
|
|
797
|
+
)
|
|
798
|
+
```
|
|
731
799
|
|
|
732
800
|
### Event Study Visualization
|
|
733
801
|
|
|
@@ -1004,6 +1072,81 @@ print(f"TOST p-value: {results['tost_p_value']:.4f}")
|
|
|
1004
1072
|
print(f"Trends equivalent: {results['equivalent']}")
|
|
1005
1073
|
```
|
|
1006
1074
|
|
|
1075
|
+
### Honest DiD Sensitivity Analysis (Rambachan-Roth)
|
|
1076
|
+
|
|
1077
|
+
Pre-trends tests have low power and can exacerbate bias. **Honest DiD** (Rambachan & Roth 2023) provides sensitivity analysis showing how robust your results are to violations of parallel trends.
|
|
1078
|
+
|
|
1079
|
+
```python
|
|
1080
|
+
from diff_diff import HonestDiD, MultiPeriodDiD
|
|
1081
|
+
|
|
1082
|
+
# First, fit a standard event study
|
|
1083
|
+
did = MultiPeriodDiD()
|
|
1084
|
+
event_results = did.fit(
|
|
1085
|
+
data,
|
|
1086
|
+
outcome='outcome',
|
|
1087
|
+
treatment='treated',
|
|
1088
|
+
time='period',
|
|
1089
|
+
post_periods=[5, 6, 7, 8, 9]
|
|
1090
|
+
)
|
|
1091
|
+
|
|
1092
|
+
# Compute honest bounds with relative magnitudes restriction
|
|
1093
|
+
# M=1 means post-treatment violations can be up to 1x the worst pre-treatment violation
|
|
1094
|
+
honest = HonestDiD(method='relative_magnitude', M=1.0)
|
|
1095
|
+
honest_results = honest.fit(event_results)
|
|
1096
|
+
|
|
1097
|
+
print(honest_results.summary())
|
|
1098
|
+
print(f"Original estimate: {honest_results.original_estimate:.4f}")
|
|
1099
|
+
print(f"Robust 95% CI: [{honest_results.ci_lb:.4f}, {honest_results.ci_ub:.4f}]")
|
|
1100
|
+
print(f"Effect robust to violations: {honest_results.is_significant}")
|
|
1101
|
+
```
|
|
1102
|
+
|
|
1103
|
+
**Sensitivity analysis over M values:**
|
|
1104
|
+
|
|
1105
|
+
```python
|
|
1106
|
+
# How do results change as we allow larger violations?
|
|
1107
|
+
sensitivity = honest.sensitivity_analysis(
|
|
1108
|
+
event_results,
|
|
1109
|
+
M_grid=[0, 0.5, 1.0, 1.5, 2.0]
|
|
1110
|
+
)
|
|
1111
|
+
|
|
1112
|
+
print(sensitivity.summary())
|
|
1113
|
+
print(f"Breakdown value: M = {sensitivity.breakdown_M}")
|
|
1114
|
+
# Breakdown = smallest M where the robust CI includes zero
|
|
1115
|
+
```
|
|
1116
|
+
|
|
1117
|
+
**Breakdown value:**
|
|
1118
|
+
|
|
1119
|
+
The breakdown value tells you how robust your conclusion is:
|
|
1120
|
+
|
|
1121
|
+
```python
|
|
1122
|
+
breakdown = honest.breakdown_value(event_results)
|
|
1123
|
+
if breakdown >= 1.0:
|
|
1124
|
+
print("Result holds even if post-treatment violations are as bad as pre-treatment")
|
|
1125
|
+
else:
|
|
1126
|
+
print(f"Result requires violations smaller than {breakdown:.1f}x pre-treatment")
|
|
1127
|
+
```
|
|
1128
|
+
|
|
1129
|
+
**Smoothness restriction (alternative approach):**
|
|
1130
|
+
|
|
1131
|
+
```python
|
|
1132
|
+
# Bounds second differences of trend violations
|
|
1133
|
+
# M=0 means linear extrapolation of pre-trends
|
|
1134
|
+
honest_smooth = HonestDiD(method='smoothness', M=0.5)
|
|
1135
|
+
smooth_results = honest_smooth.fit(event_results)
|
|
1136
|
+
```
|
|
1137
|
+
|
|
1138
|
+
**Visualization:**
|
|
1139
|
+
|
|
1140
|
+
```python
|
|
1141
|
+
from diff_diff import plot_sensitivity, plot_honest_event_study
|
|
1142
|
+
|
|
1143
|
+
# Plot sensitivity analysis
|
|
1144
|
+
plot_sensitivity(sensitivity, title="Sensitivity to Parallel Trends Violations")
|
|
1145
|
+
|
|
1146
|
+
# Event study with honest confidence intervals
|
|
1147
|
+
plot_honest_event_study(event_results, honest_results)
|
|
1148
|
+
```
|
|
1149
|
+
|
|
1007
1150
|
### Placebo Tests
|
|
1008
1151
|
|
|
1009
1152
|
Placebo tests help validate the parallel trends assumption by checking whether effects appear where they shouldn't (before treatment or in untreated groups).
|
|
@@ -1302,6 +1445,75 @@ SyntheticDiD(
|
|
|
1302
1445
|
| `get_unit_weights_df()` | Get unit weights as DataFrame |
|
|
1303
1446
|
| `get_time_weights_df()` | Get time weights as DataFrame |
|
|
1304
1447
|
|
|
1448
|
+
### HonestDiD
|
|
1449
|
+
|
|
1450
|
+
```python
|
|
1451
|
+
HonestDiD(
|
|
1452
|
+
method='relative_magnitude', # 'relative_magnitude' or 'smoothness'
|
|
1453
|
+
M=None, # Restriction parameter (default: 1.0 for RM, 0.0 for SD)
|
|
1454
|
+
alpha=0.05, # Significance level for CIs
|
|
1455
|
+
l_vec=None # Linear combination vector for target parameter
|
|
1456
|
+
)
|
|
1457
|
+
```
|
|
1458
|
+
|
|
1459
|
+
**fit() Parameters:**
|
|
1460
|
+
|
|
1461
|
+
| Parameter | Type | Description |
|
|
1462
|
+
|-----------|------|-------------|
|
|
1463
|
+
| `results` | MultiPeriodDiDResults | Results from MultiPeriodDiD.fit() |
|
|
1464
|
+
| `M` | float | Restriction parameter (overrides constructor value) |
|
|
1465
|
+
|
|
1466
|
+
**Methods:**
|
|
1467
|
+
|
|
1468
|
+
| Method | Description |
|
|
1469
|
+
|--------|-------------|
|
|
1470
|
+
| `fit(results, M)` | Compute bounds for given event study results |
|
|
1471
|
+
| `sensitivity_analysis(results, M_grid)` | Compute bounds over grid of M values |
|
|
1472
|
+
| `breakdown_value(results, tol)` | Find smallest M where CI includes zero |
|
|
1473
|
+
|
|
1474
|
+
### HonestDiDResults
|
|
1475
|
+
|
|
1476
|
+
**Attributes:**
|
|
1477
|
+
|
|
1478
|
+
| Attribute | Description |
|
|
1479
|
+
|-----------|-------------|
|
|
1480
|
+
| `original_estimate` | Point estimate under parallel trends |
|
|
1481
|
+
| `lb` | Lower bound of identified set |
|
|
1482
|
+
| `ub` | Upper bound of identified set |
|
|
1483
|
+
| `ci_lb` | Lower bound of robust confidence interval |
|
|
1484
|
+
| `ci_ub` | Upper bound of robust confidence interval |
|
|
1485
|
+
| `ci_width` | Width of robust CI |
|
|
1486
|
+
| `M` | Restriction parameter used |
|
|
1487
|
+
| `method` | Restriction method ('relative_magnitude' or 'smoothness') |
|
|
1488
|
+
| `alpha` | Significance level |
|
|
1489
|
+
| `is_significant` | True if robust CI excludes zero |
|
|
1490
|
+
|
|
1491
|
+
**Methods:**
|
|
1492
|
+
|
|
1493
|
+
| Method | Description |
|
|
1494
|
+
|--------|-------------|
|
|
1495
|
+
| `summary()` | Get formatted summary string |
|
|
1496
|
+
| `to_dict()` | Convert to dictionary |
|
|
1497
|
+
| `to_dataframe()` | Convert to pandas DataFrame |
|
|
1498
|
+
|
|
1499
|
+
### SensitivityResults
|
|
1500
|
+
|
|
1501
|
+
**Attributes:**
|
|
1502
|
+
|
|
1503
|
+
| Attribute | Description |
|
|
1504
|
+
|-----------|-------------|
|
|
1505
|
+
| `M_grid` | Array of M values analyzed |
|
|
1506
|
+
| `results` | List of HonestDiDResults for each M |
|
|
1507
|
+
| `breakdown_M` | Smallest M where CI includes zero (None if always significant) |
|
|
1508
|
+
|
|
1509
|
+
**Methods:**
|
|
1510
|
+
|
|
1511
|
+
| Method | Description |
|
|
1512
|
+
|--------|-------------|
|
|
1513
|
+
| `summary()` | Get formatted summary string |
|
|
1514
|
+
| `plot(ax)` | Plot sensitivity analysis |
|
|
1515
|
+
| `to_dataframe()` | Convert to pandas DataFrame |
|
|
1516
|
+
|
|
1305
1517
|
### Data Preparation Functions
|
|
1306
1518
|
|
|
1307
1519
|
#### generate_did_data
|
|
@@ -1525,9 +1737,23 @@ This library implements methods from the following scholarly works:
|
|
|
1525
1737
|
|
|
1526
1738
|
- **Roth, J. (2022).** "Pretest with Caution: Event-Study Estimates after Testing for Parallel Trends." *American Economic Review: Insights*, 4(3), 305-322. [https://doi.org/10.1257/aeri.20210236](https://doi.org/10.1257/aeri.20210236)
|
|
1527
1739
|
|
|
1740
|
+
- **Lakens, D. (2017).** "Equivalence Tests: A Practical Primer for t Tests, Correlations, and Meta-Analyses." *Social Psychological and Personality Science*, 8(4), 355-362. [https://doi.org/10.1177/1948550617697177](https://doi.org/10.1177/1948550617697177)
|
|
1741
|
+
|
|
1742
|
+
### Honest DiD / Sensitivity Analysis
|
|
1743
|
+
|
|
1744
|
+
The `HonestDiD` module implements sensitivity analysis methods for relaxing the parallel trends assumption:
|
|
1745
|
+
|
|
1528
1746
|
- **Rambachan, A., & Roth, J. (2023).** "A More Credible Approach to Parallel Trends." *The Review of Economic Studies*, 90(5), 2555-2591. [https://doi.org/10.1093/restud/rdad018](https://doi.org/10.1093/restud/rdad018)
|
|
1529
1747
|
|
|
1530
|
-
|
|
1748
|
+
This paper introduces the "Honest DiD" framework implemented in our `HonestDiD` class:
|
|
1749
|
+
- **Relative Magnitudes (ΔRM)**: Bounds post-treatment violations by a multiple of observed pre-treatment violations
|
|
1750
|
+
- **Smoothness (ΔSD)**: Bounds on second differences of trend violations, allowing for linear extrapolation of pre-trends
|
|
1751
|
+
- **Breakdown Analysis**: Finding the smallest violation magnitude that would overturn conclusions
|
|
1752
|
+
- **Robust Confidence Intervals**: Valid inference under partial identification
|
|
1753
|
+
|
|
1754
|
+
- **Roth, J., & Sant'Anna, P. H. C. (2023).** "When Is Parallel Trends Sensitive to Functional Form?" *Econometrica*, 91(2), 737-747. [https://doi.org/10.3982/ECTA19402](https://doi.org/10.3982/ECTA19402)
|
|
1755
|
+
|
|
1756
|
+
Discusses functional form sensitivity in parallel trends assumptions, relevant to understanding when smoothness restrictions are appropriate.
|
|
1531
1757
|
|
|
1532
1758
|
### Multi-Period and Staggered Adoption
|
|
1533
1759
|
|
|
@@ -1541,6 +1767,16 @@ This library implements methods from the following scholarly works:
|
|
|
1541
1767
|
|
|
1542
1768
|
- **Goodman-Bacon, A. (2021).** "Difference-in-Differences with Variation in Treatment Timing." *Journal of Econometrics*, 225(2), 254-277. [https://doi.org/10.1016/j.jeconom.2021.03.014](https://doi.org/10.1016/j.jeconom.2021.03.014)
|
|
1543
1769
|
|
|
1770
|
+
### Power Analysis
|
|
1771
|
+
|
|
1772
|
+
- **Bloom, H. S. (1995).** "Minimum Detectable Effects: A Simple Way to Report the Statistical Power of Experimental Designs." *Evaluation Review*, 19(5), 547-556. [https://doi.org/10.1177/0193841X9501900504](https://doi.org/10.1177/0193841X9501900504)
|
|
1773
|
+
|
|
1774
|
+
- **Burlig, F., Preonas, L., & Woerman, M. (2020).** "Panel Data and Experimental Design." *Journal of Development Economics*, 144, 102458. [https://doi.org/10.1016/j.jdeveco.2020.102458](https://doi.org/10.1016/j.jdeveco.2020.102458)
|
|
1775
|
+
|
|
1776
|
+
Essential reference for power analysis in panel DiD designs. Discusses how serial correlation (ICC) affects power and provides formulas for panel data settings.
|
|
1777
|
+
|
|
1778
|
+
- **Djimeu, E. W., & Houndolo, D.-G. (2016).** "Power Calculation for Causal Inference in Social Science: Sample Size and Minimum Detectable Effect Determination." *Journal of Development Effectiveness*, 8(4), 508-527. [https://doi.org/10.1080/19439342.2016.1244555](https://doi.org/10.1080/19439342.2016.1244555)
|
|
1779
|
+
|
|
1544
1780
|
### General Causal Inference
|
|
1545
1781
|
|
|
1546
1782
|
- **Imbens, G. W., & Rubin, D. B. (2015).** *Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction*. Cambridge University Press.
|
|
@@ -74,9 +74,25 @@ Signif. codes: '***' 0.001, '**' 0.01, '*' 0.05, '.' 0.1
|
|
|
74
74
|
- **Synthetic DiD**: Combined DiD with synthetic control for improved robustness
|
|
75
75
|
- **Event study plots**: Publication-ready visualization of treatment effects
|
|
76
76
|
- **Parallel trends testing**: Multiple methods including equivalence tests
|
|
77
|
+
- **Goodman-Bacon decomposition**: Diagnose TWFE bias by decomposing into 2x2 comparisons
|
|
77
78
|
- **Placebo tests**: Comprehensive diagnostics including fake timing, fake group, permutation, and leave-one-out tests
|
|
79
|
+
- **Honest DiD sensitivity analysis**: Rambachan-Roth (2023) bounds and breakdown analysis for parallel trends violations
|
|
80
|
+
- **Power analysis**: MDE, sample size, and power calculations for study design; simulation-based power for any estimator
|
|
78
81
|
- **Data prep utilities**: Helper functions for common data preparation tasks
|
|
79
82
|
|
|
83
|
+
## Tutorials
|
|
84
|
+
|
|
85
|
+
We provide Jupyter notebook tutorials in `docs/tutorials/`:
|
|
86
|
+
|
|
87
|
+
| Notebook | Description |
|
|
88
|
+
|----------|-------------|
|
|
89
|
+
| `01_basic_did.ipynb` | Basic 2x2 DiD, formula interface, covariates, fixed effects, cluster-robust SE, wild bootstrap |
|
|
90
|
+
| `02_staggered_did.ipynb` | Staggered adoption with Callaway-Sant'Anna, group-time effects, aggregation methods, Bacon decomposition |
|
|
91
|
+
| `03_synthetic_did.ipynb` | Synthetic DiD, unit/time weights, inference methods, regularization |
|
|
92
|
+
| `04_parallel_trends.ipynb` | Testing parallel trends, equivalence tests, placebo tests, diagnostics |
|
|
93
|
+
| `05_honest_did.ipynb` | Honest DiD sensitivity analysis, bounds, breakdown values, visualization |
|
|
94
|
+
| `06_power_analysis.ipynb` | Power analysis, MDE, sample size calculations, simulation-based power |
|
|
95
|
+
|
|
80
96
|
## Data Preparation
|
|
81
97
|
|
|
82
98
|
diff-diff provides utility functions to help prepare your data for DiD analysis. These functions handle common data transformation tasks like creating treatment indicators, reshaping panel data, and validating data formats.
|
|
@@ -685,14 +701,66 @@ CallawaySantAnna(
|
|
|
685
701
|
estimation_method='dr', # 'dr', 'ipw', or 'reg'
|
|
686
702
|
alpha=0.05, # Significance level
|
|
687
703
|
cluster=None, # Column for cluster SEs
|
|
688
|
-
n_bootstrap=0, #
|
|
704
|
+
n_bootstrap=0, # Bootstrap iterations (0 = analytical SEs)
|
|
705
|
+
bootstrap_weight_type='rademacher', # 'rademacher', 'mammen', or 'webb'
|
|
689
706
|
seed=None # Random seed
|
|
690
707
|
)
|
|
691
708
|
```
|
|
692
709
|
|
|
693
|
-
**
|
|
694
|
-
|
|
695
|
-
|
|
710
|
+
**Multiplier bootstrap for inference:**
|
|
711
|
+
|
|
712
|
+
With few clusters or when analytical standard errors may be unreliable, use the multiplier bootstrap for valid inference. This implements the approach from Callaway & Sant'Anna (2021).
|
|
713
|
+
|
|
714
|
+
```python
|
|
715
|
+
# Bootstrap inference with 999 iterations
|
|
716
|
+
cs = CallawaySantAnna(
|
|
717
|
+
n_bootstrap=999,
|
|
718
|
+
bootstrap_weight_type='rademacher', # or 'mammen', 'webb'
|
|
719
|
+
seed=42
|
|
720
|
+
)
|
|
721
|
+
results = cs.fit(
|
|
722
|
+
data,
|
|
723
|
+
outcome='sales',
|
|
724
|
+
unit='firm_id',
|
|
725
|
+
time='year',
|
|
726
|
+
first_treat='first_treat',
|
|
727
|
+
aggregate='event_study'
|
|
728
|
+
)
|
|
729
|
+
|
|
730
|
+
# Access bootstrap results
|
|
731
|
+
print(f"Overall ATT: {results.overall_att:.3f}")
|
|
732
|
+
print(f"Bootstrap SE: {results.bootstrap_results.overall_att_se:.3f}")
|
|
733
|
+
print(f"Bootstrap 95% CI: {results.bootstrap_results.overall_att_ci}")
|
|
734
|
+
print(f"Bootstrap p-value: {results.bootstrap_results.overall_att_p_value:.4f}")
|
|
735
|
+
|
|
736
|
+
# Event study bootstrap inference
|
|
737
|
+
for rel_time, se in results.bootstrap_results.event_study_ses.items():
|
|
738
|
+
ci = results.bootstrap_results.event_study_cis[rel_time]
|
|
739
|
+
print(f"e={rel_time}: SE={se:.3f}, 95% CI=[{ci[0]:.3f}, {ci[1]:.3f}]")
|
|
740
|
+
```
|
|
741
|
+
|
|
742
|
+
**Bootstrap weight types:**
|
|
743
|
+
- `'rademacher'` - Default, ±1 with p=0.5, good for most cases
|
|
744
|
+
- `'mammen'` - Two-point distribution matching first 3 moments
|
|
745
|
+
- `'webb'` - Six-point distribution, recommended for very few clusters (<10)
|
|
746
|
+
|
|
747
|
+
**Covariate adjustment for conditional parallel trends:**
|
|
748
|
+
|
|
749
|
+
When parallel trends only holds conditional on covariates, use the `covariates` parameter:
|
|
750
|
+
|
|
751
|
+
```python
|
|
752
|
+
# Doubly robust estimation with covariates
|
|
753
|
+
cs = CallawaySantAnna(estimation_method='dr') # 'dr', 'ipw', or 'reg'
|
|
754
|
+
results = cs.fit(
|
|
755
|
+
data,
|
|
756
|
+
outcome='sales',
|
|
757
|
+
unit='firm_id',
|
|
758
|
+
time='year',
|
|
759
|
+
first_treat='first_treat',
|
|
760
|
+
covariates=['size', 'age', 'industry'], # Covariates for conditional PT
|
|
761
|
+
aggregate='event_study'
|
|
762
|
+
)
|
|
763
|
+
```
|
|
696
764
|
|
|
697
765
|
### Event Study Visualization
|
|
698
766
|
|
|
@@ -969,6 +1037,81 @@ print(f"TOST p-value: {results['tost_p_value']:.4f}")
|
|
|
969
1037
|
print(f"Trends equivalent: {results['equivalent']}")
|
|
970
1038
|
```
|
|
971
1039
|
|
|
1040
|
+
### Honest DiD Sensitivity Analysis (Rambachan-Roth)
|
|
1041
|
+
|
|
1042
|
+
Pre-trends tests have low power and can exacerbate bias. **Honest DiD** (Rambachan & Roth 2023) provides sensitivity analysis showing how robust your results are to violations of parallel trends.
|
|
1043
|
+
|
|
1044
|
+
```python
|
|
1045
|
+
from diff_diff import HonestDiD, MultiPeriodDiD
|
|
1046
|
+
|
|
1047
|
+
# First, fit a standard event study
|
|
1048
|
+
did = MultiPeriodDiD()
|
|
1049
|
+
event_results = did.fit(
|
|
1050
|
+
data,
|
|
1051
|
+
outcome='outcome',
|
|
1052
|
+
treatment='treated',
|
|
1053
|
+
time='period',
|
|
1054
|
+
post_periods=[5, 6, 7, 8, 9]
|
|
1055
|
+
)
|
|
1056
|
+
|
|
1057
|
+
# Compute honest bounds with relative magnitudes restriction
|
|
1058
|
+
# M=1 means post-treatment violations can be up to 1x the worst pre-treatment violation
|
|
1059
|
+
honest = HonestDiD(method='relative_magnitude', M=1.0)
|
|
1060
|
+
honest_results = honest.fit(event_results)
|
|
1061
|
+
|
|
1062
|
+
print(honest_results.summary())
|
|
1063
|
+
print(f"Original estimate: {honest_results.original_estimate:.4f}")
|
|
1064
|
+
print(f"Robust 95% CI: [{honest_results.ci_lb:.4f}, {honest_results.ci_ub:.4f}]")
|
|
1065
|
+
print(f"Effect robust to violations: {honest_results.is_significant}")
|
|
1066
|
+
```
|
|
1067
|
+
|
|
1068
|
+
**Sensitivity analysis over M values:**
|
|
1069
|
+
|
|
1070
|
+
```python
|
|
1071
|
+
# How do results change as we allow larger violations?
|
|
1072
|
+
sensitivity = honest.sensitivity_analysis(
|
|
1073
|
+
event_results,
|
|
1074
|
+
M_grid=[0, 0.5, 1.0, 1.5, 2.0]
|
|
1075
|
+
)
|
|
1076
|
+
|
|
1077
|
+
print(sensitivity.summary())
|
|
1078
|
+
print(f"Breakdown value: M = {sensitivity.breakdown_M}")
|
|
1079
|
+
# Breakdown = smallest M where the robust CI includes zero
|
|
1080
|
+
```
|
|
1081
|
+
|
|
1082
|
+
**Breakdown value:**
|
|
1083
|
+
|
|
1084
|
+
The breakdown value tells you how robust your conclusion is:
|
|
1085
|
+
|
|
1086
|
+
```python
|
|
1087
|
+
breakdown = honest.breakdown_value(event_results)
|
|
1088
|
+
if breakdown >= 1.0:
|
|
1089
|
+
print("Result holds even if post-treatment violations are as bad as pre-treatment")
|
|
1090
|
+
else:
|
|
1091
|
+
print(f"Result requires violations smaller than {breakdown:.1f}x pre-treatment")
|
|
1092
|
+
```
|
|
1093
|
+
|
|
1094
|
+
**Smoothness restriction (alternative approach):**
|
|
1095
|
+
|
|
1096
|
+
```python
|
|
1097
|
+
# Bounds second differences of trend violations
|
|
1098
|
+
# M=0 means linear extrapolation of pre-trends
|
|
1099
|
+
honest_smooth = HonestDiD(method='smoothness', M=0.5)
|
|
1100
|
+
smooth_results = honest_smooth.fit(event_results)
|
|
1101
|
+
```
|
|
1102
|
+
|
|
1103
|
+
**Visualization:**
|
|
1104
|
+
|
|
1105
|
+
```python
|
|
1106
|
+
from diff_diff import plot_sensitivity, plot_honest_event_study
|
|
1107
|
+
|
|
1108
|
+
# Plot sensitivity analysis
|
|
1109
|
+
plot_sensitivity(sensitivity, title="Sensitivity to Parallel Trends Violations")
|
|
1110
|
+
|
|
1111
|
+
# Event study with honest confidence intervals
|
|
1112
|
+
plot_honest_event_study(event_results, honest_results)
|
|
1113
|
+
```
|
|
1114
|
+
|
|
972
1115
|
### Placebo Tests
|
|
973
1116
|
|
|
974
1117
|
Placebo tests help validate the parallel trends assumption by checking whether effects appear where they shouldn't (before treatment or in untreated groups).
|
|
@@ -1267,6 +1410,75 @@ SyntheticDiD(
|
|
|
1267
1410
|
| `get_unit_weights_df()` | Get unit weights as DataFrame |
|
|
1268
1411
|
| `get_time_weights_df()` | Get time weights as DataFrame |
|
|
1269
1412
|
|
|
1413
|
+
### HonestDiD
|
|
1414
|
+
|
|
1415
|
+
```python
|
|
1416
|
+
HonestDiD(
|
|
1417
|
+
method='relative_magnitude', # 'relative_magnitude' or 'smoothness'
|
|
1418
|
+
M=None, # Restriction parameter (default: 1.0 for RM, 0.0 for SD)
|
|
1419
|
+
alpha=0.05, # Significance level for CIs
|
|
1420
|
+
l_vec=None # Linear combination vector for target parameter
|
|
1421
|
+
)
|
|
1422
|
+
```
|
|
1423
|
+
|
|
1424
|
+
**fit() Parameters:**
|
|
1425
|
+
|
|
1426
|
+
| Parameter | Type | Description |
|
|
1427
|
+
|-----------|------|-------------|
|
|
1428
|
+
| `results` | MultiPeriodDiDResults | Results from MultiPeriodDiD.fit() |
|
|
1429
|
+
| `M` | float | Restriction parameter (overrides constructor value) |
|
|
1430
|
+
|
|
1431
|
+
**Methods:**
|
|
1432
|
+
|
|
1433
|
+
| Method | Description |
|
|
1434
|
+
|--------|-------------|
|
|
1435
|
+
| `fit(results, M)` | Compute bounds for given event study results |
|
|
1436
|
+
| `sensitivity_analysis(results, M_grid)` | Compute bounds over grid of M values |
|
|
1437
|
+
| `breakdown_value(results, tol)` | Find smallest M where CI includes zero |
|
|
1438
|
+
|
|
1439
|
+
### HonestDiDResults
|
|
1440
|
+
|
|
1441
|
+
**Attributes:**
|
|
1442
|
+
|
|
1443
|
+
| Attribute | Description |
|
|
1444
|
+
|-----------|-------------|
|
|
1445
|
+
| `original_estimate` | Point estimate under parallel trends |
|
|
1446
|
+
| `lb` | Lower bound of identified set |
|
|
1447
|
+
| `ub` | Upper bound of identified set |
|
|
1448
|
+
| `ci_lb` | Lower bound of robust confidence interval |
|
|
1449
|
+
| `ci_ub` | Upper bound of robust confidence interval |
|
|
1450
|
+
| `ci_width` | Width of robust CI |
|
|
1451
|
+
| `M` | Restriction parameter used |
|
|
1452
|
+
| `method` | Restriction method ('relative_magnitude' or 'smoothness') |
|
|
1453
|
+
| `alpha` | Significance level |
|
|
1454
|
+
| `is_significant` | True if robust CI excludes zero |
|
|
1455
|
+
|
|
1456
|
+
**Methods:**
|
|
1457
|
+
|
|
1458
|
+
| Method | Description |
|
|
1459
|
+
|--------|-------------|
|
|
1460
|
+
| `summary()` | Get formatted summary string |
|
|
1461
|
+
| `to_dict()` | Convert to dictionary |
|
|
1462
|
+
| `to_dataframe()` | Convert to pandas DataFrame |
|
|
1463
|
+
|
|
1464
|
+
### SensitivityResults
|
|
1465
|
+
|
|
1466
|
+
**Attributes:**
|
|
1467
|
+
|
|
1468
|
+
| Attribute | Description |
|
|
1469
|
+
|-----------|-------------|
|
|
1470
|
+
| `M_grid` | Array of M values analyzed |
|
|
1471
|
+
| `results` | List of HonestDiDResults for each M |
|
|
1472
|
+
| `breakdown_M` | Smallest M where CI includes zero (None if always significant) |
|
|
1473
|
+
|
|
1474
|
+
**Methods:**
|
|
1475
|
+
|
|
1476
|
+
| Method | Description |
|
|
1477
|
+
|--------|-------------|
|
|
1478
|
+
| `summary()` | Get formatted summary string |
|
|
1479
|
+
| `plot(ax)` | Plot sensitivity analysis |
|
|
1480
|
+
| `to_dataframe()` | Convert to pandas DataFrame |
|
|
1481
|
+
|
|
1270
1482
|
### Data Preparation Functions
|
|
1271
1483
|
|
|
1272
1484
|
#### generate_did_data
|
|
@@ -1490,9 +1702,23 @@ This library implements methods from the following scholarly works:
|
|
|
1490
1702
|
|
|
1491
1703
|
- **Roth, J. (2022).** "Pretest with Caution: Event-Study Estimates after Testing for Parallel Trends." *American Economic Review: Insights*, 4(3), 305-322. [https://doi.org/10.1257/aeri.20210236](https://doi.org/10.1257/aeri.20210236)
|
|
1492
1704
|
|
|
1705
|
+
- **Lakens, D. (2017).** "Equivalence Tests: A Practical Primer for t Tests, Correlations, and Meta-Analyses." *Social Psychological and Personality Science*, 8(4), 355-362. [https://doi.org/10.1177/1948550617697177](https://doi.org/10.1177/1948550617697177)
|
|
1706
|
+
|
|
1707
|
+
### Honest DiD / Sensitivity Analysis
|
|
1708
|
+
|
|
1709
|
+
The `HonestDiD` module implements sensitivity analysis methods for relaxing the parallel trends assumption:
|
|
1710
|
+
|
|
1493
1711
|
- **Rambachan, A., & Roth, J. (2023).** "A More Credible Approach to Parallel Trends." *The Review of Economic Studies*, 90(5), 2555-2591. [https://doi.org/10.1093/restud/rdad018](https://doi.org/10.1093/restud/rdad018)
|
|
1494
1712
|
|
|
1495
|
-
|
|
1713
|
+
This paper introduces the "Honest DiD" framework implemented in our `HonestDiD` class:
|
|
1714
|
+
- **Relative Magnitudes (ΔRM)**: Bounds post-treatment violations by a multiple of observed pre-treatment violations
|
|
1715
|
+
- **Smoothness (ΔSD)**: Bounds on second differences of trend violations, allowing for linear extrapolation of pre-trends
|
|
1716
|
+
- **Breakdown Analysis**: Finding the smallest violation magnitude that would overturn conclusions
|
|
1717
|
+
- **Robust Confidence Intervals**: Valid inference under partial identification
|
|
1718
|
+
|
|
1719
|
+
- **Roth, J., & Sant'Anna, P. H. C. (2023).** "When Is Parallel Trends Sensitive to Functional Form?" *Econometrica*, 91(2), 737-747. [https://doi.org/10.3982/ECTA19402](https://doi.org/10.3982/ECTA19402)
|
|
1720
|
+
|
|
1721
|
+
Discusses functional form sensitivity in parallel trends assumptions, relevant to understanding when smoothness restrictions are appropriate.
|
|
1496
1722
|
|
|
1497
1723
|
### Multi-Period and Staggered Adoption
|
|
1498
1724
|
|
|
@@ -1506,6 +1732,16 @@ This library implements methods from the following scholarly works:
|
|
|
1506
1732
|
|
|
1507
1733
|
- **Goodman-Bacon, A. (2021).** "Difference-in-Differences with Variation in Treatment Timing." *Journal of Econometrics*, 225(2), 254-277. [https://doi.org/10.1016/j.jeconom.2021.03.014](https://doi.org/10.1016/j.jeconom.2021.03.014)
|
|
1508
1734
|
|
|
1735
|
+
### Power Analysis
|
|
1736
|
+
|
|
1737
|
+
- **Bloom, H. S. (1995).** "Minimum Detectable Effects: A Simple Way to Report the Statistical Power of Experimental Designs." *Evaluation Review*, 19(5), 547-556. [https://doi.org/10.1177/0193841X9501900504](https://doi.org/10.1177/0193841X9501900504)
|
|
1738
|
+
|
|
1739
|
+
- **Burlig, F., Preonas, L., & Woerman, M. (2020).** "Panel Data and Experimental Design." *Journal of Development Economics*, 144, 102458. [https://doi.org/10.1016/j.jdeveco.2020.102458](https://doi.org/10.1016/j.jdeveco.2020.102458)
|
|
1740
|
+
|
|
1741
|
+
Essential reference for power analysis in panel DiD designs. Discusses how serial correlation (ICC) affects power and provides formulas for panel data settings.
|
|
1742
|
+
|
|
1743
|
+
- **Djimeu, E. W., & Houndolo, D.-G. (2016).** "Power Calculation for Causal Inference in Social Science: Sample Size and Minimum Detectable Effect Determination." *Journal of Development Effectiveness*, 8(4), 508-527. [https://doi.org/10.1080/19439342.2016.1244555](https://doi.org/10.1080/19439342.2016.1244555)
|
|
1744
|
+
|
|
1509
1745
|
### General Causal Inference
|
|
1510
1746
|
|
|
1511
1747
|
- **Imbens, G. W., & Rubin, D. B. (2015).** *Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction*. Cambridge University Press.
|