Pymkp 0.1.0__tar.gz

pymkp-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025, Yangyang (Claire) Meng
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

pymkp-0.1.0/MANIFEST.in

@@ -0,0 +1,10 @@
+include CONTRIBUTING.md
+include HISTORY.md
+include LICENSE
+include README.md
+
+recursive-include tests *
+recursive-exclude * __pycache__
+recursive-exclude * *.py[co]
+
+recursive-include docs *.md Makefile *.jpg *.png *.gif

pymkp-0.1.0/PKG-INFO

@@ -0,0 +1,402 @@
+Metadata-Version: 2.4
+Name: Pymkp
+Version: 0.1.0
+Summary: A Python toolkit for estimating firm-level markups using production function-based marginal cost recovery.
+Author-email: "Yangyang (Claire) Meng" <ym3593@nyu.edu>
+Maintainer-email: "Yangyang (Claire) Meng" <ym3593@nyu.edu>
+License: MIT
+Project-URL: homepage, https://github.com/immortalsRDJ/PyMarkup
+Project-URL: repository, https://github.com/immortalsRDJ/PyMarkup
+Project-URL: documentation, https://github.com/immortalsRDJ/PyMarkup#readme
+Project-URL: bugs, https://github.com/immortalsRDJ/PyMarkup/issues
+Project-URL: changelog, https://github.com/immortalsRDJ/PyMarkup/blob/main/CHANGELOG.md
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pandas>=2.0.0
+Requires-Dist: numpy<2.0,>=1.24.0
+Requires-Dist: scipy<1.14,>=1.10.0
+Requires-Dist: statsmodels>=0.14.0
+Requires-Dist: linearmodels>=5.0
+Requires-Dist: pydantic>=2.0
+Requires-Dist: typer>=0.9.0
+Requires-Dist: rich>=13.0
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: matplotlib>=3.7.0
+Requires-Dist: seaborn>=0.12.0
+Requires-Dist: openpyxl>=3.0.0
+Requires-Dist: requests>=2.32.5
+Requires-Dist: fredapi>=0.5.2
+Requires-Dist: wrds>=3.1.6
+Provides-Extra: test
+Requires-Dist: coverage[toml]>=7.0; extra == "test"
+Requires-Dist: pytest>=7.0; extra == "test"
+Requires-Dist: pytest-cov>=4.0; extra == "test"
+Requires-Dist: hypothesis>=6.0; extra == "test"
+Requires-Dist: ruff>=0.1.0; extra == "test"
+Requires-Dist: ipdb>=0.13; extra == "test"
+Provides-Extra: dev
+Requires-Dist: Pymkp[test]; extra == "dev"
+Requires-Dist: jupyter>=1.0; extra == "dev"
+Requires-Dist: ipython>=8.0; extra == "dev"
+Provides-Extra: wrds
+Requires-Dist: wrds>=3.1.0; extra == "wrds"
+Dynamic: license-file
+
+# PyMarkup
+
+A Python toolkit for estimating firm-level markups using production function-based marginal cost recovery.
+
+## Installation
+
+```bash
+git clone https://github.com/immortalsRDJ/PyMarkup
+cd PyMarkup
+uv sync
+```
+
+For WRDS data downloads, add the `wrds` extra:
+
+```bash
+uv sync --extra wrds
+```
+
+## Quick Start
+
+### Option 1: Command Line (Recommended)
+
+```bash
+# 1. Set up config file
+cp config.example.yaml config.yaml
+# Edit config.yaml with your API keys and settings
+
+# 2. Run the full pipeline
+uv run pymarkup run-all --config config.yaml
+
+# Or skip data download if you already have the data
+uv run pymarkup run-all --config config.yaml --skip-download
+```
+
+### Option 2: Python Script
+
+```python
+from PyMarkup import MarkupPipeline, PipelineConfig, EstimatorConfig
+
+config = PipelineConfig(
+    compustat_path="Input/DLEU/Compustat_annual.csv",
+    macro_vars_path="Input/DLEU/macro_vars_new.xlsx",
+    estimator=EstimatorConfig(method="wooldridge_iv"),
+)
+
+pipeline = MarkupPipeline(config)
+results = pipeline.run()
+results.save(output_dir="Output/", format="csv")
+```
+
+## Command Line Reference
+
+### Full Pipeline
+
+```bash
+# Run everything (download + estimate + figures)
+uv run pymarkup run-all --config config.yaml
+
+# Skip all downloads (use existing data)
+uv run pymarkup run-all --config config.yaml --skip-download
+
+# Skip only Compustat download (no WRDS credentials needed)
+uv run pymarkup run-all --config config.yaml --skip-compustat
+
+# Skip figure generation
+uv run pymarkup run-all --config config.yaml --no-figures
+
+# Verbose output for debugging
+uv run pymarkup run-all --config config.yaml -v
+```
+
+### Individual Commands
+
+```bash
+# Download data only
+uv run pymarkup download ppi                        # PPI (no credentials needed)
+uv run pymarkup download cpi --config config.yaml   # CPI (needs FRED API key)
+uv run pymarkup download all --config config.yaml   # All datasets
+
+# Run estimation only (requires existing data)
+uv run pymarkup estimate --config config.yaml
+
+# Validate input data
+uv run pymarkup validate Input/DLEU/Compustat_annual.csv
+
+# Check version
+uv run pymarkup version
+```
+
+## Configuration
+
+### Setting Up Credentials
+
+1. Copy the example config file:
+   ```bash
+   cp config.example.yaml config.yaml
+   ```
+
+2. Edit `config.yaml` with your credentials:
+   ```yaml
+   fred_api_key: "your-fred-api-key"
+   wrds_username: "your-wrds-username"
+   ```
+
+Alternatively, set environment variables: `FRED_API_KEY`, `WRDS_USERNAME`
+
+### Data Requirements
+
+| Data Source | Credentials | How to Get |
+|-------------|-------------|------------|
+| Compustat (WRDS) | WRDS account | Register at [WRDS](https://wrds-www.wharton.upenn.edu/) |
+| CPI (FRED) | FRED API key | Free at [FRED](https://fred.stlouisfed.org/docs/api/api_key.html) |
+| PPI (BLS) | None | Public data from [BLS](https://download.bls.gov/pub/time.series/pc/) |
+| Macro variables | N/A | Included in repo: `Input/DLEU/macro_vars_new.xlsx` |
+| NAICS descriptions | N/A | Included in repo: `Input/Other/NAICS_2D_Description.xlsx` |
+| DEU observations | N/A | Optional: Original DLEU paper firm-year sample (see below) |
+
+## Pipeline Overview
+
+```
+Download -> Data Preparation -> Elasticity Estimation -> Markup Calculation -> Figures & Decomposition
+```
+
+### 1. Data Download
+
+Downloads raw data from external sources:
+
+```python
+from PyMarkup.data import download_compustat, download_cpi, download_ppi, load_config
+
+config = load_config("config.yaml")
+download_ppi(config)        # No credentials needed
+download_cpi(config)        # Requires FRED API key
+download_compustat(config)  # Requires WRDS credentials
+```
+
+**Data Sources:**
+- **PPI**: Bureau of Labor Statistics Producer Price Index data from https://download.bls.gov/pub/time.series/pc/
+- **CPI**: Federal Reserve Economic Data (FRED) Consumer Price Index
+- **Compustat**: WRDS Compustat Fundamentals Annual/Quarterly
+
+### 2. Data Preparation
+
+Cleans and prepares the Compustat panel:
+- Deduplicates firm-year observations
+- Extracts NAICS industry codes
+- Deflates monetary values by GDP
+- Computes market shares
+- Trims outliers
+
+### 3. Elasticity Estimation
+
+Estimates output elasticity of variable inputs (θ) at the industry-year level:
+
+| Method | Class | Use Case |
+|--------|-------|----------|
+| Wooldridge IV | `WooldridgeIVEstimator` | Main method, addresses endogeneity via IV/2SLS |
+| Cost Share | `CostShareEstimator` | Fast baseline, no regression needed |
+| ACF | `ACFEstimator` | Robustness, two-stage GMM with control function |
+
+```python
+from PyMarkup.estimators import WooldridgeIVEstimator
+
+estimator = WooldridgeIVEstimator(specification="spec2")
+elasticities = estimator.estimate_elasticities(panel_data)
+```
+
+#### SG&A Configuration
+
+All three estimators support including SG&A (Selling, General & Administrative expenses) as a third input in the production function:
+
+| Estimator | Parameter | Options | Default |
+|-----------|-----------|---------|---------|
+| Wooldridge IV | `specification` | `"spec1"` (COGS+K), `"spec2"` (COGS+K+SG&A) | `"spec2"` |
+| Cost Share | `include_sga` | `True`, `False` | `False` |
+| ACF | `include_sga` | `True`, `False` | `False` |
+
+```python
+from PyMarkup.estimators import ACFEstimator, CostShareEstimator, WooldridgeIVEstimator
+
+# Wooldridge IV: use spec2 for 3-input (COGS + Capital + SG&A)
+iv_est = WooldridgeIVEstimator(specification="spec2")
+
+# Cost Share: include SG&A in cost share calculation
+cs_est = CostShareEstimator(include_sga=True)
+
+# ACF: include SG&A as third input
+acf_est = ACFEstimator(include_sga=True)
+```
+
+Via pipeline config:
+
+```python
+from PyMarkup import PipelineConfig, EstimatorConfig
+
+config = PipelineConfig(
+    compustat_path="Input/DLEU/Compustat_annual.csv",
+    macro_vars_path="Input/DLEU/macro_vars_new.xlsx",
+    estimator=EstimatorConfig(
+        method="all",
+        iv_specification="spec2",    # Wooldridge IV with SG&A
+        cs_include_sga=True,         # Cost Share with SG&A
+        acf_include_sga=True,        # ACF with SG&A
+    ),
+)
+```
+
+#### Aggregation Weights
+
+When aggregating firm-level markups to industry or economy level, you can choose the weighting scheme:
+
+| Weight Type | Formula | Use Case |
+|-------------|---------|----------|
+| `"revenue"` (default) | `firm_revenue / total_revenue` | Standard approach, larger firms weighted more |
+| `"cost"` | `firm_cogs / total_cogs` | Weight by production scale |
+
+```python
+from PyMarkup.core.markup_calculation import aggregate_markups
+
+# Revenue-weighted aggregation (default)
+agg = aggregate_markups(
+    firm_markups, by="year", method="weighted_mean",
+    weight_type="revenue", panel_data=panel_data
+)
+
+# Cost-weighted aggregation
+agg = aggregate_markups(
+    firm_markups, by="year", method="weighted_mean",
+    weight_type="cost", panel_data=panel_data
+)
+```
+
+Via pipeline config:
+```python
+config = PipelineConfig(
+    ...
+    aggregation_weight="revenue",  # or "cost"
+)
+```
+
+#### DEU Sample Filtering
+
+To replicate the original De Loecker, Eeckhout, and Unger (2020) paper results, you can filter the Compustat data to only include the firm-year observations from the original study:
+
+```yaml
+# config.yaml
+use_deu_sample: true
+deu_observations_path: "Input/DLEU/DEU_observations.dta"
+```
+
+Or via Python:
+
+```python
+config = PipelineConfig(
+    compustat_path="Input/DLEU/Compustat_annual.csv",
+    macro_vars_path="Input/DLEU/macro_vars_new.xlsx",
+    use_deu_sample=True,
+    deu_observations_path="Input/DLEU/DEU_observations.dta",
+    ...
+)
+```
+
+When enabled, the pipeline performs an inner merge on `gvkey` and `year` to filter to the original DLEU sample (approximately 242,000 firm-year observations from 1955-2016).
+
+### 4. Markup Calculation
+
+Computes firm-level markups using the De Loecker & Warzynski formula:
+
+```
+markup = θ / cost_share
+where cost_share = COGS / Revenue
+```
+
+### 5. Figures
+
+| Figure | Function | Description |
+|--------|----------|-------------|
+| Aggregate Markup | `plot_aggregate_markup()` | Time series of aggregate markups |
+| PPI vs Markup | `plot_markup_vs_ppi()` | Scatter plot with weighted OLS regression |
+
+### 6. Decomposition
+
+Dynamic Olley-Pakes decomposition of aggregate markup changes (DLEU 2020). The decomposition runs automatically in the pipeline for Wooldridge IV and Cost Share methods.
+
+**Decomposes markup growth into three components:**
+
+| Component | Description |
+|-----------|-------------|
+| **Within** | Markup changes within continuing firms |
+| **Reallocation** | Market share shifts toward high/low-markup firms |
+| **Net Entry** | Difference between entering and exiting firms |
+
+The components sum to the total markup change: `Within + Reallocation + Net Entry = Markup (benchmark)`
+
+**Output files:**
+
+| File | Description |
+|------|-------------|
+| `Output/intermediate/decomposition_wooldridge_iv.csv` | IV decomposition results |
+| `Output/intermediate/decomposition_cost_share.csv` | Cost Share decomposition results |
+| `Output/figures/Decomposition - Wooldridge IV (YYYY-YYYY).pdf` | IV decomposition figure |
+| `Output/figures/Decomposition - Cost Share (YYYY-YYYY).pdf` | Cost Share decomposition figure |
+
+**Standalone usage:**
+
+```python
+from PyMarkup.decomposition import OlleyPakesDecomposition, plot_decomposition
+
+op = OlleyPakesDecomposition(
+    firm_var="gvkey",
+    time_var="year",
+    markup_var="markup",
+    weight_var="sale_D",
+)
+decomp_results = op.decompose(firm_markups)
+
+# Plot with cumulative markup levels (DLEU Figure IV style)
+# All lines start at the same baseline and show counterfactual paths:
+# "What would markup be if only this component operated?"
+plot_decomposition(
+    decomp_results,
+    cumulative=True,
+    base_markup=1.21,  # Base period aggregate markup (e.g., 1980 value)
+    save_path="Output/decomposition.pdf",
+)
+```
+
+
+## Project Structure
+
+```
+src/PyMarkup/
+├── core/              # Data preparation, markup calculation, figures
+├── data/              # Data downloaders and loaders
+├── estimators/        # WooldridgeIV, CostShare, ACF estimators
+├── pipeline/          # MarkupPipeline orchestrator, config
+├── decomposition/     # Dynamic Olley-Pakes decomposition
+├── io/                # I/O schemas (Pydantic)
+└── cli/               # CLI commands
+
+Input/                 # Raw data (not version controlled)
+Intermediate/          # Generated datasets, theta estimates
+Output/                # Figures and tables
+```
+
+## License
+
+MIT License