data-analyst-mcp 0.4.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -2
- package/dist/adapters/_census_micro.js +2 -2
- package/dist/adapters/_census_micro.js.map +1 -1
- package/dist/adapters/_irs-soi_catalog.d.ts +42 -0
- package/dist/adapters/_irs-soi_catalog.js +318 -0
- package/dist/adapters/_irs-soi_catalog.js.map +1 -0
- package/dist/adapters/_irs-soi_eo.d.ts +10 -0
- package/dist/adapters/_irs-soi_eo.js +249 -0
- package/dist/adapters/_irs-soi_eo.js.map +1 -0
- package/dist/adapters/_irs-soi_tables.d.ts +83 -0
- package/dist/adapters/_irs-soi_tables.js +505 -0
- package/dist/adapters/_irs-soi_tables.js.map +1 -0
- package/dist/adapters/irs-soi.js +68 -0
- package/dist/adapters/irs-soi.js.map +1 -1
- package/dist/analysis/compute.d.ts +22 -10
- package/dist/analysis/compute.js +75 -16
- package/dist/analysis/compute.js.map +1 -1
- package/dist/analysis/expr.d.ts +2 -0
- package/dist/analysis/expr.js +17 -0
- package/dist/analysis/expr.js.map +1 -1
- package/dist/analysis/replicate.d.ts +15 -0
- package/dist/analysis/replicate.js +228 -0
- package/dist/analysis/replicate.js.map +1 -0
- package/dist/analysis/rerun.js +2 -1
- package/dist/analysis/rerun.js.map +1 -1
- package/dist/analysis/table.d.ts +14 -0
- package/dist/analysis/table.js +79 -28
- package/dist/analysis/table.js.map +1 -1
- package/dist/analysis/tools.js +16 -4
- package/dist/analysis/tools.js.map +1 -1
- package/dist/core/packs.d.ts +14 -5
- package/dist/core/packs.js.map +1 -1
- package/dist/{deep/_opm_parquet.d.ts → core/parquet.d.ts} +2 -1
- package/dist/{deep/_opm_parquet.js → core/parquet.js} +7 -5
- package/dist/core/parquet.js.map +1 -0
- package/dist/core/tabular.d.ts +16 -3
- package/dist/core/tabular.js +87 -4
- package/dist/core/tabular.js.map +1 -1
- package/dist/deep/_opm_io.js +2 -2
- package/dist/deep/_opm_io.js.map +1 -1
- package/dist/deep/bea.js +1 -0
- package/dist/deep/bea.js.map +1 -1
- package/dist/deep/fred.js +40 -9
- package/dist/deep/fred.js.map +1 -1
- package/dist/deep/ipums.js +1 -0
- package/dist/deep/ipums.js.map +1 -1
- package/dist/deep/opm.js +1 -1
- package/dist/deep/opm.js.map +1 -1
- package/dist/report/tools.js +2 -1
- package/dist/report/tools.js.map +1 -1
- package/dist/server.d.ts +1 -1
- package/dist/server.js +10 -6
- package/dist/server.js.map +1 -1
- package/dist/tools.d.ts +3 -1
- package/dist/tools.js +55 -3
- package/dist/tools.js.map +1 -1
- package/guide/WORKFLOW.md +6 -5
- package/guide/references/report_patterns.md +5 -2
- package/guide/references/report_schema.md +140 -0
- package/guide/references/routing.md +4 -1
- package/package.json +1 -1
- package/packs/census/source.md +8 -4
- package/packs/fred/source.md +2 -1
- package/packs/ipums/docs/report_patterns.md +1 -1
- package/packs/ipums/docs/variables.md +4 -3
- package/packs/ipums/source.md +10 -8
- package/packs/irs-soi/docs/estimate_patterns.md +95 -0
- package/packs/irs-soi/endpoints.json +96 -0
- package/packs/irs-soi/policy.json +69 -7
- package/packs/irs-soi/source.md +118 -11
- package/packs/irs-soi/tables.json +1439 -0
- package/packs/opm/docs/opm_api.md +2 -1
- package/packs/opm/source.md +2 -1
- package/dist/deep/_opm_parquet.js.map +0 -1
|
@@ -57,7 +57,7 @@ state agencies, research institutes), so there is usually a precedent to follow
|
|
|
57
57
|
- Report the unweighted cell size internally. Suppress or footnote cells with fewer than about 100 respondents; unstable estimates are not published.
|
|
58
58
|
- **Dollars:** convert to a single year's dollars and name the year and index ("in 2023 dollars, using the CPI-U-RS"; the PCE price index is a common choice for earnings and income), with CPI99 or `adjustMonetaryValues`. CPS ASEC income is for the **previous calendar year** (ASEC 2023 means 2022 income). ACS income is for the past 12 months, and IPUMS expresses it in survey-year dollars.
|
|
59
59
|
- **Top-coding:** INCWAGE is top-coded, which mostly matters for means at high education levels. Mention it.
|
|
60
|
-
- **Sampling error:** for differences that drive conclusions, note whether they are large relative to sampling error. ACS replicate weights (REPWTP, 80) and CPS ASEC replicate weights (REPWTP, 160) give proper standard errors. CPS basic monthly files have none (REPWTP is ASEC-only) and no design variables: an SE there can only be approximate (clustered by household, `SERIAL` within month).
|
|
60
|
+
- **Sampling error:** for differences that drive conclusions, note whether they are large relative to sampling error. ACS replicate weights (REPWTP, 80) and CPS ASEC replicate weights (REPWTP, 160) give proper standard errors. CPS basic monthly files have none (REPWTP is ASEC-only) and no design variables: an SE there can only be approximate (clustered by household, `SERIAL` within month). With replicate weights in the extract, `compute`'s `replicate_se` step (method `sdr`) gives the standard errors; without them, report which weight was used and that standard errors were not computed.
|
|
61
61
|
|
|
62
62
|
**Publication structure (mirrored in `build_report`, `"mode": "report"`; only when a report is asked for)**
|
|
63
63
|
|
|
@@ -51,9 +51,10 @@ The ACS starts in 2000 (large samples from 2005, group quarters from 2006). CPS
|
|
|
51
51
|
|
|
52
52
|
Pooling several years for one estimate: either average the yearly estimates, or divide the weights by the number of years so that totals stay annual.
|
|
53
53
|
|
|
54
|
-
**Standard errors.** Census recommends replicate weights.
|
|
55
|
-
|
|
56
|
-
|
|
54
|
+
**Standard errors.** Census recommends replicate weights. `compute`'s `replicate_se` step computes replicate
|
|
55
|
+
standard errors (`"weight": "PERWT", "replicates": "REPWTP1:REPWTP80", "method": "sdr"`); filtering to a domain
|
|
56
|
+
before it is fine with replicates. Design-based (linearized) standard errors are not computed here: without
|
|
57
|
+
replicates, report which weight was used and that standard errors were not computed. The methods:
|
|
57
58
|
- **USA (ACS):** request `REPWTP` to get 80 columns. They add a lot of size to multi-year extracts; a linearized SE with the automatic `STRATA` and `CLUSTER` is a fallback only: it can be far off for quantities the ACS weights are raked to (age, sex: median-age margins came out 3 to 13 times the replicate ones) and understate others. Prefer replicates; to save size, case-select a second extract with REPWTP to the areas compared.
|
|
58
59
|
- **CPS ASEC:** request `REPWTP` to get 160 columns for replicate standard errors.
|
|
59
60
|
- **NHIS, MEPS:** no replicate weights; Taylor linearization with the design variables (added automatically).
|
package/packs/ipums/source.md
CHANGED
|
@@ -109,11 +109,13 @@ weights, terms).
|
|
|
109
109
|
- **Extract gotchas**: IPUMS adds detailed versions automatically (EDUC brings EDUCD); a data-quality
|
|
110
110
|
flag or case-selection code that a sample lacks returns HTTP 400 (fix the spec and fetch again);
|
|
111
111
|
ACS 2020 1-year has experimental weights.
|
|
112
|
-
- **Standard errors**:
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
112
|
+
- **Standard errors**: request the replicate weights (`REPWTP`, `REPWT`) and use `compute`'s `replicate_se`
|
|
113
|
+
step: `{"replicate_se": {"estimates": {...}, "weight": "PERWT", "replicates": "REPWTP1:REPWTP80", "method": "sdr"}}`
|
|
114
|
+
(4/80, Census's method; ASEC `ASECWT` with `REPWTP1:REPWTP160`). Design-based (STRATA/CLUSTER) standard errors
|
|
115
|
+
are not computed here. Without standard errors, report which weight each estimate uses and that standard errors
|
|
116
|
+
were not computed, and never call a difference "statistically different" (with them, the rule is beyond 1.645
|
|
117
|
+
standard errors of the difference, 90 percent; otherwise "about the same"). The methods: ACS person estimates
|
|
118
|
+
use REPWTP (80 replicates), household estimates (tenure, household income) REPWT with HHWT, Census's own method; a
|
|
117
119
|
linearized SE with STRATA and CLUSTER (added automatically) can understate a PUMS margin (half of the
|
|
118
120
|
replicate one in a 2024 tenure check), so replicates are preferred. CPS ASEC: REPWTP (160) for person
|
|
119
121
|
estimates, or REPWT (160) with ASECWTH on householders (`RELATE == 101`) for household estimates such as
|
|
@@ -132,9 +134,9 @@ weights, terms).
|
|
|
132
134
|
every threshold or definition for sensitivity; benchmark a headline number against a published figure (Census, BLS).
|
|
133
135
|
- Public IPUMS data cannot show legal status, benefits, or full administrative controls: say so.
|
|
134
136
|
|
|
135
|
-
Analysis: build tables with `compute` on `data/ipums/ipums_extract.csv`, decoding values with `codebook.csv
|
|
136
|
-
|
|
137
|
-
|
|
137
|
+
Analysis: build tables with `compute` on `data/ipums/ipums_extract.csv`, decoding values with `codebook.csv`, and
|
|
138
|
+
standard errors with its `replicate_se` step (replicate weights in the extract). This server has no IPUMS recode
|
|
139
|
+
helpers.
|
|
138
140
|
Typical questions, conventions (populations, definitions, benchmarks) and chart forms:
|
|
139
141
|
`docs/report_patterns.md`.
|
|
140
142
|
|
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
# Estimate patterns for IRS SOI tables
|
|
2
|
+
|
|
3
|
+
Use this file when a question asks what a tax change would raise or cost, who pays a tax, how large a tax base
|
|
4
|
+
is, or how IRS collections are made up. The published tables give the base and the tax at today's law; an
|
|
5
|
+
estimate built on them is **static**: it holds incomes and behavior fixed. Say so in the answer, every time.
|
|
6
|
+
|
|
7
|
+
The method each recipe follows:
|
|
8
|
+
1. Fetch the table (`table`), long by default; keep only the rows and columns the estimate needs (`rows`,
|
|
9
|
+
`columns`, `section`).
|
|
10
|
+
2. Compute with `compute` on the fetched CSV (`filter`, `derive`, `groupby` + `rollup`, `pivot_wider`, `deflate`);
|
|
11
|
+
every number in the answer comes from a fetched table or a step recorded in the lineage.
|
|
12
|
+
3. State the year type (tax, fiscal, filing), the units (`UNITS`: usually thousands of dollars), the flags of
|
|
13
|
+
the cells used (`FLAG`), and that the estimate is static.
|
|
14
|
+
|
|
15
|
+
Never invent a parameter. A behavioral response (an elasticity of taxable income, a realization response for
|
|
16
|
+
capital gains, avoidance) is an assumption: use it only when the user gives one or a cited source does, record it
|
|
17
|
+
with `record_decisions`, and show the static number beside the adjusted one.
|
|
18
|
+
|
|
19
|
+
## Individual income tax
|
|
20
|
+
|
|
21
|
+
- **Revenue from a change in a bracket rate** (`in_3.5`, tax generated by rate). Column "Tax generated at
|
|
22
|
+
specified rate | <r> percent | Income taxed at rate" is the income taxed at that rate. A change of d
|
|
23
|
+
percentage points raises about income taxed at the rate x d / 100. Check: tax generated at the rate / income
|
|
24
|
+
taxed at the rate = the rate. Example: TY2023, 37 percent: 1,159,707,580 thousand dollars of income, 429,091,805
|
|
25
|
+
thousand of tax; one more point = about $11.6 billion a year, static, before credits interact. A change in a
|
|
26
|
+
bracket threshold needs the income near the threshold, which the table does not give: say so.
|
|
27
|
+
```json
|
|
28
|
+
{"name": "rate37", "endpoint": "table", "params": {"table": "in_3.5", "year": 2023, "rows": ["Total"], "columns": ["37 percent"]}}
|
|
29
|
+
```
|
|
30
|
+
`compute` steps: `{"filter": "COLUMN == \"Tax generated at specified rate | 37 percent | Income taxed at rate\""}`,
|
|
31
|
+
`{"derive": {"change_thousand_dollars": "VALUE * 0.01"}}`.
|
|
32
|
+
- **Average and effective tax rates by income** (`in_1.1`, `in_3.3`): total income tax / AGI by size of AGI,
|
|
33
|
+
from the same table and year. Table 1.1 has the percentages already in "Taxable returns | Total income tax |
|
|
34
|
+
Percent of | Adjusted gross income less deficit".
|
|
35
|
+
- **Who pays: shares of the top 1 percent and others** (`shares_4.1`, every year 2001 onward in one file):
|
|
36
|
+
sections "Adjusted gross income share", "Total income tax share", "Average tax rate", "Adjusted gross income
|
|
37
|
+
floor on percentiles" (current and constant dollars). Quote the share with its year and the percentile
|
|
38
|
+
definition (returns excluding dependents, positive AGI).
|
|
39
|
+
- **Credits and deductions** (`in_2.5` EITC, `in_2.1` itemized deductions, `in_1.4` sources of income and
|
|
40
|
+
adjustments): the cost of a credit change is about the change per claim x the number of claims in the affected
|
|
41
|
+
AGI classes, static. Itemized deductions are in Table 2.1 by type (SALT, mortgage interest, charitable).
|
|
42
|
+
- **Capital gains** (`in_1.4A`): gains and losses by size of AGI; rate changes on gains are where a realization
|
|
43
|
+
response matters most: never leave it out without saying so.
|
|
44
|
+
- **Over time**: one workbook per year (`start_year`/`end_year`); compare `COLUMN` text across years (column
|
|
45
|
+
numbers move); deflate with a price index fetched from bls or bea (`deflate`), and say which.
|
|
46
|
+
|
|
47
|
+
## Business taxes
|
|
48
|
+
|
|
49
|
+
- **Corporate tax base and effective rate** (`corp_2.1`, `corp_1`, `corp_5.1`; C corporations alone:
|
|
50
|
+
`corp_2.3`, `corp_3.3`, `corp_5.3`): "Net income (less deficit)", "Income subject to tax", "Total income tax
|
|
51
|
+
before credits", "Total income tax after credits". Average effective rate = tax after credits / net income (or
|
|
52
|
+
/ income subject to tax: name the denominator). Static revenue of a statutory rate change on C corporations
|
|
53
|
+
= income subject to tax x d / 100, before credits and the minimum tax; label it an upper bound.
|
|
54
|
+
- **By industry or size**: `corp_1` / `corp_5.x` rows are industries (`LEVEL` and `PARENT` give the hierarchy:
|
|
55
|
+
sum one level only); `corp_2.x` columns are asset-size classes. Cells marked `*` have high sampling
|
|
56
|
+
variability (the CVs are in `corp_1-CV`); `d` cells are deleted but counted in the totals.
|
|
57
|
+
- **Pass-through income** (`partnership_1`, `corp_2.4` / `corp_6.1` for S corporations, `soleprop_1`): net income
|
|
58
|
+
by industry. A change aimed at pass-through income (the qualified business income deduction, a surtax) starts
|
|
59
|
+
from these totals; the share that reaches individuals in each bracket is not in these tables (only microdata).
|
|
60
|
+
- **Corporate profits in the national accounts** differ from SOI net income (`historical_8`, `historical_15`
|
|
61
|
+
reconcile them); use bea for NIPA profits.
|
|
62
|
+
|
|
63
|
+
## Collections and administration (IRS Data Book, fiscal years)
|
|
64
|
+
|
|
65
|
+
- **Collections by type of tax** (`databook_1-6`: gross collections 1960 onward in one file; `databook_1-1`: gross,
|
|
66
|
+
refunds and net for the last two years) and **by state** (`databook_1-5`, `databook_1-8` refunds). Gross
|
|
67
|
+
collections are cash received in the fiscal year, of any tax year; state is the filer's address (a business
|
|
68
|
+
pays where its headquarters files), not where income was earned: say so for state comparisons.
|
|
69
|
+
- **Excise taxes by type** (`historical_20`, fiscal years 1999 onward).
|
|
70
|
+
- **Enforcement**: audit coverage and recommended additional tax (`databook_3-1`); cost of collecting $100
|
|
71
|
+
(`databook_6-2`).
|
|
72
|
+
|
|
73
|
+
## Estate and gift tax, wealth
|
|
74
|
+
|
|
75
|
+
- **Exemption or rate change** (`estate_1`, by size of gross estate and tax status): the number of taxable
|
|
76
|
+
returns, the taxable estate, the tentative tax and the net estate tax by size class. Raising the exemption
|
|
77
|
+
removes the returns below it and lowers the rest; the table gives the classes, not the returns: say how a class
|
|
78
|
+
was split, or keep to class boundaries. Filing years mix years of death.
|
|
79
|
+
- **By state** (`estate_2`), **charitable bequests** (`estate_3`), **gifts** (`gift_1`, whole dollars, not
|
|
80
|
+
thousands), **top wealth holders** (`wealth_1` ... `_6`, estimates from estate returns, millions of dollars).
|
|
81
|
+
|
|
82
|
+
## Nonprofits
|
|
83
|
+
|
|
84
|
+
- **Sector size** (`eo_1`, `eo_3`, `pf_1`, `historical_16`): revenue, expenses and assets by asset size or Code
|
|
85
|
+
section, from Form 990 samples. **One organization or a state** (`eo_bmf` registry; `eo_extract` Form 990 data
|
|
86
|
+
by EIN or state): the registry's amounts come from each organization's latest return, of different years.
|
|
87
|
+
- The charitable deduction's cost is an individual-income-tax item (`in_2.1`, charitable contributions by size
|
|
88
|
+
of AGI), not a nonprofit table.
|
|
89
|
+
|
|
90
|
+
## What to write
|
|
91
|
+
|
|
92
|
+
Lead with the number and its frame: "One more percentage point on the 37 percent rate would have raised about
|
|
93
|
+
$11.6 billion on tax year 2023 income, before any change in behavior (IRS SOI, Publication 1304 Table 3.5)."
|
|
94
|
+
Then the base, the year type, the units, the flags, and what the static estimate leaves out. Ship the fetched
|
|
95
|
+
tables and the computed exhibit; the citation names the table and year.
|
|
@@ -83,6 +83,78 @@
|
|
|
83
83
|
"paging": {"type": "none"},
|
|
84
84
|
"max_rows": 200000,
|
|
85
85
|
"flatten": false
|
|
86
|
+
},
|
|
87
|
+
"table": {
|
|
88
|
+
"path": "/{file}",
|
|
89
|
+
"method": "GET",
|
|
90
|
+
"description": "One SOI published table (an .xls/.xlsx workbook on irs.gov) as tidy rows: national individual tables by size of AGI and by percentile (Publication 1304, tax shares of the top 1%), corporations (Publication 16), partnerships, sole proprietorships, integrated business data, the IRS Data Book (collections, refunds and returns by type of tax and state), historical tables, estate tax, gift tax, personal wealth, exempt organizations and private foundations, and international (CFC, foreign tax credit, country-by-country). `table` names a catalog entry (table_list lists all 125, e.g. in_1.1, in_3.5, shares_4.1, corp_1, corp_2.1, partnership_1, soleprop_1, databook_1-6, estate_1, eo_1). The adapter finds the year's file on the table's index page, downloads it and reads the layout: header lines, the column-number row, sections, row levels, footnote markers. Long layout (default): one row per cell, TABLE, YEAR, YEAR_TYPE, SHEET, SECTION, LINE, LEVEL, PARENT, ROW_LABEL, COL (the published column number), COLUMN (the header lines joined with ' | '), VALUE (as published), FLAG (marker: * high sampling variability, ** or d or [d] suppressed/combined, [n] footnote, n.a.), UNITS (the headnote's money unit, usually thousands of dollars).",
|
|
91
|
+
"params": {
|
|
92
|
+
"table": {"in": "spec", "required": true, "desc": "catalog id (see table_list), or a product word and table number such as 'corporation 2.1' or 'data book 1-6'"},
|
|
93
|
+
"year": {"in": "spec", "desc": "tax year (fiscal year for the Data Book, filing year for estate and gift tables); default the latest posted"},
|
|
94
|
+
"tax_year": {"in": "spec", "desc": "same as year"},
|
|
95
|
+
"fiscal_year": {"in": "spec", "desc": "same as year (Data Book)"},
|
|
96
|
+
"filing_year": {"in": "spec", "desc": "same as year (estate and gift)"},
|
|
97
|
+
"start_year": {"in": "spec", "desc": "first year of a range (one workbook each)"},
|
|
98
|
+
"end_year": {"in": "spec", "desc": "last year of a range"},
|
|
99
|
+
"rows": {"in": "spec", "desc": "keep rows whose label equals one of these texts (or, for an item equal to no label, contains it), e.g. ['$1,000,000 or more']"},
|
|
100
|
+
"columns": {"in": "spec", "desc": "keep columns by published column number (e.g. [1, 3]) or header text (e.g. ['Taxable income'])"},
|
|
101
|
+
"section": {"in": "spec", "desc": "keep rows under a section label containing this text (e.g. 'Accumulated from largest', 'Average tax rate')"},
|
|
102
|
+
"layout": {"in": "spec", "enum": ["long", "wide"], "desc": "long (default: one row per cell) or wide (one row per line, a column per table column)"},
|
|
103
|
+
"sheet": {"in": "spec", "desc": "sheet name or 1-based number for multi-sheet workbooks; default every sheet with a table"},
|
|
104
|
+
"header_lines": {"in": "spec", "desc": "1-based header line numbers, only for a workbook whose header the reader gets wrong (see the notes)"}
|
|
105
|
+
},
|
|
106
|
+
"records": "$adapter",
|
|
107
|
+
"paging": {"type": "none"},
|
|
108
|
+
"max_rows": 200000,
|
|
109
|
+
"flatten": false
|
|
110
|
+
},
|
|
111
|
+
"table_list": {
|
|
112
|
+
"path": "/",
|
|
113
|
+
"method": "GET",
|
|
114
|
+
"description": "The catalog of SOI published tables the table endpoint reads (packs/irs-soi/tables.json): TABLE id, PRODUCT, NUMBER, TITLE, YEAR_TYPE, FIRST_YEAR, LAST_YEAR_VERIFIED, WHOLE_SERIES (one file holds every year), INDEX (the page listing the files). live: true also reads each index page and lists YEARS_POSTED now.",
|
|
115
|
+
"params": {
|
|
116
|
+
"product": {"in": "spec", "desc": "keep products containing this text: individual, percentile, corporations, partnerships, sole, integrated, Data Book, historical, estate, gift, wealth, exempt, international"},
|
|
117
|
+
"query": {"in": "spec", "desc": "words to find in the table titles, e.g. 'tax shares percentile' or 'excise'"},
|
|
118
|
+
"live": {"in": "spec", "desc": "true: read the index pages and list the years posted now (one request per page)"}
|
|
119
|
+
},
|
|
120
|
+
"records": "$adapter",
|
|
121
|
+
"paging": {"type": "none"},
|
|
122
|
+
"max_rows": 1000,
|
|
123
|
+
"flatten": false
|
|
124
|
+
},
|
|
125
|
+
"eo_bmf": {
|
|
126
|
+
"path": "/eo_{state}.csv",
|
|
127
|
+
"method": "GET",
|
|
128
|
+
"description": "Exempt Organizations Business Master File extract: every organization recognized as tax-exempt, one CSV per state (updated monthly; 2.0 million organizations nationally), 28 columns: EIN, NAME, address, SUBSECTION (501(c) type), FOUNDATION, RULING, DEDUCTIBILITY, ACTIVITY, NTEE_CD, ASSET_CD, INCOME_CD, ASSET_AMT, INCOME_AMT, REVENUE_AMT (whole dollars, latest return on file), TAX_PERIOD and more. Codes in https://www.irs.gov/pub/foia/ig/tege/eo-info.pdf.",
|
|
129
|
+
"params": {
|
|
130
|
+
"state": {"in": "spec", "required": true, "desc": "state(s): USPS code or name, PR, or XX (international); one file each, 10 at most"},
|
|
131
|
+
"ntee": {"in": "spec", "desc": "NTEE code prefix(es), e.g. 'E' (health) or 'B82'"},
|
|
132
|
+
"subsection": {"in": "spec", "desc": "501(c) subsection(s), e.g. '03' for 501(c)(3)"},
|
|
133
|
+
"ein": {"in": "spec", "desc": "EIN(s), 9 digits (leading zeros kept)"},
|
|
134
|
+
"name": {"in": "spec", "desc": "text to find in NAME or SORT_NAME"},
|
|
135
|
+
"columns": {"in": "spec", "desc": "columns to keep beside EIN, NAME and STATE"}
|
|
136
|
+
},
|
|
137
|
+
"records": "$adapter",
|
|
138
|
+
"paging": {"type": "none"},
|
|
139
|
+
"max_rows": 200000,
|
|
140
|
+
"flatten": false
|
|
141
|
+
},
|
|
142
|
+
"eo_extract": {
|
|
143
|
+
"path": "/{yy}eoextract{form}.zip",
|
|
144
|
+
"method": "GET",
|
|
145
|
+
"description": "SOI annual extract of Form 990, 990-EZ or 990-PF financial data (every return filed in the year; 2012-2024): one zip per form and year (57 MB for the 2024 Form 990 file, a 247 MB CSV of 246 columns), read as a stream keeping only the asked EINs, or the organizations of the asked states (EINs from the current EO BMF). Amounts in whole dollars; field names and definitions in the year's {yy}eofinextractdoc.xlsx. Some 2012-2017 files are fixed-width .dat files and are refused.",
|
|
146
|
+
"params": {
|
|
147
|
+
"year": {"in": "spec", "required": false, "desc": "filing year of the extract, 2012-2024"},
|
|
148
|
+
"filing_year": {"in": "spec", "desc": "same as year"},
|
|
149
|
+
"form": {"in": "spec", "enum": ["990", "990ez", "990pf", "990EZ", "990PF", "990-EZ", "990-PF"], "desc": "990 (default), 990ez or 990pf"},
|
|
150
|
+
"ein": {"in": "spec", "desc": "EIN(s) to keep"},
|
|
151
|
+
"state": {"in": "spec", "desc": "keep the organizations of these states (EO BMF)"},
|
|
152
|
+
"columns": {"in": "spec", "desc": "columns to keep beside EIN (names as in the extract, e.g. totrevenue, totfuncexpns)"}
|
|
153
|
+
},
|
|
154
|
+
"records": "$adapter",
|
|
155
|
+
"paging": {"type": "none"},
|
|
156
|
+
"max_rows": 200000,
|
|
157
|
+
"flatten": false
|
|
86
158
|
}
|
|
87
159
|
},
|
|
88
160
|
"catalog": {
|
|
@@ -109,6 +181,30 @@
|
|
|
109
181
|
"name": "SOI Migration Data (state-to-state and county-to-county)",
|
|
110
182
|
"endpoint": "migration",
|
|
111
183
|
"description": "taxpayer migration inflow outflow between states and counties; households, people and income moving; net migration"
|
|
184
|
+
},
|
|
185
|
+
"tables": {
|
|
186
|
+
"id": "table",
|
|
187
|
+
"name": "SOI published tables (Publication 1304, Publication 16, Data Book, ...)",
|
|
188
|
+
"endpoint": "table",
|
|
189
|
+
"description": "national tax statistics tables: income and tax by AGI size, tax shares by percentile, corporate tax base, partnerships, sole proprietorships, IRS collections by type of tax and state, estate tax, nonprofits; budget and revenue estimates"
|
|
190
|
+
},
|
|
191
|
+
"table_list": {
|
|
192
|
+
"id": "table_list",
|
|
193
|
+
"name": "Catalog of SOI published tables",
|
|
194
|
+
"endpoint": "table_list",
|
|
195
|
+
"description": "list of SOI tables, titles and years"
|
|
196
|
+
},
|
|
197
|
+
"eo_bmf": {
|
|
198
|
+
"id": "eo_bmf",
|
|
199
|
+
"name": "Exempt Organizations Business Master File",
|
|
200
|
+
"endpoint": "eo_bmf",
|
|
201
|
+
"description": "nonprofit organizations registry by state: EIN, name, 501(c) type, NTEE, assets, revenue"
|
|
202
|
+
},
|
|
203
|
+
"eo_extract": {
|
|
204
|
+
"id": "eo_extract",
|
|
205
|
+
"name": "SOI Form 990 / 990-EZ / 990-PF annual extracts",
|
|
206
|
+
"endpoint": "eo_extract",
|
|
207
|
+
"description": "nonprofit financial data from Form 990 returns: revenue, expenses, assets by organization"
|
|
112
208
|
}
|
|
113
209
|
}
|
|
114
210
|
}
|
|
@@ -1,13 +1,17 @@
|
|
|
1
1
|
{
|
|
2
2
|
"id": "irs-soi",
|
|
3
|
-
"title": "IRS Statistics of Income:
|
|
3
|
+
"title": "IRS Statistics of Income: income tax data by ZIP code, county, state and migration; national tax tables (Publication 1304, Publication 16, Data Book), business, estate, nonprofit and international statistics (irs.gov)",
|
|
4
4
|
"agency": "Internal Revenue Service, Statistics of Income Division",
|
|
5
5
|
"depth": "standard",
|
|
6
6
|
"domains": [
|
|
7
7
|
"taxes",
|
|
8
8
|
"income",
|
|
9
9
|
"migration",
|
|
10
|
-
"geography"
|
|
10
|
+
"geography",
|
|
11
|
+
"budget",
|
|
12
|
+
"business",
|
|
13
|
+
"nonprofits",
|
|
14
|
+
"wealth"
|
|
11
15
|
],
|
|
12
16
|
"keywords": [
|
|
13
17
|
"irs",
|
|
@@ -37,7 +41,51 @@
|
|
|
37
41
|
"inflow",
|
|
38
42
|
"outflow",
|
|
39
43
|
"taxpayer migration",
|
|
40
|
-
"income moving between states"
|
|
44
|
+
"income moving between states",
|
|
45
|
+
"publication 1304",
|
|
46
|
+
"individual complete report",
|
|
47
|
+
"publication 16",
|
|
48
|
+
"corporation complete report",
|
|
49
|
+
"irs data book",
|
|
50
|
+
"gross collections",
|
|
51
|
+
"tax collections by state",
|
|
52
|
+
"irs collections",
|
|
53
|
+
"refunds",
|
|
54
|
+
"tax shares",
|
|
55
|
+
"top 1 percent",
|
|
56
|
+
"average tax rate",
|
|
57
|
+
"effective tax rate",
|
|
58
|
+
"tax generated by rate",
|
|
59
|
+
"income taxed at each rate",
|
|
60
|
+
"revenue estimate",
|
|
61
|
+
"budget estimate",
|
|
62
|
+
"static score",
|
|
63
|
+
"tax base",
|
|
64
|
+
"corporate income tax",
|
|
65
|
+
"corporate tax base",
|
|
66
|
+
"income subject to tax",
|
|
67
|
+
"corporations by industry",
|
|
68
|
+
"s corporations",
|
|
69
|
+
"partnerships",
|
|
70
|
+
"pass-through income",
|
|
71
|
+
"sole proprietorships",
|
|
72
|
+
"schedule c",
|
|
73
|
+
"business receipts",
|
|
74
|
+
"estate tax",
|
|
75
|
+
"gift tax",
|
|
76
|
+
"personal wealth",
|
|
77
|
+
"top wealth holders",
|
|
78
|
+
"excise taxes",
|
|
79
|
+
"historical tax tables",
|
|
80
|
+
"tax rates history",
|
|
81
|
+
"exempt organizations",
|
|
82
|
+
"nonprofits",
|
|
83
|
+
"form 990",
|
|
84
|
+
"private foundations",
|
|
85
|
+
"eo bmf",
|
|
86
|
+
"controlled foreign corporations",
|
|
87
|
+
"foreign tax credit",
|
|
88
|
+
"country-by-country"
|
|
41
89
|
],
|
|
42
90
|
"questions": [
|
|
43
91
|
"average adjusted gross income by ZIP code in Maryland",
|
|
@@ -46,7 +94,16 @@
|
|
|
46
94
|
"earned income tax credit claims by county",
|
|
47
95
|
"how many taxpayers moved from New York to Florida and how much income they took",
|
|
48
96
|
"net migration of tax filers into Texas counties",
|
|
49
|
-
"state and local tax deduction by ZIP code"
|
|
97
|
+
"state and local tax deduction by ZIP code",
|
|
98
|
+
"how much would raising the top income tax rate by one point raise",
|
|
99
|
+
"share of federal income tax paid by the top 1 percent",
|
|
100
|
+
"average tax rate by income group",
|
|
101
|
+
"corporate income tax after credits as a share of net income by industry",
|
|
102
|
+
"IRS gross collections by type of tax and by state",
|
|
103
|
+
"how many estates pay estate tax and how much",
|
|
104
|
+
"partnership and S corporation net income by industry",
|
|
105
|
+
"revenue, expenses and assets of 501(c)(3) organizations",
|
|
106
|
+
"nonprofit organizations in a state by NTEE category"
|
|
50
107
|
],
|
|
51
108
|
"homepage": "https://www.irs.gov/statistics/soi-tax-stats-data-by-geographic-area",
|
|
52
109
|
"terms_url": "https://www.irs.gov/statistics/soi-tax-stats-about-soi",
|
|
@@ -86,7 +143,7 @@
|
|
|
86
143
|
"cache": {
|
|
87
144
|
"mode": "run_only",
|
|
88
145
|
"ttl_hours": null,
|
|
89
|
-
"why": "Each request downloads one published all-states CSV (0.1 MB for state migration up to 216 MB for ZIP code data) and keeps only the asked rows; data/ is reused within a run only."
|
|
146
|
+
"why": "Each request downloads one published all-states CSV (0.1 MB for state migration up to 216 MB for ZIP code data) and keeps only the asked rows; data/ is reused within a run only. The published tables (20-300 KB workbooks) and their index pages are read once per run."
|
|
90
147
|
},
|
|
91
148
|
"redistribution": {
|
|
92
149
|
"mode": "allowed",
|
|
@@ -99,9 +156,14 @@
|
|
|
99
156
|
"clarify": [
|
|
100
157
|
"Which tax year (ZIP, county and state data) or which pair of filing years (migration: 2022-2023 compares addresses on returns filed in 2022 and 2023)? Default: the latest posted (tax year 2023; migration 2022-2023).",
|
|
101
158
|
"All returns combined, or broken out by AGI size class (agi_stub)? Averages such as AGI per return need the all-classes total (ZIP 00000 / county 000 rows or the no-AGI-class file).",
|
|
102
|
-
"Migration: people moving into the place (inflow) or out of it (outflow), between states or between counties? Net migration needs both."
|
|
159
|
+
"Migration: people moving into the place (inflow) or out of it (outflow), between states or between counties? Net migration needs both.",
|
|
160
|
+
"Published tables: which table and year (table_list lists them; default the latest posted)? Tax year (Publication 1304, 16), fiscal year (Data Book) or filing year (estate)?",
|
|
161
|
+
"An estimate from these tables is static (no behavioral response): is that what is wanted, or is there a cited behavioral assumption to apply on top?"
|
|
103
162
|
],
|
|
104
163
|
"formats": [
|
|
105
|
-
"csv"
|
|
164
|
+
"csv",
|
|
165
|
+
"xls",
|
|
166
|
+
"xlsx",
|
|
167
|
+
"zip"
|
|
106
168
|
]
|
|
107
169
|
}
|
package/packs/irs-soi/source.md
CHANGED
|
@@ -1,6 +1,9 @@
|
|
|
1
|
-
# IRS Statistics of Income:
|
|
2
|
-
Depth: standard. Key: none. Counts and amounts from
|
|
3
|
-
county and state and by AGI size class,
|
|
1
|
+
# IRS Statistics of Income: tax statistics by geography, national tables, business, estate and nonprofits (irs.gov)
|
|
2
|
+
Depth: standard. Key: none. Counts and amounts from tax returns: individual returns (Forms 1040) by ZIP code,
|
|
3
|
+
county and state and by AGI size class, taxpayer migration between states and counties, and the SOI published
|
|
4
|
+
tables (national individual tables and tax shares by percentile, corporations, partnerships, sole
|
|
5
|
+
proprietorships, the IRS Data Book, historical tables, estate and gift tax, personal wealth, exempt organizations,
|
|
6
|
+
international), plus the Exempt Organizations Business Master File and the Form 990 extracts.
|
|
4
7
|
|
|
5
8
|
## When to use
|
|
6
9
|
- Number of returns, adjusted gross income (AGI), wages, dividends, capital gains, business income, deductions
|
|
@@ -8,12 +11,20 @@ county and state and by AGI size class, plus taxpayer migration between states a
|
|
|
8
11
|
or state; average AGI per return; the income mix of a place; returns with AGI of $200,000 or more (ZIP,
|
|
9
12
|
county) or $1 million or more (state).
|
|
10
13
|
- Households (returns), people and AGI moving into or out of a state or county between two filing years.
|
|
14
|
+
- Budget and economic estimates from published tax data (`table`): national income and tax by size of AGI, income
|
|
15
|
+
taxed at each rate (static revenue effect of a rate change), average tax rates and tax shares of the top 1%,
|
|
16
|
+
0.1% ..., the corporate tax base (net income, income subject to tax, tax after credits by industry or asset
|
|
17
|
+
size), pass-through business income (partnerships, S corporations, sole proprietorships), IRS gross collections
|
|
18
|
+
and refunds by type of tax and by state, estate and gift tax by size of estate and by state, wealth of top
|
|
19
|
+
wealth holders, the nonprofit sector (Form 990 totals, private foundations), controlled foreign corporations,
|
|
20
|
+
foreign tax credits and country-by-country reports. Recipes: `docs/estimate_patterns.md`.
|
|
21
|
+
- One nonprofit or the nonprofits of a state (`eo_bmf`: registry; `eo_extract`: Form 990 financial data).
|
|
11
22
|
- Not here: person- or household-level income, poverty and median household income (census: ACS); personal
|
|
12
|
-
income and
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
23
|
+
income, GDP and corporate profits in the national accounts (bea); federal receipts by month and the budget
|
|
24
|
+
totals (treasury, omb); projections (CBO, not covered); tax-return microdata: the Individual Public-Use
|
|
25
|
+
Microdata File is not public (it was sold under license; tax years 2012-2015 are no longer available, and SOI
|
|
26
|
+
has announced a synthetic file instead), so no row-level data; Form 990 e-file XML (100-500 MB zips of single
|
|
27
|
+
returns); tax gap estimates (PDF only); the Corporation Source Book (frozen at 2013).
|
|
17
28
|
|
|
18
29
|
## Endpoints and datasets
|
|
19
30
|
One all-states CSV per year under `https://www.irs.gov/pub/irs-soi/` (verified 2026-10-30). The adapter
|
|
@@ -37,8 +48,86 @@ not posted yet ends with `not_found` naming the file. Variable lists and layouts
|
|
|
37
48
|
the index pages (ZIP `{yy}zpdoc.docx`, county `{yy}incydocguide.docx`, state `{yy}incmdocguide.doc`,
|
|
38
49
|
migration `{y1y2}inpublicmigdoc.pdf`).
|
|
39
50
|
|
|
51
|
+
## Published tables (`table`, `table_list`)
|
|
52
|
+
SOI publishes its national statistics as .xls/.xlsx workbooks, one per table and year. `packs/irs-soi/tables.json`
|
|
53
|
+
catalogs 125 of them (checked 2026-11-08); `table_list` lists them (`product`, `query`; `live: true` adds the
|
|
54
|
+
years posted now). File names change over the years (the Data Book renumbered its tables in 2006, `97in11.xls`,
|
|
55
|
+
`ccr`/`nr`/`tabl` suffixes), so the adapter reads the table's index page and takes the year's link from it; a link
|
|
56
|
+
the page gets wrong is corrected in the catalog (`overrides`: partnership Tables 5 and 7 for 2023).
|
|
57
|
+
|
|
58
|
+
| Product (ids) | Tables | Years | Year |
|
|
59
|
+
|---|---|---|---|
|
|
60
|
+
| Individual, Publication 1304 (`in_1.1` ... `in_3.6`) | 1.1 income and tax by size of AGI, 1.2 by marital status, 1.4 sources of income, 1.4A capital gains, 1.6 age, 1.7 dependents, 2.1 itemized deductions, 2.5 EITC, 2.7 ACA, 3.1-3.6 tax computation, tax by rate (3.5: income taxed at each rate) | TY1996-2023 | tax year |
|
|
61
|
+
| Individual, percentiles (`shares_4.1`, `shares_4.2`, `shares_4.3`) | returns, AGI floor, AGI, income tax, average tax rate and shares, top 0.001% ... top 50% and bottom 50% | 2001-2023 in one file | edition |
|
|
62
|
+
| Corporations, Publication 16 (`corp_1` ... `corp_14`) | 1 minor industry, 2.1-2.4A size of total assets (2.3 C corporations, 2.4 S corporations), 3.x size of receipts, 4 size of tax, 5.x minor industry, 6-9 S corporations, 10 foreign (1120-F), 11 tax and credits, 12-14 COGS, depreciation, research credit; `corp_1-CV` coefficients of variation | TY2014-2023 (Table 1 from 1994) | tax year |
|
|
63
|
+
| Partnerships (`partnership_1` ... `_7`, `_10`, `_23`) | by industrial group: income and deductions, balance sheets, rental real estate, income allocated to partners, LLCs, receipts, foreign transactions | TY1993-2023 | tax year |
|
|
64
|
+
| Sole proprietorships (`soleprop_1` ... `_4`) | by industrial sector: receipts, deductions, payroll, net income; income statements; size of receipts; returns with Schedule C | TY1996-2023 | tax year |
|
|
65
|
+
| Integrated business data (`business_1`, `business_3`) | returns, receipts and net income by form of business, 1980-2015; by NAICS sector | to 2015 | edition / tax year |
|
|
66
|
+
| IRS Data Book (`databook_1-1` ... `_1-8`, `_3-1`, `_6-1`, `_6-2`) | collections and refunds by type of tax, returns filed, by state; gross collections by type of tax 1960 onward (1-6); audit coverage; costs; cost of collecting $100 | FY1995-2025 | fiscal year |
|
|
67
|
+
| Historical tables (`historical_1` ... `_24`) | individual series since 1913/1950, corporations since 1909, NIPA vs SOI income (8, 15), excise taxes by type FY1999 onward (20), returns filed (21), corporate rates 1909-2010 (24) | one file each | edition |
|
|
68
|
+
| Estate, gift, wealth (`estate_1` ... `_5`, `gift_1`, `wealth_1` ... `_6`) | estate returns by size of gross estate and tax status, by state, charitable bequests, beneficiaries, occupation; gift returns; top wealth holders | FY2001-2024; gift 2003-2021; wealth 2001-2019 | filing year |
|
|
69
|
+
| Exempt organizations (`eo_1` ... `_4`, `eo_excise`, `pf_1` ... `_4`) | Form 990 501(c)(3) by asset size, by Code section, 990-EZ, Form 4720 excise; private foundations | 1985-2024 | tax year |
|
|
70
|
+
| International (`cfc_1` ... `_3`, `ftc_1` ... `_3`, `cbc_1A` ... `cbc_4`) | controlled foreign corporations, corporate foreign tax credit, country-by-country reports | to 2021/2023 | tax year |
|
|
71
|
+
|
|
72
|
+
Parameters: `table` (an id, or a product word and number: "corporation 2.1", "data book 1-6"); `year` (or
|
|
73
|
+
`tax_year`, `fiscal_year`, `filing_year`; `start_year`/`end_year`, one workbook each); default the latest posted.
|
|
74
|
+
A whole-series table (`shares_4.x`, `databook_1-6`, `historical_*`, `business_1`) holds every year in one file:
|
|
75
|
+
the latest edition is read and the years are rows or columns (filter them with `rows`, `section` or `columns`).
|
|
76
|
+
`rows` keeps rows by label (an exact label, else the labels containing the text), `section` by section label,
|
|
77
|
+
`columns` by published column number or header text; `layout: "wide"` gives one row per line; `sheet` picks a
|
|
78
|
+
sheet of a multi-sheet workbook; `header_lines` overrides the header reading (rarely needed: see the notes).
|
|
79
|
+
|
|
80
|
+
How a workbook is read (no settings per table): title and headnote lines; the header lines down to the row of
|
|
81
|
+
column numbers (`1`, `(1)`, or `-1` shown as `(1)`); label-only lines become `SECTION` (a label running over two
|
|
82
|
+
lines is joined); the row level comes from the cell indent (or leading spaces) and gives `LEVEL` and `PARENT`;
|
|
83
|
+
footnote lines after the last data line go to the notes. A sheet printed as pages (1998 tables: columns 1-20 on
|
|
84
|
+
page 1, 21-40 on page 2) is read block by block; side-by-side panels (older partnership and sole proprietorship
|
|
85
|
+
tables) take each cell's label from the nearest label column to its left. A table without column numbers
|
|
86
|
+
(Publication 16 Table 2.1 for 2014, some historical tables) is read with the header down to the first line with
|
|
87
|
+
a label and a number, and says so in the notes, with `COL` empty.
|
|
88
|
+
|
|
89
|
+
Output (long, the default): `TABLE, YEAR, YEAR_TYPE, SHEET, SECTION, LINE, LEVEL, PARENT, ROW_LABEL, COL, COLUMN,
|
|
90
|
+
VALUE, FLAG, UNITS`. `COLUMN` joins the header lines with " | " ("All returns | Adjusted gross income less deficit
|
|
91
|
+
| Amount"); `COL` is the published column number, stable within a year (not across years: check `COLUMN`).
|
|
92
|
+
`VALUE` is the number as published (no thousands separators); `FLAG` holds the cell's markers: `*` estimate based
|
|
93
|
+
on few sample returns (high sampling variability; in newer workbooks the `*` is only in the number format and is
|
|
94
|
+
read from it), `**` data combined or suppressed, `d`/`[d]` deleted to avoid disclosure (included in totals), `[n]`
|
|
95
|
+
a footnote (e.g. `[2]` less than 0.05 percent), `n.a.` not available, `--`/`-` zero or not applicable as the
|
|
96
|
+
table's footnotes say. `UNITS` is the money unit of the headnote (usually thousands of dollars; gift tables whole
|
|
97
|
+
dollars; historical tables millions or billions); count, percent and average columns are as their header says.
|
|
98
|
+
|
|
99
|
+
## Exempt organizations (`eo_bmf`, `eo_extract`)
|
|
100
|
+
- `eo_bmf`: `https://www.irs.gov/pub/irs-soi/eo_{st}.csv` (one per state, PR, XX = international; updated monthly,
|
|
101
|
+
1,975,975 organizations on 2026-10-13), 28 columns. Filters: `ntee` (prefix), `subsection` (`03` = 501(c)(3)),
|
|
102
|
+
`ein`, `name`, `columns`. EIN and ZIP stay text. ASSET_AMT, INCOME_AMT, REVENUE_AMT are whole dollars from the
|
|
103
|
+
latest return on file (TAX_PERIOD differs by organization). Codes: https://www.irs.gov/pub/foia/ig/tege/eo-info.pdf.
|
|
104
|
+
- `eo_extract`: the SOI annual extract of Form 990 (`form: 990`), 990-EZ (`990ez`) or 990-PF (`990pf`) data,
|
|
105
|
+
`{yy}eoextract990.zip` etc. (2018-2024; `{yy}eofinextract...` 2012-2017), listed on
|
|
106
|
+
soi-tax-stats-annual-extract-of-tax-exempt-organization-financial-data. The 2024 Form 990 zip is 57 MB holding a
|
|
107
|
+
247 MB CSV of 246 columns; it is read as a stream keeping only `ein` or the organizations of `state` (their EINs
|
|
108
|
+
from the current EO BMF, so an organization that moved or lost exemption is missed). The 990-PF CSV drops EIN
|
|
109
|
+
leading zeros: they are restored. Field definitions: `{yy}eofinextractdoc.xlsx`. Years whose zip holds a
|
|
110
|
+
fixed-width `.dat` file (some of 2012-2017) are refused.
|
|
111
|
+
|
|
40
112
|
## Spec examples
|
|
41
113
|
```json
|
|
114
|
+
{"description": "Static revenue effect of one more point on the 37 percent rate, TY2023",
|
|
115
|
+
"requests": [{"name": "rate37", "endpoint": "table",
|
|
116
|
+
"params": {"table": "in_3.5", "year": 2023, "rows": ["Total"], "columns": ["37 percent"]}}]}
|
|
117
|
+
```
|
|
118
|
+
```json
|
|
119
|
+
{"description": "Tax shares and average tax rates of the top 1 percent, 2001-2023",
|
|
120
|
+
"requests": [{"name": "top1", "endpoint": "table",
|
|
121
|
+
"params": {"table": "shares_4.1", "columns": ["Top 1 percent"],
|
|
122
|
+
"section": ["Total income tax share", "Average tax rate", "Adjusted gross income share"]}}]}
|
|
123
|
+
```
|
|
124
|
+
```json
|
|
125
|
+
{"description": "Corporate tax base by asset size and IRS collections by type of tax",
|
|
126
|
+
"requests": [{"name": "corp", "endpoint": "table", "params": {"table": "corp_2.1", "year": 2023,
|
|
127
|
+
"rows": ["Net income (less deficit)", "Income subject to tax", "Total income tax after credits"]}},
|
|
128
|
+
{"name": "coll", "endpoint": "table", "params": {"table": "databook_1-6", "rows": ["2021", "2022", "2023", "2024", "2025"]}}]}
|
|
129
|
+
```
|
|
130
|
+
```json
|
|
42
131
|
{"description": "Maryland counties, tax year 2022: returns and AGI by AGI class",
|
|
43
132
|
"requests": [{"name": "md_county", "endpoint": "county_agi",
|
|
44
133
|
"params": {"tax_year": 2022, "state": "MD", "columns": ["N1", "N2", "A00100", "A00200"]}}]}
|
|
@@ -117,8 +206,26 @@ Manual route (`manual_bundle`): INSTRUCTIONS.md lists the direct link of each fi
|
|
|
117
206
|
e.g. `22incyallagi.csv`); download it from the index page (ZIP: soi-tax-stats-individual-income-tax-statistics-zip-code-data-soi,
|
|
118
207
|
county: soi-tax-stats-county-data, state: soi-tax-stats-historic-table-2, migration: soi-tax-stats-migration-data,
|
|
119
208
|
all under https://www.irs.gov/statistics/), upload it unchanged, and `ingest` filters it like `fetch` does.
|
|
120
|
-
Per-state Excel files
|
|
209
|
+
Per-state Excel files and years before the CSV files are manual and not read here.
|
|
210
|
+
`table`: one GET of the table's index page per run, then one GET per workbook (20-300 KB). Manual route: the index
|
|
211
|
+
page and the file pattern; upload the workbook under its published name (`23in11si.xls`), and `ingest` reads it
|
|
212
|
+
like `fetch` (the year from the name). `eo_bmf`: one GET per state file (up to about 25 MB for California);
|
|
213
|
+
`eo_extract`: one GET of the extract page, the state's EO BMF file, and the zip (10-57 MB, held in memory and
|
|
214
|
+
read as a stream). Manual: upload the zip under its published name (and `eo_<state>.csv` with `state`).
|
|
215
|
+
|
|
216
|
+
Traps for estimates from the published tables (see `docs/estimate_patterns.md`):
|
|
217
|
+
- Estimates from samples: totals are reliable; cells marked `*` are not; Publication 16 Table 1 has its
|
|
218
|
+
coefficients of variation in `corp_1-CV`.
|
|
219
|
+
- Tax year, filing year, fiscal year: Publication 1304 and Publication 16 are tax years; the Data Book is IRS
|
|
220
|
+
fiscal years (October-September) of collections; estate tables are filing years (returns filed in the year, of
|
|
221
|
+
decedents of earlier years). Do not mix them without saying so.
|
|
222
|
+
- Lag: the latest tax year is 2-3 years back (TY2023 posted in 2025-2026).
|
|
223
|
+
- Definitions change: AGI classes, the "total income tax" item, and table numbers change between years; compare
|
|
224
|
+
`COLUMN` text, not only `COL`, across years.
|
|
225
|
+
- Business tables count returns, not firms; the corporate tables include S corporations, REITs and RICs unless the
|
|
226
|
+
table is "other than Forms 1120S, 1120-REIT and 1120-RIC" (`corp_2.3`, `corp_3.3`, `corp_5.3`).
|
|
121
227
|
|
|
122
228
|
## Citation and terms
|
|
123
|
-
"Internal Revenue Service, Statistics of Income Division, <product, e.g. County Data, Tax Year 2022
|
|
124
|
-
irs.gov/statistics, retrieved <date>."
|
|
229
|
+
"Internal Revenue Service, Statistics of Income Division, <product, e.g. County Data, Tax Year 2022, or
|
|
230
|
+
Publication 1304 Table 1.1, Tax Year 2023>, irs.gov/statistics, retrieved <date>." The EO BMF: "Internal Revenue
|
|
231
|
+
Service, Exempt Organizations Business Master File Extract, <month year>". Public domain (U.S. government work); redistribution allowed.
|