data-analyst-mcp 0.8.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/dist/adapters/rma.d.ts +10 -0
- package/dist/adapters/rma.js +40 -0
- package/dist/adapters/rma.js.map +1 -0
- package/dist/adapters/sba.d.ts +20 -2
- package/dist/adapters/sba.js +104 -8
- package/dist/adapters/sba.js.map +1 -1
- package/dist/analysis/rerun_job.js +2 -0
- package/dist/analysis/rerun_job.js.map +1 -1
- package/dist/analysis/tools.d.ts +35 -4
- package/dist/analysis/tools.js +31 -12
- package/dist/analysis/tools.js.map +1 -1
- package/dist/core/context.d.ts +6 -0
- package/dist/core/context.js +14 -0
- package/dist/core/context.js.map +1 -1
- package/dist/core/generic-rest.js +7 -2
- package/dist/core/generic-rest.js.map +1 -1
- package/dist/core/http.js +6 -2
- package/dist/core/http.js.map +1 -1
- package/dist/core/packs.d.ts +3 -0
- package/dist/core/packs.js +6 -1
- package/dist/core/packs.js.map +1 -1
- package/dist/core/tabular.d.ts +2 -0
- package/dist/core/tabular.js +14 -3
- package/dist/core/tabular.js.map +1 -1
- package/dist/index.d.ts +1 -0
- package/dist/index.js +1 -1
- package/dist/index.js.map +1 -1
- package/dist/report/audit_report.js +3 -0
- package/dist/report/audit_report.js.map +1 -1
- package/dist/server.d.ts +5 -1
- package/dist/server.js +73 -18
- package/dist/server.js.map +1 -1
- package/dist/session.d.ts +2 -0
- package/dist/session.js +2 -0
- package/dist/session.js.map +1 -1
- package/guide/references/routing.md +4 -1
- package/package.json +1 -1
- package/packs/bea/policy.json +5 -1
- package/packs/cbp/source.md +1 -1
- package/packs/cdc-wonder/source.md +1 -1
- package/packs/port.json +5 -0
- package/packs/rma/endpoints.json +44 -0
- package/packs/rma/policy.json +94 -0
- package/packs/rma/source.md +78 -0
- package/packs/sba/endpoints.json +17 -0
- package/packs/sba/source.md +19 -3
- package/packs/uscourts/source.md +1 -1
package/packs/bea/policy.json
CHANGED
|
@@ -42,7 +42,11 @@
|
|
|
42
42
|
"gdi",
|
|
43
43
|
"share of gdp", "wages per full-time equivalent", "compensation per employee",
|
|
44
44
|
"percent of gdp",
|
|
45
|
-
"gdp deflator"
|
|
45
|
+
"gdp deflator",
|
|
46
|
+
"gdp price index",
|
|
47
|
+
"pce price index",
|
|
48
|
+
"chain-type price index",
|
|
49
|
+
"implicit price deflator"
|
|
46
50
|
],
|
|
47
51
|
"questions": [
|
|
48
52
|
"real GDP growth by quarter",
|
package/packs/cbp/source.md
CHANGED
|
@@ -23,7 +23,7 @@ Columns:
|
|
|
23
23
|
- By state (`-state`): Fiscal Year, Month Grouping, Month (abbv), Land Border Region, State, Demographic,
|
|
24
24
|
Citizenship, Title of Authority, Encounter Count.
|
|
25
25
|
|
|
26
|
-
## Spec
|
|
26
|
+
## Spec examples
|
|
27
27
|
```json
|
|
28
28
|
{"description": "Southwest border encounters by month, FY2024 to date (file of November 2026)",
|
|
29
29
|
"requests": [{"name": "files", "endpoint": "list_files"},
|
|
@@ -17,7 +17,7 @@ origin, as grouped in the query.
|
|
|
17
17
|
|---|---|---|
|
|
18
18
|
| `export` | the uploaded WONDER export as published: one column per Group By field (name and code columns, e.g. `Year`, `Year Code`), then `Deaths`, `Population`, `Crude Rate`, `Age Adjusted Rate` ... as chosen; `Notes` (`Total` on total rows) | `database` (required: the WONDER database name), `query` (the query in words) |
|
|
19
19
|
|
|
20
|
-
## Spec
|
|
20
|
+
## Spec examples
|
|
21
21
|
```json
|
|
22
22
|
{"description": "Age-adjusted suicide death rates by state, 2023 (WONDER export)",
|
|
23
23
|
"requests": [{"name": "suicide_2023", "endpoint": "export",
|
package/packs/port.json
CHANGED
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
{
|
|
2
|
+
"base_url": "https://pubfs-rma.fpac.usda.gov",
|
|
3
|
+
"timeout": 240,
|
|
4
|
+
"endpoints": {
|
|
5
|
+
"coverage": {
|
|
6
|
+
"path": "/pub/Web_Data_Files/Summary_of_Business/state_county_crop/sobcov_{year}.zip",
|
|
7
|
+
"method": "GET",
|
|
8
|
+
"description": "Federal crop insurance Summary of Business by commodity year, state, county, crop, insurance plan, coverage category, delivery type and coverage level: policies, units, insured quantity, liability, total premium, premium subsidy, indemnity and loss ratio, dollars (RMA, 1989 on; updated weekly for open years).",
|
|
9
|
+
"params": {
|
|
10
|
+
"year": {"in": "path", "required": true, "desc": "commodity year, 1989 on (the current and next years are partial)"},
|
|
11
|
+
"state": {"in": "spec", "desc": "keep these states (USPS abbreviation or 2-digit FIPS code; a list works)"},
|
|
12
|
+
"county": {"in": "spec", "desc": "keep these 3-digit county FIPS codes (999 is All Other Counties)"},
|
|
13
|
+
"commodity": {"in": "spec", "desc": "keep these commodities (name as published, e.g. Corn, Soybeans, or the 4-digit RMA code; case-insensitive)"},
|
|
14
|
+
"plan": {"in": "spec", "desc": "keep these insurance plans (abbreviation, e.g. RP, YP, RPHPE, APH, or the plan code)"},
|
|
15
|
+
"coverage_category": {"in": "spec", "desc": "keep A (buy-up), C (CAT), E (existing coverage policy) or L (limited)"},
|
|
16
|
+
"delivery_type": {"in": "spec", "desc": "keep RBUP, RCAT (reinsured, sold by companies), FBUP or FCAT (federal)"}
|
|
17
|
+
},
|
|
18
|
+
"records": "$zip:*.txt",
|
|
19
|
+
"delimiter": "|",
|
|
20
|
+
"columns": ["Commodity Year", "Location State Code", "Location State Abbreviation", "Location County Code",
|
|
21
|
+
"Location County Name", "Commodity Code", "Commodity Name", "Insurance Plan Code", "Insurance Plan Name Abbreviation",
|
|
22
|
+
"Coverage Category", "Delivery Type", "Coverage Level", "Policies Sold Count", "Policies Earning Premium Count",
|
|
23
|
+
"Policies Indemnified Count", "Units Earning Premium Count", "Units Indemnified Count", "Quantity Type",
|
|
24
|
+
"Net Reported Quantity", "Endorsed/Companion Acres", "Liability Amount", "Total Premium Amount", "Subsidy Amount",
|
|
25
|
+
"State/Private Subsidy", "Additional Subsidy", "EFA Premium Discount", "Indemnity Amount", "Loss Ratio"],
|
|
26
|
+
"paging": {"type": "none"},
|
|
27
|
+
"max_rows": 2000000
|
|
28
|
+
},
|
|
29
|
+
"reinsurance": {
|
|
30
|
+
"path": "/pub/Web_Data_Files/Reinsurance_Reports/standard_reinsurance_agreement_reports/{year}_Standard_Reinsurance_Report_{level}.xls",
|
|
31
|
+
"method": "GET",
|
|
32
|
+
"description": "Standard Reinsurance Agreement results by reinsurance year and fund (A assigned, C commercial), national or by state: gross liability, premium and indemnity, and the companies' retained liability, premium, indemnity and net underwriting gain or loss, dollars (RMA, 2014 on).",
|
|
33
|
+
"params": {
|
|
34
|
+
"year": {"in": "path", "required": true, "desc": "reinsurance year, 2014 on (the latest is not final)"},
|
|
35
|
+
"level": {"in": "path", "required": true, "desc": "National (totals by fund) or State (each state by fund)"},
|
|
36
|
+
"state": {"in": "spec", "desc": "keep these states (USPS abbreviation) in the State report"}
|
|
37
|
+
},
|
|
38
|
+
"records": "$xls",
|
|
39
|
+
"header_row": 7,
|
|
40
|
+
"paging": {"type": "none"},
|
|
41
|
+
"max_rows": 10000
|
|
42
|
+
}
|
|
43
|
+
}
|
|
44
|
+
}
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
{
|
|
2
|
+
"id": "rma",
|
|
3
|
+
"title": "USDA RMA federal crop insurance: Summary of Business (liability, premium, subsidy, indemnity) and reinsurance results",
|
|
4
|
+
"agency": "U.S. Department of Agriculture, Risk Management Agency",
|
|
5
|
+
"depth": "standard",
|
|
6
|
+
"domains": [
|
|
7
|
+
"agriculture",
|
|
8
|
+
"federal spending",
|
|
9
|
+
"insurance"
|
|
10
|
+
],
|
|
11
|
+
"keywords": [
|
|
12
|
+
"rma",
|
|
13
|
+
"risk management agency",
|
|
14
|
+
"crop insurance",
|
|
15
|
+
"federal crop insurance",
|
|
16
|
+
"federal crop insurance corporation",
|
|
17
|
+
"fcic",
|
|
18
|
+
"premium subsidy",
|
|
19
|
+
"crop insurance subsidy",
|
|
20
|
+
"indemnity",
|
|
21
|
+
"indemnities",
|
|
22
|
+
"loss ratio",
|
|
23
|
+
"insured acres",
|
|
24
|
+
"liability",
|
|
25
|
+
"summary of business",
|
|
26
|
+
"revenue protection",
|
|
27
|
+
"yield protection",
|
|
28
|
+
"catastrophic coverage",
|
|
29
|
+
"coverage level",
|
|
30
|
+
"standard reinsurance agreement",
|
|
31
|
+
"underwriting gains",
|
|
32
|
+
"approved insurance providers"
|
|
33
|
+
],
|
|
34
|
+
"questions": [
|
|
35
|
+
"federal crop insurance premium subsidies by state or crop",
|
|
36
|
+
"crop insurance indemnities and loss ratios over time",
|
|
37
|
+
"how much would raising the premium subsidy rate cost",
|
|
38
|
+
"insured acres and liability for corn in Iowa",
|
|
39
|
+
"crop insurance companies' underwriting gains",
|
|
40
|
+
"coverage levels farmers buy by crop"
|
|
41
|
+
],
|
|
42
|
+
"homepage": "https://www.rma.usda.gov/tools-reports/summary-of-business",
|
|
43
|
+
"terms_url": "https://pubfs-rma.fpac.usda.gov/pub/Web_Data_Files/",
|
|
44
|
+
"auth": {
|
|
45
|
+
"env": null,
|
|
46
|
+
"required": "none",
|
|
47
|
+
"placement": "none",
|
|
48
|
+
"signup_url": null,
|
|
49
|
+
"echoes_key": false
|
|
50
|
+
},
|
|
51
|
+
"limits": {
|
|
52
|
+
"rps": 1,
|
|
53
|
+
"burst": 2,
|
|
54
|
+
"max_rows": 2000000
|
|
55
|
+
},
|
|
56
|
+
"routes": {
|
|
57
|
+
"api": {
|
|
58
|
+
"base": "https://pubfs-rma.fpac.usda.gov",
|
|
59
|
+
"probe": "https://pubfs-rma.fpac.usda.gov/pub/Web_Data_Files/Summary_of_Business/state_county_crop/",
|
|
60
|
+
"hosts": [
|
|
61
|
+
"pubfs-rma.fpac.usda.gov"
|
|
62
|
+
]
|
|
63
|
+
},
|
|
64
|
+
"manual": {
|
|
65
|
+
"how": "Open https://pubfs-rma.fpac.usda.gov/pub/Web_Data_Files/Summary_of_Business/state_county_crop/ (sobcov_<year>.zip) or .../Reinsurance_Reports/standard_reinsurance_agreement_reports/ (<year>_Standard_Reinsurance_Report_National.xls or _State.xls), download the file and upload it with ingest."
|
|
66
|
+
}
|
|
67
|
+
},
|
|
68
|
+
"cache": {
|
|
69
|
+
"mode": "run_only",
|
|
70
|
+
"ttl_hours": null,
|
|
71
|
+
"why": "Open crop years are updated every week; data/ is reused within a run."
|
|
72
|
+
},
|
|
73
|
+
"redistribution": {
|
|
74
|
+
"mode": "allowed",
|
|
75
|
+
"why": "RMA data are U.S. government works in the public domain (17 U.S.C. 105)."
|
|
76
|
+
},
|
|
77
|
+
"notices": [],
|
|
78
|
+
"retention": null,
|
|
79
|
+
"citation": "U.S. Department of Agriculture, Risk Management Agency, {dataset}, pubfs-rma.fpac.usda.gov, retrieved {retrieved_long}.",
|
|
80
|
+
"audit_hooks": [],
|
|
81
|
+
"clarify": [
|
|
82
|
+
"Which money? Total premium is the full premium; the subsidy is the federal share of it (farmer-paid premium = total premium - subsidy - state/private subsidy - additional subsidy - EFA discount); indemnities are losses paid. Program cost also includes delivery expense reimbursements and companies' underwriting gains (reinsurance), which the Summary of Business does not hold. Default: say which of these the number is.",
|
|
83
|
+
"Crop year or fiscal year? The Summary of Business is by commodity (crop) year, and reinsurance results by reinsurance year (July-June); federal outlays are by fiscal year (budget: treasury, omb). Default: crop year, said as such, and never added to fiscal-year outlays without saying so."
|
|
84
|
+
],
|
|
85
|
+
"formats": [
|
|
86
|
+
"csv"
|
|
87
|
+
],
|
|
88
|
+
"aliases": [
|
|
89
|
+
"RMA",
|
|
90
|
+
"FCIC",
|
|
91
|
+
"Summary of Business",
|
|
92
|
+
"crop insurance"
|
|
93
|
+
]
|
|
94
|
+
}
|
|
@@ -0,0 +1,78 @@
|
|
|
1
|
+
# USDA RMA federal crop insurance: Summary of Business (liability, premium, subsidy, indemnity) and reinsurance results
|
|
2
|
+
Depth: standard. Key: none. The Risk Management Agency's public data files (pubfs-rma.fpac.usda.gov), which
|
|
3
|
+
run the Federal Crop Insurance Corporation's program:
|
|
4
|
+
- the Summary of Business by commodity year, state, county, crop, insurance plan, coverage category, delivery
|
|
5
|
+
type and coverage level: policies, units, insured quantity, liability, total premium, premium subsidy,
|
|
6
|
+
indemnity and loss ratio (1989 on);
|
|
7
|
+
- the Standard Reinsurance Agreement results by reinsurance year and fund, national or by state: gross and
|
|
8
|
+
retained liability, premium and indemnity, and the insurance companies' net underwriting gain or loss
|
|
9
|
+
(2014 on).
|
|
10
|
+
|
|
11
|
+
## When to use
|
|
12
|
+
- Crop insurance cost inputs: premium subsidies (the federal share of premium) by state, crop, plan or coverage
|
|
13
|
+
level; indemnities and loss ratios over time; liability and insured acres; the effect of a change in subsidy
|
|
14
|
+
rates by coverage level; companies' underwriting gains, a program cost beside subsidies.
|
|
15
|
+
- Not here: crop insurance outlays by fiscal year as the budget records them (treasury MTS, omb; USDA's
|
|
16
|
+
budget account is the Federal Crop Insurance Corporation Fund); crop production, prices and acreage on all
|
|
17
|
+
farms (usda-nass, usda-outlook); farm commodity program payments.
|
|
18
|
+
|
|
19
|
+
## Endpoints
|
|
20
|
+
| Endpoint | Rows | Filters |
|
|
21
|
+
|---|---|---|
|
|
22
|
+
| `coverage` (`year`, a commodity year; 3 to 30 MB a year, 150,000 rows recently) | one per year, state, county, crop, plan, coverage category, delivery type and coverage level | `state` (abbreviation or FIPS), `county` (FIPS), `commodity` (name or code), `plan` (abbreviation or code), `coverage_category`, `delivery_type` |
|
|
23
|
+
| `reinsurance` (`year`, a reinsurance year; `level` National or State) | one per fund (and state), plus the published Total row | `state` |
|
|
24
|
+
|
|
25
|
+
Columns of `coverage`, as the layout document names them: Commodity Year, Location State Code, Location State
|
|
26
|
+
Abbreviation, Location County Code, Location County Name, Commodity Code, Commodity Name, Insurance Plan Code,
|
|
27
|
+
Insurance Plan Name Abbreviation, Coverage Category, Delivery Type, Coverage Level, Policies Sold Count,
|
|
28
|
+
Policies Earning Premium Count, Policies Indemnified Count, Units Earning Premium Count, Units Indemnified Count,
|
|
29
|
+
Quantity Type, Net Reported Quantity, Endorsed/Companion Acres, Liability Amount, Total Premium Amount, Subsidy
|
|
30
|
+
Amount, State/Private Subsidy, Additional Subsidy, EFA Premium Discount, Indemnity Amount, Loss Ratio.
|
|
31
|
+
|
|
32
|
+
## Spec examples
|
|
33
|
+
```json
|
|
34
|
+
{"description": "Iowa corn crop insurance by plan and coverage level, crop year 2024",
|
|
35
|
+
"requests": [{"name": "ia_corn_2024", "endpoint": "coverage", "params": {"year": "2024", "state": "IA", "commodity": "Corn"}}]}
|
|
36
|
+
```
|
|
37
|
+
```json
|
|
38
|
+
{"description": "Standard Reinsurance Agreement results by fund, national, reinsurance year 2024",
|
|
39
|
+
"requests": [{"name": "sra_2024", "endpoint": "reinsurance", "params": {"year": "2024", "level": "National"}}]}
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
## Data format and traps
|
|
43
|
+
- Dollars, as published (whole dollars in `coverage`; reinsurance sheets carry decimals). Loss Ratio is
|
|
44
|
+
indemnity / total premium, as a decimal (`.33`); recompute it from summed dollars, never average it.
|
|
45
|
+
- Total Premium Amount is the full premium. The federal premium subsidy is Subsidy Amount. The farmer-paid
|
|
46
|
+
premium is total premium less Subsidy Amount, State/Private Subsidy, Additional Subsidy (the 2021-2022
|
|
47
|
+
Pandemic Cover Crop Program) and EFA Premium Discount. The subsidy rate (subsidy / premium) is set by law for
|
|
48
|
+
each coverage level and unit structure; read it from the data, never assume it.
|
|
49
|
+
- The Summary of Business holds premium, subsidy and indemnity only. The program's full cost also includes
|
|
50
|
+
delivery expense (A&O) reimbursements to companies and their underwriting gains under the reinsurance
|
|
51
|
+
agreement (`reinsurance`: Net Gain/Loss, positive is a company gain), which are not in `coverage`.
|
|
52
|
+
- Years: commodity (crop) year in `coverage`; reinsurance year (July to June) in `reinsurance`. Neither is the
|
|
53
|
+
federal fiscal year. Open years (the current and next crop years) are updated every week and late years still
|
|
54
|
+
change as claims settle; give the file date (the zip's member date) with the numbers.
|
|
55
|
+
- Quantity Type is the unit of Net Reported Quantity (Acres, Colonies, Trees, Vines, NotReported...): never add
|
|
56
|
+
quantities across types. Insured acres are counted once per plan line, so acres across crops add, but the
|
|
57
|
+
Endorsed/Companion Acres of an SCO, STAX or ECO endorsement are on top of the underlying policy's acres.
|
|
58
|
+
- Indemnities can be negative (adjustments: 25 rows in the 2024 file of 2026-11-11). Sum them with the rest.
|
|
59
|
+
- Policy and unit counts are not available for CLIP and some area plans (zeros), so counts do not add across
|
|
60
|
+
plans to a farm count.
|
|
61
|
+
- County 999 (All Other Counties) and commodity 9999 (All Other Crops) are published groupings; keep them in
|
|
62
|
+
totals.
|
|
63
|
+
- `reinsurance` has a Total row (sum of the funds; State: all states): drop it before summing. Retained figures
|
|
64
|
+
include the 6.5 percent net book quota share (the sheet's footnote). Fund A is assigned (mostly ceded to
|
|
65
|
+
FCIC), fund C commercial.
|
|
66
|
+
- Older years and other files on the server: by type, practice and unit structure (sobtpu, 1999 on), cause of
|
|
67
|
+
loss by month (colsom), livestock (LRP, LGM, DRP). Not read here.
|
|
68
|
+
|
|
69
|
+
## Access
|
|
70
|
+
No key. `fetch` reads pubfs-rma.fpac.usda.gov/pub/Web_Data_Files (verified 2026-11-11: sobcov zips up to
|
|
71
|
+
3 MB compressed, 28 MB of text; reinsurance .xls 30 to 50 KB) and filters the rows here. Manual: the same
|
|
72
|
+
folders in a browser, then `ingest` the zip or workbook. The layout documents sit beside the files
|
|
73
|
+
(SOB_State_County_Crop_with_Coverage_Level_1989_Forward.pdf).
|
|
74
|
+
|
|
75
|
+
## Citation and terms
|
|
76
|
+
"U.S. Department of Agriculture, Risk Management Agency, Summary of Business (State/County/Crop with Coverage
|
|
77
|
+
Level), sobcov_<year>.zip, pubfs-rma.fpac.usda.gov, retrieved <date>." For reinsurance: "..., Standard
|
|
78
|
+
Reinsurance Agreement Report, <year> <National|State> ...". Public domain (17 U.S.C. 105).
|
package/packs/sba/endpoints.json
CHANGED
|
@@ -29,6 +29,23 @@
|
|
|
29
29
|
"records": "$csv",
|
|
30
30
|
"paging": {"type": "none"},
|
|
31
31
|
"max_rows": 1000000
|
|
32
|
+
},
|
|
33
|
+
"cohorts": {
|
|
34
|
+
"path": "/sites/default/files/{file}.csv",
|
|
35
|
+
"method": "GET",
|
|
36
|
+
"description": "Totals per approval fiscal year (cohort) of one FOIA file, optionally by more columns (group_by), summed while reading with the same filters as foia (none needed): loan counts by status, GrossApproval and SBAGuaranteedApproval dollars (all loans and disbursed loans; 504 files have no guaranty column), and GrossChargeOffAmount. One row per ApprovalFY x group.",
|
|
37
|
+
"params": {
|
|
38
|
+
"file": {"in": "path", "raw": true, "required": true, "desc": "the file's path under /sites/default/files without .csv, from list_files, e.g. uploaded_resources/FOIA_7a_FY2010_FY2019_asof_260930"},
|
|
39
|
+
"state": {"in": "spec", "desc": "keep loans with this ProjectState (two letters; a list works)"},
|
|
40
|
+
"fiscal_year": {"in": "spec", "desc": "keep these ApprovalFY values (a year or a list)"},
|
|
41
|
+
"from": {"in": "spec", "desc": "first ApprovalFY to keep"},
|
|
42
|
+
"to": {"in": "spec", "desc": "last ApprovalFY to keep"},
|
|
43
|
+
"naics": {"in": "spec", "desc": "keep NaicsCode values starting with these digits (e.g. 72; a list works)"},
|
|
44
|
+
"group_by": {"in": "spec", "desc": "extra grouping columns, any of BorrState, ProjectState, ProcessingMethod, naics2 (first 2 NAICS digits), NaicsCode, BusinessType, BusinessAge, FixedorVariableInterestInd, TermInMonths, RevolverStatus, SoldSecMrktInd, loan_status_code"}
|
|
45
|
+
},
|
|
46
|
+
"records": "$csv",
|
|
47
|
+
"paging": {"type": "none"},
|
|
48
|
+
"max_rows": 200000
|
|
32
49
|
}
|
|
33
50
|
}
|
|
34
51
|
}
|
package/packs/sba/source.md
CHANGED
|
@@ -7,6 +7,8 @@ Depth: standard. Key: none. The U.S. Small Business Administration's loan data o
|
|
|
7
7
|
## When to use
|
|
8
8
|
- SBA 7(a) or 504 loans by state, county, congressional district, industry (NAICS), lender, fiscal year; loan
|
|
9
9
|
sizes; charge-offs and paid-in-full status; franchise loans.
|
|
10
|
+
- Realized charge-off rates by approval cohort for a whole program (`cohorts`), to set against the default
|
|
11
|
+
assumptions in a credit subsidy estimate or reestimate (omb Federal Credit Supplement).
|
|
10
12
|
- Not here: all federal loans and grants by recipient (usaspending); PPP loans (not in these files); bank
|
|
11
13
|
lending in general (fdic, fed-board); business counts (census CBP).
|
|
12
14
|
|
|
@@ -15,17 +17,25 @@ Depth: standard. Key: none. The U.S. Small Business Administration's loan data o
|
|
|
15
17
|
|---|---|---|
|
|
16
18
|
| `list_files` | every file in data.sba.gov's catalog: `dataset`, `title`, `format`, `file` (the path for `foia`, CSVs only), `url`, `modified` | `search` (e.g. `7(a) & 504 FOIA`) |
|
|
17
19
|
| `foia` | the loans of one FOIA file that pass the filters, columns as published | `file` (from `list_files`), and at least one of `state` (ProjectState), `fiscal_year`, `from`/`to` (ApprovalFY), `naics` (prefix) |
|
|
20
|
+
| `cohorts` | totals per ApprovalFY (and `group_by` columns) of one FOIA file: `as_of`, `loans`, `gross_approval`, `sba_guaranteed` (7(a) only), `loans_disbursed`, `disbursed_gross_approval`, `disbursed_sba_guaranteed` (7(a) only), `loans_cancld`, `loans_chgoff`, `loans_closed`, `loans_exempt`, `loans_pif`, `loans_undisbursed`, `gross_chargeoff_amount`, `loans_other_status` | `file`, the `foia` filters (none required), `group_by` (BorrState, ProjectState, ProcessingMethod, naics2, NaicsCode, BusinessType, BusinessAge, FixedorVariableInterestInd, TermInMonths, RevolverStatus, SoldSecMrktInd, loan_status_code) |
|
|
18
21
|
|
|
19
22
|
Files of 2026-11-07 (as of September 30, 2026): `uploaded_resources/FOIA_7a_FY1991_FY1999_asof_260930`,
|
|
20
23
|
`..._FY2000_FY2009_...`, `..._FY2010_FY2019_...`, `..._FY2020_Present_...`, `uploaded_resources/FOIA_504_FY1991_FY2009_asof_260930`,
|
|
21
24
|
`..._504_FY2010_Present_...`. The as-of part changes each quarter: run `list_files` first.
|
|
22
25
|
|
|
23
|
-
## Spec
|
|
26
|
+
## Spec examples
|
|
24
27
|
```json
|
|
25
|
-
{"description": "504 loans in Vermont approved FY2024-2025",
|
|
28
|
+
{"description": "504 loans in Vermont approved FY2024-2025, and 504 totals per approval cohort FY2010-2020",
|
|
26
29
|
"requests": [{"name": "files", "endpoint": "list_files", "params": {"search": "7(a) & 504 FOIA"}},
|
|
27
30
|
{"name": "vt_504", "endpoint": "foia", "params": {"file": "uploaded_resources/FOIA_504_FY2010_Present_asof_260930",
|
|
28
|
-
"state": "VT", "from": "2024", "to": "2025"}}
|
|
31
|
+
"state": "VT", "from": "2024", "to": "2025"}},
|
|
32
|
+
{"name": "cohorts_504", "endpoint": "cohorts", "params": {"file": "uploaded_resources/FOIA_504_FY2010_Present_asof_260930",
|
|
33
|
+
"from": "2010", "to": "2020"}}]}
|
|
34
|
+
```
|
|
35
|
+
```json
|
|
36
|
+
{"description": "7(a) charge-off rates by approval cohort and processing method, FY2010-2019",
|
|
37
|
+
"requests": [{"name": "cohorts_7a", "endpoint": "cohorts", "params": {"file": "uploaded_resources/FOIA_7a_FY2010_FY2019_asof_260930",
|
|
38
|
+
"group_by": ["ProcessingMethod"]}}]}
|
|
29
39
|
```
|
|
30
40
|
|
|
31
41
|
## Data format and traps
|
|
@@ -45,6 +55,12 @@ Files of 2026-11-07 (as of September 30, 2026): `uploaded_resources/FOIA_7a_FY19
|
|
|
45
55
|
`SBAGuaranteedApproval` is SBA's guaranteed part of `GrossApproval`: say which you sum.
|
|
46
56
|
- `LoanStatus` and charge-off fields are as of the file's `AsOfDate`: recent loans have had little time to
|
|
47
57
|
default. Compare charge-off rates by approval cohort, not across all years.
|
|
58
|
+
- Charge-off rates by cohort (`cohorts`): the dollar rate is `gross_chargeoff_amount` / `disbursed_gross_approval`
|
|
59
|
+
(the charged-off balance against what was lent), the count rate `loans_chgoff` / `loans_disbursed`. Cancelled and undisbursed loans (`loans_cancld`,
|
|
60
|
+
`loans_undisbursed`: 7(a) COMMIT, 504 NOT FUNDED) were never lent and are left out of the disbursed totals.
|
|
61
|
+
`loan_status_code` is LoanStatus without spaces (7(a) `P I F` is PIF). These are realized losses to date, gross
|
|
62
|
+
of recoveries: not the lifetime default rate in a credit subsidy estimate, which a cohort reaches only when its
|
|
63
|
+
loans have run their term.
|
|
48
64
|
- Borrowers are named businesses (sometimes sole proprietors with a home address). Analyse in aggregate; do not
|
|
49
65
|
single out individuals.
|
|
50
66
|
- Size: the files are 60 to 255 MB. `fetch` streams them and keeps only the rows that pass the filters
|
package/packs/uscourts/source.md
CHANGED
|
@@ -24,7 +24,7 @@ The file name is on the table's entry in https://www.uscourts.gov/statistics-rep
|
|
|
24
24
|
June 30, 2026.
|
|
25
25
|
- The date part is not zero-padded the same way in every series: copy it.
|
|
26
26
|
|
|
27
|
-
## Spec
|
|
27
|
+
## Spec examples
|
|
28
28
|
```json
|
|
29
29
|
{"description": "Bankruptcy filings by chapter and district, 12 months to September 30, 2026 (table F-2)",
|
|
30
30
|
"requests": [{"name": "f2", "endpoint": "table",
|