data-analyst-mcp 0.3.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +286 -262
- package/dist/adapters/bjs.d.ts +13 -0
- package/dist/adapters/bjs.js +28 -0
- package/dist/adapters/bjs.js.map +1 -0
- package/dist/adapters/cbp.d.ts +5 -0
- package/dist/adapters/cbp.js +64 -0
- package/dist/adapters/cbp.js.map +1 -0
- package/dist/adapters/cdc-wonder.d.ts +5 -0
- package/dist/adapters/cdc-wonder.js +68 -0
- package/dist/adapters/cdc-wonder.js.map +1 -0
- package/dist/adapters/cpsc.d.ts +4 -0
- package/dist/adapters/cpsc.js +125 -0
- package/dist/adapters/cpsc.js.map +1 -0
- package/dist/adapters/fhwa.d.ts +9 -0
- package/dist/adapters/fhwa.js +16 -0
- package/dist/adapters/fhwa.js.map +1 -0
- package/dist/adapters/foreign-assistance.d.ts +9 -0
- package/dist/adapters/foreign-assistance.js +42 -0
- package/dist/adapters/foreign-assistance.js.map +1 -0
- package/dist/adapters/hud-ahar.d.ts +3 -0
- package/dist/adapters/hud-ahar.js +80 -0
- package/dist/adapters/hud-ahar.js.map +1 -0
- package/dist/adapters/nces.d.ts +10 -0
- package/dist/adapters/nces.js +32 -0
- package/dist/adapters/nces.js.map +1 -0
- package/dist/adapters/nsf.d.ts +11 -0
- package/dist/adapters/nsf.js +25 -0
- package/dist/adapters/nsf.js.map +1 -0
- package/dist/adapters/onrr.d.ts +9 -0
- package/dist/adapters/onrr.js +35 -0
- package/dist/adapters/onrr.js.map +1 -0
- package/dist/adapters/sba.d.ts +10 -0
- package/dist/adapters/sba.js +134 -0
- package/dist/adapters/sba.js.map +1 -0
- package/dist/adapters/treasury-tic.d.ts +30 -0
- package/dist/adapters/treasury-tic.js +149 -0
- package/dist/adapters/treasury-tic.js.map +1 -0
- package/dist/adapters/uscourts.d.ts +8 -0
- package/dist/adapters/uscourts.js +8 -0
- package/dist/adapters/uscourts.js.map +1 -0
- package/dist/analysis/cite.d.ts +9 -1
- package/dist/analysis/cite.js +9 -2
- package/dist/analysis/cite.js.map +1 -1
- package/dist/bin/cli.d.ts +2 -0
- package/dist/bin/cli.js +17 -0
- package/dist/bin/cli.js.map +1 -1
- package/dist/core/generic-rest.js +42 -17
- package/dist/core/generic-rest.js.map +1 -1
- package/dist/core/guard.d.ts +32 -0
- package/dist/core/guard.js +176 -0
- package/dist/core/guard.js.map +1 -0
- package/dist/core/http.d.ts +10 -0
- package/dist/core/http.js +36 -7
- package/dist/core/http.js.map +1 -1
- package/dist/core/keys.d.ts +13 -3
- package/dist/core/keys.js +67 -7
- package/dist/core/keys.js.map +1 -1
- package/dist/core/package.d.ts +12 -3
- package/dist/core/package.js +56 -12
- package/dist/core/package.js.map +1 -1
- package/dist/core/secret_patterns.d.ts +6 -0
- package/dist/core/secret_patterns.js +20 -0
- package/dist/core/secret_patterns.js.map +1 -0
- package/dist/core/tabular.d.ts +43 -1
- package/dist/core/tabular.js +149 -6
- package/dist/core/tabular.js.map +1 -1
- package/dist/envelope.js +3 -1
- package/dist/envelope.js.map +1 -1
- package/dist/index.d.ts +29 -22
- package/dist/index.js +45 -46
- package/dist/index.js.map +1 -1
- package/dist/remote/app.js +87 -31
- package/dist/remote/app.js.map +1 -1
- package/dist/remote/oauth.js +8 -1
- package/dist/remote/oauth.js.map +1 -1
- package/dist/remote/pages.d.ts +3 -7
- package/dist/remote/pages.js +49 -19
- package/dist/remote/pages.js.map +1 -1
- package/dist/remote/sign.d.ts +7 -1
- package/dist/remote/sign.js +33 -7
- package/dist/remote/sign.js.map +1 -1
- package/dist/remote/store.js +14 -5
- package/dist/remote/store.js.map +1 -1
- package/dist/report/audit_report.js +29 -7
- package/dist/report/audit_report.js.map +1 -1
- package/dist/report/build_report.js +34 -4
- package/dist/report/build_report.js.map +1 -1
- package/dist/report/method_audit.js +10 -1
- package/dist/report/method_audit.js.map +1 -1
- package/dist/report/scan_secrets.d.ts +4 -1
- package/dist/report/scan_secrets.js +38 -18
- package/dist/report/scan_secrets.js.map +1 -1
- package/dist/server.d.ts +3 -1
- package/dist/server.js +46 -13
- package/dist/server.js.map +1 -1
- package/dist/tools.js +27 -6
- package/dist/tools.js.map +1 -1
- package/guide/WORKFLOW.md +134 -134
- package/guide/assets/disclaimer.txt +1 -1
- package/guide/assets/download_data.ps1 +164 -162
- package/guide/assets/download_data.sh +8 -2
- package/guide/assets/report_template.html +597 -597
- package/guide/references/routing.md +198 -171
- package/guide/references/style_guide.md +398 -398
- package/package.json +56 -56
- package/packs/acf/endpoints.json +114 -114
- package/packs/acf/policy.json +104 -104
- package/packs/actuaries/endpoints.json +82 -82
- package/packs/actuaries/policy.json +109 -109
- package/packs/bea/assets/download_bea.ps1 +182 -182
- package/packs/bea/docs/bea_api.md +461 -461
- package/packs/bea/docs/datasets.md +157 -157
- package/packs/bea/docs/report_patterns.md +112 -112
- package/packs/bea/policy.json +170 -170
- package/packs/bea/source.md +152 -152
- package/packs/bjs/endpoints.json +150 -0
- package/packs/bjs/policy.json +114 -0
- package/packs/bjs/source.md +98 -0
- package/packs/bls/assets/download_bls.ps1 +152 -152
- package/packs/bls/docs/analysis_patterns.md +108 -108
- package/packs/bls/docs/bls_api.md +73 -73
- package/packs/bls/docs/series_ids.md +149 -149
- package/packs/bls/policy.json +238 -238
- package/packs/bts/endpoints.json +346 -346
- package/packs/bts/policy.json +94 -94
- package/packs/cbp/endpoints.json +31 -0
- package/packs/cbp/policy.json +85 -0
- package/packs/cbp/source.md +57 -0
- package/packs/cdc/endpoints.json +197 -197
- package/packs/cdc/policy.json +83 -83
- package/packs/cdc/source.md +1 -1
- package/packs/cdc-wonder/endpoints.json +26 -0
- package/packs/cdc-wonder/policy.json +84 -0
- package/packs/cdc-wonder/source.md +53 -0
- package/packs/census/endpoints.json +99 -99
- package/packs/cfpb/endpoints.json +326 -326
- package/packs/cfpb/policy.json +119 -119
- package/packs/clinical-trials/endpoints.json +166 -166
- package/packs/clinical-trials/policy.json +84 -84
- package/packs/cms/endpoints.json +69 -69
- package/packs/cms/policy.json +96 -96
- package/packs/college-scorecard/endpoints.json +252 -252
- package/packs/college-scorecard/policy.json +106 -106
- package/packs/college-scorecard/source.md +1 -1
- package/packs/congress/endpoints.json +1701 -1701
- package/packs/congress/policy.json +117 -117
- package/packs/cpsc/endpoints.json +91 -0
- package/packs/cpsc/policy.json +91 -0
- package/packs/cpsc/source.md +76 -0
- package/packs/dhs-ohss/endpoints.json +66 -66
- package/packs/dhs-ohss/policy.json +107 -107
- package/packs/dhs-ohss/source.md +1 -1
- package/packs/doj-news/endpoints.json +59 -59
- package/packs/doj-news/policy.json +96 -96
- package/packs/doj-news/source.md +1 -1
- package/packs/dol/endpoints.json +817 -817
- package/packs/dol/policy.json +135 -135
- package/packs/eia/endpoints.json +717 -717
- package/packs/eia/policy.json +112 -112
- package/packs/epa/endpoints.json +291 -291
- package/packs/epa/policy.json +99 -99
- package/packs/epa-aqs/endpoints.json +173 -173
- package/packs/epa-aqs/policy.json +96 -96
- package/packs/fbi/endpoints.json +1841 -1841
- package/packs/fbi/policy.json +118 -118
- package/packs/fbi/source.md +3 -3
- package/packs/fda/endpoints.json +237 -237
- package/packs/fda/policy.json +121 -121
- package/packs/fdic/endpoints.json +416 -416
- package/packs/fdic/policy.json +83 -83
- package/packs/fec/endpoints.json +707 -707
- package/packs/fec/policy.json +93 -93
- package/packs/fed-board/endpoints.json +89 -89
- package/packs/fed-board/policy.json +136 -136
- package/packs/federal-register/endpoints.json +143 -143
- package/packs/federal-register/policy.json +86 -86
- package/packs/fema/endpoints.json +645 -645
- package/packs/fema/policy.json +103 -103
- package/packs/fhfa/endpoints.json +105 -105
- package/packs/fhfa/policy.json +109 -109
- package/packs/fhwa/endpoints.json +48 -0
- package/packs/fhwa/policy.json +91 -0
- package/packs/fhwa/source.md +59 -0
- package/packs/foreign-assistance/endpoints.json +166 -0
- package/packs/foreign-assistance/policy.json +80 -0
- package/packs/foreign-assistance/source.md +76 -0
- package/packs/fred/assets/download_fred.ps1 +122 -122
- package/packs/fred/docs/analysis_patterns.md +124 -124
- package/packs/fred/docs/fred_api.md +94 -94
- package/packs/fred/docs/series.md +188 -188
- package/packs/fred/policy.json +253 -253
- package/packs/fsa/endpoints.json +150 -150
- package/packs/fsa/policy.json +99 -99
- package/packs/fsa/source.md +1 -1
- package/packs/govinfo/endpoints.json +519 -519
- package/packs/gsa-calc/endpoints.json +93 -93
- package/packs/gsa-calc/policy.json +97 -97
- package/packs/hud/endpoints.json +146 -146
- package/packs/hud/policy.json +84 -84
- package/packs/hud/source.md +1 -1
- package/packs/hud-ahar/endpoints.json +92 -0
- package/packs/hud-ahar/policy.json +88 -0
- package/packs/hud-ahar/source.md +72 -0
- package/packs/ipums/docs/report_patterns.md +117 -117
- package/packs/irs-soi/endpoints.json +114 -114
- package/packs/irs-soi/policy.json +107 -107
- package/packs/medicaid/endpoints.json +412 -412
- package/packs/medicaid/policy.json +108 -108
- package/packs/naep/endpoints.json +217 -217
- package/packs/naep/policy.json +100 -100
- package/packs/naep/source.md +1 -1
- package/packs/nces/endpoints.json +101 -0
- package/packs/nces/policy.json +99 -0
- package/packs/nces/source.md +75 -0
- package/packs/nhtsa/endpoints.json +132 -132
- package/packs/nhtsa/policy.json +93 -93
- package/packs/nhtsa/source.md +1 -1
- package/packs/nih/endpoints.json +452 -452
- package/packs/nih/policy.json +86 -86
- package/packs/noaa/endpoints.json +431 -431
- package/packs/noaa/policy.json +95 -95
- package/packs/nrel/endpoints.json +327 -327
- package/packs/nrel/policy.json +96 -96
- package/packs/nsf/endpoints.json +33 -0
- package/packs/nsf/policy.json +83 -0
- package/packs/nsf/source.md +59 -0
- package/packs/nws/endpoints.json +271 -271
- package/packs/nws/policy.json +82 -82
- package/packs/nyfed/endpoints.json +80 -80
- package/packs/nyfed/policy.json +92 -92
- package/packs/omb/endpoints.json +460 -460
- package/packs/omb/policy.json +106 -106
- package/packs/onrr/endpoints.json +198 -0
- package/packs/onrr/policy.json +91 -0
- package/packs/onrr/source.md +55 -0
- package/packs/open-payments/endpoints.json +297 -297
- package/packs/open-payments/policy.json +101 -101
- package/packs/opm/assets/download_opm.ps1 +125 -125
- package/packs/opm/docs/fields.md +184 -184
- package/packs/opm/docs/opm_api.md +71 -71
- package/packs/opm/docs/report_patterns.md +83 -83
- package/packs/opm/lib/data_dictionary.csv +73 -73
- package/packs/pbgc/endpoints.json +483 -483
- package/packs/pbgc/policy.json +94 -94
- package/packs/philfed/endpoints.json +85 -85
- package/packs/philfed/policy.json +94 -94
- package/packs/port.json +65 -0
- package/packs/regulations/endpoints.json +287 -287
- package/packs/regulations/policy.json +86 -86
- package/packs/sba/endpoints.json +34 -0
- package/packs/sba/policy.json +86 -0
- package/packs/sba/source.md +60 -0
- package/packs/sec/endpoints.json +236 -236
- package/packs/sec/policy.json +86 -86
- package/packs/senate-lobbying/endpoints.json +553 -553
- package/packs/senate-lobbying/policy.json +99 -99
- package/packs/snapqc/endpoints.json +1426 -1426
- package/packs/snapqc/policy.json +92 -92
- package/packs/treasury/catalog.json +2267 -2267
- package/packs/treasury/docs/api_guide.md +245 -245
- package/packs/treasury/docs/datasets.md +56 -56
- package/packs/treasury/docs/report_patterns.md +126 -126
- package/packs/treasury/policy.json +159 -159
- package/packs/treasury/source.md +170 -165
- package/packs/treasury-tic/endpoints.json +77 -0
- package/packs/treasury-tic/policy.json +89 -0
- package/packs/treasury-tic/source.md +80 -0
- package/packs/treasury-yields/endpoints.json +226 -226
- package/packs/treasury-yields/policy.json +86 -86
- package/packs/usaspending/endpoints.json +1315 -1315
- package/packs/usaspending/policy.json +109 -109
- package/packs/uscourts/endpoints.json +22 -0
- package/packs/uscourts/policy.json +87 -0
- package/packs/uscourts/source.md +56 -0
- package/packs/usda-fna/policy.json +133 -133
- package/packs/usda-fooddata/endpoints.json +180 -180
- package/packs/usda-fooddata/policy.json +94 -94
- package/packs/usda-nass/endpoints.json +323 -323
- package/packs/usda-nass/policy.json +95 -95
- package/packs/usda-outlook/endpoints.json +86 -86
- package/packs/usda-outlook/policy.json +109 -109
- package/packs/usgs/endpoints.json +313 -313
- package/packs/usgs/policy.json +95 -95
- package/packs/usitc/endpoints.json +74 -74
- package/packs/usitc/policy.json +111 -111
- package/packs/uspto/endpoints.json +141 -141
- package/packs/uspto/policy.json +95 -95
- package/packs/uspto/source.md +1 -1
- package/packs/va/endpoints.json +414 -414
- package/packs/va/policy.json +101 -101
- package/packs/world-bank/endpoints.json +122 -122
- package/packs/world-bank/policy.json +101 -101
package/packs/naep/policy.json
CHANGED
|
@@ -1,100 +1,100 @@
|
|
|
1
|
-
{
|
|
2
|
-
"id": "naep",
|
|
3
|
-
"title": "NAEP, The Nation's Report Card (NAEP Data Service)",
|
|
4
|
-
"agency": "National Center for Education Statistics (U.S. Department of Education), for the National Assessment Governing Board",
|
|
5
|
-
"depth": "standard",
|
|
6
|
-
"domains": [
|
|
7
|
-
"education",
|
|
8
|
-
"k-12",
|
|
9
|
-
"student achievement"
|
|
10
|
-
],
|
|
11
|
-
"keywords": [
|
|
12
|
-
"naep",
|
|
13
|
-
"nation's report card",
|
|
14
|
-
"test scores",
|
|
15
|
-
"reading scores",
|
|
16
|
-
"math scores",
|
|
17
|
-
"achievement",
|
|
18
|
-
"proficient",
|
|
19
|
-
"proficiency",
|
|
20
|
-
"achievement gap",
|
|
21
|
-
"4th grade",
|
|
22
|
-
"8th grade",
|
|
23
|
-
"12th grade",
|
|
24
|
-
"grade 4",
|
|
25
|
-
"grade 8",
|
|
26
|
-
"student performance",
|
|
27
|
-
"school",
|
|
28
|
-
"learning loss",
|
|
29
|
-
"science scores",
|
|
30
|
-
"civics",
|
|
31
|
-
"us history",
|
|
32
|
-
"tuda",
|
|
33
|
-
"urban district"
|
|
34
|
-
],
|
|
35
|
-
"questions": [
|
|
36
|
-
"4th grade reading scores by state",
|
|
37
|
-
"did math scores fall since 2019",
|
|
38
|
-
"percent of 8th graders proficient in math",
|
|
39
|
-
"achievement gap by race or school lunch eligibility",
|
|
40
|
-
"how a state compares with the national average",
|
|
41
|
-
"NAEP results for a large city district"
|
|
42
|
-
],
|
|
43
|
-
"homepage": "https://www.nationsreportcard.gov/",
|
|
44
|
-
"terms_url": "https://nces.ed.gov/help/copyright.asp",
|
|
45
|
-
"auth": {
|
|
46
|
-
"env": null,
|
|
47
|
-
"required": "none",
|
|
48
|
-
"placement": "none",
|
|
49
|
-
"signup_url": null,
|
|
50
|
-
"echoes_key": false
|
|
51
|
-
},
|
|
52
|
-
"limits": {
|
|
53
|
-
"rps": 2,
|
|
54
|
-
"burst": 4,
|
|
55
|
-
"max_rows": 50000
|
|
56
|
-
},
|
|
57
|
-
"routes": {
|
|
58
|
-
"api": {
|
|
59
|
-
"base": "https://www.nationsreportcard.gov",
|
|
60
|
-
"probe": "https://www.nationsreportcard.gov/Dataservice/GetAdhocData.aspx?type=data&subject=reading&grade=4&subscale=RRPCM&variable=TOTAL&jurisdiction=NP&stattype=MN:MN&Year=Current",
|
|
61
|
-
"hosts": [
|
|
62
|
-
"www.nationsreportcard.gov"
|
|
63
|
-
]
|
|
64
|
-
},
|
|
65
|
-
"manual": {
|
|
66
|
-
"how": "Open each link in manual_download/INSTRUCTIONS.md in a browser (each returns JSON; save it under the name shown) and upload the files. Without the links: the NAEP Data Explorer (https://www.nationsreportcard.gov/ndecore/landing) builds the same tables (choose subject, grade, jurisdictions, variables, statistics; Download > Excel); upload the export and say which table it is. The report cards at https://www.nationsreportcard.gov/ give the headline numbers with significance marks."
|
|
67
|
-
}
|
|
68
|
-
},
|
|
69
|
-
"cache": {
|
|
70
|
-
"mode": "run_only",
|
|
71
|
-
"ttl_hours": null,
|
|
72
|
-
"why": "Small queries; results change only when a new assessment is released (about every two years), but data/ is reused only within a run."
|
|
73
|
-
},
|
|
74
|
-
"redistribution": {
|
|
75
|
-
"mode": "allowed",
|
|
76
|
-
"why": "NAEP results are published by NCES, a U.S. government agency; its materials are in the public domain (17 U.S.C. 105) unless marked otherwise, and NCES asks that the source be cited."
|
|
77
|
-
},
|
|
78
|
-
"notices": [],
|
|
79
|
-
"retention": null,
|
|
80
|
-
"citation": "U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), {dataset}, NAEP Data Service (nationsreportcard.gov), retrieved {retrieved_long}.",
|
|
81
|
-
"audit_hooks": [
|
|
82
|
-
{
|
|
83
|
-
"id": "naep_uncertainty",
|
|
84
|
-
"kind": "require_text",
|
|
85
|
-
"any": [
|
|
86
|
-
"standard error",
|
|
87
|
-
"significant"
|
|
88
|
-
],
|
|
89
|
-
"why": "NAEP scores are sample estimates with standard errors; a report must say whether differences are statistically significant (NAEP's own tests), not compare point estimates."
|
|
90
|
-
}
|
|
91
|
-
],
|
|
92
|
-
"clarify": [
|
|
93
|
-
"Which subject, grade(s) and year(s)? Default: reading and mathematics, grades 4 and 8, the latest year and 2019 (pre-pandemic).",
|
|
94
|
-
"Average scale score or achievement level (percent at or above Proficient)? They can move differently; default: report both.",
|
|
95
|
-
"Public schools only (NP, the comparison base for states) or all schools (NT)? Default NP when states are involved."
|
|
96
|
-
],
|
|
97
|
-
"formats": [
|
|
98
|
-
"json"
|
|
99
|
-
]
|
|
100
|
-
}
|
|
1
|
+
{
|
|
2
|
+
"id": "naep",
|
|
3
|
+
"title": "NAEP, The Nation's Report Card (NAEP Data Service)",
|
|
4
|
+
"agency": "National Center for Education Statistics (U.S. Department of Education), for the National Assessment Governing Board",
|
|
5
|
+
"depth": "standard",
|
|
6
|
+
"domains": [
|
|
7
|
+
"education",
|
|
8
|
+
"k-12",
|
|
9
|
+
"student achievement"
|
|
10
|
+
],
|
|
11
|
+
"keywords": [
|
|
12
|
+
"naep",
|
|
13
|
+
"nation's report card",
|
|
14
|
+
"test scores",
|
|
15
|
+
"reading scores",
|
|
16
|
+
"math scores",
|
|
17
|
+
"achievement",
|
|
18
|
+
"proficient",
|
|
19
|
+
"proficiency",
|
|
20
|
+
"achievement gap",
|
|
21
|
+
"4th grade",
|
|
22
|
+
"8th grade",
|
|
23
|
+
"12th grade",
|
|
24
|
+
"grade 4",
|
|
25
|
+
"grade 8",
|
|
26
|
+
"student performance",
|
|
27
|
+
"school",
|
|
28
|
+
"learning loss",
|
|
29
|
+
"science scores",
|
|
30
|
+
"civics",
|
|
31
|
+
"us history",
|
|
32
|
+
"tuda",
|
|
33
|
+
"urban district"
|
|
34
|
+
],
|
|
35
|
+
"questions": [
|
|
36
|
+
"4th grade reading scores by state",
|
|
37
|
+
"did math scores fall since 2019",
|
|
38
|
+
"percent of 8th graders proficient in math",
|
|
39
|
+
"achievement gap by race or school lunch eligibility",
|
|
40
|
+
"how a state compares with the national average",
|
|
41
|
+
"NAEP results for a large city district"
|
|
42
|
+
],
|
|
43
|
+
"homepage": "https://www.nationsreportcard.gov/",
|
|
44
|
+
"terms_url": "https://nces.ed.gov/help/copyright.asp",
|
|
45
|
+
"auth": {
|
|
46
|
+
"env": null,
|
|
47
|
+
"required": "none",
|
|
48
|
+
"placement": "none",
|
|
49
|
+
"signup_url": null,
|
|
50
|
+
"echoes_key": false
|
|
51
|
+
},
|
|
52
|
+
"limits": {
|
|
53
|
+
"rps": 2,
|
|
54
|
+
"burst": 4,
|
|
55
|
+
"max_rows": 50000
|
|
56
|
+
},
|
|
57
|
+
"routes": {
|
|
58
|
+
"api": {
|
|
59
|
+
"base": "https://www.nationsreportcard.gov",
|
|
60
|
+
"probe": "https://www.nationsreportcard.gov/Dataservice/GetAdhocData.aspx?type=data&subject=reading&grade=4&subscale=RRPCM&variable=TOTAL&jurisdiction=NP&stattype=MN:MN&Year=Current",
|
|
61
|
+
"hosts": [
|
|
62
|
+
"www.nationsreportcard.gov"
|
|
63
|
+
]
|
|
64
|
+
},
|
|
65
|
+
"manual": {
|
|
66
|
+
"how": "Open each link in manual_download/INSTRUCTIONS.md in a browser (each returns JSON; save it under the name shown) and upload the files. Without the links: the NAEP Data Explorer (https://www.nationsreportcard.gov/ndecore/landing) builds the same tables (choose subject, grade, jurisdictions, variables, statistics; Download > Excel); upload the export and say which table it is. The report cards at https://www.nationsreportcard.gov/ give the headline numbers with significance marks."
|
|
67
|
+
}
|
|
68
|
+
},
|
|
69
|
+
"cache": {
|
|
70
|
+
"mode": "run_only",
|
|
71
|
+
"ttl_hours": null,
|
|
72
|
+
"why": "Small queries; results change only when a new assessment is released (about every two years), but data/ is reused only within a run."
|
|
73
|
+
},
|
|
74
|
+
"redistribution": {
|
|
75
|
+
"mode": "allowed",
|
|
76
|
+
"why": "NAEP results are published by NCES, a U.S. government agency; its materials are in the public domain (17 U.S.C. 105) unless marked otherwise, and NCES asks that the source be cited."
|
|
77
|
+
},
|
|
78
|
+
"notices": [],
|
|
79
|
+
"retention": null,
|
|
80
|
+
"citation": "U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), {dataset}, NAEP Data Service (nationsreportcard.gov), retrieved {retrieved_long}.",
|
|
81
|
+
"audit_hooks": [
|
|
82
|
+
{
|
|
83
|
+
"id": "naep_uncertainty",
|
|
84
|
+
"kind": "require_text",
|
|
85
|
+
"any": [
|
|
86
|
+
"standard error",
|
|
87
|
+
"significant"
|
|
88
|
+
],
|
|
89
|
+
"why": "NAEP scores are sample estimates with standard errors; a report must say whether differences are statistically significant (NAEP's own tests), not compare point estimates."
|
|
90
|
+
}
|
|
91
|
+
],
|
|
92
|
+
"clarify": [
|
|
93
|
+
"Which subject, grade(s) and year(s)? Default: reading and mathematics, grades 4 and 8, the latest year and 2019 (pre-pandemic).",
|
|
94
|
+
"Average scale score or achievement level (percent at or above Proficient)? They can move differently; default: report both.",
|
|
95
|
+
"Public schools only (NP, the comparison base for states) or all schools (NT)? Default NP when states are involved."
|
|
96
|
+
],
|
|
97
|
+
"formats": [
|
|
98
|
+
"json"
|
|
99
|
+
]
|
|
100
|
+
}
|
package/packs/naep/source.md
CHANGED
|
@@ -8,7 +8,7 @@ student group, with standard errors and NAEP's own significance tests.
|
|
|
8
8
|
science, writing, civics, U.S. history, geography, economics, technology and engineering literacy,
|
|
9
9
|
music. States versus the nation, change since 2019, gaps by race, sex, economic disadvantage.
|
|
10
10
|
- Not here: state test results (each state's own exams, not comparable across states), graduation rates
|
|
11
|
-
and school finance (nces
|
|
11
|
+
and school finance (nces), college outcomes (college-scorecard).
|
|
12
12
|
- NAEP is a sample survey: every number is an estimate. It reports groups, never students or schools.
|
|
13
13
|
|
|
14
14
|
## Endpoints and datasets
|
|
@@ -0,0 +1,101 @@
|
|
|
1
|
+
{
|
|
2
|
+
"base_url": "https://nces.ed.gov",
|
|
3
|
+
"timeout": 180,
|
|
4
|
+
"endpoints": {
|
|
5
|
+
"digest_table": {
|
|
6
|
+
"path": "/programs/digest/d{edition}/tables/xls/tabn{table}.xlsx",
|
|
7
|
+
"method": "GET",
|
|
8
|
+
"description": "One table of the Digest of Education Statistics as its Excel file: enrollment, teachers, finances, degrees, outcomes, from pre-K to postsecondary. Default: every line numbered (row, col1...) to find the header lines; then header_lines (usually '3-4') names the columns.",
|
|
9
|
+
"params": {
|
|
10
|
+
"edition": {
|
|
11
|
+
"in": "path",
|
|
12
|
+
"required": true,
|
|
13
|
+
"desc": "two-digit Digest edition, e.g. 25 (Digest 2025); not every table is in every edition"
|
|
14
|
+
},
|
|
15
|
+
"table": {
|
|
16
|
+
"in": "path",
|
|
17
|
+
"required": true,
|
|
18
|
+
"desc": "table number, e.g. 203.10 (public school enrollment by grade)"
|
|
19
|
+
},
|
|
20
|
+
"header_lines": {
|
|
21
|
+
"in": "records",
|
|
22
|
+
"desc": "the header line numbers, e.g. '3-4'"
|
|
23
|
+
},
|
|
24
|
+
"header_row": {
|
|
25
|
+
"in": "records",
|
|
26
|
+
"desc": "one header line instead; 0 (default) returns every line numbered"
|
|
27
|
+
},
|
|
28
|
+
"sheet": {
|
|
29
|
+
"in": "records",
|
|
30
|
+
"desc": "sheet name or number (default 1)"
|
|
31
|
+
},
|
|
32
|
+
"carry": {
|
|
33
|
+
"in": "records",
|
|
34
|
+
"desc": "which header lines span to the right over blank cells: top (default), all, none; check the names against the numbered lines"
|
|
35
|
+
}
|
|
36
|
+
},
|
|
37
|
+
"defaults": {
|
|
38
|
+
"header_row": 0
|
|
39
|
+
},
|
|
40
|
+
"records": "$xlsx",
|
|
41
|
+
"paging": {
|
|
42
|
+
"type": "none"
|
|
43
|
+
},
|
|
44
|
+
"max_rows": 20000
|
|
45
|
+
},
|
|
46
|
+
"ipeds": {
|
|
47
|
+
"path": "/ipeds/datacenter/data/{file}.zip",
|
|
48
|
+
"method": "GET",
|
|
49
|
+
"description": "An IPEDS complete data file (one row per institution, or institution by category), e.g. HD2023 directory, EFFY2023 12-month enrollment, C2023_A completions by CIP code, ADM2023 admissions, SFA2223 student aid, F2223_F2 finance. The revised (_rv) version is read when the zip has one.",
|
|
50
|
+
"params": {
|
|
51
|
+
"file": {
|
|
52
|
+
"in": "path",
|
|
53
|
+
"required": true,
|
|
54
|
+
"desc": "IPEDS file name as on the Data Center's Complete Data Files page, e.g. HD2023, EFFY2023, C2023_A"
|
|
55
|
+
},
|
|
56
|
+
"unitid": {
|
|
57
|
+
"in": "spec",
|
|
58
|
+
"desc": "keep only these institutions (UNITID, a list works)"
|
|
59
|
+
},
|
|
60
|
+
"state": {
|
|
61
|
+
"in": "spec",
|
|
62
|
+
"desc": "keep only these states (two-letter STABBR; files that have that column, e.g. HD)"
|
|
63
|
+
},
|
|
64
|
+
"member": {
|
|
65
|
+
"in": "records",
|
|
66
|
+
"desc": "the CSV in the zip, e.g. c2023_a.csv for the original instead of the revised file"
|
|
67
|
+
}
|
|
68
|
+
},
|
|
69
|
+
"records": "$zip:*.csv",
|
|
70
|
+
"prefer": "_rv[.]csv$",
|
|
71
|
+
"paging": {
|
|
72
|
+
"type": "none"
|
|
73
|
+
},
|
|
74
|
+
"max_rows": 1000000
|
|
75
|
+
},
|
|
76
|
+
"ipeds_dictionary": {
|
|
77
|
+
"path": "/ipeds/datacenter/data/{file}_Dict.zip",
|
|
78
|
+
"method": "GET",
|
|
79
|
+
"description": "The data dictionary of an IPEDS file: variable names and titles (sheet Varlist), code values and labels (Frequencies), long descriptions (Description).",
|
|
80
|
+
"params": {
|
|
81
|
+
"file": {
|
|
82
|
+
"in": "path",
|
|
83
|
+
"required": true,
|
|
84
|
+
"desc": "the IPEDS file name, e.g. HD2023"
|
|
85
|
+
},
|
|
86
|
+
"sheet": {
|
|
87
|
+
"in": "records",
|
|
88
|
+
"desc": "Varlist (default), Frequencies, Description, Introduction"
|
|
89
|
+
}
|
|
90
|
+
},
|
|
91
|
+
"defaults": {
|
|
92
|
+
"sheet": "Varlist"
|
|
93
|
+
},
|
|
94
|
+
"records": "$zip:*.xlsx",
|
|
95
|
+
"paging": {
|
|
96
|
+
"type": "none"
|
|
97
|
+
},
|
|
98
|
+
"max_rows": 100000
|
|
99
|
+
}
|
|
100
|
+
}
|
|
101
|
+
}
|
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
{
|
|
2
|
+
"id": "nces",
|
|
3
|
+
"title": "National Center for Education Statistics: Digest of Education Statistics tables and IPEDS data files",
|
|
4
|
+
"agency": "U.S. Department of Education, National Center for Education Statistics",
|
|
5
|
+
"depth": "standard",
|
|
6
|
+
"domains": [
|
|
7
|
+
"education",
|
|
8
|
+
"higher education"
|
|
9
|
+
],
|
|
10
|
+
"keywords": [
|
|
11
|
+
"nces",
|
|
12
|
+
"national center for education statistics",
|
|
13
|
+
"digest of education statistics",
|
|
14
|
+
"school enrollment",
|
|
15
|
+
"public school enrollment",
|
|
16
|
+
"enrollment by grade",
|
|
17
|
+
"teachers",
|
|
18
|
+
"pupil teacher ratio",
|
|
19
|
+
"per-pupil expenditure",
|
|
20
|
+
"school finance",
|
|
21
|
+
"high school graduates",
|
|
22
|
+
"graduation rate",
|
|
23
|
+
"degrees conferred",
|
|
24
|
+
"ipeds",
|
|
25
|
+
"college enrollment",
|
|
26
|
+
"fall enrollment",
|
|
27
|
+
"completions",
|
|
28
|
+
"degrees by field",
|
|
29
|
+
"cip code",
|
|
30
|
+
"institutional characteristics",
|
|
31
|
+
"tuition and fees",
|
|
32
|
+
"admissions",
|
|
33
|
+
"student financial aid",
|
|
34
|
+
"college finance"
|
|
35
|
+
],
|
|
36
|
+
"questions": [
|
|
37
|
+
"public school enrollment by grade over time",
|
|
38
|
+
"number of teachers and pupil/teacher ratio",
|
|
39
|
+
"current expenditure per pupil by state",
|
|
40
|
+
"bachelor's degrees conferred by field",
|
|
41
|
+
"enrollment at a college or all colleges in a state (IPEDS)",
|
|
42
|
+
"completions by CIP code and award level",
|
|
43
|
+
"admissions and yield at a university"
|
|
44
|
+
],
|
|
45
|
+
"homepage": "https://nces.ed.gov/",
|
|
46
|
+
"terms_url": "https://nces.ed.gov/ipeds/datacenter/DataFiles.aspx",
|
|
47
|
+
"auth": {
|
|
48
|
+
"env": null,
|
|
49
|
+
"required": "none",
|
|
50
|
+
"placement": "none",
|
|
51
|
+
"signup_url": null,
|
|
52
|
+
"echoes_key": false
|
|
53
|
+
},
|
|
54
|
+
"limits": {
|
|
55
|
+
"rps": 2,
|
|
56
|
+
"burst": 4,
|
|
57
|
+
"max_rows": 1000000
|
|
58
|
+
},
|
|
59
|
+
"routes": {
|
|
60
|
+
"api": {
|
|
61
|
+
"base": "https://nces.ed.gov",
|
|
62
|
+
"probe": "https://nces.ed.gov/ipeds/datacenter/data/HD2023_Dict.zip",
|
|
63
|
+
"hosts": [
|
|
64
|
+
"nces.ed.gov"
|
|
65
|
+
]
|
|
66
|
+
},
|
|
67
|
+
"manual": {
|
|
68
|
+
"how": "Digest: open the table page (https://nces.ed.gov/programs/digest/current_tables.asp, then the table) and click its Excel link, or https://nces.ed.gov/programs/digest/d<yy>/tables/xls/tabn<table>.xlsx. IPEDS: download https://nces.ed.gov/ipeds/datacenter/data/<FILE>.zip from the Complete Data Files page (https://nces.ed.gov/ipeds/datacenter/DataFiles.aspx). Upload the file as saved with ingest."
|
|
69
|
+
}
|
|
70
|
+
},
|
|
71
|
+
"cache": {
|
|
72
|
+
"mode": "run_only",
|
|
73
|
+
"ttl_hours": null,
|
|
74
|
+
"why": "Files are replaced when NCES revises them; data/ is reused within a run."
|
|
75
|
+
},
|
|
76
|
+
"redistribution": {
|
|
77
|
+
"mode": "allowed",
|
|
78
|
+
"why": "NCES publications and IPEDS data are U.S. government works in the public domain (17 U.S.C. 105)."
|
|
79
|
+
},
|
|
80
|
+
"notices": [],
|
|
81
|
+
"retention": null,
|
|
82
|
+
"citation": "U.S. Department of Education, National Center for Education Statistics, {dataset}, nces.ed.gov, retrieved {retrieved_long}.",
|
|
83
|
+
"audit_hooks": [],
|
|
84
|
+
"clarify": [
|
|
85
|
+
"Published table or institution file? Digest tables give national (and often state) totals as NCES published them; IPEDS files give one row per institution to filter and sum. Default: a Digest table for national or state totals, IPEDS for named colleges.",
|
|
86
|
+
"Which year? Digest tables are labelled by school year or fall; IPEDS files by collection year (EFFY2023 = 12-month enrollment for 2022-23). Say which, and whether the IPEDS file is provisional or revised (_rv)."
|
|
87
|
+
],
|
|
88
|
+
"formats": [
|
|
89
|
+
"xlsx",
|
|
90
|
+
"zip",
|
|
91
|
+
"csv"
|
|
92
|
+
],
|
|
93
|
+
"aliases": [
|
|
94
|
+
"NCES",
|
|
95
|
+
"IPEDS",
|
|
96
|
+
"Digest of Education Statistics",
|
|
97
|
+
"Integrated Postsecondary Education Data System"
|
|
98
|
+
]
|
|
99
|
+
}
|
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
# National Center for Education Statistics: Digest of Education Statistics tables and IPEDS data files
|
|
2
|
+
Depth: standard. Key: none.
|
|
3
|
+
- The Digest of Education Statistics: NCES's compendium of published tables (enrollment, teachers, finances,
|
|
4
|
+
graduates, degrees, outcomes, pre-K to postsecondary), as each table's Excel file.
|
|
5
|
+
- IPEDS complete data files: one row per postsecondary institution, or institution by category, for every
|
|
6
|
+
survey component (directory, enrollment, completions, admissions, student aid, finance, staff), with their
|
|
7
|
+
data dictionaries.
|
|
8
|
+
|
|
9
|
+
## When to use
|
|
10
|
+
- National or state totals as NCES published them: public school enrollment by grade, teachers and
|
|
11
|
+
pupil/teacher ratios, current expenditure per pupil, high school completers, degrees conferred by field and
|
|
12
|
+
level, enrollment trends.
|
|
13
|
+
- A named college or every college in a state: fall or 12-month enrollment, completions by CIP code and award
|
|
14
|
+
level, admissions and test scores, tuition, aid, finance (IPEDS).
|
|
15
|
+
- Not here: earnings and debt of graduates by program and institution (college-scorecard); federal student
|
|
16
|
+
loans and Pell totals (fsa); test scores of K-12 students (naep); school-age population, poverty or
|
|
17
|
+
attainment from surveys (census ACS).
|
|
18
|
+
|
|
19
|
+
## Endpoints
|
|
20
|
+
| Endpoint | Returns | Parameters |
|
|
21
|
+
|---|---|---|
|
|
22
|
+
| `digest_table` | one Digest table: without `header_lines`, every line numbered (`row`, `col1`...); with it, named columns | `edition` (two digits, e.g. `25`), `table` (e.g. `203.10`), `header_lines` (usually `3-4`), `header_row`, `sheet` |
|
|
23
|
+
| `ipeds` | an IPEDS complete data file (the revised `_rv` CSV when the zip has one) | `file` (e.g. `HD2023`, `EFFY2023`, `C2023_A`), `unitid`, `state` (filters), `member` |
|
|
24
|
+
| `ipeds_dictionary` | that file's dictionary sheet | `file`, `sheet` (`Varlist` default, `Frequencies` for code labels, `Description`) |
|
|
25
|
+
|
|
26
|
+
## Spec examples
|
|
27
|
+
```json
|
|
28
|
+
{"description": "Public school enrollment by grade, fall 1980-2024 (Digest 2025 table 203.10)",
|
|
29
|
+
"requests": [{"name": "t203_10", "endpoint": "digest_table", "params": {"edition": "25", "table": "203.10", "header_lines": "3-4"}}]}
|
|
30
|
+
```
|
|
31
|
+
```json
|
|
32
|
+
{"description": "Colleges in Vermont (IPEDS directory 2023-24) and the code labels",
|
|
33
|
+
"requests": [{"name": "vt_colleges", "endpoint": "ipeds", "params": {"file": "HD2023", "state": "VT"}},
|
|
34
|
+
{"name": "hd_codes", "endpoint": "ipeds_dictionary", "params": {"file": "HD2023", "sheet": "Frequencies"}}]}
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
## Data format and traps
|
|
38
|
+
- Digest tables:
|
|
39
|
+
- Line 1 is the title, line 2 the units ("[In thousands]": multiply), lines 3-4 the stacked header, and the
|
|
40
|
+
first data line is the row of column numbers ("1", "2", ...): drop it. Notes and the source follow the last
|
|
41
|
+
data line. Verified on table 203.10, Digest 2025; other tables differ, so read the raw lines first.
|
|
42
|
+
- Columns named `<header>_colN` hold footnote marks for the value to their left, or nothing. In table
|
|
43
|
+
203.10's 2024 row, `\3,4\` points to footnotes 3 and 4 below the table. Keep the marks with the value they
|
|
44
|
+
qualify.
|
|
45
|
+
- Values are text with thousands separators. Footnote marks also sit inside header cells (`Ungraded\1\`), and
|
|
46
|
+
some cells start with a line break ("\n1993"): trim them. Symbols such as `†` and `—`, used in many
|
|
47
|
+
tables, are defined in each table's notes; keep them and never treat them as 0. Projections are marked in
|
|
48
|
+
the notes.
|
|
49
|
+
- Each edition renumbers or drops tables, and a table that is not in that edition answers 404. Use the
|
|
50
|
+
current edition's table list (https://nces.ed.gov/programs/digest/current_tables.asp) to find the number.
|
|
51
|
+
- IPEDS:
|
|
52
|
+
- Names. Files are named by collection year: HD2023 is the 2023-24 directory, EFFY2023 the 12-month enrollment
|
|
53
|
+
of 2022-23, C2023_A the completions of July 2022-June 2023. Read the dictionary's Introduction sheet to be
|
|
54
|
+
sure.
|
|
55
|
+
- Revisions. When a zip has a revised file (`_rv`, the final release), it is read instead of the provisional
|
|
56
|
+
one; `member` picks the other.
|
|
57
|
+
- Codes. Columns are coded (`SECTOR`, `CONTROL`, `ICLEVEL`; `X`-prefixed imputation flags): label them from
|
|
58
|
+
the dictionary's Frequencies sheet. Negative codes such as -1 and -2 mean not reported or not applicable;
|
|
59
|
+
never sum them.
|
|
60
|
+
- Joining files. UNITID joins files; a parent/child system (`F1SYSTYP` ...) can report for several campuses.
|
|
61
|
+
- Size. Completions files are large (C2023_A has two 60 MB CSVs): filter by `unitid` or `state`. `state`
|
|
62
|
+
works only on files with `STABBR`, so join HD for state.
|
|
63
|
+
- Counting. Enrollment and completions rows are by category (all students, by race, by sex, by CIP and award
|
|
64
|
+
level), and totals are rows of their own (CIPCODE 99, MAJORNUM 1): pick the total row; never sum across
|
|
65
|
+
categories.
|
|
66
|
+
|
|
67
|
+
## Access
|
|
68
|
+
No key. `fetch` reads nces.ed.gov directly (verified 2026-11-07). Manual: the Digest table's Excel link, or the
|
|
69
|
+
IPEDS Complete Data Files page (https://nces.ed.gov/ipeds/datacenter/DataFiles.aspx); upload the file as saved;
|
|
70
|
+
then `ingest`.
|
|
71
|
+
|
|
72
|
+
## Citation and terms
|
|
73
|
+
"U.S. Department of Education, National Center for Education Statistics, Digest of Education Statistics 2025,
|
|
74
|
+
table 203.10" or "... Integrated Postsecondary Education Data System (IPEDS), <file> (<release>), nces.ed.gov,
|
|
75
|
+
retrieved <date>." Public domain (17 U.S.C. 105).
|