data-analyst-mcp 0.3.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (292) hide show
  1. package/README.md +286 -262
  2. package/dist/adapters/bjs.d.ts +13 -0
  3. package/dist/adapters/bjs.js +28 -0
  4. package/dist/adapters/bjs.js.map +1 -0
  5. package/dist/adapters/cbp.d.ts +5 -0
  6. package/dist/adapters/cbp.js +64 -0
  7. package/dist/adapters/cbp.js.map +1 -0
  8. package/dist/adapters/cdc-wonder.d.ts +5 -0
  9. package/dist/adapters/cdc-wonder.js +68 -0
  10. package/dist/adapters/cdc-wonder.js.map +1 -0
  11. package/dist/adapters/cpsc.d.ts +4 -0
  12. package/dist/adapters/cpsc.js +125 -0
  13. package/dist/adapters/cpsc.js.map +1 -0
  14. package/dist/adapters/fhwa.d.ts +9 -0
  15. package/dist/adapters/fhwa.js +16 -0
  16. package/dist/adapters/fhwa.js.map +1 -0
  17. package/dist/adapters/foreign-assistance.d.ts +9 -0
  18. package/dist/adapters/foreign-assistance.js +42 -0
  19. package/dist/adapters/foreign-assistance.js.map +1 -0
  20. package/dist/adapters/hud-ahar.d.ts +3 -0
  21. package/dist/adapters/hud-ahar.js +80 -0
  22. package/dist/adapters/hud-ahar.js.map +1 -0
  23. package/dist/adapters/nces.d.ts +10 -0
  24. package/dist/adapters/nces.js +32 -0
  25. package/dist/adapters/nces.js.map +1 -0
  26. package/dist/adapters/nsf.d.ts +11 -0
  27. package/dist/adapters/nsf.js +25 -0
  28. package/dist/adapters/nsf.js.map +1 -0
  29. package/dist/adapters/onrr.d.ts +9 -0
  30. package/dist/adapters/onrr.js +35 -0
  31. package/dist/adapters/onrr.js.map +1 -0
  32. package/dist/adapters/sba.d.ts +10 -0
  33. package/dist/adapters/sba.js +134 -0
  34. package/dist/adapters/sba.js.map +1 -0
  35. package/dist/adapters/treasury-tic.d.ts +30 -0
  36. package/dist/adapters/treasury-tic.js +149 -0
  37. package/dist/adapters/treasury-tic.js.map +1 -0
  38. package/dist/adapters/uscourts.d.ts +8 -0
  39. package/dist/adapters/uscourts.js +8 -0
  40. package/dist/adapters/uscourts.js.map +1 -0
  41. package/dist/analysis/cite.d.ts +9 -1
  42. package/dist/analysis/cite.js +9 -2
  43. package/dist/analysis/cite.js.map +1 -1
  44. package/dist/bin/cli.d.ts +2 -0
  45. package/dist/bin/cli.js +17 -0
  46. package/dist/bin/cli.js.map +1 -1
  47. package/dist/core/generic-rest.js +42 -17
  48. package/dist/core/generic-rest.js.map +1 -1
  49. package/dist/core/guard.d.ts +32 -0
  50. package/dist/core/guard.js +176 -0
  51. package/dist/core/guard.js.map +1 -0
  52. package/dist/core/http.d.ts +10 -0
  53. package/dist/core/http.js +36 -7
  54. package/dist/core/http.js.map +1 -1
  55. package/dist/core/keys.d.ts +13 -3
  56. package/dist/core/keys.js +67 -7
  57. package/dist/core/keys.js.map +1 -1
  58. package/dist/core/package.d.ts +12 -3
  59. package/dist/core/package.js +56 -12
  60. package/dist/core/package.js.map +1 -1
  61. package/dist/core/secret_patterns.d.ts +6 -0
  62. package/dist/core/secret_patterns.js +20 -0
  63. package/dist/core/secret_patterns.js.map +1 -0
  64. package/dist/core/tabular.d.ts +43 -1
  65. package/dist/core/tabular.js +149 -6
  66. package/dist/core/tabular.js.map +1 -1
  67. package/dist/envelope.js +3 -1
  68. package/dist/envelope.js.map +1 -1
  69. package/dist/index.d.ts +29 -22
  70. package/dist/index.js +45 -46
  71. package/dist/index.js.map +1 -1
  72. package/dist/remote/app.js +87 -31
  73. package/dist/remote/app.js.map +1 -1
  74. package/dist/remote/oauth.js +8 -1
  75. package/dist/remote/oauth.js.map +1 -1
  76. package/dist/remote/pages.d.ts +3 -7
  77. package/dist/remote/pages.js +49 -19
  78. package/dist/remote/pages.js.map +1 -1
  79. package/dist/remote/sign.d.ts +7 -1
  80. package/dist/remote/sign.js +33 -7
  81. package/dist/remote/sign.js.map +1 -1
  82. package/dist/remote/store.js +14 -5
  83. package/dist/remote/store.js.map +1 -1
  84. package/dist/report/audit_report.js +29 -7
  85. package/dist/report/audit_report.js.map +1 -1
  86. package/dist/report/build_report.js +34 -4
  87. package/dist/report/build_report.js.map +1 -1
  88. package/dist/report/method_audit.js +10 -1
  89. package/dist/report/method_audit.js.map +1 -1
  90. package/dist/report/scan_secrets.d.ts +4 -1
  91. package/dist/report/scan_secrets.js +38 -18
  92. package/dist/report/scan_secrets.js.map +1 -1
  93. package/dist/server.d.ts +3 -1
  94. package/dist/server.js +46 -13
  95. package/dist/server.js.map +1 -1
  96. package/dist/tools.js +27 -6
  97. package/dist/tools.js.map +1 -1
  98. package/guide/WORKFLOW.md +134 -134
  99. package/guide/assets/disclaimer.txt +1 -1
  100. package/guide/assets/download_data.ps1 +164 -162
  101. package/guide/assets/download_data.sh +8 -2
  102. package/guide/assets/report_template.html +597 -597
  103. package/guide/references/routing.md +198 -171
  104. package/guide/references/style_guide.md +398 -398
  105. package/package.json +56 -56
  106. package/packs/acf/endpoints.json +114 -114
  107. package/packs/acf/policy.json +104 -104
  108. package/packs/actuaries/endpoints.json +82 -82
  109. package/packs/actuaries/policy.json +109 -109
  110. package/packs/bea/assets/download_bea.ps1 +182 -182
  111. package/packs/bea/docs/bea_api.md +461 -461
  112. package/packs/bea/docs/datasets.md +157 -157
  113. package/packs/bea/docs/report_patterns.md +112 -112
  114. package/packs/bea/policy.json +170 -170
  115. package/packs/bea/source.md +152 -152
  116. package/packs/bjs/endpoints.json +150 -0
  117. package/packs/bjs/policy.json +114 -0
  118. package/packs/bjs/source.md +98 -0
  119. package/packs/bls/assets/download_bls.ps1 +152 -152
  120. package/packs/bls/docs/analysis_patterns.md +108 -108
  121. package/packs/bls/docs/bls_api.md +73 -73
  122. package/packs/bls/docs/series_ids.md +149 -149
  123. package/packs/bls/policy.json +238 -238
  124. package/packs/bts/endpoints.json +346 -346
  125. package/packs/bts/policy.json +94 -94
  126. package/packs/cbp/endpoints.json +31 -0
  127. package/packs/cbp/policy.json +85 -0
  128. package/packs/cbp/source.md +57 -0
  129. package/packs/cdc/endpoints.json +197 -197
  130. package/packs/cdc/policy.json +83 -83
  131. package/packs/cdc/source.md +1 -1
  132. package/packs/cdc-wonder/endpoints.json +26 -0
  133. package/packs/cdc-wonder/policy.json +84 -0
  134. package/packs/cdc-wonder/source.md +53 -0
  135. package/packs/census/endpoints.json +99 -99
  136. package/packs/cfpb/endpoints.json +326 -326
  137. package/packs/cfpb/policy.json +119 -119
  138. package/packs/clinical-trials/endpoints.json +166 -166
  139. package/packs/clinical-trials/policy.json +84 -84
  140. package/packs/cms/endpoints.json +69 -69
  141. package/packs/cms/policy.json +96 -96
  142. package/packs/college-scorecard/endpoints.json +252 -252
  143. package/packs/college-scorecard/policy.json +106 -106
  144. package/packs/college-scorecard/source.md +1 -1
  145. package/packs/congress/endpoints.json +1701 -1701
  146. package/packs/congress/policy.json +117 -117
  147. package/packs/cpsc/endpoints.json +91 -0
  148. package/packs/cpsc/policy.json +91 -0
  149. package/packs/cpsc/source.md +76 -0
  150. package/packs/dhs-ohss/endpoints.json +66 -66
  151. package/packs/dhs-ohss/policy.json +107 -107
  152. package/packs/dhs-ohss/source.md +1 -1
  153. package/packs/doj-news/endpoints.json +59 -59
  154. package/packs/doj-news/policy.json +96 -96
  155. package/packs/doj-news/source.md +1 -1
  156. package/packs/dol/endpoints.json +817 -817
  157. package/packs/dol/policy.json +135 -135
  158. package/packs/eia/endpoints.json +717 -717
  159. package/packs/eia/policy.json +112 -112
  160. package/packs/epa/endpoints.json +291 -291
  161. package/packs/epa/policy.json +99 -99
  162. package/packs/epa-aqs/endpoints.json +173 -173
  163. package/packs/epa-aqs/policy.json +96 -96
  164. package/packs/fbi/endpoints.json +1841 -1841
  165. package/packs/fbi/policy.json +118 -118
  166. package/packs/fbi/source.md +3 -3
  167. package/packs/fda/endpoints.json +237 -237
  168. package/packs/fda/policy.json +121 -121
  169. package/packs/fdic/endpoints.json +416 -416
  170. package/packs/fdic/policy.json +83 -83
  171. package/packs/fec/endpoints.json +707 -707
  172. package/packs/fec/policy.json +93 -93
  173. package/packs/fed-board/endpoints.json +89 -89
  174. package/packs/fed-board/policy.json +136 -136
  175. package/packs/federal-register/endpoints.json +143 -143
  176. package/packs/federal-register/policy.json +86 -86
  177. package/packs/fema/endpoints.json +645 -645
  178. package/packs/fema/policy.json +103 -103
  179. package/packs/fhfa/endpoints.json +105 -105
  180. package/packs/fhfa/policy.json +109 -109
  181. package/packs/fhwa/endpoints.json +48 -0
  182. package/packs/fhwa/policy.json +91 -0
  183. package/packs/fhwa/source.md +59 -0
  184. package/packs/foreign-assistance/endpoints.json +166 -0
  185. package/packs/foreign-assistance/policy.json +80 -0
  186. package/packs/foreign-assistance/source.md +76 -0
  187. package/packs/fred/assets/download_fred.ps1 +122 -122
  188. package/packs/fred/docs/analysis_patterns.md +124 -124
  189. package/packs/fred/docs/fred_api.md +94 -94
  190. package/packs/fred/docs/series.md +188 -188
  191. package/packs/fred/policy.json +253 -253
  192. package/packs/fsa/endpoints.json +150 -150
  193. package/packs/fsa/policy.json +99 -99
  194. package/packs/fsa/source.md +1 -1
  195. package/packs/govinfo/endpoints.json +519 -519
  196. package/packs/gsa-calc/endpoints.json +93 -93
  197. package/packs/gsa-calc/policy.json +97 -97
  198. package/packs/hud/endpoints.json +146 -146
  199. package/packs/hud/policy.json +84 -84
  200. package/packs/hud/source.md +1 -1
  201. package/packs/hud-ahar/endpoints.json +92 -0
  202. package/packs/hud-ahar/policy.json +88 -0
  203. package/packs/hud-ahar/source.md +72 -0
  204. package/packs/ipums/docs/report_patterns.md +117 -117
  205. package/packs/irs-soi/endpoints.json +114 -114
  206. package/packs/irs-soi/policy.json +107 -107
  207. package/packs/medicaid/endpoints.json +412 -412
  208. package/packs/medicaid/policy.json +108 -108
  209. package/packs/naep/endpoints.json +217 -217
  210. package/packs/naep/policy.json +100 -100
  211. package/packs/naep/source.md +1 -1
  212. package/packs/nces/endpoints.json +101 -0
  213. package/packs/nces/policy.json +99 -0
  214. package/packs/nces/source.md +75 -0
  215. package/packs/nhtsa/endpoints.json +132 -132
  216. package/packs/nhtsa/policy.json +93 -93
  217. package/packs/nhtsa/source.md +1 -1
  218. package/packs/nih/endpoints.json +452 -452
  219. package/packs/nih/policy.json +86 -86
  220. package/packs/noaa/endpoints.json +431 -431
  221. package/packs/noaa/policy.json +95 -95
  222. package/packs/nrel/endpoints.json +327 -327
  223. package/packs/nrel/policy.json +96 -96
  224. package/packs/nsf/endpoints.json +33 -0
  225. package/packs/nsf/policy.json +83 -0
  226. package/packs/nsf/source.md +59 -0
  227. package/packs/nws/endpoints.json +271 -271
  228. package/packs/nws/policy.json +82 -82
  229. package/packs/nyfed/endpoints.json +80 -80
  230. package/packs/nyfed/policy.json +92 -92
  231. package/packs/omb/endpoints.json +460 -460
  232. package/packs/omb/policy.json +106 -106
  233. package/packs/onrr/endpoints.json +198 -0
  234. package/packs/onrr/policy.json +91 -0
  235. package/packs/onrr/source.md +55 -0
  236. package/packs/open-payments/endpoints.json +297 -297
  237. package/packs/open-payments/policy.json +101 -101
  238. package/packs/opm/assets/download_opm.ps1 +125 -125
  239. package/packs/opm/docs/fields.md +184 -184
  240. package/packs/opm/docs/opm_api.md +71 -71
  241. package/packs/opm/docs/report_patterns.md +83 -83
  242. package/packs/opm/lib/data_dictionary.csv +73 -73
  243. package/packs/pbgc/endpoints.json +483 -483
  244. package/packs/pbgc/policy.json +94 -94
  245. package/packs/philfed/endpoints.json +85 -85
  246. package/packs/philfed/policy.json +94 -94
  247. package/packs/port.json +65 -0
  248. package/packs/regulations/endpoints.json +287 -287
  249. package/packs/regulations/policy.json +86 -86
  250. package/packs/sba/endpoints.json +34 -0
  251. package/packs/sba/policy.json +86 -0
  252. package/packs/sba/source.md +60 -0
  253. package/packs/sec/endpoints.json +236 -236
  254. package/packs/sec/policy.json +86 -86
  255. package/packs/senate-lobbying/endpoints.json +553 -553
  256. package/packs/senate-lobbying/policy.json +99 -99
  257. package/packs/snapqc/endpoints.json +1426 -1426
  258. package/packs/snapqc/policy.json +92 -92
  259. package/packs/treasury/catalog.json +2267 -2267
  260. package/packs/treasury/docs/api_guide.md +245 -245
  261. package/packs/treasury/docs/datasets.md +56 -56
  262. package/packs/treasury/docs/report_patterns.md +126 -126
  263. package/packs/treasury/policy.json +159 -159
  264. package/packs/treasury/source.md +170 -165
  265. package/packs/treasury-tic/endpoints.json +77 -0
  266. package/packs/treasury-tic/policy.json +89 -0
  267. package/packs/treasury-tic/source.md +80 -0
  268. package/packs/treasury-yields/endpoints.json +226 -226
  269. package/packs/treasury-yields/policy.json +86 -86
  270. package/packs/usaspending/endpoints.json +1315 -1315
  271. package/packs/usaspending/policy.json +109 -109
  272. package/packs/uscourts/endpoints.json +22 -0
  273. package/packs/uscourts/policy.json +87 -0
  274. package/packs/uscourts/source.md +56 -0
  275. package/packs/usda-fna/policy.json +133 -133
  276. package/packs/usda-fooddata/endpoints.json +180 -180
  277. package/packs/usda-fooddata/policy.json +94 -94
  278. package/packs/usda-nass/endpoints.json +323 -323
  279. package/packs/usda-nass/policy.json +95 -95
  280. package/packs/usda-outlook/endpoints.json +86 -86
  281. package/packs/usda-outlook/policy.json +109 -109
  282. package/packs/usgs/endpoints.json +313 -313
  283. package/packs/usgs/policy.json +95 -95
  284. package/packs/usitc/endpoints.json +74 -74
  285. package/packs/usitc/policy.json +111 -111
  286. package/packs/uspto/endpoints.json +141 -141
  287. package/packs/uspto/policy.json +95 -95
  288. package/packs/uspto/source.md +1 -1
  289. package/packs/va/endpoints.json +414 -414
  290. package/packs/va/policy.json +101 -101
  291. package/packs/world-bank/endpoints.json +122 -122
  292. package/packs/world-bank/policy.json +101 -101
@@ -1,100 +1,100 @@
1
- {
2
- "id": "naep",
3
- "title": "NAEP, The Nation's Report Card (NAEP Data Service)",
4
- "agency": "National Center for Education Statistics (U.S. Department of Education), for the National Assessment Governing Board",
5
- "depth": "standard",
6
- "domains": [
7
- "education",
8
- "k-12",
9
- "student achievement"
10
- ],
11
- "keywords": [
12
- "naep",
13
- "nation's report card",
14
- "test scores",
15
- "reading scores",
16
- "math scores",
17
- "achievement",
18
- "proficient",
19
- "proficiency",
20
- "achievement gap",
21
- "4th grade",
22
- "8th grade",
23
- "12th grade",
24
- "grade 4",
25
- "grade 8",
26
- "student performance",
27
- "school",
28
- "learning loss",
29
- "science scores",
30
- "civics",
31
- "us history",
32
- "tuda",
33
- "urban district"
34
- ],
35
- "questions": [
36
- "4th grade reading scores by state",
37
- "did math scores fall since 2019",
38
- "percent of 8th graders proficient in math",
39
- "achievement gap by race or school lunch eligibility",
40
- "how a state compares with the national average",
41
- "NAEP results for a large city district"
42
- ],
43
- "homepage": "https://www.nationsreportcard.gov/",
44
- "terms_url": "https://nces.ed.gov/help/copyright.asp",
45
- "auth": {
46
- "env": null,
47
- "required": "none",
48
- "placement": "none",
49
- "signup_url": null,
50
- "echoes_key": false
51
- },
52
- "limits": {
53
- "rps": 2,
54
- "burst": 4,
55
- "max_rows": 50000
56
- },
57
- "routes": {
58
- "api": {
59
- "base": "https://www.nationsreportcard.gov",
60
- "probe": "https://www.nationsreportcard.gov/Dataservice/GetAdhocData.aspx?type=data&subject=reading&grade=4&subscale=RRPCM&variable=TOTAL&jurisdiction=NP&stattype=MN:MN&Year=Current",
61
- "hosts": [
62
- "www.nationsreportcard.gov"
63
- ]
64
- },
65
- "manual": {
66
- "how": "Open each link in manual_download/INSTRUCTIONS.md in a browser (each returns JSON; save it under the name shown) and upload the files. Without the links: the NAEP Data Explorer (https://www.nationsreportcard.gov/ndecore/landing) builds the same tables (choose subject, grade, jurisdictions, variables, statistics; Download > Excel); upload the export and say which table it is. The report cards at https://www.nationsreportcard.gov/ give the headline numbers with significance marks."
67
- }
68
- },
69
- "cache": {
70
- "mode": "run_only",
71
- "ttl_hours": null,
72
- "why": "Small queries; results change only when a new assessment is released (about every two years), but data/ is reused only within a run."
73
- },
74
- "redistribution": {
75
- "mode": "allowed",
76
- "why": "NAEP results are published by NCES, a U.S. government agency; its materials are in the public domain (17 U.S.C. 105) unless marked otherwise, and NCES asks that the source be cited."
77
- },
78
- "notices": [],
79
- "retention": null,
80
- "citation": "U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), {dataset}, NAEP Data Service (nationsreportcard.gov), retrieved {retrieved_long}.",
81
- "audit_hooks": [
82
- {
83
- "id": "naep_uncertainty",
84
- "kind": "require_text",
85
- "any": [
86
- "standard error",
87
- "significant"
88
- ],
89
- "why": "NAEP scores are sample estimates with standard errors; a report must say whether differences are statistically significant (NAEP's own tests), not compare point estimates."
90
- }
91
- ],
92
- "clarify": [
93
- "Which subject, grade(s) and year(s)? Default: reading and mathematics, grades 4 and 8, the latest year and 2019 (pre-pandemic).",
94
- "Average scale score or achievement level (percent at or above Proficient)? They can move differently; default: report both.",
95
- "Public schools only (NP, the comparison base for states) or all schools (NT)? Default NP when states are involved."
96
- ],
97
- "formats": [
98
- "json"
99
- ]
100
- }
1
+ {
2
+ "id": "naep",
3
+ "title": "NAEP, The Nation's Report Card (NAEP Data Service)",
4
+ "agency": "National Center for Education Statistics (U.S. Department of Education), for the National Assessment Governing Board",
5
+ "depth": "standard",
6
+ "domains": [
7
+ "education",
8
+ "k-12",
9
+ "student achievement"
10
+ ],
11
+ "keywords": [
12
+ "naep",
13
+ "nation's report card",
14
+ "test scores",
15
+ "reading scores",
16
+ "math scores",
17
+ "achievement",
18
+ "proficient",
19
+ "proficiency",
20
+ "achievement gap",
21
+ "4th grade",
22
+ "8th grade",
23
+ "12th grade",
24
+ "grade 4",
25
+ "grade 8",
26
+ "student performance",
27
+ "school",
28
+ "learning loss",
29
+ "science scores",
30
+ "civics",
31
+ "us history",
32
+ "tuda",
33
+ "urban district"
34
+ ],
35
+ "questions": [
36
+ "4th grade reading scores by state",
37
+ "did math scores fall since 2019",
38
+ "percent of 8th graders proficient in math",
39
+ "achievement gap by race or school lunch eligibility",
40
+ "how a state compares with the national average",
41
+ "NAEP results for a large city district"
42
+ ],
43
+ "homepage": "https://www.nationsreportcard.gov/",
44
+ "terms_url": "https://nces.ed.gov/help/copyright.asp",
45
+ "auth": {
46
+ "env": null,
47
+ "required": "none",
48
+ "placement": "none",
49
+ "signup_url": null,
50
+ "echoes_key": false
51
+ },
52
+ "limits": {
53
+ "rps": 2,
54
+ "burst": 4,
55
+ "max_rows": 50000
56
+ },
57
+ "routes": {
58
+ "api": {
59
+ "base": "https://www.nationsreportcard.gov",
60
+ "probe": "https://www.nationsreportcard.gov/Dataservice/GetAdhocData.aspx?type=data&subject=reading&grade=4&subscale=RRPCM&variable=TOTAL&jurisdiction=NP&stattype=MN:MN&Year=Current",
61
+ "hosts": [
62
+ "www.nationsreportcard.gov"
63
+ ]
64
+ },
65
+ "manual": {
66
+ "how": "Open each link in manual_download/INSTRUCTIONS.md in a browser (each returns JSON; save it under the name shown) and upload the files. Without the links: the NAEP Data Explorer (https://www.nationsreportcard.gov/ndecore/landing) builds the same tables (choose subject, grade, jurisdictions, variables, statistics; Download > Excel); upload the export and say which table it is. The report cards at https://www.nationsreportcard.gov/ give the headline numbers with significance marks."
67
+ }
68
+ },
69
+ "cache": {
70
+ "mode": "run_only",
71
+ "ttl_hours": null,
72
+ "why": "Small queries; results change only when a new assessment is released (about every two years), but data/ is reused only within a run."
73
+ },
74
+ "redistribution": {
75
+ "mode": "allowed",
76
+ "why": "NAEP results are published by NCES, a U.S. government agency; its materials are in the public domain (17 U.S.C. 105) unless marked otherwise, and NCES asks that the source be cited."
77
+ },
78
+ "notices": [],
79
+ "retention": null,
80
+ "citation": "U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), {dataset}, NAEP Data Service (nationsreportcard.gov), retrieved {retrieved_long}.",
81
+ "audit_hooks": [
82
+ {
83
+ "id": "naep_uncertainty",
84
+ "kind": "require_text",
85
+ "any": [
86
+ "standard error",
87
+ "significant"
88
+ ],
89
+ "why": "NAEP scores are sample estimates with standard errors; a report must say whether differences are statistically significant (NAEP's own tests), not compare point estimates."
90
+ }
91
+ ],
92
+ "clarify": [
93
+ "Which subject, grade(s) and year(s)? Default: reading and mathematics, grades 4 and 8, the latest year and 2019 (pre-pandemic).",
94
+ "Average scale score or achievement level (percent at or above Proficient)? They can move differently; default: report both.",
95
+ "Public schools only (NP, the comparison base for states) or all schools (NT)? Default NP when states are involved."
96
+ ],
97
+ "formats": [
98
+ "json"
99
+ ]
100
+ }
@@ -8,7 +8,7 @@ student group, with standard errors and NAEP's own significance tests.
8
8
  science, writing, civics, U.S. history, geography, economics, technology and engineering literacy,
9
9
  music. States versus the nation, change since 2019, gaps by race, sex, economic disadvantage.
10
10
  - Not here: state test results (each state's own exams, not comparable across states), graduation rates
11
- and school finance (nces or ed packs if present), college outcomes (college-scorecard).
11
+ and school finance (nces), college outcomes (college-scorecard).
12
12
  - NAEP is a sample survey: every number is an estimate. It reports groups, never students or schools.
13
13
 
14
14
  ## Endpoints and datasets
@@ -0,0 +1,101 @@
1
+ {
2
+ "base_url": "https://nces.ed.gov",
3
+ "timeout": 180,
4
+ "endpoints": {
5
+ "digest_table": {
6
+ "path": "/programs/digest/d{edition}/tables/xls/tabn{table}.xlsx",
7
+ "method": "GET",
8
+ "description": "One table of the Digest of Education Statistics as its Excel file: enrollment, teachers, finances, degrees, outcomes, from pre-K to postsecondary. Default: every line numbered (row, col1...) to find the header lines; then header_lines (usually '3-4') names the columns.",
9
+ "params": {
10
+ "edition": {
11
+ "in": "path",
12
+ "required": true,
13
+ "desc": "two-digit Digest edition, e.g. 25 (Digest 2025); not every table is in every edition"
14
+ },
15
+ "table": {
16
+ "in": "path",
17
+ "required": true,
18
+ "desc": "table number, e.g. 203.10 (public school enrollment by grade)"
19
+ },
20
+ "header_lines": {
21
+ "in": "records",
22
+ "desc": "the header line numbers, e.g. '3-4'"
23
+ },
24
+ "header_row": {
25
+ "in": "records",
26
+ "desc": "one header line instead; 0 (default) returns every line numbered"
27
+ },
28
+ "sheet": {
29
+ "in": "records",
30
+ "desc": "sheet name or number (default 1)"
31
+ },
32
+ "carry": {
33
+ "in": "records",
34
+ "desc": "which header lines span to the right over blank cells: top (default), all, none; check the names against the numbered lines"
35
+ }
36
+ },
37
+ "defaults": {
38
+ "header_row": 0
39
+ },
40
+ "records": "$xlsx",
41
+ "paging": {
42
+ "type": "none"
43
+ },
44
+ "max_rows": 20000
45
+ },
46
+ "ipeds": {
47
+ "path": "/ipeds/datacenter/data/{file}.zip",
48
+ "method": "GET",
49
+ "description": "An IPEDS complete data file (one row per institution, or institution by category), e.g. HD2023 directory, EFFY2023 12-month enrollment, C2023_A completions by CIP code, ADM2023 admissions, SFA2223 student aid, F2223_F2 finance. The revised (_rv) version is read when the zip has one.",
50
+ "params": {
51
+ "file": {
52
+ "in": "path",
53
+ "required": true,
54
+ "desc": "IPEDS file name as on the Data Center's Complete Data Files page, e.g. HD2023, EFFY2023, C2023_A"
55
+ },
56
+ "unitid": {
57
+ "in": "spec",
58
+ "desc": "keep only these institutions (UNITID, a list works)"
59
+ },
60
+ "state": {
61
+ "in": "spec",
62
+ "desc": "keep only these states (two-letter STABBR; files that have that column, e.g. HD)"
63
+ },
64
+ "member": {
65
+ "in": "records",
66
+ "desc": "the CSV in the zip, e.g. c2023_a.csv for the original instead of the revised file"
67
+ }
68
+ },
69
+ "records": "$zip:*.csv",
70
+ "prefer": "_rv[.]csv$",
71
+ "paging": {
72
+ "type": "none"
73
+ },
74
+ "max_rows": 1000000
75
+ },
76
+ "ipeds_dictionary": {
77
+ "path": "/ipeds/datacenter/data/{file}_Dict.zip",
78
+ "method": "GET",
79
+ "description": "The data dictionary of an IPEDS file: variable names and titles (sheet Varlist), code values and labels (Frequencies), long descriptions (Description).",
80
+ "params": {
81
+ "file": {
82
+ "in": "path",
83
+ "required": true,
84
+ "desc": "the IPEDS file name, e.g. HD2023"
85
+ },
86
+ "sheet": {
87
+ "in": "records",
88
+ "desc": "Varlist (default), Frequencies, Description, Introduction"
89
+ }
90
+ },
91
+ "defaults": {
92
+ "sheet": "Varlist"
93
+ },
94
+ "records": "$zip:*.xlsx",
95
+ "paging": {
96
+ "type": "none"
97
+ },
98
+ "max_rows": 100000
99
+ }
100
+ }
101
+ }
@@ -0,0 +1,99 @@
1
+ {
2
+ "id": "nces",
3
+ "title": "National Center for Education Statistics: Digest of Education Statistics tables and IPEDS data files",
4
+ "agency": "U.S. Department of Education, National Center for Education Statistics",
5
+ "depth": "standard",
6
+ "domains": [
7
+ "education",
8
+ "higher education"
9
+ ],
10
+ "keywords": [
11
+ "nces",
12
+ "national center for education statistics",
13
+ "digest of education statistics",
14
+ "school enrollment",
15
+ "public school enrollment",
16
+ "enrollment by grade",
17
+ "teachers",
18
+ "pupil teacher ratio",
19
+ "per-pupil expenditure",
20
+ "school finance",
21
+ "high school graduates",
22
+ "graduation rate",
23
+ "degrees conferred",
24
+ "ipeds",
25
+ "college enrollment",
26
+ "fall enrollment",
27
+ "completions",
28
+ "degrees by field",
29
+ "cip code",
30
+ "institutional characteristics",
31
+ "tuition and fees",
32
+ "admissions",
33
+ "student financial aid",
34
+ "college finance"
35
+ ],
36
+ "questions": [
37
+ "public school enrollment by grade over time",
38
+ "number of teachers and pupil/teacher ratio",
39
+ "current expenditure per pupil by state",
40
+ "bachelor's degrees conferred by field",
41
+ "enrollment at a college or all colleges in a state (IPEDS)",
42
+ "completions by CIP code and award level",
43
+ "admissions and yield at a university"
44
+ ],
45
+ "homepage": "https://nces.ed.gov/",
46
+ "terms_url": "https://nces.ed.gov/ipeds/datacenter/DataFiles.aspx",
47
+ "auth": {
48
+ "env": null,
49
+ "required": "none",
50
+ "placement": "none",
51
+ "signup_url": null,
52
+ "echoes_key": false
53
+ },
54
+ "limits": {
55
+ "rps": 2,
56
+ "burst": 4,
57
+ "max_rows": 1000000
58
+ },
59
+ "routes": {
60
+ "api": {
61
+ "base": "https://nces.ed.gov",
62
+ "probe": "https://nces.ed.gov/ipeds/datacenter/data/HD2023_Dict.zip",
63
+ "hosts": [
64
+ "nces.ed.gov"
65
+ ]
66
+ },
67
+ "manual": {
68
+ "how": "Digest: open the table page (https://nces.ed.gov/programs/digest/current_tables.asp, then the table) and click its Excel link, or https://nces.ed.gov/programs/digest/d<yy>/tables/xls/tabn<table>.xlsx. IPEDS: download https://nces.ed.gov/ipeds/datacenter/data/<FILE>.zip from the Complete Data Files page (https://nces.ed.gov/ipeds/datacenter/DataFiles.aspx). Upload the file as saved with ingest."
69
+ }
70
+ },
71
+ "cache": {
72
+ "mode": "run_only",
73
+ "ttl_hours": null,
74
+ "why": "Files are replaced when NCES revises them; data/ is reused within a run."
75
+ },
76
+ "redistribution": {
77
+ "mode": "allowed",
78
+ "why": "NCES publications and IPEDS data are U.S. government works in the public domain (17 U.S.C. 105)."
79
+ },
80
+ "notices": [],
81
+ "retention": null,
82
+ "citation": "U.S. Department of Education, National Center for Education Statistics, {dataset}, nces.ed.gov, retrieved {retrieved_long}.",
83
+ "audit_hooks": [],
84
+ "clarify": [
85
+ "Published table or institution file? Digest tables give national (and often state) totals as NCES published them; IPEDS files give one row per institution to filter and sum. Default: a Digest table for national or state totals, IPEDS for named colleges.",
86
+ "Which year? Digest tables are labelled by school year or fall; IPEDS files by collection year (EFFY2023 = 12-month enrollment for 2022-23). Say which, and whether the IPEDS file is provisional or revised (_rv)."
87
+ ],
88
+ "formats": [
89
+ "xlsx",
90
+ "zip",
91
+ "csv"
92
+ ],
93
+ "aliases": [
94
+ "NCES",
95
+ "IPEDS",
96
+ "Digest of Education Statistics",
97
+ "Integrated Postsecondary Education Data System"
98
+ ]
99
+ }
@@ -0,0 +1,75 @@
1
+ # National Center for Education Statistics: Digest of Education Statistics tables and IPEDS data files
2
+ Depth: standard. Key: none.
3
+ - The Digest of Education Statistics: NCES's compendium of published tables (enrollment, teachers, finances,
4
+ graduates, degrees, outcomes, pre-K to postsecondary), as each table's Excel file.
5
+ - IPEDS complete data files: one row per postsecondary institution, or institution by category, for every
6
+ survey component (directory, enrollment, completions, admissions, student aid, finance, staff), with their
7
+ data dictionaries.
8
+
9
+ ## When to use
10
+ - National or state totals as NCES published them: public school enrollment by grade, teachers and
11
+ pupil/teacher ratios, current expenditure per pupil, high school completers, degrees conferred by field and
12
+ level, enrollment trends.
13
+ - A named college or every college in a state: fall or 12-month enrollment, completions by CIP code and award
14
+ level, admissions and test scores, tuition, aid, finance (IPEDS).
15
+ - Not here: earnings and debt of graduates by program and institution (college-scorecard); federal student
16
+ loans and Pell totals (fsa); test scores of K-12 students (naep); school-age population, poverty or
17
+ attainment from surveys (census ACS).
18
+
19
+ ## Endpoints
20
+ | Endpoint | Returns | Parameters |
21
+ |---|---|---|
22
+ | `digest_table` | one Digest table: without `header_lines`, every line numbered (`row`, `col1`...); with it, named columns | `edition` (two digits, e.g. `25`), `table` (e.g. `203.10`), `header_lines` (usually `3-4`), `header_row`, `sheet` |
23
+ | `ipeds` | an IPEDS complete data file (the revised `_rv` CSV when the zip has one) | `file` (e.g. `HD2023`, `EFFY2023`, `C2023_A`), `unitid`, `state` (filters), `member` |
24
+ | `ipeds_dictionary` | that file's dictionary sheet | `file`, `sheet` (`Varlist` default, `Frequencies` for code labels, `Description`) |
25
+
26
+ ## Spec examples
27
+ ```json
28
+ {"description": "Public school enrollment by grade, fall 1980-2024 (Digest 2025 table 203.10)",
29
+ "requests": [{"name": "t203_10", "endpoint": "digest_table", "params": {"edition": "25", "table": "203.10", "header_lines": "3-4"}}]}
30
+ ```
31
+ ```json
32
+ {"description": "Colleges in Vermont (IPEDS directory 2023-24) and the code labels",
33
+ "requests": [{"name": "vt_colleges", "endpoint": "ipeds", "params": {"file": "HD2023", "state": "VT"}},
34
+ {"name": "hd_codes", "endpoint": "ipeds_dictionary", "params": {"file": "HD2023", "sheet": "Frequencies"}}]}
35
+ ```
36
+
37
+ ## Data format and traps
38
+ - Digest tables:
39
+ - Line 1 is the title, line 2 the units ("[In thousands]": multiply), lines 3-4 the stacked header, and the
40
+ first data line is the row of column numbers ("1", "2", ...): drop it. Notes and the source follow the last
41
+ data line. Verified on table 203.10, Digest 2025; other tables differ, so read the raw lines first.
42
+ - Columns named `<header>_colN` hold footnote marks for the value to their left, or nothing. In table
43
+ 203.10's 2024 row, `\3,4\` points to footnotes 3 and 4 below the table. Keep the marks with the value they
44
+ qualify.
45
+ - Values are text with thousands separators. Footnote marks also sit inside header cells (`Ungraded\1\`), and
46
+ some cells start with a line break ("\n1993"): trim them. Symbols such as `†` and `—`, used in many
47
+ tables, are defined in each table's notes; keep them and never treat them as 0. Projections are marked in
48
+ the notes.
49
+ - Each edition renumbers or drops tables, and a table that is not in that edition answers 404. Use the
50
+ current edition's table list (https://nces.ed.gov/programs/digest/current_tables.asp) to find the number.
51
+ - IPEDS:
52
+ - Names. Files are named by collection year: HD2023 is the 2023-24 directory, EFFY2023 the 12-month enrollment
53
+ of 2022-23, C2023_A the completions of July 2022-June 2023. Read the dictionary's Introduction sheet to be
54
+ sure.
55
+ - Revisions. When a zip has a revised file (`_rv`, the final release), it is read instead of the provisional
56
+ one; `member` picks the other.
57
+ - Codes. Columns are coded (`SECTOR`, `CONTROL`, `ICLEVEL`; `X`-prefixed imputation flags): label them from
58
+ the dictionary's Frequencies sheet. Negative codes such as -1 and -2 mean not reported or not applicable;
59
+ never sum them.
60
+ - Joining files. UNITID joins files; a parent/child system (`F1SYSTYP` ...) can report for several campuses.
61
+ - Size. Completions files are large (C2023_A has two 60 MB CSVs): filter by `unitid` or `state`. `state`
62
+ works only on files with `STABBR`, so join HD for state.
63
+ - Counting. Enrollment and completions rows are by category (all students, by race, by sex, by CIP and award
64
+ level), and totals are rows of their own (CIPCODE 99, MAJORNUM 1): pick the total row; never sum across
65
+ categories.
66
+
67
+ ## Access
68
+ No key. `fetch` reads nces.ed.gov directly (verified 2026-11-07). Manual: the Digest table's Excel link, or the
69
+ IPEDS Complete Data Files page (https://nces.ed.gov/ipeds/datacenter/DataFiles.aspx); upload the file as saved;
70
+ then `ingest`.
71
+
72
+ ## Citation and terms
73
+ "U.S. Department of Education, National Center for Education Statistics, Digest of Education Statistics 2025,
74
+ table 203.10" or "... Integrated Postsecondary Education Data System (IPEDS), <file> (<release>), nces.ed.gov,
75
+ retrieved <date>." Public domain (17 U.S.C. 105).