@cliwant/mcp-sam-gov 1.0.0 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.ja.md +3 -2
- package/README.ko.md +3 -2
- package/README.md +32 -3
- package/dist/census-economic.d.ts +93 -0
- package/dist/census-economic.d.ts.map +1 -0
- package/dist/census-economic.js +355 -0
- package/dist/census-economic.js.map +1 -0
- package/dist/fred.d.ts +108 -0
- package/dist/fred.d.ts.map +1 -0
- package/dist/fred.js +373 -0
- package/dist/fred.js.map +1 -0
- package/dist/keys.d.ts +83 -0
- package/dist/keys.d.ts.map +1 -0
- package/dist/keys.js +173 -0
- package/dist/keys.js.map +1 -0
- package/dist/server.d.ts.map +1 -1
- package/dist/server.js +143 -1
- package/dist/server.js.map +1 -1
- package/dist/snapshot.d.ts +33 -16
- package/dist/snapshot.d.ts.map +1 -1
- package/dist/snapshot.js +46 -17
- package/dist/snapshot.js.map +1 -1
- package/package.json +1 -1
- package/src/census-economic.ts +425 -0
- package/src/fred.ts +464 -0
- package/src/keys.ts +216 -0
- package/src/server.ts +167 -1
- package/src/snapshot.ts +51 -20
package/README.ja.md
CHANGED
|
@@ -188,8 +188,9 @@ npm install --omit=dev
|
|
|
188
188
|
本サーバーの原則は一つ: **もっともらしい捏造より誠実な失敗。** 以下はすべて公開データの*可用性*に関するものであり、いかなるアクセス制御も回避しません。
|
|
189
189
|
|
|
190
190
|
- **キーレス優先、ダウンしたソースは例外を*投げる*。** すべてのソースが API キーなしで動作します。ソースが rate-limit・ブロック・ダウンした場合、ツールは**型付きエラー**(`rate_limited` / `upstream_unavailable` / `schema_drift` …)を返し、行を捏造したりダウンしたサービスを「結果 0」/「見つからない」と報告しません。本物の空結果と障害は常に区別できます。
|
|
191
|
-
-
|
|
192
|
-
-
|
|
191
|
+
- **オフラインスナップショット (既定 on)。** ゆっくり変わる参照データ(toptier 機関一覧、上位 NAICS ツリー、USAspending 用語集、SBA 規模基準、最新 Treasury「Debt to the Penny」)は、ライブの連邦ソースが egress から一時的に到達不能なとき、サーバーが既定で `raw.githubusercontent.com/cliwant/mcp-sam-gov/snapshots` にホストされた**公開・週次更新スナップショット**へフォールバックします。ライブの**ハード障害**(障害 / IP 評判ブロック)時のみ取得し、通常運用中は決して取得しません — 公開データ、テレメトリなし。スナップショットが提供されるとき**決してライブとして表示しません** — 応答に `_meta.dataPath: "snapshot"` + `asOf` タイムスタンプが付き、`complete` は強制的に off。rate limit(429)は常に**尊重**し、ミラーへ回避しません。
|
|
192
|
+
- **無効化(純ライブ専用):** `SAMGOV_SNAPSHOT_BASE_URL=off` を設定するとスナップショット経路は追加されず、ライブ専用クライアントと byte-identical。
|
|
193
|
+
- **自前ミラーを指定:** `SAMGOV_SNAPSHOT_BASE_URL` を自分の base URL に設定すれば、公開既定値の代わりに自前ホスティング。
|
|
193
194
|
- **スナップショットのビルド:** ブロックされていないクリーンな egress(ノート PC / 自宅 / クリーンな CI)から `node scripts/build-snapshots.mjs` を実行。**ソース別到達性を自己診断**し、reachability 表 + `manifest.json` を出力します。部分カバレッジでは到達できるソースのみ更新し、残りは **last-good ファイルをそのまま残します**(古くても誠実、決して空にしない)。*すべての*ソースが到達不能なときのみ非ゼロ終了(egress 全面ブロックの合図 — よりクリーンな egress で再実行)。
|
|
194
195
|
- **誠実な境界。** これは**公開データの可用性のみ**を扱います。ビルダーは公開・再配布可能(public-domain / CC0)なデータのみ取り込み、リーダーは `accessLevel: "public"` でない封筒の提供を拒否します。**rate limit を尊重**(429 を回避しない)し、**プロキシ・IP ローテーション・認証/ペイウォール/CAPTCHA 回避なし**、off-host リダイレクトも拒否します。ブロックされた場合の誠実な解決策は、よりクリーンな egress からビルドすることであり、ブロックの回避ではありません。
|
|
195
196
|
|
package/README.ko.md
CHANGED
|
@@ -188,8 +188,9 @@ npm install --omit=dev
|
|
|
188
188
|
이 서버의 원칙은 하나입니다: **그럴듯한 조작보다 정직한 실패.** 아래는 모두 공개 데이터의 *가용성*에 관한 것이며, 어떤 접근 통제도 우회하지 않습니다.
|
|
189
189
|
|
|
190
190
|
- **Keyless 우선, 다운된 소스는 예외를 *던진다*.** 모든 소스가 API 키 없이 동작합니다. 소스가 rate-limit·차단·다운되면 도구는 **타입이 지정된 에러**(`rate_limited` / `upstream_unavailable` / `schema_drift` …)를 반환하며, 행을 지어내거나 다운된 서비스를 "결과 0" / "없음"으로 보고하지 않습니다. 진짜 빈 결과와 장애는 항상 구별됩니다.
|
|
191
|
-
-
|
|
192
|
-
-
|
|
191
|
+
- **오프라인 스냅샷 (기본 on).** 느리게 바뀌는 참조 데이터(toptier 기관 목록, 상위 NAICS 트리, USAspending 용어집, SBA 규모 기준, 최신 Treasury "Debt to the Penny")는, 라이브 연방 소스가 egress 에서 잠시 도달 불가일 때 서버가 기본적으로 `raw.githubusercontent.com/cliwant/mcp-sam-gov/snapshots` 에 호스팅된 **공개·주간 갱신 스냅샷**으로 폴백합니다. 라이브 **하드 실패**(장애 / IP 평판 차단) 시에만 가져오며 평상시엔 절대 아닙니다 — 공개 데이터, 텔레메트리 없음. 스냅샷이 서빙되면 **절대 라이브처럼 표시하지 않습니다** — 응답에 `_meta.dataPath: "snapshot"` + `asOf` 타임스탬프가 붙고 `complete` 는 강제로 꺼집니다. rate limit(429)은 항상 **준수**하며 미러로 우회하지 않습니다.
|
|
192
|
+
- **끄기(순수 라이브 전용):** `SAMGOV_SNAPSHOT_BASE_URL=off` 설정 시 스냅샷 경로가 추가되지 않고 라이브 전용 클라이언트와 byte-identical.
|
|
193
|
+
- **자체 미러 지정:** `SAMGOV_SNAPSHOT_BASE_URL` 을 자신의 base URL 로 설정하면 공개 기본값 대신 직접 호스팅.
|
|
193
194
|
- **스냅샷 빌드:** 차단되지 않은 깨끗한 egress(노트북 / 집 / 깨끗한 CI)에서 `node scripts/build-snapshots.mjs` 실행. **소스별 도달성을 자가 진단**하고 reachability 표 + `manifest.json` 을 출력합니다. 부분 커버리지에서는 도달 가능한 소스만 갱신하고 나머지는 **last-good 파일을 그대로 둡니다**(오래됐어도 정직, 절대 비우지 않음). *모든* 소스가 도달 불가일 때만 non-zero 종료(egress 전면 차단 신호 — 더 깨끗한 egress 에서 재실행).
|
|
194
195
|
- **정직한 경계.** 이것은 **공개-데이터 가용성만** 다룹니다. 빌더는 공개·재배포 가능(public-domain / CC0) 데이터만 수집하고, 리더는 `accessLevel: "public"` 이 아닌 봉투는 서빙을 거부합니다. **rate limit 을 준수**(429 를 우회하지 않음)하고 **프록시·IP 로테이션·인증/페이월/CAPTCHA 우회 없음**, off-host 리다이렉트도 거부합니다. 차단되면 정직한 해법은 더 깨끗한 egress 에서 빌드하는 것이지 차단을 회피하는 것이 아닙니다.
|
|
195
196
|
|
package/README.md
CHANGED
|
@@ -340,6 +340,34 @@ A handful of sources ride the shared **api.data.gov** gateway — Congress.gov,
|
|
|
340
340
|
|
|
341
341
|
Get one free (instant, no wait) at [api.data.gov/signup](https://api.data.gov/signup). The same key is accepted across all api.data.gov / api.gsa.gov sources. Like the SAM key, it is sent only on the wire (never logged); unset simply means `DEMO_KEY`. BLS sources similarly accept an optional free `BLS_API_KEY` to lift their daily quota.
|
|
342
342
|
|
|
343
|
+
### Keys & higher limits — the full inventory
|
|
344
|
+
|
|
345
|
+
**Most tools are keyless.** Only **Census** (`census_business_patterns`) and **FRED** (`fred_search_series`, `fred_series_observations`) *require* a key — those sources have no keyless tier, so the tool throws without one. The other five keys are *optional*: they only raise a rate limit or unlock a single filter. **Every key below is free.**
|
|
346
|
+
|
|
347
|
+
| Env var | Required? | What it unlocks | Free signup |
|
|
348
|
+
|---|---|---|---|
|
|
349
|
+
| `CENSUS_API_KEY` | **Required** | `census_business_patterns` (no keyless tier — throws without it) | [api.census.gov/data/key_signup.html](https://api.census.gov/data/key_signup.html) |
|
|
350
|
+
| `FRED_API_KEY` | **Required** | the 2 FRED tools (no keyless tier — throw without it) | [fred.stlouisfed.org/docs/api/api_key.html](https://fred.stlouisfed.org/docs/api/api_key.html) |
|
|
351
|
+
| `DATA_GOV_API_KEY` | Optional | higher limits on all api.data.gov sources (Regulations.gov, FAC, NPPES, CMS, data.gov catalog, GSA per-diem) — lifts the shared `DEMO_KEY` cap | [api.data.gov/signup](https://api.data.gov/signup/) |
|
|
352
|
+
| `SAM_GOV_API_KEY` | Optional | authenticated SAM.gov v2 search + the organization-name filter | [open.gsa.gov/api/get-opportunities-public-api](https://open.gsa.gov/api/get-opportunities-public-api/) |
|
|
353
|
+
| `BLS_API_KEY` | Optional | the BLS v2 tier (~500 queries/day vs keyless ~25/day) | [data.bls.gov/registrationEngine](https://data.bls.gov/registrationEngine/) |
|
|
354
|
+
| `NVD_API_KEY` | Optional | a higher NVD rate limit (`cve_lookup`) | [nvd.nist.gov/developers/request-an-api-key](https://nvd.nist.gov/developers/request-an-api-key) |
|
|
355
|
+
| `SOCRATA_APP_TOKEN` | Optional | higher Socrata throttling limits | [evergreen.data.socrata.com/signup](https://evergreen.data.socrata.com/signup) |
|
|
356
|
+
|
|
357
|
+
**Two ways to set any key** — pick one:
|
|
358
|
+
|
|
359
|
+
1. **Host env block** — the `"env": { … }` object shown in the examples above.
|
|
360
|
+
2. **A `.env` file** in the server's working directory — configure your keys **once**:
|
|
361
|
+
```
|
|
362
|
+
CENSUS_API_KEY=your-key-here
|
|
363
|
+
FRED_API_KEY=your-key-here
|
|
364
|
+
# optional — raise limits / unlock filters
|
|
365
|
+
SAM_GOV_API_KEY=your-key-here
|
|
366
|
+
```
|
|
367
|
+
The server auto-loads `.env` at startup. A real environment variable always wins over `.env` (standard precedence), and `.env` is git-ignored so your keys never get committed.
|
|
368
|
+
|
|
369
|
+
**Ask the server which keys it needs.** The keyless **`api_key_status`** tool lists every key, whether it's required or optional, the free signup URL + what it unlocks, and whether each is **currently configured** (a boolean — the key value is never shown). Creating the account at the signup URL is your one manual step; the server automates *discovery* (`api_key_status`) and *configuration* (`.env`). To confirm a key actually works, call that source's own tool.
|
|
370
|
+
|
|
343
371
|
---
|
|
344
372
|
|
|
345
373
|
## Tool catalog (111 tools)
|
|
@@ -525,13 +553,14 @@ This server is built around one rule: **honest failure over confident fabricatio
|
|
|
525
553
|
|
|
526
554
|
**Keyless-first, and a down source *throws*.** Every source works with no API key. When a source is rate-limited, blocked, or down, the tool returns a **typed error** (`rate_limited` / `upstream_unavailable` / `schema_drift` / …) — it never invents rows and never reports a DOWN service as "0 results" or "not found". A genuine empty result and an outage are always distinguishable.
|
|
527
555
|
|
|
528
|
-
**
|
|
556
|
+
**Offline snapshots (on by default).** Some reference data changes slowly — the toptier-agency list, the top-level NAICS tree, the USAspending glossary, SBA size standards, the latest Treasury "Debt to the Penny." By default, when a live federal source is briefly unreachable from your egress, the server falls back to a **public, weekly-refreshed snapshot** of that slow-changing reference data, hosted at `raw.githubusercontent.com/cliwant/mcp-sam-gov/snapshots`. It only fetches on a **live hard-failure** (an outage / IP-reputation block), never during normal operation — public data, no telemetry. A served snapshot is **never presented as live** — the response carries `_meta.dataPath: "snapshot"` plus an `asOf` timestamp, and `complete` is forced off, so an AI agent (and you) always see the staleness. A rate limit (429) is always **honored**, never routed around onto the mirror.
|
|
529
557
|
|
|
530
|
-
- **
|
|
558
|
+
- **Disable it (pure live-only):** set `SAMGOV_SNAPSHOT_BASE_URL=off`. Then no snapshot path is ever added and behavior is byte-for-byte identical to a live-only client.
|
|
559
|
+
- **Point at your own mirror:** set `SAMGOV_SNAPSHOT_BASE_URL` to your base URL to host the snapshots yourself instead of using the public default.
|
|
531
560
|
|
|
532
561
|
```json
|
|
533
562
|
{ "mcpServers": { "sam-gov": { "command": "mcp-sam-gov",
|
|
534
|
-
"env": { "SAMGOV_SNAPSHOT_BASE_URL": "
|
|
563
|
+
"env": { "SAMGOV_SNAPSHOT_BASE_URL": "off" } } } }
|
|
535
564
|
```
|
|
536
565
|
|
|
537
566
|
- **Build the snapshots:** run `node scripts/build-snapshots.mjs` from any clean, non-blocked egress (a laptop / home / clean CI runner). It **self-diagnoses per-source reachability**, prints a reachability table, and writes a `manifest.json`. On partial coverage it refreshes only the sources it can reach and **leaves the last-good file in place** for the rest (stale-but-honest, never blanked). It exits non-zero only when *zero* sources were reachable (a fully blocked egress — the signal to re-run from a cleaner one).
|
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* US Census — County Business Patterns (CBP) — the MARKET-SIZING lane
|
|
3
|
+
* (ADR-0047, Wave-4 source #1). NAICS × geography establishments / employment /
|
|
4
|
+
* annual payroll — the demand-side complement to BLS-QCEW + USAspending in the
|
|
5
|
+
* B2G market-sizing set.
|
|
6
|
+
*
|
|
7
|
+
* ★ THIS IS THE SERVER'S FIRST KEY-REQUIRED SOURCE. The Census Data API removed
|
|
8
|
+
* its keyless tier — a request WITHOUT a key is 302-redirected to a "Missing
|
|
9
|
+
* Key" HTML page. So, honestly: with NO `CENSUS_API_KEY` this tool THROWS an
|
|
10
|
+
* `invalid_input` config error BEFORE any fetch (never a fake-empty, never a
|
|
11
|
+
* keyless-pretend). The other 111 tools stay keyless — this key is scoped to
|
|
12
|
+
* this one source. (Contrast the OPTIONAL keys of datagov/bls/nvd, which lift a
|
|
13
|
+
* tier but are not required.)
|
|
14
|
+
*
|
|
15
|
+
* DIFFERENT HOST than census.ts (the geocoder, geocoding.geo.census.gov): the
|
|
16
|
+
* DATA API is `api.census.gov/data/{year}/cbp`. This module COPIES (does NOT
|
|
17
|
+
* import) the census.ts fixed-host SSRF idiom (a single host const + a
|
|
18
|
+
* post-construction `new URL().hostname` assertion + `redirect:"error"`) and does
|
|
19
|
+
* NOT touch census_geocode.
|
|
20
|
+
*
|
|
21
|
+
* Coercion/meta code is REUSED (`driftError`, `errorFromResponse`, `num`
|
|
22
|
+
* coerce.ts null-never-0, `withMeta`/`buildMeta`). The ONE bespoke bit is the
|
|
23
|
+
* fetch: a single `fetch(redirect:"manual")` (NOT the shared getJson) so a
|
|
24
|
+
* missing/invalid-key 302 surfaces as an INSPECTABLE opaque-redirect → honest
|
|
25
|
+
* invalid_input, instead of undici's redirect:"error" TypeError that
|
|
26
|
+
* fetchWithRetry would mask as a retryable outage (see the fetch block). The
|
|
27
|
+
* optional-key leak discipline is MIRRORED from datagovKey.ts/bls.ts — but here
|
|
28
|
+
* the key is REQUIRED and rides ONLY in the `&key=` query param, NOWHERE else
|
|
29
|
+
* (never the label, `_meta.source`, notes, or a log — the K-test).
|
|
30
|
+
*
|
|
31
|
+
* GET https://api.census.gov/data/{year}/cbp
|
|
32
|
+
* ?get=NAME,NAICS2017_LABEL,ESTAB,EMP,PAYANN,GEO_ID
|
|
33
|
+
* &for=<geoClause> (us:* | state:* | state:NN | county:*)
|
|
34
|
+
* &in=state:NN (county queries only)
|
|
35
|
+
* &NAICS2017=<naics> (optional NAICS filter)
|
|
36
|
+
* &key=<CENSUS_API_KEY> (REQUIRED)
|
|
37
|
+
* → a 2D JSON ARRAY: row 0 = column headers, rows 1..N = data. e.g.
|
|
38
|
+
* [["NAME","ESTAB","EMP","PAYANN","NAICS2017","NAICS2017_LABEL","state"],
|
|
39
|
+
* ["California","39755","...","...","5415","...","06"], ...]
|
|
40
|
+
*
|
|
41
|
+
* ★ HONESTY (ADR-0047 P1–P5):
|
|
42
|
+
* [KEY] no key ⇒ invalid_input THROW pre-fetch (0 fetch); the message names
|
|
43
|
+
* CENSUS_API_KEY + the free-signup URL. A wire 302 (a key that IS set but is
|
|
44
|
+
* invalid → the Missing-Key page) is caught via redirect:"manual" as an
|
|
45
|
+
* opaque-redirect ⇒ invalid_input "check CENSUS_API_KEY" (never a
|
|
46
|
+
* fake-empty, never a masked outage).
|
|
47
|
+
* [P1] CBP returns the COMPLETE geography set for the filter (no server
|
|
48
|
+
* pagination) ⇒ totalAvailable = the row count, complete:true. NEVER
|
|
49
|
+
* fabricated (RED if totalAvailable = header-length or invented).
|
|
50
|
+
* [P3] ★the sentinel→null crux: Census suppresses/withholds cells with large
|
|
51
|
+
* NEGATIVE sentinels (-999999999 / -888888888 / -666666666 …). `censusNum`
|
|
52
|
+
* maps any value ≤ -100000000 to **null** (withheld) — NEVER a negative
|
|
53
|
+
* number, NEVER 0. A genuine 0 stays 0. `annualPayrollUsd = PAYANN×1000`
|
|
54
|
+
* (PAYANN is in $1,000 units), null-preserving.
|
|
55
|
+
* [P2] a 302 ⇒ invalid_input (key); a header-only body ⇒ honest empty
|
|
56
|
+
* (returned:0, complete:true); a 5xx ⇒ upstream_unavailable THROW; a 200
|
|
57
|
+
* non-JSON ⇒ schema_drift.
|
|
58
|
+
* [P4] a body that is not an array, or whose row 0 is not a string[] header
|
|
59
|
+
* row ⇒ driftError (never a fabricated empty).
|
|
60
|
+
* [SSRF] fixed host; `year` re-guarded ^\d{4}$ (it rides in the PATH); `naics`
|
|
61
|
+
* ^\d{2,6}$; `state` ^\d{2}$; geography enum {us,state,county}. All
|
|
62
|
+
* predicate VALUES ride in URLSearchParams. The key rides `&key=` ONLY.
|
|
63
|
+
*/
|
|
64
|
+
import { num } from "./coerce.js";
|
|
65
|
+
import { type MetaBundle } from "./meta.js";
|
|
66
|
+
export { num };
|
|
67
|
+
/** Read CENSUS_API_KEY from env; trim; return the value or undefined (unset/blank). */
|
|
68
|
+
export declare function censusApiKey(): string | undefined;
|
|
69
|
+
export type CbpRow = {
|
|
70
|
+
name: string | null;
|
|
71
|
+
geoId: string | null;
|
|
72
|
+
naicsCode: string | null;
|
|
73
|
+
naicsLabel: string | null;
|
|
74
|
+
establishments: number | null;
|
|
75
|
+
employees: number | null;
|
|
76
|
+
annualPayrollUsd: number | null;
|
|
77
|
+
state: string | null;
|
|
78
|
+
};
|
|
79
|
+
export type CensusBusinessPatternsArgs = {
|
|
80
|
+
naics?: string;
|
|
81
|
+
geography?: string;
|
|
82
|
+
state?: string;
|
|
83
|
+
year?: string;
|
|
84
|
+
limit?: number;
|
|
85
|
+
};
|
|
86
|
+
/**
|
|
87
|
+
* Fetch County Business Patterns rows for a NAICS × geography filter → normalized
|
|
88
|
+
* establishment / employment / annual-payroll rows + honest `_meta`. REQUIRES
|
|
89
|
+
* CENSUS_API_KEY (throws invalid_input pre-fetch when unset). The 2D-array body is
|
|
90
|
+
* parsed by HEADER NAME (order-independent); suppressed cells map to null.
|
|
91
|
+
*/
|
|
92
|
+
export declare function businessPatterns(args: CensusBusinessPatternsArgs): Promise<MetaBundle>;
|
|
93
|
+
//# sourceMappingURL=census-economic.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"census-economic.d.ts","sourceRoot":"","sources":["../src/census-economic.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA8DG;AAIH,OAAO,EAAE,GAAG,EAAO,MAAM,aAAa,CAAC;AACvC,OAAO,EAAY,KAAK,UAAU,EAAqB,MAAM,WAAW,CAAC;AAKzE,OAAO,EAAE,GAAG,EAAE,CAAC;AA+Bf,uFAAuF;AACvF,wBAAgB,YAAY,IAAI,MAAM,GAAG,SAAS,CAIjD;AAGD,MAAM,MAAM,MAAM,GAAG;IACnB,IAAI,EAAE,MAAM,GAAG,IAAI,CAAC;IACpB,KAAK,EAAE,MAAM,GAAG,IAAI,CAAC;IACrB,SAAS,EAAE,MAAM,GAAG,IAAI,CAAC;IACzB,UAAU,EAAE,MAAM,GAAG,IAAI,CAAC;IAC1B,cAAc,EAAE,MAAM,GAAG,IAAI,CAAC;IAC9B,SAAS,EAAE,MAAM,GAAG,IAAI,CAAC;IACzB,gBAAgB,EAAE,MAAM,GAAG,IAAI,CAAC;IAChC,KAAK,EAAE,MAAM,GAAG,IAAI,CAAC;CACtB,CAAC;AAiBF,MAAM,MAAM,0BAA0B,GAAG;IACvC,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,KAAK,CAAC,EAAE,MAAM,CAAC;CAChB,CAAC;AAEF;;;;;GAKG;AACH,wBAAsB,gBAAgB,CACpC,IAAI,EAAE,0BAA0B,GAC/B,OAAO,CAAC,UAAU,CAAC,CA+QrB"}
|
|
@@ -0,0 +1,355 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* US Census — County Business Patterns (CBP) — the MARKET-SIZING lane
|
|
3
|
+
* (ADR-0047, Wave-4 source #1). NAICS × geography establishments / employment /
|
|
4
|
+
* annual payroll — the demand-side complement to BLS-QCEW + USAspending in the
|
|
5
|
+
* B2G market-sizing set.
|
|
6
|
+
*
|
|
7
|
+
* ★ THIS IS THE SERVER'S FIRST KEY-REQUIRED SOURCE. The Census Data API removed
|
|
8
|
+
* its keyless tier — a request WITHOUT a key is 302-redirected to a "Missing
|
|
9
|
+
* Key" HTML page. So, honestly: with NO `CENSUS_API_KEY` this tool THROWS an
|
|
10
|
+
* `invalid_input` config error BEFORE any fetch (never a fake-empty, never a
|
|
11
|
+
* keyless-pretend). The other 111 tools stay keyless — this key is scoped to
|
|
12
|
+
* this one source. (Contrast the OPTIONAL keys of datagov/bls/nvd, which lift a
|
|
13
|
+
* tier but are not required.)
|
|
14
|
+
*
|
|
15
|
+
* DIFFERENT HOST than census.ts (the geocoder, geocoding.geo.census.gov): the
|
|
16
|
+
* DATA API is `api.census.gov/data/{year}/cbp`. This module COPIES (does NOT
|
|
17
|
+
* import) the census.ts fixed-host SSRF idiom (a single host const + a
|
|
18
|
+
* post-construction `new URL().hostname` assertion + `redirect:"error"`) and does
|
|
19
|
+
* NOT touch census_geocode.
|
|
20
|
+
*
|
|
21
|
+
* Coercion/meta code is REUSED (`driftError`, `errorFromResponse`, `num`
|
|
22
|
+
* coerce.ts null-never-0, `withMeta`/`buildMeta`). The ONE bespoke bit is the
|
|
23
|
+
* fetch: a single `fetch(redirect:"manual")` (NOT the shared getJson) so a
|
|
24
|
+
* missing/invalid-key 302 surfaces as an INSPECTABLE opaque-redirect → honest
|
|
25
|
+
* invalid_input, instead of undici's redirect:"error" TypeError that
|
|
26
|
+
* fetchWithRetry would mask as a retryable outage (see the fetch block). The
|
|
27
|
+
* optional-key leak discipline is MIRRORED from datagovKey.ts/bls.ts — but here
|
|
28
|
+
* the key is REQUIRED and rides ONLY in the `&key=` query param, NOWHERE else
|
|
29
|
+
* (never the label, `_meta.source`, notes, or a log — the K-test).
|
|
30
|
+
*
|
|
31
|
+
* GET https://api.census.gov/data/{year}/cbp
|
|
32
|
+
* ?get=NAME,NAICS2017_LABEL,ESTAB,EMP,PAYANN,GEO_ID
|
|
33
|
+
* &for=<geoClause> (us:* | state:* | state:NN | county:*)
|
|
34
|
+
* &in=state:NN (county queries only)
|
|
35
|
+
* &NAICS2017=<naics> (optional NAICS filter)
|
|
36
|
+
* &key=<CENSUS_API_KEY> (REQUIRED)
|
|
37
|
+
* → a 2D JSON ARRAY: row 0 = column headers, rows 1..N = data. e.g.
|
|
38
|
+
* [["NAME","ESTAB","EMP","PAYANN","NAICS2017","NAICS2017_LABEL","state"],
|
|
39
|
+
* ["California","39755","...","...","5415","...","06"], ...]
|
|
40
|
+
*
|
|
41
|
+
* ★ HONESTY (ADR-0047 P1–P5):
|
|
42
|
+
* [KEY] no key ⇒ invalid_input THROW pre-fetch (0 fetch); the message names
|
|
43
|
+
* CENSUS_API_KEY + the free-signup URL. A wire 302 (a key that IS set but is
|
|
44
|
+
* invalid → the Missing-Key page) is caught via redirect:"manual" as an
|
|
45
|
+
* opaque-redirect ⇒ invalid_input "check CENSUS_API_KEY" (never a
|
|
46
|
+
* fake-empty, never a masked outage).
|
|
47
|
+
* [P1] CBP returns the COMPLETE geography set for the filter (no server
|
|
48
|
+
* pagination) ⇒ totalAvailable = the row count, complete:true. NEVER
|
|
49
|
+
* fabricated (RED if totalAvailable = header-length or invented).
|
|
50
|
+
* [P3] ★the sentinel→null crux: Census suppresses/withholds cells with large
|
|
51
|
+
* NEGATIVE sentinels (-999999999 / -888888888 / -666666666 …). `censusNum`
|
|
52
|
+
* maps any value ≤ -100000000 to **null** (withheld) — NEVER a negative
|
|
53
|
+
* number, NEVER 0. A genuine 0 stays 0. `annualPayrollUsd = PAYANN×1000`
|
|
54
|
+
* (PAYANN is in $1,000 units), null-preserving.
|
|
55
|
+
* [P2] a 302 ⇒ invalid_input (key); a header-only body ⇒ honest empty
|
|
56
|
+
* (returned:0, complete:true); a 5xx ⇒ upstream_unavailable THROW; a 200
|
|
57
|
+
* non-JSON ⇒ schema_drift.
|
|
58
|
+
* [P4] a body that is not an array, or whose row 0 is not a string[] header
|
|
59
|
+
* row ⇒ driftError (never a fabricated empty).
|
|
60
|
+
* [SSRF] fixed host; `year` re-guarded ^\d{4}$ (it rides in the PATH); `naics`
|
|
61
|
+
* ^\d{2,6}$; `state` ^\d{2}$; geography enum {us,state,county}. All
|
|
62
|
+
* predicate VALUES ride in URLSearchParams. The key rides `&key=` ONLY.
|
|
63
|
+
*/
|
|
64
|
+
import { ToolErrorCarrier, errorFromResponse } from "./errors.js";
|
|
65
|
+
import { driftError } from "./datasource.js";
|
|
66
|
+
import { num, str } from "./coerce.js";
|
|
67
|
+
import { withMeta } from "./meta.js";
|
|
68
|
+
// Re-export the shared honesty coercion (single audited copy in ./coerce.js —
|
|
69
|
+
// ADR-0005 v2 FIX-C) so a num regression fails together across sources. NO local
|
|
70
|
+
// num/str.
|
|
71
|
+
export { num };
|
|
72
|
+
// ─── SSRF core: the single fixed host (DIFFERENT from census.ts) ──
|
|
73
|
+
const CENSUS_DATA_HOST = "api.census.gov";
|
|
74
|
+
const CENSUS_DATA_LABEL = "census:/data/cbp"; // host-only ToolError surface; NO token, NO key
|
|
75
|
+
// ─── Validation charclasses (SSRF + "verify the input" honesty) ───
|
|
76
|
+
const YEAR_RE = /^\d{4}$/; // rides in the PATH — strict 4-digit (no path injection)
|
|
77
|
+
const NAICS_RE = /^\d{2,6}$/; // 2–6 digit NAICS-2017 sector/code
|
|
78
|
+
const STATE_FIPS_RE = /^\d{2}$/; // 2-digit state FIPS
|
|
79
|
+
const GEOGRAPHIES = new Set(["us", "state", "county"]);
|
|
80
|
+
// The Census suppression/withhold sentinel floor. Census encodes a suppressed or
|
|
81
|
+
// unavailable cell as a large NEGATIVE value (-999999999 / -888888888 /
|
|
82
|
+
// -666666666 …). Establishment/employment/payroll counts are non-negative, so any
|
|
83
|
+
// value at/below this floor is a sentinel, NOT data.
|
|
84
|
+
const CENSUS_SENTINEL_FLOOR = -100000000;
|
|
85
|
+
const DEFAULT_YEAR = "2022"; // the latest confirmed CBP vintage (ADR-0047)
|
|
86
|
+
// ─── Honesty notes (ADR-0047 required set) ────────────────────────
|
|
87
|
+
const KEY_REQUIRED_NOTE = "This source REQUIRES a free CENSUS_API_KEY (the Census Data API has no keyless tier). The key is sent ONLY as the &key= query parameter to api.census.gov and is NEVER logged, echoed, or placed in this response.";
|
|
88
|
+
const PAYROLL_UNITS_NOTE = "annualPayrollUsd is ANNUAL payroll in US dollars, converted from the Census PAYANN field's $1,000 units (×1000). establishments and employees are integer counts (as-of the reference year).";
|
|
89
|
+
const SUPPRESSED_NOTE = "Census suppresses cells for confidentiality/reliability using large negative sentinels (e.g. -999999999); such values are mapped to null (withheld) — NEVER a negative number and NEVER 0. A genuine 0 is preserved as 0.";
|
|
90
|
+
const NO_PAGINATION_NOTE = "CBP returns the COMPLETE set of geographies matching the filter (no server-side pagination); totalAvailable equals the number of rows returned. Narrow with naics / geography to reduce the row count.";
|
|
91
|
+
// ─── The key seam (REQUIRED; value NEVER leaked past the &key= param) ──
|
|
92
|
+
/** Read CENSUS_API_KEY from env; trim; return the value or undefined (unset/blank). */
|
|
93
|
+
export function censusApiKey() {
|
|
94
|
+
const raw = process.env.CENSUS_API_KEY;
|
|
95
|
+
const trimmed = typeof raw === "string" ? raw.trim() : "";
|
|
96
|
+
return trimmed ? trimmed : undefined;
|
|
97
|
+
}
|
|
98
|
+
/** num(), but map the Census negative suppression sentinel family → null (withheld). */
|
|
99
|
+
function censusNum(v) {
|
|
100
|
+
const n = num(v);
|
|
101
|
+
if (n === null)
|
|
102
|
+
return null;
|
|
103
|
+
// A large-negative sentinel is a WITHHELD/suppressed cell, never data. A genuine
|
|
104
|
+
// 0 (n > floor) passes through as 0.
|
|
105
|
+
if (n <= CENSUS_SENTINEL_FLOOR)
|
|
106
|
+
return null;
|
|
107
|
+
return n;
|
|
108
|
+
}
|
|
109
|
+
/** Annual payroll: PAYANN is in $1,000 units → ×1000; null-preserving. */
|
|
110
|
+
function mul1000(n) {
|
|
111
|
+
return n === null ? null : n * 1000;
|
|
112
|
+
}
|
|
113
|
+
/**
|
|
114
|
+
* Fetch County Business Patterns rows for a NAICS × geography filter → normalized
|
|
115
|
+
* establishment / employment / annual-payroll rows + honest `_meta`. REQUIRES
|
|
116
|
+
* CENSUS_API_KEY (throws invalid_input pre-fetch when unset). The 2D-array body is
|
|
117
|
+
* parsed by HEADER NAME (order-independent); suppressed cells map to null.
|
|
118
|
+
*/
|
|
119
|
+
export async function businessPatterns(args) {
|
|
120
|
+
// ── [KEY] REQUIRED key — throw an honest config error BEFORE any fetch. ──
|
|
121
|
+
const key = censusApiKey();
|
|
122
|
+
if (key === undefined) {
|
|
123
|
+
throw new ToolErrorCarrier({
|
|
124
|
+
kind: "invalid_input",
|
|
125
|
+
retryable: false,
|
|
126
|
+
message: "Census Data API requires a free key. Get one at https://api.census.gov/data/key_signup.html and set CENSUS_API_KEY.",
|
|
127
|
+
upstreamEndpoint: CENSUS_DATA_LABEL,
|
|
128
|
+
});
|
|
129
|
+
}
|
|
130
|
+
// ── Validate + default the inputs (belt-and-suspenders behind the server Zod;
|
|
131
|
+
// a DIRECT handler call bypasses Zod). ──
|
|
132
|
+
const year = args.year ?? DEFAULT_YEAR;
|
|
133
|
+
if (!YEAR_RE.test(year)) {
|
|
134
|
+
throw new ToolErrorCarrier({
|
|
135
|
+
kind: "invalid_input",
|
|
136
|
+
retryable: false,
|
|
137
|
+
message: `Invalid year ${JSON.stringify(year)} — expected a 4-digit year (^\\d{4}$), e.g. "2022". (year rides in the request PATH; it is strictly validated.)`,
|
|
138
|
+
upstreamEndpoint: CENSUS_DATA_LABEL,
|
|
139
|
+
});
|
|
140
|
+
}
|
|
141
|
+
const geography = args.geography ?? "us";
|
|
142
|
+
if (!GEOGRAPHIES.has(geography)) {
|
|
143
|
+
throw new ToolErrorCarrier({
|
|
144
|
+
kind: "invalid_input",
|
|
145
|
+
retryable: false,
|
|
146
|
+
message: `Invalid geography ${JSON.stringify(geography)} — expected one of us, state, county.`,
|
|
147
|
+
upstreamEndpoint: CENSUS_DATA_LABEL,
|
|
148
|
+
});
|
|
149
|
+
}
|
|
150
|
+
if (args.state !== undefined && !STATE_FIPS_RE.test(args.state)) {
|
|
151
|
+
throw new ToolErrorCarrier({
|
|
152
|
+
kind: "invalid_input",
|
|
153
|
+
retryable: false,
|
|
154
|
+
message: `Invalid state ${JSON.stringify(args.state)} — expected a 2-digit state FIPS code (^\\d{2}$), e.g. "06" (California).`,
|
|
155
|
+
upstreamEndpoint: CENSUS_DATA_LABEL,
|
|
156
|
+
});
|
|
157
|
+
}
|
|
158
|
+
if (args.naics !== undefined && !NAICS_RE.test(args.naics)) {
|
|
159
|
+
throw new ToolErrorCarrier({
|
|
160
|
+
kind: "invalid_input",
|
|
161
|
+
retryable: false,
|
|
162
|
+
message: `Invalid naics ${JSON.stringify(args.naics)} — expected a 2–6 digit NAICS-2017 code (^\\d{2,6}$), e.g. "5415" (Computer Systems Design).`,
|
|
163
|
+
upstreamEndpoint: CENSUS_DATA_LABEL,
|
|
164
|
+
});
|
|
165
|
+
}
|
|
166
|
+
// ── Resolve the geography clause (for=… [+ in=state:…]). ──
|
|
167
|
+
let forClause;
|
|
168
|
+
let inClause;
|
|
169
|
+
let geoFilter;
|
|
170
|
+
if (geography === "us") {
|
|
171
|
+
forClause = "us:*";
|
|
172
|
+
geoFilter = "geography:us";
|
|
173
|
+
}
|
|
174
|
+
else if (geography === "state") {
|
|
175
|
+
forClause = args.state !== undefined ? `state:${args.state}` : "state:*";
|
|
176
|
+
geoFilter = `geography:state:${args.state ?? "*"}`;
|
|
177
|
+
}
|
|
178
|
+
else {
|
|
179
|
+
// county — requires a state (the CBP `in=state:` predicate is mandatory).
|
|
180
|
+
if (args.state === undefined) {
|
|
181
|
+
throw new ToolErrorCarrier({
|
|
182
|
+
kind: "invalid_input",
|
|
183
|
+
retryable: false,
|
|
184
|
+
message: "geography 'county' requires `state` (a 2-digit FIPS) — CBP county queries need an `in=state:NN` predicate. Pass state, e.g. state:'06'.",
|
|
185
|
+
upstreamEndpoint: CENSUS_DATA_LABEL,
|
|
186
|
+
});
|
|
187
|
+
}
|
|
188
|
+
forClause = "county:*";
|
|
189
|
+
inClause = `state:${args.state}`;
|
|
190
|
+
geoFilter = `geography:county:* in state:${args.state}`;
|
|
191
|
+
}
|
|
192
|
+
// ── Build the query (all VALUES via URLSearchParams — no host/path steer; the
|
|
193
|
+
// REQUIRED key rides ONLY here in &key=). ──
|
|
194
|
+
const params = new URLSearchParams();
|
|
195
|
+
params.set("get", "NAME,NAICS2017_LABEL,ESTAB,EMP,PAYANN,GEO_ID");
|
|
196
|
+
params.set("for", forClause);
|
|
197
|
+
if (inClause !== undefined)
|
|
198
|
+
params.set("in", inClause);
|
|
199
|
+
if (args.naics !== undefined)
|
|
200
|
+
params.set("NAICS2017", args.naics);
|
|
201
|
+
params.set("key", key);
|
|
202
|
+
const url = `https://${CENSUS_DATA_HOST}/data/${year}/cbp?${params.toString()}`;
|
|
203
|
+
// Belt-and-suspenders: the fixed host + strictly-validated path leave nothing to
|
|
204
|
+
// steer the authority; assert the built URL cannot have been moved off-host.
|
|
205
|
+
const built = new URL(url);
|
|
206
|
+
if (built.hostname !== CENSUS_DATA_HOST || built.protocol !== "https:") {
|
|
207
|
+
throw new ToolErrorCarrier({
|
|
208
|
+
kind: "invalid_input",
|
|
209
|
+
retryable: false,
|
|
210
|
+
message: `Constructed Census Data URL host ${JSON.stringify(built.hostname)} (${built.protocol}) is not ${CENSUS_DATA_HOST} over https — refusing to fetch (SSRF safety).`,
|
|
211
|
+
upstreamEndpoint: CENSUS_DATA_LABEL,
|
|
212
|
+
});
|
|
213
|
+
}
|
|
214
|
+
// ── Fetch with redirect:"manual" — the KEY-ERROR DETECTION crux. A missing or
|
|
215
|
+
// invalid key makes the Census Data API 302-redirect to its "Missing Key" page
|
|
216
|
+
// (live-verified: `HTTP 302` + `Location: /data/missing_key.html` +
|
|
217
|
+
// `X-DataWebAPI-KeyError: 1`). We must NOT use redirect:"error" via getJson
|
|
218
|
+
// here: undici rejects an "error"-mode redirect with a TypeError that the
|
|
219
|
+
// shared fetchWithRetry catch reclassifies to a *retryable upstream_unavailable*
|
|
220
|
+
// — masking a key-config error as a transient outage. redirect:"manual" instead
|
|
221
|
+
// yields an INSPECTABLE opaque-redirect (type "opaqueredirect", status 0), so
|
|
222
|
+
// the missing/invalid key surfaces as an honest invalid_input. This is a
|
|
223
|
+
// SINGLE classified attempt (no auto-retry): a transient 5xx THROWS
|
|
224
|
+
// upstream_unavailable — re-invoke the tool to retry. The 5xx/404/400 taxonomy
|
|
225
|
+
// is delegated to the shared errors.ts `errorFromResponse`; a 200 non-JSON body
|
|
226
|
+
// ⇒ r.json() SyntaxError ⇒ schema_drift. The redirect is NEVER followed, so no
|
|
227
|
+
// off-host hop can occur (a stronger SSRF posture than redirect:"error"). ──
|
|
228
|
+
let res;
|
|
229
|
+
try {
|
|
230
|
+
res = await fetch(built.toString(), {
|
|
231
|
+
redirect: "manual",
|
|
232
|
+
signal: AbortSignal.timeout(15_000),
|
|
233
|
+
});
|
|
234
|
+
}
|
|
235
|
+
catch (e) {
|
|
236
|
+
// Timeout/abort ⇒ non-retryable (the same aborted signal would re-reject);
|
|
237
|
+
// a genuine network TypeError ⇒ retryable upstream_unavailable. NEVER empty.
|
|
238
|
+
if (e instanceof Error &&
|
|
239
|
+
(e.name === "TimeoutError" || e.name === "AbortError")) {
|
|
240
|
+
throw new ToolErrorCarrier({
|
|
241
|
+
kind: "upstream_unavailable",
|
|
242
|
+
message: `Request to ${CENSUS_DATA_LABEL} timed out.`,
|
|
243
|
+
retryable: false,
|
|
244
|
+
upstreamEndpoint: CENSUS_DATA_LABEL,
|
|
245
|
+
});
|
|
246
|
+
}
|
|
247
|
+
throw new ToolErrorCarrier({
|
|
248
|
+
kind: "upstream_unavailable",
|
|
249
|
+
message: `Network error reaching ${CENSUS_DATA_LABEL}: ${e instanceof Error ? e.message : String(e)}`,
|
|
250
|
+
retryable: true,
|
|
251
|
+
retryAfterSeconds: 30,
|
|
252
|
+
upstreamEndpoint: CENSUS_DATA_LABEL,
|
|
253
|
+
});
|
|
254
|
+
}
|
|
255
|
+
// [KEY] A redirect (opaque-redirect via redirect:"manual", or a raw 3xx status) ⇒
|
|
256
|
+
// the missing/invalid-key "Missing Key" page ⇒ an honest key-config error, NEVER
|
|
257
|
+
// a fake-empty (swallowing this as empty ⇒ RED in the fault suite).
|
|
258
|
+
if (res.type === "opaqueredirect" || (res.status >= 300 && res.status < 400)) {
|
|
259
|
+
throw new ToolErrorCarrier({
|
|
260
|
+
kind: "invalid_input",
|
|
261
|
+
retryable: false,
|
|
262
|
+
message: "Census Data API redirected the request to its 'Missing Key' page — CENSUS_API_KEY is missing or invalid. Get or check a free key at https://api.census.gov/data/key_signup.html.",
|
|
263
|
+
upstreamEndpoint: CENSUS_DATA_LABEL,
|
|
264
|
+
});
|
|
265
|
+
}
|
|
266
|
+
// [P2] 404/429/5xx/4xx ⇒ the shared errors.ts taxonomy (a DOWN service is NEVER
|
|
267
|
+
// an empty result; 400 → invalid_input, 5xx → upstream_unavailable, …).
|
|
268
|
+
if (!res.ok) {
|
|
269
|
+
throw new ToolErrorCarrier(errorFromResponse(res, CENSUS_DATA_LABEL));
|
|
270
|
+
}
|
|
271
|
+
// [P4] 200 ⇒ parse JSON; a non-JSON body (an HTML error page at 200) makes
|
|
272
|
+
// r.json() throw a SyntaxError ⇒ schema_drift (never read as an empty result).
|
|
273
|
+
let body;
|
|
274
|
+
try {
|
|
275
|
+
body = await res.json();
|
|
276
|
+
}
|
|
277
|
+
catch (e) {
|
|
278
|
+
if (e instanceof SyntaxError) {
|
|
279
|
+
throw driftError(CENSUS_DATA_LABEL, "Census CBP returned a non-JSON body at HTTP 200 (likely an HTML 'Missing Key' / error page) — treating as schema drift (never read as an empty result).");
|
|
280
|
+
}
|
|
281
|
+
throw e;
|
|
282
|
+
}
|
|
283
|
+
// ── [P4] Parse the 2D array: row 0 = string[] header, rows 1..N = data. ──
|
|
284
|
+
if (!Array.isArray(body) || body.length === 0) {
|
|
285
|
+
throw driftError(CENSUS_DATA_LABEL, "Census CBP returned a body that is not a non-empty 2D array — treating as schema drift (never a fabricated empty).");
|
|
286
|
+
}
|
|
287
|
+
const header = body[0];
|
|
288
|
+
if (!Array.isArray(header) ||
|
|
289
|
+
header.length === 0 ||
|
|
290
|
+
!header.every((h) => typeof h === "string")) {
|
|
291
|
+
throw driftError(CENSUS_DATA_LABEL, "Census CBP row 0 is not a string[] header row — treating as schema drift (the 2D-array contract changed; never a fabricated empty).");
|
|
292
|
+
}
|
|
293
|
+
// Header-name → column index (order-independent; a missing column ⇒ index -1 ⇒
|
|
294
|
+
// the field maps to null, never a positional mis-read).
|
|
295
|
+
const idx = new Map();
|
|
296
|
+
header.forEach((h, i) => idx.set(h, i));
|
|
297
|
+
const col = (row, name) => {
|
|
298
|
+
const i = idx.get(name);
|
|
299
|
+
return i === undefined || i < 0 ? undefined : row[i];
|
|
300
|
+
};
|
|
301
|
+
const allRows = [];
|
|
302
|
+
for (let i = 1; i < body.length; i++) {
|
|
303
|
+
const raw = body[i];
|
|
304
|
+
if (!Array.isArray(raw)) {
|
|
305
|
+
throw driftError(CENSUS_DATA_LABEL, `Census CBP data row ${i} is not an array — treating as schema drift (never a fabricated empty).`);
|
|
306
|
+
}
|
|
307
|
+
const row = raw;
|
|
308
|
+
allRows.push({
|
|
309
|
+
name: str(col(row, "NAME")),
|
|
310
|
+
geoId: str(col(row, "GEO_ID")),
|
|
311
|
+
naicsCode: str(col(row, "NAICS2017")),
|
|
312
|
+
naicsLabel: str(col(row, "NAICS2017_LABEL")),
|
|
313
|
+
establishments: censusNum(col(row, "ESTAB")),
|
|
314
|
+
employees: censusNum(col(row, "EMP")),
|
|
315
|
+
annualPayrollUsd: mul1000(censusNum(col(row, "PAYANN"))),
|
|
316
|
+
state: str(col(row, "state")),
|
|
317
|
+
});
|
|
318
|
+
}
|
|
319
|
+
// ── [P1] The COMPLETE set for the filter (no server pagination). An OPTIONAL
|
|
320
|
+
// client-side top-N slice is disclosed (totalAvailable stays the full count,
|
|
321
|
+
// so buildMeta derives truncated/complete honestly). ──
|
|
322
|
+
const totalAvailable = allRows.length;
|
|
323
|
+
const notes = [
|
|
324
|
+
KEY_REQUIRED_NOTE,
|
|
325
|
+
PAYROLL_UNITS_NOTE,
|
|
326
|
+
SUPPRESSED_NOTE,
|
|
327
|
+
NO_PAGINATION_NOTE,
|
|
328
|
+
];
|
|
329
|
+
let rows = allRows;
|
|
330
|
+
if (typeof args.limit === "number" &&
|
|
331
|
+
Number.isFinite(args.limit) &&
|
|
332
|
+
args.limit >= 0 &&
|
|
333
|
+
args.limit < allRows.length) {
|
|
334
|
+
rows = allRows.slice(0, args.limit);
|
|
335
|
+
notes.push(`Returned the first ${rows.length} of ${totalAvailable} rows (client-side limit=${args.limit}); CBP has NO server-side pagination, so the remaining ${totalAvailable - rows.length} are not fetched separately — raise limit or narrow the filter to see them.`);
|
|
336
|
+
}
|
|
337
|
+
const filtersApplied = [
|
|
338
|
+
args.naics !== undefined ? `naics:${args.naics}` : "naics:(all)",
|
|
339
|
+
geoFilter,
|
|
340
|
+
`year:${year}`,
|
|
341
|
+
];
|
|
342
|
+
const meta = {
|
|
343
|
+
// MODE only — never the key value (K-test).
|
|
344
|
+
source: `api.census.gov /data/${year}/cbp (County Business Patterns; CENSUS_API_KEY)`,
|
|
345
|
+
keylessMode: false, // ★KEYED — the first key-required source
|
|
346
|
+
returned: rows.length,
|
|
347
|
+
totalAvailable,
|
|
348
|
+
filtersApplied,
|
|
349
|
+
filtersDropped: [],
|
|
350
|
+
fieldsUnavailable: [],
|
|
351
|
+
notes,
|
|
352
|
+
};
|
|
353
|
+
return withMeta({ rows }, meta);
|
|
354
|
+
}
|
|
355
|
+
//# sourceMappingURL=census-economic.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"census-economic.js","sourceRoot":"","sources":["../src/census-economic.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA8DG;AAEH,OAAO,EAAE,gBAAgB,EAAE,iBAAiB,EAAE,MAAM,aAAa,CAAC;AAClE,OAAO,EAAE,UAAU,EAAE,MAAM,iBAAiB,CAAC;AAC7C,OAAO,EAAE,GAAG,EAAE,GAAG,EAAE,MAAM,aAAa,CAAC;AACvC,OAAO,EAAE,QAAQ,EAAsC,MAAM,WAAW,CAAC;AAEzE,8EAA8E;AAC9E,iFAAiF;AACjF,WAAW;AACX,OAAO,EAAE,GAAG,EAAE,CAAC;AAEf,qEAAqE;AACrE,MAAM,gBAAgB,GAAG,gBAAgB,CAAC;AAC1C,MAAM,iBAAiB,GAAG,kBAAkB,CAAC,CAAC,gDAAgD;AAE9F,qEAAqE;AACrE,MAAM,OAAO,GAAG,SAAS,CAAC,CAAC,yDAAyD;AACpF,MAAM,QAAQ,GAAG,WAAW,CAAC,CAAC,mCAAmC;AACjE,MAAM,aAAa,GAAG,SAAS,CAAC,CAAC,qBAAqB;AACtD,MAAM,WAAW,GAAG,IAAI,GAAG,CAAC,CAAC,IAAI,EAAE,OAAO,EAAE,QAAQ,CAAC,CAAC,CAAC;AAEvD,iFAAiF;AACjF,wEAAwE;AACxE,kFAAkF;AAClF,qDAAqD;AACrD,MAAM,qBAAqB,GAAG,CAAC,SAAS,CAAC;AAEzC,MAAM,YAAY,GAAG,MAAM,CAAC,CAAC,8CAA8C;AAE3E,qEAAqE;AACrE,MAAM,iBAAiB,GACrB,oNAAoN,CAAC;AACvN,MAAM,kBAAkB,GACtB,8LAA8L,CAAC;AACjM,MAAM,eAAe,GACnB,2NAA2N,CAAC;AAC9N,MAAM,kBAAkB,GACtB,wMAAwM,CAAC;AAE3M,0EAA0E;AAC1E,uFAAuF;AACvF,MAAM,UAAU,YAAY;IAC1B,MAAM,GAAG,GAAG,OAAO,CAAC,GAAG,CAAC,cAAc,CAAC;IACvC,MAAM,OAAO,GAAG,OAAO,GAAG,KAAK,QAAQ,CAAC,CAAC,CAAC,GAAG,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;IAC1D,OAAO,OAAO,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,SAAS,CAAC;AACvC,CAAC;AAcD,wFAAwF;AACxF,SAAS,SAAS,CAAC,CAAU;IAC3B,MAAM,CAAC,GAAG,GAAG,CAAC,CAAC,CAAC,CAAC;IACjB,IAAI,CAAC,KAAK,IAAI;QAAE,OAAO,IAAI,CAAC;IAC5B,iFAAiF;IACjF,qCAAqC;IACrC,IAAI,CAAC,IAAI,qBAAqB;QAAE,OAAO,IAAI,CAAC;IAC5C,OAAO,CAAC,CAAC;AACX,CAAC;AAED,0EAA0E;AAC1E,SAAS,OAAO,CAAC,CAAgB;IAC/B,OAAO,CAAC,KAAK,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,GAAG,IAAI,CAAC;AACtC,CAAC;AAUD;;;;;GAKG;AACH,MAAM,CAAC,KAAK,UAAU,gBAAgB,CACpC,IAAgC;IAEhC,4EAA4E;IAC5E,MAAM,GAAG,GAAG,YAAY,EAAE,CAAC;IAC3B,IAAI,GAAG,KAAK,SAAS,EAAE,CAAC;QACtB,MAAM,IAAI,gBAAgB,CAAC;YACzB,IAAI,EAAE,eAAe;YACrB,SAAS,EAAE,KAAK;YAChB,OAAO,EACL,qHAAqH;YACvH,gBAAgB,EAAE,iBAAiB;SACpC,CAAC,CAAC;IACL,CAAC;IAED,+EAA+E;IAC/E,6CAA6C;IAC7C,MAAM,IAAI,GAAG,IAAI,CAAC,IAAI,IAAI,YAAY,CAAC;IACvC,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;QACxB,MAAM,IAAI,gBAAgB,CAAC;YACzB,IAAI,EAAE,eAAe;YACrB,SAAS,EAAE,KAAK;YAChB,OAAO,EAAE,gBAAgB,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC,iHAAiH;YAC9J,gBAAgB,EAAE,iBAAiB;SACpC,CAAC,CAAC;IACL,CAAC;IAED,MAAM,SAAS,GAAG,IAAI,CAAC,SAAS,IAAI,IAAI,CAAC;IACzC,IAAI,CAAC,WAAW,CAAC,GAAG,CAAC,SAAS,CAAC,EAAE,CAAC;QAChC,MAAM,IAAI,gBAAgB,CAAC;YACzB,IAAI,EAAE,eAAe;YACrB,SAAS,EAAE,KAAK;YAChB,OAAO,EAAE,qBAAqB,IAAI,CAAC,SAAS,CAAC,SAAS,CAAC,uCAAuC;YAC9F,gBAAgB,EAAE,iBAAiB;SACpC,CAAC,CAAC;IACL,CAAC;IAED,IAAI,IAAI,CAAC,KAAK,KAAK,SAAS,IAAI,CAAC,aAAa,CAAC,IAAI,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC;QAChE,MAAM,IAAI,gBAAgB,CAAC;YACzB,IAAI,EAAE,eAAe;YACrB,SAAS,EAAE,KAAK;YAChB,OAAO,EAAE,iBAAiB,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC,KAAK,CAAC,2EAA2E;YAC/H,gBAAgB,EAAE,iBAAiB;SACpC,CAAC,CAAC;IACL,CAAC;IAED,IAAI,IAAI,CAAC,KAAK,KAAK,SAAS,IAAI,CAAC,QAAQ,CAAC,IAAI,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC;QAC3D,MAAM,IAAI,gBAAgB,CAAC;YACzB,IAAI,EAAE,eAAe;YACrB,SAAS,EAAE,KAAK;YAChB,OAAO,EAAE,iBAAiB,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC,KAAK,CAAC,8FAA8F;YAClJ,gBAAgB,EAAE,iBAAiB;SACpC,CAAC,CAAC;IACL,CAAC;IAED,6DAA6D;IAC7D,IAAI,SAAiB,CAAC;IACtB,IAAI,QAA4B,CAAC;IACjC,IAAI,SAAiB,CAAC;IACtB,IAAI,SAAS,KAAK,IAAI,EAAE,CAAC;QACvB,SAAS,GAAG,MAAM,CAAC;QACnB,SAAS,GAAG,cAAc,CAAC;IAC7B,CAAC;SAAM,IAAI,SAAS,KAAK,OAAO,EAAE,CAAC;QACjC,SAAS,GAAG,IAAI,CAAC,KAAK,KAAK,SAAS,CAAC,CAAC,CAAC,SAAS,IAAI,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,SAAS,CAAC;QACzE,SAAS,GAAG,mBAAmB,IAAI,CAAC,KAAK,IAAI,GAAG,EAAE,CAAC;IACrD,CAAC;SAAM,CAAC;QACN,0EAA0E;QAC1E,IAAI,IAAI,CAAC,KAAK,KAAK,SAAS,EAAE,CAAC;YAC7B,MAAM,IAAI,gBAAgB,CAAC;gBACzB,IAAI,EAAE,eAAe;gBACrB,SAAS,EAAE,KAAK;gBAChB,OAAO,EACL,yIAAyI;gBAC3I,gBAAgB,EAAE,iBAAiB;aACpC,CAAC,CAAC;QACL,CAAC;QACD,SAAS,GAAG,UAAU,CAAC;QACvB,QAAQ,GAAG,SAAS,IAAI,CAAC,KAAK,EAAE,CAAC;QACjC,SAAS,GAAG,+BAA+B,IAAI,CAAC,KAAK,EAAE,CAAC;IAC1D,CAAC;IAED,+EAA+E;IAC/E,gDAAgD;IAChD,MAAM,MAAM,GAAG,IAAI,eAAe,EAAE,CAAC;IACrC,MAAM,CAAC,GAAG,CAAC,KAAK,EAAE,8CAA8C,CAAC,CAAC;IAClE,MAAM,CAAC,GAAG,CAAC,KAAK,EAAE,SAAS,CAAC,CAAC;IAC7B,IAAI,QAAQ,KAAK,SAAS;QAAE,MAAM,CAAC,GAAG,CAAC,IAAI,EAAE,QAAQ,CAAC,CAAC;IACvD,IAAI,IAAI,CAAC,KAAK,KAAK,SAAS;QAAE,MAAM,CAAC,GAAG,CAAC,WAAW,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC;IAClE,MAAM,CAAC,GAAG,CAAC,KAAK,EAAE,GAAG,CAAC,CAAC;IAEvB,MAAM,GAAG,GAAG,WAAW,gBAAgB,SAAS,IAAI,QAAQ,MAAM,CAAC,QAAQ,EAAE,EAAE,CAAC;IAChF,iFAAiF;IACjF,6EAA6E;IAC7E,MAAM,KAAK,GAAG,IAAI,GAAG,CAAC,GAAG,CAAC,CAAC;IAC3B,IAAI,KAAK,CAAC,QAAQ,KAAK,gBAAgB,IAAI,KAAK,CAAC,QAAQ,KAAK,QAAQ,EAAE,CAAC;QACvE,MAAM,IAAI,gBAAgB,CAAC;YACzB,IAAI,EAAE,eAAe;YACrB,SAAS,EAAE,KAAK;YAChB,OAAO,EAAE,oCAAoC,IAAI,CAAC,SAAS,CAAC,KAAK,CAAC,QAAQ,CAAC,KAAK,KAAK,CAAC,QAAQ,YAAY,gBAAgB,gDAAgD;YAC1K,gBAAgB,EAAE,iBAAiB;SACpC,CAAC,CAAC;IACL,CAAC;IAED,+EAA+E;IAC/E,kFAAkF;IAClF,uEAAuE;IACvE,+EAA+E;IAC/E,6EAA6E;IAC7E,oFAAoF;IACpF,mFAAmF;IACnF,iFAAiF;IACjF,4EAA4E;IAC5E,uEAAuE;IACvE,kFAAkF;IAClF,mFAAmF;IACnF,kFAAkF;IAClF,gFAAgF;IAChF,IAAI,GAAa,CAAC;IAClB,IAAI,CAAC;QACH,GAAG,GAAG,MAAM,KAAK,CAAC,KAAK,CAAC,QAAQ,EAAE,EAAE;YAClC,QAAQ,EAAE,QAAQ;YAClB,MAAM,EAAE,WAAW,CAAC,OAAO,CAAC,MAAM,CAAC;SACpC,CAAC,CAAC;IACL,CAAC;IAAC,OAAO,CAAC,EAAE,CAAC;QACX,2EAA2E;QAC3E,6EAA6E;QAC7E,IACE,CAAC,YAAY,KAAK;YAClB,CAAC,CAAC,CAAC,IAAI,KAAK,cAAc,IAAI,CAAC,CAAC,IAAI,KAAK,YAAY,CAAC,EACtD,CAAC;YACD,MAAM,IAAI,gBAAgB,CAAC;gBACzB,IAAI,EAAE,sBAAsB;gBAC5B,OAAO,EAAE,cAAc,iBAAiB,aAAa;gBACrD,SAAS,EAAE,KAAK;gBAChB,gBAAgB,EAAE,iBAAiB;aACpC,CAAC,CAAC;QACL,CAAC;QACD,MAAM,IAAI,gBAAgB,CAAC;YACzB,IAAI,EAAE,sBAAsB;YAC5B,OAAO,EAAE,0BAA0B,iBAAiB,KAAK,CAAC,YAAY,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE;YACrG,SAAS,EAAE,IAAI;YACf,iBAAiB,EAAE,EAAE;YACrB,gBAAgB,EAAE,iBAAiB;SACpC,CAAC,CAAC;IACL,CAAC;IAED,kFAAkF;IAClF,iFAAiF;IACjF,oEAAoE;IACpE,IAAI,GAAG,CAAC,IAAI,KAAK,gBAAgB,IAAI,CAAC,GAAG,CAAC,MAAM,IAAI,GAAG,IAAI,GAAG,CAAC,MAAM,GAAG,GAAG,CAAC,EAAE,CAAC;QAC7E,MAAM,IAAI,gBAAgB,CAAC;YACzB,IAAI,EAAE,eAAe;YACrB,SAAS,EAAE,KAAK;YAChB,OAAO,EACL,kLAAkL;YACpL,gBAAgB,EAAE,iBAAiB;SACpC,CAAC,CAAC;IACL,CAAC;IAED,gFAAgF;IAChF,wEAAwE;IACxE,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,CAAC;QACZ,MAAM,IAAI,gBAAgB,CAAC,iBAAiB,CAAC,GAAG,EAAE,iBAAiB,CAAC,CAAC,CAAC;IACxE,CAAC;IAED,2EAA2E;IAC3E,+EAA+E;IAC/E,IAAI,IAAa,CAAC;IAClB,IAAI,CAAC;QACH,IAAI,GAAG,MAAM,GAAG,CAAC,IAAI,EAAE,CAAC;IAC1B,CAAC;IAAC,OAAO,CAAC,EAAE,CAAC;QACX,IAAI,CAAC,YAAY,WAAW,EAAE,CAAC;YAC7B,MAAM,UAAU,CACd,iBAAiB,EACjB,yJAAyJ,CAC1J,CAAC;QACJ,CAAC;QACD,MAAM,CAAC,CAAC;IACV,CAAC;IAED,4EAA4E;IAC5E,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,IAAI,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC9C,MAAM,UAAU,CACd,iBAAiB,EACjB,oHAAoH,CACrH,CAAC;IACJ,CAAC;IACD,MAAM,MAAM,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;IACvB,IACE,CAAC,KAAK,CAAC,OAAO,CAAC,MAAM,CAAC;QACtB,MAAM,CAAC,MAAM,KAAK,CAAC;QACnB,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,OAAO,CAAC,KAAK,QAAQ,CAAC,EAC3C,CAAC;QACD,MAAM,UAAU,CACd,iBAAiB,EACjB,qIAAqI,CACtI,CAAC;IACJ,CAAC;IAED,+EAA+E;IAC/E,wDAAwD;IACxD,MAAM,GAAG,GAAG,IAAI,GAAG,EAAkB,CAAC;IACrC,MAAmB,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC;IACtD,MAAM,GAAG,GAAG,CAAC,GAAc,EAAE,IAAY,EAAW,EAAE;QACpD,MAAM,CAAC,GAAG,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;QACxB,OAAO,CAAC,KAAK,SAAS,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC;IACvD,CAAC,CAAC;IAEF,MAAM,OAAO,GAAa,EAAE,CAAC;IAC7B,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QACrC,MAAM,GAAG,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;QACpB,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,GAAG,CAAC,EAAE,CAAC;YACxB,MAAM,UAAU,CACd,iBAAiB,EACjB,uBAAuB,CAAC,yEAAyE,CAClG,CAAC;QACJ,CAAC;QACD,MAAM,GAAG,GAAG,GAAgB,CAAC;QAC7B,OAAO,CAAC,IAAI,CAAC;YACX,IAAI,EAAE,GAAG,CAAC,GAAG,CAAC,GAAG,EAAE,MAAM,CAAC,CAAC;YAC3B,KAAK,EAAE,GAAG,CAAC,GAAG,CAAC,GAAG,EAAE,QAAQ,CAAC,CAAC;YAC9B,SAAS,EAAE,GAAG,CAAC,GAAG,CAAC,GAAG,EAAE,WAAW,CAAC,CAAC;YACrC,UAAU,EAAE,GAAG,CAAC,GAAG,CAAC,GAAG,EAAE,iBAAiB,CAAC,CAAC;YAC5C,cAAc,EAAE,SAAS,CAAC,GAAG,CAAC,GAAG,EAAE,OAAO,CAAC,CAAC;YAC5C,SAAS,EAAE,SAAS,CAAC,GAAG,CAAC,GAAG,EAAE,KAAK,CAAC,CAAC;YACrC,gBAAgB,EAAE,OAAO,CAAC,SAAS,CAAC,GAAG,CAAC,GAAG,EAAE,QAAQ,CAAC,CAAC,CAAC;YACxD,KAAK,EAAE,GAAG,CAAC,GAAG,CAAC,GAAG,EAAE,OAAO,CAAC,CAAC;SAC9B,CAAC,CAAC;IACL,CAAC;IAED,8EAA8E;IAC9E,gFAAgF;IAChF,2DAA2D;IAC3D,MAAM,cAAc,GAAG,OAAO,CAAC,MAAM,CAAC;IACtC,MAAM,KAAK,GAAa;QACtB,iBAAiB;QACjB,kBAAkB;QAClB,eAAe;QACf,kBAAkB;KACnB,CAAC;IAEF,IAAI,IAAI,GAAG,OAAO,CAAC;IACnB,IACE,OAAO,IAAI,CAAC,KAAK,KAAK,QAAQ;QAC9B,MAAM,CAAC,QAAQ,CAAC,IAAI,CAAC,KAAK,CAAC;QAC3B,IAAI,CAAC,KAAK,IAAI,CAAC;QACf,IAAI,CAAC,KAAK,GAAG,OAAO,CAAC,MAAM,EAC3B,CAAC;QACD,IAAI,GAAG,OAAO,CAAC,KAAK,CAAC,CAAC,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC;QACpC,KAAK,CAAC,IAAI,CACR,sBAAsB,IAAI,CAAC,MAAM,OAAO,cAAc,4BAA4B,IAAI,CAAC,KAAK,0DAA0D,cAAc,GAAG,IAAI,CAAC,MAAM,6EAA6E,CAChQ,CAAC;IACJ,CAAC;IAED,MAAM,cAAc,GAAG;QACrB,IAAI,CAAC,KAAK,KAAK,SAAS,CAAC,CAAC,CAAC,SAAS,IAAI,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,aAAa;QAChE,SAAS;QACT,QAAQ,IAAI,EAAE;KACf,CAAC;IAEF,MAAM,IAAI,GAA0B;QAClC,4CAA4C;QAC5C,MAAM,EAAE,wBAAwB,IAAI,iDAAiD;QACrF,WAAW,EAAE,KAAK,EAAE,yCAAyC;QAC7D,QAAQ,EAAE,IAAI,CAAC,MAAM;QACrB,cAAc;QACd,cAAc;QACd,cAAc,EAAE,EAAE;QAClB,iBAAiB,EAAE,EAAE;QACrB,KAAK;KACN,CAAC;IAEF,OAAO,QAAQ,CAAC,EAAE,IAAI,EAAE,EAAE,IAAI,CAAC,CAAC;AAClC,CAAC"}
|